Friday, 5 July 2013

Is this a defect?

Is this defect?  When i opened FB just now, i witnessed this behavior.

What can be a reason for this behavior?  Have to analyse !!

This is my Environment.
  • Web Browser - Chrome 27.0.1453.116
  • Operating System - Microsoft Windows Vista
  • Screen Resolution - 1280 x 800
  • Browser Size - 1280 x 642
  • Javascript - Enabled
  • Flash Version - 11.7.700
  • Colour Depth - 32 bit

Now, How to get the above details in an easy way.  Just open www.supportdetails.net in your browser.

Thursday, 4 July 2013

Exploratory Testing – Learner’s Series -1

What is Exploratory Testing??


It is Quick Loop of “What You Learn influences what you Test & What You Test feeds your Learning

Exploratory Testing can be done with detailed analysis which drives the testing in a structured way and in real time we learn about what we test.

Let’s take a problem/puzzle (Puzzle 1) to improve our Exploratory Testing skills.

Puzzle

 Puzzle 1 - 

When i played around with the puzzle for the very first time, i could observe (I missed important points too)

Puzzle Elements
  • Yellow Button, 4 Blue Buttons, Vertical Transparent Button, Question mark & Drum at the Right Hand Side, 1 Red & Blue button in the Horizontal lines
  • Right click on the puzzle, we can understand that it is a Flash app
  • Zooming is possible, Quality can be modified (in Chrome), Result of Refresh

Initial Observation

  1. Yellow button just moves red button 1 step and brings back to original position,
  2. Red button jumps in equal steps in the horizontal meter irrespective of  vertical meter button position,
  3.  Red button by any way does not reach the end in its horizontal meter,
  4. Vertical meter's button position influences blue button's distance movement(at the top, makes jump twice than its previous step)  When in top blue button moves to end of its meter
  5. Blue vertical button pressing direction influences the blue horizontal button's movement, down to up or up to down


Analysis

When you are having a problem in your hand(while testing), we need to
  • Analyse the Default position
  • Identify the elements present in the problem
  • Observe, Analyze, Change One factor at a time(OFAT) on the problem
  • Observe, Analyze, Change Multi factor at a time(MFAT) on the problem
  • Check whether it is possible to get some Clues on the problem from different sources?
  • Check whether there is a Consistency in behavior?

You might need to take screenshots, Video, save logs, collect environment details etc for your bug advocacy.

Observations

When testing, we might need to

  • Take Screenshot of the problem(if possible & needed)
  • Observe the elements in your problem(this is the structure)
  • Press buttons one by one understand how they work(Take screen shots or Video)
  • Analyze whether different buttons have relationship between them or the order of button press influence the result.
  • Look for clues – Explore the Question mark  & read through, Clues are helpful in jump-starting our testing or improving our understanding
  • Check whether the operation is same across different browsers?
  • Disable Flash & check how the puzzle is displayed?  Are you observing Error Handling?? Refresh the page & what you observe?
  • Give a early heads-up to your stake holders to check whether you are going in the right direction
  • Check whether keeping idle of the operations influences or affects the puzzle results?
  • Check whether the puzzle can be influenced by Key board inputs?

Whether the points above changed your perception of taking your first step when given a problem?

Let’s Explore & Learn.

Happy Testing!!!!!!!!!!

Reference: -

Wednesday, 3 July 2013

Cross Browser Testing - Basics - Learner's Series - 5

Cross Browser Testing (Checking) Tools

Depending on the project context, you might want consider these tools for BCT.

·         BrowserShots
·         Adobe Browser lab
·         Browsera
·         Crossbrowsertesting
·         Spoon
·         Browerstack
·         Browsercam

Some of these tools check compatibility by taking & comparing screenshots of the web pages displayed across different browser/OS combinations.(Literally a Cross browser Testing as a Service)

For more details on, how the individual tools work, please refer their website or contact their customer support.

In some project environments, the stakeholders might not be interested in exposing their web pages to external services due to various business or environment factors. 

Just a few examples – Pros & cons of different tools can be analysed(in detail) based on your project context)

v  Home grown Automation Suite for BCT
Example –
Selenium Grid to drive different browser OS combinations in parallel
Automation Framework to run across different web pages based on scenarios and comparing the screenshots or working through the browser’s DOM regarding placements of different GUI elements and comparing their placements with other browsers.

·         Tricentis Tosca Cross Browser Testing (http://www.tricentis.com/en/tosca/x-browser)

Test build Scope

Depending on your Project context & Development model (waterfall, Agile), your Execution build scope can be planned.
Example: -
For your Execution Scope for BCT of your application your scope can be planned something like this,

Browser
Priority
Build 1++
Build 2
Build 3
IE 8
High
Functional (50% of Progression features )and the relevant user experience
Functional (Remaining 50% of Progression features )and the relevant user experience
Bug Testing
Firefox 11, 12
IE 9
Safari 5
Medium

High Risk End to End (E2E) Scenarios, Part of user documentation,  Data Management, Configuration, User Management, Testability  & Pages Covering different Page Templates

Bug Testing
Chrome 18
Firefox 3.x
Low
High Risk E2E Scenarios, Data Handling, User Management & Pages covering different Page Templates

Bug Testing
Chrome 12
Firefox 4.x
Low
E2E Scenarios & Pages covering different Page Templates

Bug Testing

++ - Depending on the Project context,  build scope might include Regression Test cases, lesser builds, more Sprints etc.

References

http://mattarcherblog.wordpress.com/2011/11/08/risk-based-cross-browser-testing-with-scrum/

Cross Browser Testing - Basics - Learner's Series - 4

Browser Selection

To shortlist browsers & their OS versions, we can consider a combination of data from

  • Browser Trend Analysis
  • Technical analysis to remove browser redundancy
  • Risk Based Browser/OS matrix


Output of Browser Trend Analysis can be used to analyze the similarities between different browsers based on

·         Layout Engine
·         Shell Features
·         JavaScript engine
·         Plug-ins

Different browser versions can be grouped and few of them can be considered for the execution scope, based on the risk attributes of the browser/OS combination & prioritizing them.

Risk Based OS – Browser Priority

To identify each of the supported browser’s priority,

Priority = (probability of the Bug occurring) x (Expected loss)

Probability of the Bug

Variety of factors has to be considered for estimating the “probability” of a bug being discovered in a website while using a particular browser.

·        Age of the Browser
Age of the browser is meant, when the most stable browser version was released to the market/users. Ex: - IE 6.0 is released in 2001 and is the default browser shipped with Windows XP, which used to have a large market share.  Probability of Bug in an IE 6.x version is higher, due to the lack of compatibility with modern web standards.
·         Default Browser used by the Dev Team(Example – Firefox 3.6.x)
·         Similarities between Default(Example – Firefox 3.6.x) & the Targeted browsers(Example – Firefox 3.6.y, Safari 5)

Expected Loss

Estimating the “expected loss” can be found out by the popularity of a browser on which the app is mostly run (No. of users & Type of usage) and the stability of the browser with respect to the client technologies used.
An application can be used by users who are Internal/External/Both. 

This type of usage along with the probable number of users who access the application decides the impact of the browser /OS version (Expected Loss)

The analysis ((probability of the Bug occurring) x (expected loss)) will result in the identification of priority as “high”, “medium” and “low” categories for a particular application.

Browser**
Probability of Bug
Expected Loss
Priority
IE 6
High
Low
Low
IE 7
High
Medium
High
IE 8
Medium
High
High
IE 9
Medium
Medium
Medium
Firefox 3.x
Low
Medium
Low*
Firefox 4.x
Low
Low
Low*
Firefox 5.x+
Low
High
Low
Firefox 11
Medium
Medium
Medium
Firefox 12
Medium
Medium
Medium
Apple Safari  5
Medium
Medium
Medium
Google Chrome 12
Low
Medium
Low*
Google Chrome 13+
Low
High
Low
Google Chrome 18
Low
High
Low

Low* - Low Priority/Importance browser.
** - For the above analysis, Microsoft Windows OS is considered & new versions of Browser/OS/devices might have been released by now.+ - Since Firefox & Google Chrome browser versions are getting iterated fast, Test coverage is recommended for the latest major, stable version of the browser & analytics data from your web application.

All the credits for this post go to,

Matt Archer - mattarcherblog.wordpress.com
Brent Strange - http://QAInsight.net

as I have just collated the information from their sites, to give a basic idea on browser selection.

Both of them have posted detailed & more analyzed information on Cross Browser Testing.

For info on, Risk Based, Cross Browser Testing with Scrum, Please refer

For info on, various browser’s technology background, Please refer

Cross Browser Testing - Basics - Learner's Series - 3

BCT for User Experience

To test user experience, we might need to consider the following: 

Check
Remarks
Font size validation
Text Size validations through Browser
Page layout in different resolutions
640x480, 800x600, 1024x768, 1366x768(as applicable, based on the resolutions supported) with respect to Oses supported
All images and alignment
Are all graphics properly aligned?
Assure that command buttons are all of similar size and shape, and same font & font size.
Banner style & size & display exact same as existing windows
Does text wrap properly around pictures/graphics?
Time & Date(Display as well as Functionality)
1) Time outs(for any operation which needs to be completed with in a duration)( Online Bank, Transport Reservation, Email account etc)
2) Time Zone difference
3) Year 3K(Y2K issue)
4) Day Light Savings
5) Crossing Time Zones( + Machines in different time zones/Geographies)
6) Leap Days
7) Leap Days in Non Leap Year
8) Always Invalid days( February 30, 31, 31st on Odd months)
9) Year end + January 1st
10) Previous Centuries
11) Different Time / Date formats
12) Reset Clock forward or backward
13) Country Specific public holidays
Page zoom-in and zoom-out functionality

GUI Colors
Are hyperlink colors Uniform across the application?
Are the field backgrounds the correct color?
Are the field prompts the correct color?
Are all the buttons are in standard format and size?
Is the page background (color) distraction free?
Content
All fonts to be the same
Are all the screen prompts specified in the correct screen font?
Does content remain if you need to go back to a previous page, or if you move forward to another new page?
Is all text properly aligned?
Is the text in all fields specified in the correct screen font?
Is all the heading are left aligned?
Does the first letter of the second word appears in lowercase?
Instructions
Is all the micro-help text(i.e. tool tip) spelt correctly on this screen?(as applicable)
Progress messages on load of tabbed(active screens) screens
Navigation
Are all disabled fields avoided in the TAB sequence?
Are all read-only fields avoided in the TAB sequence?
Can all screens accessible via buttons on this screen be accessed correctly?
Does a scrollbar appear if required?
Does the Tab Order specified on the screen go in sequence from Top Left to bottom right? This is the default unless otherwise specified.
Is there a link to home on every single page?
On open of tab focus will be on first editable field
When an error message occurs does the focus return to the field in error when the user cancels it?
Usability
Are all the field prompts spelt correctly?
Assure that option boxes, option buttons, and command buttons are logically grouped together in clearly demarcated areas “Group Box”
Do pages print legibly without cutting off text?
Does the site convey a clear sense of its intended audience?
Few more testing can be done with respect to CSS validation, HTML or XHTML validation, Ajax and JQeury functionality, Header and footer sections, Page content alignment to center, LHS or RHS, Special characters with HTML character encoding, Page zoom-in and zoom-out functionality, CSS Reset
Preferences
JavaScript Off,   Cookies Off,   Security( High, Medium,  Low), Pop Up blocker, Download File Blocker,   Delete Browsing History


When?

Based on your Project context, it is better to consider the below points when deciding about the schedule of BCT

  • Important & Complex functionality of the applications tested and is in stable condition
  • GUI Design is relatively stable and is not scheduled for a major haul
  • Contains browser plug-in’s related to Flash, Flex, Silver light etc


Depending up on the Application stability, Environment availability BCT is generally done towards the end of the project or just before UAT; however there is no defined time during which you need to do BCT. 

The suggestion would be consider your project context to decide, when you would do BCT.

Cross Browser Testing - Basics - Learner's Series - 2

When there is scope for BCT in your application, you might want to consider


  • Features/ Functionalities that will be covered 
  • User experience issues that might be faced
  • When you would do BCT?
  • Selecting the right or closest possible Browser/OS/Device Matrix
  • Test cases that would be needed for BCT?
  • Cross Browser Test Tools
  • Execution Build Scope

BCT Scope - Functional Test Areas


From the Major areas of functional testing, variety of specific test scenarios can be reused or designed by combining one or more areas mentioned below for BCT.

E2E Scenario Testing
Testing that cover multiple activities connected end-to-end across various functionalities of an application.

User documentation
Some web applications have a Help guide which is part of the functionality explains about the ways to use the application, GUI etc, which might be necessary for Customer Satisfaction.

Data Management
Some web applications displays different types of data which might be very important(Exam results, Bank Account Transactions, Travel Reservations etc)
  •        Input data that is processed by the application.
  •        Output data that results from processing by the application.
  •        Preset data that is supplied as part of the application, or otherwise built into it, such as databases, default values, etc and will be displayed as part of administration/configuration etc
  •       Persistent data that is stored internally and expected to persist over multiple operations. This includes modes or states of the application, such as options settings, view modes, contents of documents, etc and displayed to the user.

Testability
Testing that covers the quality of Transaction/Error Logs (only if it is stored in Database and local disk), Error Messages, and All pop-up messages of an application.

User Management
Testing that covers the User Management – Creation, Updating (username, password, Profiles etc), removing users (if user management is not part of the E2E Scenarios testing) and is used by different user groups on a regular basis.

Configuration
Testing that covers the Administration pages, Application Configuration (User-level) etc of an application. This testing will be applicable only if the application has administration/different user levels as its features.

Page template testing
Testing that covers features from at least one web page from different web page templates of an application. This testing will be applicable only if the application has many page templates.

The above testing might cover the below combined with functionality
·         GUI functionality- Displaying and working of various GUI controls with Functionality
·         Client-side validations
Mandatory Fields which is needs to be entered while using an application. 
Example: - In Some web applications, the functionality might require the mandatory fields are Name, Country etc.  to be entered. When the data is missed, client side application itself will validate instead of the server on the correctness.

·         Effect of browser settings on functionality of the application
Required settings with respect to Cookies, JavaScript, and Security Levels will have an effect on the functionality of the application.

·         Level of Accessibility implementation( Administrator, Different User levels, Single Sign-On)

·         Browser’s ability to inter-operate with other software programs(PDF Reader, Office document(Microsoft, Open Office)

Tuesday, 2 July 2013

Cross Browser Testing - Basics - Learner's Series - 1

Public facing web sites are intended to reach wider audience, who tend to browse the site with their available or preferred OS/Browser/device Combinations

These web applications (hosted in these sites) should get rendered/displayed as uniform as possible across these Browser/OS combinations (as expected by the Customers/users), however the browsers have differences among them with respect to the

  • Layout Engine
  • Shell Features
  •  JavaScript engine

 Below is a sample list of few browsers from different vendors/opensource, their differences.
Browser
JavaScript Eng. -Comp. version
Layout Engine
OS Support
Microsoft IE 6.0
JScript 5.6-
JavaScript 1.5
Trident 6.0.x
Win 98, 98SE, ME, NT4, 2000, XP
IE 7.0
JScript 5.7-
JavaScript 1.5
Trident 7.0.x
XP SP2, Vista
IE 8.0
JScript 5.8-
JavaScript 1.5
Trident 8.0.x
XP, 2003, Vista, 2008, Windows 7
IE 9.0
Jscript 9.0-
JavaScript 1.8.1
Trident 9.0.x
 Windows 7, 2008 R2, Vista SP2
Mozilla Firefox 5.0
SpiderMonkey-JavaScript 1.8.5
Gecko 5.0.x
Windows Vista, 2000, XP, Server 2003, 2008,  Mac OSX, Linux and Unix
Firefox 11.0
SpiderMonkey-JavaScript 1.8.5
Gecko 11.0.x
Windows 7, Vista, 2000, XP, Server 2008, Mac OSX, Linux and Unix
Firefox 12.0
SpiderMonkey-JavaScript 1.8.5
Gecko 12.0.x
Windows 7, Vista, 2000, XP, Server 2008, Mac OSX, Linux and Unix
Apple Safari 3.0b
JavaScriptCore
JavaScript 1.?
WebKit  522
Mac OS 10.4.9 or later
Safari 3.0.4
JavaScriptCore
JavaScript 1.?
WebKit  523
Mac OS 10.5
Safari 4.x
Squirrel Fish JavaScript 1.?
WebKit 530
Mac OS 10.6
Safari 5.x
Squirrel Fish JavaScript 1.?
WebKit 534
Mac OS 10.7

Browser compatibility testing (BCT) is the testing of “implementation of client-side technologies” of browser. BCT tests whether the business critical functionality is delivered in the same manner across multiple browser and OS combinations.

"Cross Browser compatibility issues stems from the differences in rendering of HTML, XHTML, XML, etc by different browsers and OS combinations into a formatted web page"

"Though the Browser Translation Rules/Standards are set by the World Wide Web Consortium (W3C), the standards are interpreted differently by different organizations resulting in varied user experiences"

Scope

BCT test scope will be driven by the need of each individual application requirements based on

·         No. of Users
·         Type of Usage (Internal/External/Both) – Private/Public Facing websites
·         Usage of various browser & OS variants

Also, it may not be feasible to test all the available test cases or functionality on every OS / Browser combination due to constraints imposed by size of test suite, Test Scope, Schedule / Cost constraints, Tester Skill sets, Environment availability, Stakeholder expectations etc.
BCT of the application might not include:
·         Performance Testing of functionality in different OS/Browser Combinations
·         Security and vulnerability testing of the application
·         508 Compliance testing of the apps.

Browser usage Trend Analysis


Browser usage trend analysis for any application can be analysed in different ways.  Please discuss with your web master/administrator (In house or External)

  • Configuring the web server, to log the Browser/OS combinations, which access the web site. Depending on the web server/host, there might be a need to restart the server, which you might not prefer. For best options,
             Example: -
             By changing httpd.conf in Apache web server.  This is a configuration change; however this technique      requires analyzing the logs either manually or by writing a script to filter out the Browser/OS details
  • Design /Code Change to add Servlet filter in the application code and intercepting all requests received by application server.  This technique will intercept all URLs and then in the filter write code to record counts of different browsers. Performance overhead of adding Servlet filters might require discussion with your Design Team.


If you do not wish to change your web server configuration or Application code, then browser usage data (world wide usage) due to various factors can be obtained from (some services might need to be bought)



References
pP.S.  Currently i do not have any data regarding Phone, Tablet or any other hand held device's browser/OS combination details