- Impact
- 2
World Wide Web has revolutionized the traditional ways of acquiring and sharing information. Consequently, the nature and behaviour of the websites have introduced vital software testing challenges. Normally in depth coverage is given to functionality aspect of the website, while the non-functional testing remains un-explored.
This article describes the major aspects that need to be focused while conducting non-functional testing. The author does not intend to delve deep into actually how the testing will be conducted, rather just attempted to acquaint the readers at an abstract level by highlighting what areas need to be tested.
Non Functional Testing suggests "HOW WELL" the product behaves. This HOW WELL definition is with reference to ease of use, reliability, performance, security etc. The scope of this article is limited to the following non-functional testing types:
Security Testing
Performance Testing
Load Testing
Usability Testing
Security Testing
Security Testing refers to testing of software in order to keep company data and resources secure from mistaken/accidental users, hackers, and other malevolent attackers. In this context, following are the major areas to focus on:
Database-driven websites are very common, and the parameters of a database query can often be seen in the URL of web pages. Here comes the time to test for Data Leakage. This refers to the fact that when information is passed in the query string i.e. in URL (e.g. using the "Get" method), an attacker may be able to access the database by iterating through the sequence of parameters passed in query string that forms part of the URL. For instance, a customer may somehow manage to alter the price of an order, or mark an unpaid order as paid. Special consideration needs to be given to this aspect while testing security related scenarios of websites.
While testing a website from a security perspective, make sure to test for SQL Injection Attacks. This is the name given to a vulnerability caused by poor input validation in an application, resulting in running unintended SQL scripts. For details on how to test for the mentioned vulnerability, refer to http://www.unixwiz.net/techtips/sql-injection.html.
Weakness in Control Panel of a shopping cart is another domain to focus on. In the Control Panel, roles and rights need to be clearly defined. Restricted access needs to be given to users e.g. in case of an admin control panel for a shopping cart, the right to view and alter the transactions needs to be restricted to a particular role and should not be given to multiple users. Only the super user should have access to alter the transaction details. Moreover, thorough testing is required to test the access rights definition feature. This is important because it may happen that a user with limited access may somehow manage to elevate his/her rights and gain the rights of any other user (say super user).
Incase of credit card transactions, if credit card numbers are to be stored in the database, a tester needs to ensure that a proper encryption algorithm is in place to encrypt those numbers and later on decrypt them.
Performance Testing
Performance testing is testing that is performed to determine how fast some aspect of a system performs under a particular workload. This is specifically related to response times. With reference to this, following are the main points of considerations:
Web hits should cater for handling peak hit rate, not the average rate.
Response time should be catered with respect to download speed and complexity of the web page e.g. graphics displayed on the web page. A poorly-designed highly graphical dynamic website will be considered ‘slow’ and this element needs to be catered for seriously while testing performance of websites. This is specially required incase of broadband connections; since in this case pages that don’t appear instantly will perhaps never be seen at all.
Database indexing technology needs to be tested incase you have switched from a moderately populated database to an over populated database, e.g. if you move from a database with 50 rows in its tables to 50,000 rows, response time should not increase too much; rather it should be a matter of milliseconds. If however, response time increases too much this implies indexing has not been implemented on the corresponding columns, which ultimately is a performance issue and needs to be rectified.
The web application needs to be load tested to check the scalability of the website. To cater for this aspect of performance testing, refer to Load Testing.
Load Testing
Load testing refers to testing the system behavior and performance under stressful conditions.
As mentioned in the context of performance testing, load testing is done to analyze scalability of the website.
[Web application scalability is the ability to sustain the required number of simultaneous users and/or transactions, while maintaining adequate response times to end users].
Now the question arises, why is load testing so important? The reason is, most performance issues arise only when the server is stressed with a high user load. This means that you should perform load testing to know how many concurrent visitors your site can serve flawlessly. It can be difficult to organize such testing without the help of a group of real users. The best approach would be to use an automated load test tool which would serve the purpose of simulating many different users coming from different IP addresses, each with their own parameters: cookies, input data for various page forms, name and password, connection speed and their own specific path through the site. Such an automated tool may also help in testing the website at a constant load through out the entire test time or increase/decrease load after specified time intervals. Eventually detailed reports and graphs may be generated to depict test results with respect to performance issues, response time, bandwidth usage, errors generated etc.
For detail on website testing tools, refer to http://www.softwareqatest.com/qatweb1.html.
Usability Testing
Usability testing refers to evaluating the ease with which users can learn and use a product (such as a website) for its intended purpose.
If usability testing uncovers difficulties, such as people having difficulty in understanding instructions, or interpreting feedback, then this aspect needs to be highlighted and design needs to be improved and re-tested.
Different usability evaluation techniques may be used. For instance, formal testing (such as lab testing, questionnaires etc) or informal testing (such as walkthroughs, peer reviews etc).
Following points may help in performing formal testing:
While testing for usability issues, the first and the foremost thing to note is check whether the company’s home page is displaying information regarding what the company does and its purpose.
It is a good practice to have the name and logo placed on every page and the logo can be a link to the home page (except on the home page itself, where the logo should not be a link: never have a link that points right back to the current page).
Each page should be structured so as to break long text into smaller units. e.g. grouping or subheadings can be used to help ignore large chunks of the page in a single glance.
Another important usability issue is to check for link titles, which provide users with a preview of where each link will take them, before they have clicked on it.
Test for broken links.
The behavior of websites with respect to supported resolution(s) can be tested to ensure that changing resolution does not distort the layout of web pages.
The layout of all the pages should be in accordance with the design specified. All pages need to be consistent with respect to the following:
Layout of controls
Color scheme of pages
Font style, size and case of forms and reports
Font style and size of text displayed on all the pages of website.
Font style, size and case of column headings incase of tables throughout the website
The navigation links in all the subsequent pages need to depict the entire path followed to reach a specific page. Clicking any of the links should lead to the corresponding page. Remember, if you are testing with the intent to improve the usability of the website, you should not have the feeling "Ok, where to go now?" or "Where did I start from?".
On all ‘Submit’ buttons, there should be a proper indication to the user corresponding to the action performed (success/failure).
Test for semantic and syntactic errors incase of text displayed on all the pages.
On long forms (e.g. while adding users, providing company information etc), a ‘Clear’ button has to be provided to reset all the fields.
It would be a good idea to provide a proper welcome message along with username when the user logs in.
If multiple records are to be displayed per page, there should be a limit on the maximum number of records displayed per page (e.g. 10 or 20). An alternative could be to ask the user to specify his/her own limit. Moreover, distribution of records in terms of page numbers may be provided to increase readability. (e.g. Page: 1 2 3 4).
Test for navigation within a form i.e. all the fields within a form should have proper tab-order.
Different colors may be used to distinguish visited and un-visited links.
Posted by Ghulam Kubra at http://www.binarytrends.com/article_details/12/Testing-Guidelines-For-Non-Functional-Testing
This article describes the major aspects that need to be focused while conducting non-functional testing. The author does not intend to delve deep into actually how the testing will be conducted, rather just attempted to acquaint the readers at an abstract level by highlighting what areas need to be tested.
Non Functional Testing suggests "HOW WELL" the product behaves. This HOW WELL definition is with reference to ease of use, reliability, performance, security etc. The scope of this article is limited to the following non-functional testing types:
Security Testing
Performance Testing
Load Testing
Usability Testing
Security Testing
Security Testing refers to testing of software in order to keep company data and resources secure from mistaken/accidental users, hackers, and other malevolent attackers. In this context, following are the major areas to focus on:
Database-driven websites are very common, and the parameters of a database query can often be seen in the URL of web pages. Here comes the time to test for Data Leakage. This refers to the fact that when information is passed in the query string i.e. in URL (e.g. using the "Get" method), an attacker may be able to access the database by iterating through the sequence of parameters passed in query string that forms part of the URL. For instance, a customer may somehow manage to alter the price of an order, or mark an unpaid order as paid. Special consideration needs to be given to this aspect while testing security related scenarios of websites.
While testing a website from a security perspective, make sure to test for SQL Injection Attacks. This is the name given to a vulnerability caused by poor input validation in an application, resulting in running unintended SQL scripts. For details on how to test for the mentioned vulnerability, refer to http://www.unixwiz.net/techtips/sql-injection.html.
Weakness in Control Panel of a shopping cart is another domain to focus on. In the Control Panel, roles and rights need to be clearly defined. Restricted access needs to be given to users e.g. in case of an admin control panel for a shopping cart, the right to view and alter the transactions needs to be restricted to a particular role and should not be given to multiple users. Only the super user should have access to alter the transaction details. Moreover, thorough testing is required to test the access rights definition feature. This is important because it may happen that a user with limited access may somehow manage to elevate his/her rights and gain the rights of any other user (say super user).
Incase of credit card transactions, if credit card numbers are to be stored in the database, a tester needs to ensure that a proper encryption algorithm is in place to encrypt those numbers and later on decrypt them.
Performance Testing
Performance testing is testing that is performed to determine how fast some aspect of a system performs under a particular workload. This is specifically related to response times. With reference to this, following are the main points of considerations:
Web hits should cater for handling peak hit rate, not the average rate.
Response time should be catered with respect to download speed and complexity of the web page e.g. graphics displayed on the web page. A poorly-designed highly graphical dynamic website will be considered ‘slow’ and this element needs to be catered for seriously while testing performance of websites. This is specially required incase of broadband connections; since in this case pages that don’t appear instantly will perhaps never be seen at all.
Database indexing technology needs to be tested incase you have switched from a moderately populated database to an over populated database, e.g. if you move from a database with 50 rows in its tables to 50,000 rows, response time should not increase too much; rather it should be a matter of milliseconds. If however, response time increases too much this implies indexing has not been implemented on the corresponding columns, which ultimately is a performance issue and needs to be rectified.
The web application needs to be load tested to check the scalability of the website. To cater for this aspect of performance testing, refer to Load Testing.
Load Testing
Load testing refers to testing the system behavior and performance under stressful conditions.
As mentioned in the context of performance testing, load testing is done to analyze scalability of the website.
[Web application scalability is the ability to sustain the required number of simultaneous users and/or transactions, while maintaining adequate response times to end users].
Now the question arises, why is load testing so important? The reason is, most performance issues arise only when the server is stressed with a high user load. This means that you should perform load testing to know how many concurrent visitors your site can serve flawlessly. It can be difficult to organize such testing without the help of a group of real users. The best approach would be to use an automated load test tool which would serve the purpose of simulating many different users coming from different IP addresses, each with their own parameters: cookies, input data for various page forms, name and password, connection speed and their own specific path through the site. Such an automated tool may also help in testing the website at a constant load through out the entire test time or increase/decrease load after specified time intervals. Eventually detailed reports and graphs may be generated to depict test results with respect to performance issues, response time, bandwidth usage, errors generated etc.
For detail on website testing tools, refer to http://www.softwareqatest.com/qatweb1.html.
Usability Testing
Usability testing refers to evaluating the ease with which users can learn and use a product (such as a website) for its intended purpose.
If usability testing uncovers difficulties, such as people having difficulty in understanding instructions, or interpreting feedback, then this aspect needs to be highlighted and design needs to be improved and re-tested.
Different usability evaluation techniques may be used. For instance, formal testing (such as lab testing, questionnaires etc) or informal testing (such as walkthroughs, peer reviews etc).
Following points may help in performing formal testing:
While testing for usability issues, the first and the foremost thing to note is check whether the company’s home page is displaying information regarding what the company does and its purpose.
It is a good practice to have the name and logo placed on every page and the logo can be a link to the home page (except on the home page itself, where the logo should not be a link: never have a link that points right back to the current page).
Each page should be structured so as to break long text into smaller units. e.g. grouping or subheadings can be used to help ignore large chunks of the page in a single glance.
Another important usability issue is to check for link titles, which provide users with a preview of where each link will take them, before they have clicked on it.
Test for broken links.
The behavior of websites with respect to supported resolution(s) can be tested to ensure that changing resolution does not distort the layout of web pages.
The layout of all the pages should be in accordance with the design specified. All pages need to be consistent with respect to the following:
Layout of controls
Color scheme of pages
Font style, size and case of forms and reports
Font style and size of text displayed on all the pages of website.
Font style, size and case of column headings incase of tables throughout the website
The navigation links in all the subsequent pages need to depict the entire path followed to reach a specific page. Clicking any of the links should lead to the corresponding page. Remember, if you are testing with the intent to improve the usability of the website, you should not have the feeling "Ok, where to go now?" or "Where did I start from?".
On all ‘Submit’ buttons, there should be a proper indication to the user corresponding to the action performed (success/failure).
Test for semantic and syntactic errors incase of text displayed on all the pages.
On long forms (e.g. while adding users, providing company information etc), a ‘Clear’ button has to be provided to reset all the fields.
It would be a good idea to provide a proper welcome message along with username when the user logs in.
If multiple records are to be displayed per page, there should be a limit on the maximum number of records displayed per page (e.g. 10 or 20). An alternative could be to ask the user to specify his/her own limit. Moreover, distribution of records in terms of page numbers may be provided to increase readability. (e.g. Page: 1 2 3 4).
Test for navigation within a form i.e. all the fields within a form should have proper tab-order.
Different colors may be used to distinguish visited and un-visited links.
Posted by Ghulam Kubra at http://www.binarytrends.com/article_details/12/Testing-Guidelines-For-Non-Functional-Testing
