Sunday, June 6, 2010

Low Cost Performance Testing with free tools - Front-End Performance Testing

In this article I will share information on conducting Front-End Performance Testing (FPT) which is low cost yet very effective and does not need any commercial tools to be used. Before starting with the core topic I should thank Steve Souders for writing great books High Performance Web Sites and Even Faster Websites and Scott Barber for making it easier by bringing in the heuristic approach.

So, what is Front-End Performance Testing?

The Front-End Performance Testing is Monitoring and measuring the user experience from  the performance point of view and validating the front-end design with respect to the practices for high performance websites without any focus on the server side load. FPT is conducted to find any client side performance problems in the application.

You may ask why do we need to conduct FPT?

FPT is important for the reasons listed below (few of the points I have taken it from Scott Barber’s article on FPT)

  • 50-90% of a web page's response time results from front-end design and implementation  (Taken from Scott Barber’s article)
  • The front-end design and development of websites is conducted with little to no thought on performance (Other than, possibly reducing the size of the graphics)
  • Usually neglected by development considering the client system not in control. Also, neglected by test team during testing
  • Checking/Testing for potentially largest and cheapest performance improvements
  • Relatively easy to conduct tests and investigate the Front-End performance issues (in comparison with backend performance tests)
  • Minimal learning curve, can be tested by developers or testers [Capturing the data may not need any experience but effective analysis of the captured data needs reasonable performance testing skills and understanding]
  • The client side processing and activities done at the client side by Java Script and Silverlight or not captured by LoadRunner like performance test tools that work on the HTTP/HTTPs layer (E.G. You would have seen in Google that suggests the search criteria on typing every single letter in the search box. This is a JSON request that should be quick enough not to let the user know that the response is from the server. Measuring the response time and user experience of this type of scenarios could be real hard without FPT)
  • It needs less effort and can be done with Functional Testing, Can be automated as well. [For testing and reporting a website it would take about 4 hours based on my experience (here it also matters how big is the website) :-)]. I usually combine it with manual exploration during initial navigation through the application for learning it. In fact the information captured by HTTP sniffers is worth taking a look as you can understand about the application much better and device your tests.(not only performance but also other tests such as security and even functional or validation etc.,)

The objective of FPT is to

  • Find client side performance problems without any focus on server loading or resource utilization by the server
  • To understand the end user experience

Few considerations to be made while conducting FPT include the following

  • It is important to consider all the web pages in the application including any alternative flows
  • Use Free HTTP Sniffer tools and Browser plug-ins to gather information about the end user experience
  • Document any functional defects that you may find during FPT

Now, let us see how to do the real test. The strategy that I follow for FPT is based on the heuristic created by Scott Barber.

Test Strategy

Use the heuristic SCORN for comprehensive Front-End Performance Testing. The heuristic can be defined as below (I have changed a few points based on my understanding but you can see the original heuristic as defined by Scott at Use "SCORN" to test the front end of a website for performance):

  • Size – Focus on testing for
    • Uncompressed graphics and media
    • Object or code duplication
    • Script and styles living outside of the base HTML
    • Code "minification."
  • Caching – Focus on testing for
    • Expires settings,
    • Etags
  • Order – Focus on testing for the order sequence of components
    • Styles/style sheets
    • Critical content (i.e. what the user came to see the page)
    • Relevant media (i.e. graphics related to the critical content)
    • Incidental content (i.e. non-critical graphics)
    • Scripts
  • Response Codes – Test for
    • Requests for objects that don't actually exist (Any exception is costly - MSDN)
    • Superfluous redirects
    • Errors that are not obvious from the browser
    • Redundant Requests
    • Invalid URL references (E.g. http://:)
    • Unused Request
  • Number – Ask questions related to
    • Number of Requests
    • 1 Heavy Graphic Vs many small graphics
    • Inline Scripts Vs 1 External Script
    • Inline Styles Vs 1 External Style Sheet

Testing Procedure

  • Use browser plug-ins or online tools to capture page load times.
    • YSLOW – FireFox FireBug Add-on
    • Episodes – YSLOW Add-on
    • HammerHead – YSLOW Add-on
    • HTTPWatch Basic
    • Page Speed – FireBug Add-on
    • Microsoft Visual Round Trip Analyzer
    • Fiddler with nXpert – Performance Analysis Plug-in
  • Conduct FPT with functional tests and/or user acceptance tests
  • Devise the tests based on the project context and criteria – Pick the tool based on the need, best for the context
  • Additionally, for monitoring the resource utilization during the test you can leverage on Perfmon (for windows based systems ) and NMon (Unix based systems). PAL is a good tool for analysis of Perfmon Logs.

My favorite tools for FPT are Net tab of FireBug, YSLOW, Episodes, HTTPWatch and VRTA. 

Reporting

The performance errors and slow pages need to be analyzed and reported for both executive and technical audience.

Although, I call it as Low cost performance test in the title of this article. FTP is not a substitute for comprehensive performance test. In fact the FPT is part of the complete performance testing strategy. When customer is not ready to invest for performance testing and in situations wherein you have customer complaints on the performance of the applications, FPT can be the first step of performance testing.

Hope this information helps you to device your performance tests. In the next post I will plan to write about automation of FPT with different tools such as QTP, TestPartner, TestComplete  and also without using any tools but VBScript.

--LN

4 comments:

Anonymous said...

Nice and useful article LN and yes value added for clients who do not want to invest on actual performance testing with load. :)

In one the section you have mentioned that
"Additionally, for monitoring the resource utilization during the test you can leverage on Perfmon (for windows based systems ) and NMon (Unix based systems). PAL is a good tool for analysis of Perfmon Logs."

Since there would be no load generated while conducting FPT is it good to monitor resource utilization?
Please correct me.

Regards,
Mahendra

Lakshminarasimha Manjunatha Mohan said...

Hi Mahendra,
Thanks for reading through the post.

Monitoring of resource utilization is important for some situations where in there is resource intensive processes are being executed. E.g. Silverlight component making a sort or search or filter operation at the client side.

This monitoring is at the client side and this data complimented with the FPT results should give a great amount of valuable informaiton related to performance of the webpage.

Hope this clarifies.

Regards,
LN

Unknown said...

I recently came across your blog and have been reading along. I thought I would leave my first comment. I don't know what to say except that I have enjoyed reading. Nice blog. I will keep visiting this blog very often.
Software Testing Tools

Unknown said...

Cool post. Thanks for getting it done so soon and not ONE WEEK :)

Points about handling such kind MS updates which you have mentioned is a good learning.

Looking forward for more cool posts like this.

I have updated my post by adding "Related Post" by linking to your blog post of MS Updates as my blog readers would find it helpful.

Cheers,
Santhosh Shivanand Tuppad
macapijnbestrijding