Testing for Exceptions in Web Application – An Approach

Last week when I had been to Norway, one of my colleague test manager casually asked question on testing a requirement for a banking application. I think it is interesting challenge to conduct such a test and this article is an attempt to provide testing approach for such a situation.

Requirement: Application to log every exception it encounters and fail gracefully (of course securely without any sensitive information disclosure)

Question: How to test and plan for testing the exceptions in a Banking and Finance application?

Solution:

This is not that easy testing that can be done with ease as simulating exceptions is not always possible.

Before thinking about the solution it is pivotal to understand the requirement clearly. In an effort to understand the requirements we need answer the below questions.

• How are the exception stack traces logged by the application? Usually it logged to a secure database or logged with audit trail logs.
• What exceptions are caught as specific exceptions in code and what exceptions are caught as generic exceptions? This is important to understand what need to be done to simulate an exception.

Now, having understood the requirement reasonably well we can think about testing.

It is hard to anticipate different types of exceptions caught on a specific event. However, 2 approaches to test this situation are described below.

Black box approach: This is best suited for the manual functional testers with limited competency in programming. As seen by the black box tester any exception is thrown only on an event such as page submission or request either GET or POST of a request.

• List all the scenarios in the application where there is GET or POST, predominantly form submissions.
• For each and every request list down the possible exceptions based on the functionality being processed and data input. For example on page where integer inputs are made it is necessary to consider IntergerOverflow, IndexOutOfBound and NullPointerException
• Design test cases to simulate each of the exceptions by relevant test input data and expected results. It is important to consider the Post Conditions or Follow-ups after the exception.
• Execute the test cases and record the results with defects

The advantage of this approach it does not need programming skills for a tester to conduct the tests although it requires basic programming knowledge for designing the tests. The cons of this approach are it does not ensure coverage as many exceptions may not be practically possible to simulate.

Grey box approach: This is the approach that ensures greater coverage and best suited for testers with programming knowledge. I recommend this approach. It is good to plan and estimate for this testing during test design and planning phase.

• Review the code with focus on the catch clause
• Verify if all the specific exceptions are handled in the catch clause depending on the scenario under context
• Verify that there is a generic catch clause that takes care of all the exceptions not handled by specific exceptions
• Here, it is important to validate the exception reports for accuracy and consistency. Usually the exception are report to a secure database or to a log file stored securely.
• It is also important to look that exception report for sensitive information that can be juicy for a hacker.
• The stack trace of the exception should not be transferred as clear text in the HTTP requests and responses. It should be encrypted with a strong cipher.

The testing approach is described based on my experience in testing such scenarios with an intention that it could be of some help to the testers community. Comments and feedback on this are appreciated.

-- LN

TESTER by INSTINCT, not by CHANCE.