Archive for February 2011
- Test automation is expensive and takes ages to justify the investments.
- Automation resources are top-heavy and would mean a lot of upfront costs.
- Product roadmap is not clearly defined and we aren’t sure if automation is appropriate for us?
These are some of the initial reactions that we get to hear whenever we think of test automation. These reactions aren’t unfounded and it merits careful assessment.
Some questions to consider before deciding to automate are:
- Is your product fairly stable and do you have paying customers for its initial version?
- Does the product have a clearly defined roadmap?
- Does every version upgrade, both minor and major warrant running all the functional tests over and over again? Essentially, the test scripts would be re-used multiple times justifying the investments
- In most ISVs, developers double up us testers and there is no formal testing process in place
- Due attention is not given to QA and is looked at only when schedule and time permits
This is most likely to result in increased rework, decreased customer satisfaction, increased support and product maintenance costs. Independent testing or 3rd party testing, as it is famously called can help ISVs in addressing the pains listed above.
Some of the reasons that substantiate usage of 3rd party testing by ISVs are:
- Good software development practice calls for separating the development team from the testing team. This allows provision of unbiased view of the product quality and greater visibility of product issues. It also reduces direct staff costs for the ISVs who do not have to staff QA internally.
Although proper testing requires substantial amount of time and effort in planning and execution, there are situations when time is limited and full-fledged testing runs against the time limitation. How can testers handle such situations?
When facing a limited time frame available for testing, we need to effectively use the available time and resources. Starting the testing of the project with an assumption that “We can’t test everything, no matter what” will really help in prioritizing tasks. Do a risk analysis to identify functionalities with the highest risks and functionalities that will be used by the maximum number of users. Do an analysis about what to test first and in which sequence.
Also, preparing a checklist that focuses on major key areas covered during testing will help testers, by ensuring that they are not missed out in the tight scheduled testing. The checklist should cover