2008 IIST Best Practices Award

TESTCo Agile Offshore Software Testing

Best Practices

TESTCo uses a set of best practices for offshore agile software testing in a U.S.-based agile development environment that allow the development team to continually expand and update multiple applications while maintaining continuous release readiness throughout the development cycle. Through continuous testing and working closely with the  development organization, multiple teams are able to focus on specific applications with a common code base, and produce a higher volume of new features than the industry norm without destabilizing the existing code base. By implementing best practices on one project alone, TESTCo has been able to complete an average of 8 stories per 10 person-days while eliminating the previous defect backlog of 1000 defects.

Test Requirements and Design:

• A functional map of the application is created and used to identify risk areas, partially or poorly covered functional areas, and priorities for future test automation efforts
• During iteration planning the stories are identified
• The product owner, test and development team work together prior to developing stories to ensure the stories are concise, clear, and specific
• The testers identify any potential problems or ambiguous requirements by focusing on acceptance criteria
• The test team reviews each story and signs off on the acceptance criteria prior to development of the story
• The test team creates test cases prior to coding by the developer

Test Execution:

• TESTCo uses and recommends Rally and story-based requirement systems for test case management, tracking results and for traceability to requirements
• The Test Engineer and Developer work together to verify that the test case is appropriate
• As each story completes, the test team verifies successful testing
• During the iteration the story is developed and the test case is executed manually
• 80% of all test cases on new functionality are passed the first time
• For the 20% that do not pass the first time, they are brought up the following morning on that team's scrum and the test engineer and developer set up a one on one conversation to resolve the issue
• After a story is accepted, the test case is automated and added to the regression suite as part of the following iteration

Test Management:

• A daily scrum of scrums meeting is held with representatives from all of the scrum teams
• Test case results are tracked daily and how many test cases pass and fail are reported to the entire team
• Issues blocking a test case from being successfully executed are raised on the daily scrums, and on the scrum of scrums if it is not immediately resolved
• Metrics are tracked on a daily basis for the number of failed test cases, percentage of stories accepted, number of remaining stories to be completed or accepted, number of stories that are blocked, and the number of accepted story points per team member
• During iteration planning all stories are given a priority rank, which indicates what order the completed stories are tested
• Rally is used to manage and maintain a regression suite of test scenarios, which are executed through automation scripts during each iteration
• Management includes variable staffing with schedule overlap to address challenging issues by using cross-functional teams

About TESTCo - TESTCo customers eliminate software testing problems, slash support costs by up to 50% and achieve their release date commitments ... all without the risk of hunting down the best offshore vendor, the pain and frustration of learning how to properly manage them and the unpredictable results so common with most offshore vendors.