Challenges of Test Estimation in Agile Manifesto

Estimating size of the project (i.e. test estimation) is always the toughest part, especially when it comes to sizing testing efforts. Success of the whole project depends on how much accurate, realistic and actionable the estimates are.

Stakeholders simply ask how long testing will take? For manager this question means;

1. How many testers we need?
2. How many test cases should we write?
3. How many bugs we can find?
4. How much experienced resource we need?
5. How much the schedule would re-adjust for later sprints?
6. What are the probable risks we may face?

1. Estimating the Experience and Domain Knowledge of Test Designer

A good manager knows answers to all the above questions and can make precise and doable estimate. Estimates are required on two major stages i.e. “Planning” and “Execution” for every sprint or cycle. Planning is further divided into designing of tests and test setup configurations. Relatively the easier part is setting up test environment and test data, but designing tests is difficult and its highly dependent on the resource allocated for test designing. That is skills and experience required for test designing, it varies from engineer to engineer, and project to project.

Why? If the test designer leaves the company or is allocated some other important project, new resource is not experienced in this domain, the intern needs training and you don’t have time to hire new resource. So experience and domain knowledge of test designer is a challenge.

2. Estimating the Details required in Test Cases

The second stage is the execution of test cases, bugs reporting and regression. It all depends on how well the test cases are written, what is the probability of finding bugs? It can only be achieved successfully by maintaining a balance between number of test cases and how much detailed they are. A large number of test cases are written with details, or few test cases written like high level scenarios. It is very obvious that large number of test cases written in details are not suitable for a small application. Most of the time gets wasted in writing details of test cases, maintaining them, and recording test results etc. Although there may be more probability of finding bugs, but how much significant those bugs are? Here high level scenarios would work more effectively. Keep in mind that since we are testing small application so exploratory testing would also work.

Why? Large number of test cases written in detail, large number of test cases with very little detail, few test cases written in detail, or few test cases with very little detail. Need to find a suitable point among these four corners. So the challenge is how much detailed test cases should be sufficient, that they are capable of finding number of defects.

3. Estimating the Velocity of the Test Team

Test execution is not as simple as many would think. No matter how well the test cases are written, testers’ attention to details, complete the coverage as defined in scope. Testing goes parallel with the development needs to maintain test execution in limited time slot. That is the velocity of the test team. Every sprint comes with new stories, frequent changes in requirements, scope is increased, new version of operating system in market is also required to test. Client demand for demonstration and project will be ready by end of the week. These kinds of issues can be handled or managed somehow. But as a result of these hitches either quality is compromised or schedule is disturbed, or resources would be overworked.

Why? In order to keep the testing team synchronized with development team, we may need to add more testers. For testing the application with details, complete coverage and speed, we need to maintain the velocity with development team.

It is impossible to foresee things like estimating the size of the project and measuring the effort required to test etc long before the project is expected to start. You can keep some time ¬¬as a buffer. That’s good, but when these things are changed, during the sprint, you are in trouble. It also varies from project to project. Small projects are easy to estimate than the larger ones. In simple words; testing effort keeps on changing for every sprint, every cycle, every phase and every project.

Stay tuned for the next blog to learn how these challenges can be overcome rapidly !

[1] Source of the Diagram: http://agilemodeling.com/essays/agileArchitecture.htm