Test case management

A good test suite with test cases is of inestimable value. During the creation of an application, it is the basis for the reporting of test coverage and progress. A good test is of inestimable value. During the creation of an application, it is the basis for the reporting of test coverage and progress. During maintenance and releases it is a store of knowledge about the application and a source of ready-made test cases for regression testing.

create testcase.png

The creation of a well thought-out test is easier said than done. Most often people are not used to create test plans, suites and cases as a project deliverable, which will last for the complete life cycle of the application. With the availability of tests as a work item in Visual Studio and the registration in the repository, test cases get at the same level as sources, they are a project deliverable. Not only tests are used for the proof of quality but they can be used through out the lifecycle of the project.

With tests available, within projects, which are project deliverables, than we also need treat them as project deliverables and create, manage and support them as deliverable. An interesting question is “what are the acceptance criteria for tests?”

Test Cube

An approach for building,using, and managing tests is: Test cube

Test cube is a coherent set of principles with which it is possible to:
  • to specify and test cases on the basis of establishing priorities;
  • rapid and appropriate to report on the progress of test specification and/or test run;
  • test routes to plan and budget;
  • quickly and variable regression tests;
  • changes in the test object is easy to handle in the test.

test kubus.png
PDF

Essential in the test cube is the classification in to categories of the test cases. This rating indicates the ‘weight’ of the test case in the test. In particular it makes it possible to use a variable depth to risk based (regression) test. This works as follows, for example, in three main categories:

risk cat.png

each underlying category is a collection of the other.

Each object (to test) has test cases in each category. The derivation between the different categories are within some margins in practice (for example, category 1 = 10-15% of the total number of test cases, category 2 = 60-70% and category 3 = 100% ). The us of this categories, for example, in the preparation of regression tests, is as follows:
  • By selecting only the test cases for category 1 for an object, a subset of the test cases is created. This subset is used for an object in which no adjustments have been made (or for an intake test on a new or substantially amended part object).
  • The test cases of category 2 (= including category 1) are making a normal regression test set, for example, a part of the object in which adjustments have been made.
  • The test cases of category 3 (= including category 1 and 2) cover the total part object are applied to new or major amended part objects.

Prioritizing test cases

While Microsoft Test and Lab Manager has the ability to advice testers to re-run test cases based on changes in the object, this method of putting test cases in different categories for regression sets is very valuable.Not during the selection and execution of test cases, but also for the planning of the testing effort. for example category 1 test cases which test high risk user stories or requirements are very important test cases. These test cases must be executed. By providing the test manager with information about executed high priority test case for high risk functionality is unbearable information when organizing the test effort. Also for bug reporting this is important information, we can create a matrix with – user story risk- test case category and bug priority. where for sure high risk user story with a failed category 1 test case and a major bug is many times more important than a high priority bug for a failed category 3 test case for a low risk user story.

This information can be added to the work items user story, bug and test case

image placeholder

and reports can be created based on this information, giving the test manager valuable information for managing the testing effort.

image placeholder reports

Test case management with Microsoft Test and Lab manger


-- working on this story --

Last edited Nov 11, 2009 at 11:40 AM by ClemensReijnen, version 3

Comments

No comments yet.