What are the different parts of a Test suite? What are the right names to designate such or such part? How can different parts be assembled into just one software? Let's try to define the important pieces of a test suite.
A test suite minimally comprises:
- a set of test cases
- a description of how they are supposed to be used
- author - the person(s) or organization responsible for creating and managing the information
- @@@ what else?
- description of the objectives, scope of the test suite
- identification of the Recommentation(s) or specification(s) being tested [lynnerosenthal]
- Do all agree on what a "test case" comprises? In very general terms, there is a stimulus and response. When the product under test is stimulated as indicated by the stimulus, the response must be "equivalent" to the provided response. How the stimulus and product are brought together depends on the Class of Product. -- David Marston
(the upcoming TestGL will probably have something about this?)
In addition, it may comprise:
- a test execution framework (test harness?)
- a test reporting system
- a test cases management system [@@@ is this really a part of a test suite?]
- test suite design
- adjudication process - how to handle inqueries and/or disputes
- disclaimers
- branding (icon) criteria [lynnerosenthal]
Here is my idea of what a test execution framework does, and with a try at defining its functional units:
svg dia (@@@ send updates to dom@w3.org - or create an SVG wiki file for this?)
Needs to define better labels. How do that relate to ISO/IEC-9646? esp. where do the Upper Tester/Lower Test concepts fit in this? Needs to add the Points of Controls (POC in ISO terminology)
Examples:
- a Makefile-based Test suite would have the first filters made through a set of targets, with the dependencies encoded in the targets lists
