4.2. Position of test phases

Consider on the types of tests you will use, when (during which phase) you will conduct them, the data that is needed in order to conduct the tests and the requirements of the testing environment.
These points need to be considered early in the project, as they are involved in confirming whether quality targets have been achieved and in the QCD of the testing phase and the project as a whole.
The requirements you confirm at this point will be refined later to consider specific points such as the test schedule and the requirements and transitions of the test environments.

Overview of points to be considered

Define the tests that will be conducted for each test phase, ensuring that there is cohesion between each test and those before and after, and in the content being verified throughout the testing phase as a whole.
This document presumes that users will also use “Test Type & Viewpoint Catalog” as a reference, and that the phases for testing such as integration testing and system testing have already been established in the phase plan.

Decide on the types of tests to be used in each test phase and then determine the following points for each test phase and combination of test types.

  • Scope
  • Objective
  • Test data requirements
  • Test environments
  • Test type in preceding testing
    • In cases where multiple types of tests will be conducted successively during the same phases, such as those where performance testing is conducted based on the results of functional testing, this needs to be clearly indicated, along with the type of test that needs to be started or finished before conducting the test in question.

Relationship between this topic and other topics in the overall test planning phase

This topic is mainly related to the following topics as shown below.

Relationship between this topic and other topics

Consideration method

Summary

  1. Select the types of tests to be conducted.
  2. Confirm the phases during which tests will be conducted.
  3. Decide on the scope of each test type and allocate phases.
  4. Decide on the objective and the necessary details for each combination of test types and phases.
  5. Define test environment requirements for each combination of test types and phases.
  6. Decide how test types within a phase will be linked to the tests before and after.

Details

✓1. Select the types of tests to be conducted.

  • Select the types of tests that are needed from “the test type catalog” included in “Test Type & Viewpoint Catalog” (reference) and confirm the priority order.
  • If the quality that needs to be achieved and quality characteristics that need to be considered are defined as specific quality targets, follow those.
    If they are not defined, here are some examples of points to consider.

    • Overview and requirements of the project or system.
      • The purpose of the project or the system being created.
      • Whether there are software or a system for migration source or comparison.
      • The intended users and the environments in which operation needs to be guaranteed.
      • Areas where additions or changes will be made, e.g. changes to settings files that will not modify the program.
      • The processing architecture (web application, batch application, etc.)
    • Product risks.
    • Bugs that have previously occurred in similar systems.
    • Test policies.

✓2. Confirm the testing phase.

  • Use details such as the phase plan in the project plan for this confirmation.

✓3. Decide on the scope of each test type and allocate each test type to phases.

  • Decide on the elements to be tested (each feature, interface between subsystems, etc.).
  • Consider the objective of potential phases and how they will be performed when allocating test types to phases.
    • Another method is to gradually increase the scope and confirm this for multiple phases, so that any problems that arise can be easily addressed.
  • Check that the following points are suited to the objective of each test phase and the period during which it will be conducted.
    • The objective of each test type and the period during which it will be possible to prepare such as creating test cases and execute tests.
      • The “Input materials” column in the test type catalog (reference material) is likely to be useful.
        This column indicates the requirement definitions and design specifications that are needed in order to consider what will be verified in each type of test.
        Without these materials, it is not possible to consider on test cases. Confirming whether these materials are available therefore makes it possible to confirm the feasibility of the planned timing.
        See “Test Type & Viewpoint Catalog” (reference material), where “the test type catalog” is located, for details.
    • Priority order.
      • Consider on the priority order based on the quality targets, quality plan and the testing policies considered on for the overall test plan.

✓4. Decide on the objective of each combination of test types and phases.

  • Establish the objectives of the tests.
    • Clearly establish the objectives so that the relevant personnel know what kind of verification is needed for each combination of test types and phases.
      • The objectives need to reflect the quality targets to be achieved in the tests and the reason why each type of test is needed.

✓5. Establish the details needed to execute each combination of test types and phases and the test environment requirements based on these.

  1. Establish any dependency relationships between tests or between tests and migrations or migration rehearsals.
    • This is applicable in cases such as those where data created in one test will be used in other tests.
  2. Establish the requirements of test data that is needed to execute tests.
    • Determine whether variations of data need to be equivalent to production.
    • Determine the number of data.
    • Determine how the data will be obtained (extracted from a previous system, created with a tool or similar, created manually, etc.)
      • If migration data or data for the production environment will be used, it is also necessary to confirm the timing at which the data will be used.
      • In cases such as those where the production environment or another specific environment will be used for the preparation, the data preparation environment needs to be organized in addition to the test environments.
    • It is also necessary to confirm points where caution is needed after acquiring the data, such as whether masking or encryption needs to be used and the method and timing for registration of data in the test environments.
  3. Confirm the method for executing the tests. Your next consideration, a consideration about the test environments, will depend on this.
    • Determine whether the tests will be automated.
      • Identify any tests that cannot be executed easily without automation or, conversely, tests that will only fulfill their objective if they are conducted manually.
    • For tests that will be automated, determine the tools that are needed and how to procure them.
  4. Confirm the following points that will affect the requirements of the test environments, based on what was considered in your previous consideration.
    • Determine areas requiring the same conditions as the production environment and areas where the production environment needs to be used.
      • These might include the test data, the memory of the computer and/or the system to which the system being developed will be connected.
    • Determine tools or middleware required to run the test targets, execute the test and confirm the test results.
      • These might include a job management system to be used in batch application testing and/or an electronic report viewer required for report output testing.
    • Determine environments and networks required to run the test targets, execute the test and confirm the test results.
      • These include the system to which the system being developed will be connected, the network required to connect to that system and the environment required to confirm the expected results of the comparative test between the old and new system.
    • Determine security restrictions.
      • Determine whether any tests require data that cannot be taken out of a specific area or can only be accessed in a specific area.
    • Determine who will access the test environments and where the work will be carried out.

✓6. Determine test relationships within each phase (relationships between data test types or between one test and the test before or after).

  • Establish the type of test that will be conducted first based on the priority order and the input/output relationship of the test data.
  • Take the following points into account when deciding this.
    • Which test types need to be prioritized to meet the quality targets that are important to the stakeholders?
    • Which test types are prerequisites for other test types or generate test data that can be used by other test types?
      • Functional testing needs to be completed soon in cases such as those where all of the functional testing needs to be conducted to confirm operability before proceeding to performance testing and those where operational scenario testing is conducted according to test data created during the functional testing phase.

Reference

  • In some cases, it may be necessary to revise a previously established phase plan during the consideration-making phase for this topic.
    Revisions such as splitting some phases are recommended if it is judged that testing cannot be carried out efficiently for purposes such as the necessary test management and execution with the division and definitions in the current phase plan.
  • If there are any points that particularly need to be verified, considerations need to be made from more specific test perspectives than test type when establishing the overall test plan.