Test engineering methodology integrated in the Project System Engineering

1. Abstract

Lifecycle (development, production, maintenance and operational use) Tests Infrastructure is a crucial part of a project and consume significant budget and effort.

Definition and selection the right Tests concept by considering the overall Lifecycle tests aspects in early stage of the project, can reduce the Project Life Cycle Cost (LCC), improves the Project profit and will deliver a better product to the Customer.

The importance of the System Tests Engineering (STE) activities is well known for many years but it is not implemented in most of the companies. The main reasons for this are

1. At early stage of the development there are many uncertainties and information gaps.  The project team feel that they don’t have the knowledge to define in this stage the System test concept.
2. There are many aspects which should be taken in consideration for the Test concept definition, Maintenance, Safety, production, Operation Use, development phase etc.  What will be the process to achieve  the optimal Tests concept technically  and commercially.

If such STE activities exist in a project then usually it is done intuitively by the Integrated Project Team (IPT) and probably it will not lead to the best and the optimal solution.

This article describes a methodical tool which can be integrated in the Project STE activities in-order to define the optimal Lifecycle tests infrastructure at early stage of the Project.

The concept is integrated in the System engineering activities from the Project Kickoff and step by step in a structural and clear way defines the optimal Test concept technically and commercially.

The process connects the Development tasks, Maintenance concept, product safety aspects, production requirements and the Customer Operation uses with the STE.

Since this STE activity starts from the Project start-up when the development activities are still on goings, it is an iterative process which progress continually with the product development till a well-defined stop condition is met and the final Test concept is frozen.

Some examples for Project tests optimization which can be analyzed in this STE process are:

1. Tests which will be performed by the product Built In Test (BIT) vs  Tests which will be performed by External Testers
2. Commercial Optimization

• System Level Tests only vs including Subsystem level Tests in the Production and Maintenance activities
• Dedicated Testers for each Lifecycle phase vs Common Testers.
• Complicate and Automatic Testers vs Simple and Manual Testers 

1. Introduction

During Project startup the Project team focuses on initiating the Project activities, such as contract evaluation, system engineering, program planning, etc. Their priority is to meet the technical requirements of the Project, and less the System Testability which is considered at this time as not urgent. This causes usually the Tests issue to be left aside for a later stage.

The results of this will be that:

1. Testability coverage gaps of the product will be detected late in the development process, or sometime after completion the development, while changes in the product are complicates and expensive.
2. The Test Infrastructure (the Built In Tests (BIT) and the Testers) will be delayed, and the production people will not be able to test the product properly.

This methodical STE which is described in this article evaluates the tests requirements for the overall product Lifecycle, as part of the  Project System Engineering activities from the Project kickoff. The STE defines the Lifecycle Test Concept (LTC) iteratively in parallel to the progress of the development .

It is performed step by step in a structural and clear way to achieve the optimal and the most efficient Test concept technically and commercially.

The STE is performed by a professional Test Engineer (TE) who is part of the System Engineering team of the Project, or for bigger programs, by a test engineers team.