Test, Modélisation, Validation, Agilité: More complexity, but more quality insurance either !

The increase in complexity of IT project carries inexorably an increase in the odds of errors and defects in the development of these projects.

On the other hand, while IT systems are becoming more and more critical, the cost of uncorrected or belatedly corrected errors and defects will grow exponentially woth the size of the project. Moreover, it is known since a long time, almost three decades, since Barry Boehm in 1981 that the more an error is discovered lately, the more it will be expensive to correct. The figure 1 below shows the increase in cost correction of an error depending on the stage in which it is discovered.

Figure 1 : Increase in error correction cost for each development stage error

Fortunately, today, we have quality insurance tools that are made to verify the system at each step of its development.

The SMV method is a quality insurance method that will, such as the V & V, be used with renowned tools in the software lifecycle development in order to reduce as soon as in the requirement stage, functional and technical errors present in all IT projects.

1. Towards an increase of complexity in IT projects

1.1. Innovation, a prerequisite for company growth

Companies are bound to innovate. The company that doesn't innovate will create fewer added value and will lose its competitiveness. On the other hand, the company that innovates will keep its competitiveness but will face other problems such as IT maintenance problems or IT system upgrade. As Pierre Bonnet [1] says: " The IT system follows a complexity curve indexed on the innovation and the change curves". This increase in complexity will inexorably lead to an increase in cost of development, tests and validation of the final system.

1.2. Technical drifting, a technical risk proportional to the size of an IT project

According to the NIST (National Institute of Standards and Technology), 80% of the development cost of an IT project is dedicated to error identification and correction, and the later an error is corrected, the more expensive it will be to correct. In 2002, IT errors and defects cost the American economy not less than 60 billion dollars [2] or in other words 0,57% of the American GDP.

Waiting for the test stages to begin error correction in IT projects is ineffective and will become more dangerous as IT systems are getting more complex. The Standish Group doesn't make a very optimistic analysis of the IT project situation either. In fact, in its report titled "Extreme Chaos", the Standish Group classifies IT projects in three categories:

The successful projects, projects released in time, on budget and with all the expected functionalities,
The challenged projects, those released behind schedule, over budget or with less functionalities than expected,
The failed projects, those that were never finished or never used.

Since 1994, the Standish Group holds statistics on all these three categories of projects. For the 1994-2009 period in the United States, the figures are those of the table 1 and figure 2 :

Figure 2 : Success rate of IT projects between 1994 and 2009 in the United States according to the Standish Group

Table 1 : Evolution in IT project success rate (in %) for the 1994-2009 period according to the Standish Group [3]

	1994	1996	1998	2000	2002	2004	2006	2009
Succeeded	16	27	26	28	34	29	35	32
Challenged	53	33	46	49	51	53	46	44
Failed	31	40	28	23	15	18	19	24

The stagnation of the success rate of IT projects since the year 2000 around 31,6% and the increase in the failure rate since 2002 shows the ineffectiveness, inadequacy or the lack of actual quality insurance method that are unable to insure the good completion of IT projects.
Those three observations, the increase of complexity in IT projects, the exponential increase of the error correction cost in the later stage of the project and the difficulty to finish a project according to the requirement and the original estimates shows the necessity of using effective quality insurance methods and procedure made to control quality from the beginning of the project and identify error before the software testing step by applying control and recording methods in the requirements and modeling stages.

2. Towards an improvement of quality insurance in IT software

In 2007, the spendings in IT quality of French companies represented 535 million Euros. Moreover, 53% of French companies consider that the quality insurance of their IT project will have an increasingly important impact on their business results. These two observations shows that we are no longer thinking about how much will cost an IT system, but rather how much will cost a defective system [4]. The quality insurance is an essential process to add to IT business development.

2.1. A definition of quality insurance

Quality insurance is: "A whole set of pre established and systematic actions that are required to give the necessary confidence in what a product or a service must satisfy to quality related requirements". It is a set of processes acting at each stage of an IT project and allowing each stakeholder to know at anytime the exact status of the project and its preceding states.
If the processes of the quality insurances comply with the needs of the V & V (Verification and Validation), are they enough to answer the increasing problem of IT project due to complexity and project size?

Quality insurance tools are, for instance, reviews, audits, cross readings, peer reviews, technical reviews, quality inspection or others. But theses tools, even if they identify some of the errors and omissions, they won't identify them early enough to save 80% of the budget of an IT development.
In order to solve the errors upstream, where they are less expensive to correct, it is necessary to implement technical add-ons, as soon as in the business needs analysis stage, to clarify the project requirement and help all the stakeholders follow their project with precision.

2.2. Verification and validation, a technical process of the quality insurance

When they are applied at each step of a project, verification and validation answers the problems of quality insurance from the inside of iterative project development steps.

The verification: Its aim is to show that the activity has been well done, that it was compliant to the realization plan and that it didn't introduce any defects in the result. In order to achieve those 3 objectives, it needs to have access, at every step of the project, to executable tests for model and the system itself.
The validation: Its aim is to show that the activity complied with its objectives. In order to succeed in this, we need to dispose of tests created from the requirements to execute them at each step of the project to verify that the product itself is compliant with its requirements.

The main advantage of the V & V is that it can be fully integrated in the software development lifecycle. And on the contrary of other quality insurance methods, it proposes technical solutions that helps every stakeholders of a project to better understand its development.

References:

Test, Modélisation, Validation, Agilité

Pages

lundi 14 février 2011

More complexity, but more quality insurance either !