Vidéo de tests automatisés d'un site de e-commerce

Voici une vidéo en 2 parties montrant l'exécution de tests automatiques sur un site e-commerce de démonstration. La première partie présente le site et montre sa modélisation et la gestion d'exigences. La deuxième partie montre comment créer et exécuter les tests automatiques ainsi que la visualisation des résultats Bon visionnage

10 points à savoir pour réduire les bugs informatiques

Voici 10 points identifiés par Barry Boehm et Victor R. Basili à savoir pour améliorer la qualité des logiciels et systèmes en réduisant les bugs informatiques.

1. Identifier et corriger un problème informatique après livraison est le plus souvent 100 fois plus cher que l’identifier et le corriger durant les phases de spécifications et de conceptions.
2. Les projets informatiques actuels passent entre 40 et 50 % de leur temps sur des refontes évitables.
3. Environ 80% des refontes proviennent de 20% des défauts.
4. Environ 80% des défauts proviennent de 20% des fonctions du système. Et la moitié des fonctions sont le plus souvent sans erreurs ni défauts.
5. Environ 90% des crashs proviennent d'au plus 10% des défauts.
6. Les revues des pairs identifient 60% des défauts.
7. La validation par scénarios identifie 35% de défauts en plus qu’une validation point à point.
8. De bonnes méthodes de développement peuvent réduire l’introduction de défauts d’environ 75%.
9. Toutes choses égales par ailleurs, cela coûte 50% plus cher de concevoir des systèmes à confiance élevée par rapport à des systèmes à confiance moindre. Toutefois, cet investissement vaut largement le coût si le système implique de large coût d’opération et de maintenance.
10. Environ 40 à 50% des programmes informatiques contiennent des défauts non triviaux.

Source: http://www.cs.umd.edu/projects/SoftEng/ESEG/papers/82.78.pdf

La Validation Automatisée avec VINCI Autoroutes

Faites comme VINCI Autoroutes, utilisez la validation automatisée avec COGENIT pour assurer la qualité de vos systèmes.

VINCI Autoroutes a fait appel à COGENIT pour l'aider à valider le systéme de contrôle du tunnel Duplex de l'A86

Les solutions Xcarecrows 4 SMV de COGENIT et Rational d'IBM vous permettent de vérifier et valider par itération votre projet ainsi que ses spécifications.

Vous désirez innover pour rester compétitif, mais les coûts de vérification et de validation de vos outils réduisent votre ROI. N'hésitez plus à rentrer dans la démarche qualité proposée par COGENIT avec Xcarecrows et les outils Rational d'IBM pour une validation amont de tous vos projets.

La solution Xcarecrows 4 SMV de COGENIT, associée à des méthodes agiles, vous permet de relier automatiquement vos exigences aux tests fonctionnels grâce à des modèles exécutables.

• Xcarecrows et Rational RequisitePro ou Rational Requirement Composer, tracent vos exigences de leur genèse jusqu'aux modèles et aux tests.
• Xcarecrows et Rational Software Modeler, transforment vos modèles en exécutables et en tests.
• Xcarecrows, Rational Functional Tester et Rational Quality Manager, transforment vos modèles en tests exécutables à la fois sur le projet modélisé et sur le produit final.

Grâce à ces solutions , vous:

• Maîtrisez la qualité : Le lien entre les spécifications, le modèle et les tests de la solution Xcarecroxs 4 SMV vous permet à chaque étape du projet de rejoindre les exigences à la validation finale.
• Réduisez les risques : L'agilité de la solution Xcarecrows 4 SMV réduit les risques de vos projets informatiques grâce à des cycles d'itérations courts qui, à chaque étape relient les tests aux exigences sans passer obligatoirement par le développement lourd de l'application final. Les imprécisions du cahier des charges sont mises en exergue à chaque itération.
• Maîtrisez vos coûts : Vous n'avez plus besoin de développer le système final avant de commencer la préparation de la phase de validation. Les erreurs dans les projets sont identifiées très tôt grâce à la liaison entre le modèle et les exigences. Vous réduisez d'en moyenne 80 % le coût de production de vos projets informatiques en identifiant les erreurs possibles dès les premières phases du projet.

The SMV method: Specification, Design, Validation

1. The objectives

The main objective of the SMV method is to offer the stakeholders a set of tools, methods and processes to reduce the technical and functional risks in their IT projects by improving the quality management. It assesses on the requirement in order to:

• Create an executable model of the system
• Create automatic functional test sequences
• Offer the developers, clear, accurate and complete requirements

When used with agile development methods, the SMV method iteratively complies with the V & V (Verification and Validation) during the development of an IT system by creating the three above objectives to answer the questions:

• Do my requirements exactly reflect my needs?
• Do my system answers to the requirements of the specification plan?

The SMV method, in order to reduce the development cost of IT projects, identifies the errors and the omissions in the specification plan and reduce the risks implied by the technical and functional difficulties. Thus shortening the time to market and optimize the return on investment of IT projects.

1.1. Why creating an executable model?

The SMV method creates, exclusively from the owner's requirements, a system of the model first in UML, then in MDA in order to obtain an executable file capable of being simulated. This model represents the whole set of requirements already defined in the project and will, with the appropriate test sequences, identify the errors and omissions in the owner's specification plan.
The principal benefit of the creation of a testable and executable model is to get a precise and error free specification plan that can be transmitted to the project leader with all the documentation written for it. The second benefit is that this model will answer the first V (Verification) of the V & V. It will verify at each step of the project if the requirements defines exclusively and completely the system being developed. The model will answer the question: "Am I doing the desired product?"

1.2. Why creating automatic functional test sequences?

The test sequences will validate the system during its development. The automatic functional test sequences created from the requirements will answer the second V (Validation) of the V & V. They will validate the system at each step in order to verify that the system matches the requirements. They will answer the question: "Am I doing my product correctly?"
Another benefit of automatic test sequences in iterative project is the earning due to the reuse of the older automatic test sequences for the new step of the project thus reducing the cost of test creation at each execution of the automatic functional test.

1.3. Why offer clear, accurate and complete requirements?

Having detailed, tested and complete requirements enables to pursue a software development with precise information thus avoiding the risks due to inaccurate specification plan or incomplete documentation. Furthermore, the transformation of literal specification in modeled specification offers an unambiguous interpretation of the project owner's needs.

2. A solution centered on three types of software

In order to answer the three previous questions, the SMV method uses three types of software. The first is the Eclipse IDE used as a Java support for the whole SMV method. Over Eclipse, we use the Open Source Xcarecrows software developed by COGENIT and IBM^® Rational^® tools provided by IBM^®.

2.1. IBM^® Rational^® software

• Rational^® RequisitePro^™ allows, thanks to a database and a set of traceability matrix and traceability trees, to follow up the changes at each step of the iterative requirement management.
• Rational^® Software Modeler^™ is used as a UML modeler to design, for example, the class diagrams, the use case diagrams or the sequence diagrams.
• Rational^® Functional Tester^™ is used as a test automaton on which we apply in an automatic manner the test created from the sequence diagram to execute them on the executable model and the developed software.

2.2. Xcarecrows

Xcarecrows 4 XML

This tool completes Eclipse with XML edition tasks. It offers an XML GUI, an XSD schema, an XSL style sheet editor, an XML graphical tree comparator, an integrated XSD scheme verifier and a set of tools to transform the XSL.

Xcarecrows 4 MDA

This tool offers a full set of tools to model and verify the behavior of all types applications thanks to a hierarchical approach with state machines.

Xcarecrows 4 MDI

This tool offers a GUI to model software with the UML language and execute them according to the MDA process defined by the OMG^®.

Xcarecrows 4 SSL edition MIDP

It is an autonomous Java library to create secure SSL/TLS connections over the CLDC 1.1 and MIDP 2.0 APIs.

Xcarecrows 4 UTP

It is a forerunner tool in the test activity. It describes explicitly the test architecture to simplify the creation of automatic functional test sequences to validate the developed system.

Xcarecrows 4 Developers Edition

This tool contains, in a single package, the full set of Xcarecrows tools to develop MDA applications using Xcarecrows 4 MDI and Xcarecrows 4 MDA, to edit XML documents, XSD schemes and XSL style sheet using Xcarecrows 4 XML.

3. An approach centered on three axes

The SMV method is centered on three axes: the iterative requirement management, the iterative test management and the iterative model management.

3.1. The iterative requirement management

It is a requirement based model development method that helps creating and tracing with traceability matrices, three kinds of diagrams: the use case diagrams, the class diagrams and the sequence diagrams. All these diagrams will show in a structured language, understandable by any stakeholder of the project, the project owner needs.

Figure 1 : Principal of the iterative requirement management

The use case diagrams

The use case diagrams are the first step of the methodology. They consist in transforming the client needs in a structured diagram with all the necessary information to understand the need. It is done with a data collection, a business analysis and a business needs management to list all the important functionalities of the system. They are the language interface between the project owner and the project manager.

Figure 2 : The use case diagram of an E-commerce website in development

Every oval of these diagrams reflects a functionality identified by the project owner. They serve as a foundation for the SMV method as they transform in a model the business needs and as they help creating the class diagrams that represent the internal structure of the system.

The class diagrams

They show the internal structure of the system. The more the requirements are complete, the less business knowledge is necessary to model a complete class diagram. In the best case, we get a "1 for 1" relationship between the class diagram and the functionalities of the use case diagrams.

Figure 3 : The class diagram of an E-commerce website in development

The sequence diagrams

They translate the functional requirements in ordered methods and actions to apply to the system. They describe exactly the progress to do the stakeholders desired objectives.

Figure 4 : Example of a sequence diagram for a user registration of an E-commerce website in development

You can find one or more sequence diagrams per system functionalities. They serve for two utilities:

• They offer the developers clear, accurate and complete requirements,
• They offer the testers automatic test sequences applicable on any platform.

This short set of UML models offers a great start for the developers and to continue the SMV method for the iterative model management with the class diagrams and the iterative test management with the sequence diagrams and the UTP (UML Testing Profile) profile.

Figure 5 : The complete diagram of the iterative requirement management

The use of Rational RequisitePro in the iterative requirement management

The main objective of Rational RequisitePro is to record the project owner's requirements and their change history. It is also used to trace the modification between the requirements and the model and to generate documents to explain the progress of the project to every stakeholder.

Linking each UML model with a RequisitePro attribute allows us to create 4 traceability matrices that record the changes in each step of the project. This use of RequisitePro answers to the Verification, the first requirement of the V & V (Verification and Validation), by ensuring that, at each step of the project, the activity has been well done, in accordance of the realization plan, and that no defect has been introduced on the system.

Figure 6 : Diagram of the traceability matrices of the iterative requirement management

3.2. The iterative model management

The iterative model management uses the class diagrams and the iterative requirement management associated to the MDA (Model-Driven Architecture) paradigm to create an executable model. This executable model of the system is generated, in addition of the previous class diagrams, by object and state machine diagrams injected in Xcarecrows 4 MDA and Xcarecrows 4 MDI. This model will be used to validate test sequences and verify the specification plan.

The MDA paradigm, the insurance of a validated system

In the SMV method, the MDA paradigm uses 2 diagrams. Object diagrams and state machine diagrams. The object diagram shows the instances of the state machines and the data flow of the system while the state machine diagrams shows the transition in the model.

Figure 7 : Example of an object diagram for the MDA paradigm of an E-Commerce website in development

Figure 8 : Example of a state machine diagram for an E-Commerce website in development

This model, made from the stakeholder's requirements, shows the completeness and the quality of those requirements. If the model works without any flows, we can estimate that the requirements are complete. If the model has black holes, it means that information is missing in the requirements and it need to be added or clarified before starting the coding stage.
On the other hand, the model can also be used to test the test sequences created in the iterative test management before executing them on the final system. This action will validate the convergence of the system development, the model design and the requirement management at each step of the project.

The MDA paradigm in the SMV method isn't used to generate the code and the replace the developers. It is used to get in a few steps a simplified model of the system to verify the requirements, validate the test sequences and the convergence between the design and the coding stage.

The diagram of the iterative model management

Figure 9 : Principal of the iterative model management

3.3. The iterative test management

The last step of the SMV method is the iterative test. After having translated the requirements in sequence diagrams, having created an executable model on which we can verify the coherence and the completeness of the specification plan, we only need to transform the sequence diagrams into test sequences that will validate the executable model and the final system at each iteration in order to comply with the second requirement of the V & V, the validation.
However, the iterative test management cannot be made without the help of the UML testing Profile (UTP)

The UML Testing Profile

The UML Testing Profile [1] (UTP) helps the testers and the developers speak the same language. It offers them a full set of stereotypes to organize and execute the tests to validate the desired system. The figure 10 shows the complete set of UTP stereotypes proposed by the OMG^®.

Figure 10 : The full set of stereotypes defined by the OMG^® for the UTP profile

The complete diagram of the iterative test manager

Figure 11 : Principal of the iterative test management

The test scripts creation

From the sequence diagrams and the UTP transformation, we get a full set of executable test scripts on a test automate such as Rational Functional Tester. Those tests, directly created from the system's requirements verify at each step of the project the changes in the system by looking at the compliance between the system and the business needs.

4. How the SMV method reduces the technical and functional risks in IT projects

The SMV method is made to support the software development. It reduces the risks inherent in the creation and the modernization of IT systems. It complies perfectly with the V & V because it's a technical methodology that applies to the technical processes of software development by creating a unique referential from the stakeholder's requirements to get a verification and validation foundation. This referential helps:

• Reduce the risks due to a defective architecture: As the architecture is already designed, and that tests created from the requirements have been executed on it, the architecture given to the developers is already validated for a more robust coding stage,
• Better estimate the workload of the project and so the necessary timeline: Disposing of a complete model helps to know the critical points that will create difficulties in a project. This knowledge helps to better share the assets to optimize the realization time,
• Clarify the specification plan: Creating the executable model exclusively from the specification plan helps identify the errors and omissions of the specification plan. We get as soon as in the design stage a clear knowledge of the accuracy of the specification plan. The errors are corrected in the design stage avoiding the cost of an requirement error correction in the coding stage or worth during maintenance,
• Reduce the risks due to last minute changes: Applying last minute changes on an executable model before applying them to the developed system helps to verify and correct the errors generated by the last minute changes. They are no longer last minute changes applied to a system in development, but new validated needs that can be safely coded on the system.

The SMV method reduces the risk of misunderstanding in IT project by creating sequence of actions directly from the requirements. These sequences, linked to the model of the system, are the literal translation of the project owner's needs understandable by any project manager.

Finally, the sequences, direct translation of the project owner's requirements, are transformed, thanks to the UML Testing Profile (UTP), in test scripts made to validate the system according to the principal of the V & V by verifying the compliance of the system with the project owner's requirements. The use of the UTP to transform the test sequences helps the testers and the developers speak the same language all along the software lifecycle development (SDLC). Thus reducing the risks due to the lack of experimented testers and also insure the conformity of the project with the specified requirements. It also results in the guarantee of a correct validation because the project owner's needs are linked to the test scripts at each step of the project.

References :

1. The UML testing profile has been specified by the OMG^® in this document: UML Testing Profile, Version 1.0 formal/05-07-07 in July 2005.

Avoidable risks in IT projects

We can differentiate possible risks in IT projects in 2 categories: The technical and functional risks. The first kind of risks is du to internal programming team error such as wrong coding method or lack of version control. The second kind of risk is du to bad process around the IT project that implies all the stakeholders, such as the communication obstacle between the project development team and the project owner.

1. The technical risks of IT projects

According to an article of software-testing-zone.blogspot.com [1], nine technical risks can impact an IT project:

• Communication problems,
• An unrealistic calendar,
• A wrong definition of the architecture,
• Bad coding methods,
• A lack of version control,
• Buggy external tools,
• A lack of experimented testers,
• Last minute changes,
• The human factor.

Let's evaluate each risks in order to identify the possible responsible. If we use a typical "Project Owner" "Project Development Team" organization with eventually a "Project Management Support" which job is to assist the "Project Owner" in the achievement of its project, we can order these technical risks into 3 categories:

• The first category is constituted of the bad coding methods and the lack of version control. It is a category where the responsibility goes to the project development team. A knowledgeable project development team must use a version control tool and have most of the time formal coding methods taught at each engineer working with the project development team
• The second category is at charge of the project owner. They are made of the choices of the project owner. The risks in this category are, the use of an unrealistic calendar, the provision of buggy or unqualified external tools or too many last minute changes
• The third and final category is at charge of the project management support. The risks in this category are communication problems between the project development team and the project owner, the lack of experimented testers or a wrong definition of the architecture of the project.

The last unclassified technical risk, the human factor, can be ascribed to all 3 categories. It is also an irreducible risk common to every project made by man.

Figure 1 : Technical risks in IT project classified in 4 categories

This classification shows that technical risks in IT project are du to every stakeholder of the project. That is why it is important to use quality improvement methods understandable by every actor of the project. The proposition made in the following of this article goes in the direction of putting every actor on a same structured communication framework.

1.1. How to reduce the technical risks of IT projects?

The project development team's technical risks

As shown in the previous classification, the project management technical risks are du to wrong coding methods and the lack of version control in IT project. The only way to reduce those risks is to explain the utility of existing and proven useful methods and tools made increase the security and the productivity gain of IT systems.

The project owner's technical risks

Three risks were identified in this category: An unrealistic calendar, too many last minute changes and the provision of buggy or unqualified third party tools.

An unrealistic calendar

It is complicated to evaluate the workload of a project simply from the project owner's needs. As the Golub’s Laws of Computerdom writes it: "A carelessly planned project takes three times longer to complete than expected; a carefully planned project takes only twice as long." This shows in a humoristic way, the difficulty of planning the workload of a project. That's why it is important and very useful for an IT project to use methods such as the redaction in the specification plan of a clear tested architecture of the project before putting the project development team at work.

Too many last minute changes

These changes imply last minute coding and increase the risks of defects in the final system. Last minute coding have more chance to add regression in an IT project and force the changing of already coded and tested part of the system. An easy way to reduce last minutes changes risks is to use very short agile development cycle where short part of the system are coded, tested, integrated, validated and version controlled in a regular way and transform last minute changes of an iteration into early needs of the next agile iteration.
What about changes that generate incoherencies in previous requirements? A solution is to use requirement based testable model to apply the new requirements and verify if they don't imply to many changes in the previous requirements.

Buggy or unqualified external tools

Few can be done on a methodological basis to reduce the risks of buggy unqualified external tools. As for IT systems, the best way to ensure that an external tool is going to answer the needs of the project is to apply on it validation tests that will ensure that the external tool adds productivity instead of reducing it.

The project management support's technical risks

There are three technical risks at charge of the project management support: The communication problems risks, the lack of experimented testers risks and finally, the risks du to a wrong definition of the project architecture.

Communication problems

It is clear that, in IT projects, business needs analysis and their feasibility analyses are optimizable processes. In a usual IT project, the project owner does the business needs analysis while the project development team does the feasibility analysis. It adds roundtrip reunions and communication risks. A solution is to analyze the requirements upstream and to write them in a structured language understandable by the project development team. Furthermore, modeling in a simple way the requirements before the beginning of the coding stage will reflect the completeness of the specification plan and also the incoherence between them.

Lack of experimented testers

Test represents an important part of the software development. According to www.basildev.com, test represents between 20 to 50% [2] of the cost of a project. Testers occupy an important part in the software development lifecycle. Without prejudging of the quality of the testers, having automatic test sequences corresponding to the system requirement before the validation stage is an asset for the testers who can prepare themselves and for the developers who can use these test sequence to better understand the meaning of a requirement. It will accelerate the software development lifecycle and increase in the same time the project's quality.

A wrong definition of the architecture

A correct architecture definition is induced by a good knowledge of the business. Here resides the difficulty to offer the project development team a correct architecture. Model testing is a great help for this. Testing a model exclusively made from requirements ensures its quality and its accuracy. It can tell us if the architecture and the specification plan are complete.

1.2. Four recommendations to reduce risks in IT projects

The analysis of the technical risks and their solutions leads us to 4 recommendations that help the risk-agility couple in IT projects:

• Model the system from the requirement in order to verify the completeness of the specification plan, but also the incoherence between 2 requirements,
• Transform a textual requirement into an executable requirement written in a structured and precise language understandable by any project development team,
• Execute each new requirement on the model made with the existing requirements. It will avoid the incoherence of the business needs
• Create automatic test sequences corresponding to the system requirements being released before the release of a first version.

2. The functional risks of IT projects

These risks are different than the technical risk of the previous chapter in a way that we can find them in the project management on the contrary of the first risks that can be found in the project development stages. According to Nathan Hattab, consultant in IT systems and member off the AFAI [3], there are 5 functional risks [4] in an IT project:

• The inexperience of the project leader or his inability to lead a project,
• The inaccuracy of the specification plan,
• The lack of maturity of the project owner,
• The lack of mastery in the engineering process of a third party tool,
• The wrong estimation of the workload of the project.

As for the technical risks chapter, we can classify those five functional risks into the three same categories: the project owner category, the Project management support and the project management categories.

Figure 2: The 5 functional risks in IT project classified in 3 categories

2.1. How to reduce the functional risks of IT projects?

The lack of maturity, the main functional risks of the project owner

The project owner maturity only depends on him. However, giving him recommendations that will help him reduce the risks induced by his need is a win-win situation. If this risk reduction can be made at an affordable cost, it would be a mistake not to apply it.

The estimation of the work load, a risky set for the project management support

Here is a set that can be improved by analyzing the accuracy of the specification plan. In computer science, the sooner we start, the later we finish. This means that it is often quicker to take the necessary time to analyze a specification plan than start coding instead of starting to code without analyzing the specification plan. The accuracy of the specification plan will help estimating the workload and will remove information that would lead to avoidable coding work. Thus spending some time to model, in a simple way, the specification plan of the desired system will help giving an insight of the completeness and the quality of the specification plan and, in the same time, will help estimating the work load of the project thus putting the project development team in the best condition to start coding.

Project management and project tools, the work of the project development team

As for the technical risks, it is very difficult to tell a project development team how to work. Reducing risk inherent to the project development team is made when he must give the proof of his ability to conduct a project. However, giving him structured document easy to understand and without incoherencies will help him lead the project in the best conditions.

2.2. Modeling, a simple way to reduce functional risks in IT projects

According to the risks identified by Nathan Hattab, we can see two key functional risks that can be reduced to improve IT system quality, the inaccuracy of the specification plan and the wrong estimate of the project load. An upstream modeling of the system will help reduce these two key functional risks. The IT system modeling guarantees a good communication between all the stakeholders of a project. It allows the transformation of the literal language of the project owner needs in a structured language understandable by the project development teams.

To resume, modeling the system exclusively from the requirements allows the verification of the completeness of the specification plan, as well as the identification of the incoherencies between two requirements. It will also help in:

• Clarifying the specification plan by testing upstream the requirement of the system,
• Better estimating the work load by having before the estimation a tested and qualified architecture,
• Reducing the risks induced by the project leader by offering him detailed requirements, a qualified model and a set of test to help him lead his project,
• Improving the project owner maturity by showing him how his choice of upstream analysis allowed in increase in the quality of the whole project.

References:

1. http://software-testing-zone.blogspot.com/2008/12/why-are-bugsdefects-in-software.html
2. www.BasilDev.com, Automating the functional tests of embedded real time software
3. AFAI : French Association of Audit and Computer Consulting
4. http://www.afai.asso.fr/index.php?m=76

More complexity, but more quality insurance either !

The increase in complexity of IT project carries inexorably an increase in the odds of errors and defects in the development of these projects.

On the other hand, while IT systems are becoming more and more critical, the cost of uncorrected or belatedly corrected errors and defects will grow exponentially woth the size of the project. Moreover, it is known since a long time, almost three decades, since Barry Boehm in 1981 that the more an error is discovered lately, the more it will be expensive to correct. The figure 1 below shows the increase in cost correction of an error depending on the stage in which it is discovered.

Figure 1 : Increase in error correction cost for each development stage error

Fortunately, today, we have quality insurance tools that are made to verify the system at each step of its development.

The SMV method is a quality insurance method that will, such as the V & V, be used with renowned tools in the software lifecycle development in order to reduce as soon as in the requirement stage, functional and technical errors present in all IT projects.

1. Towards an increase of complexity in IT projects

1.1. Innovation, a prerequisite for company growth

Companies are bound to innovate. The company that doesn't innovate will create fewer added value and will lose its competitiveness. On the other hand, the company that innovates will keep its competitiveness but will face other problems such as IT maintenance problems or IT system upgrade. As Pierre Bonnet [1] says: " The IT system follows a complexity curve indexed on the innovation and the change curves". This increase in complexity will inexorably lead to an increase in cost of development, tests and validation of the final system.

1.2. Technical drifting, a technical risk proportional to the size of an IT project

According to the NIST (National Institute of Standards and Technology), 80% of the development cost of an IT project is dedicated to error identification and correction, and the later an error is corrected, the more expensive it will be to correct. In 2002, IT errors and defects cost the American economy not less than 60 billion dollars [2] or in other words 0,57% of the American GDP.

Waiting for the test stages to begin error correction in IT projects is ineffective and will become more dangerous as IT systems are getting more complex. The Standish Group doesn't make a very optimistic analysis of the IT project situation either. In fact, in its report titled "Extreme Chaos", the Standish Group classifies IT projects in three categories:

• The successful projects, projects released in time, on budget and with all the expected functionalities,
• The challenged projects, those released behind schedule, over budget or with less functionalities than expected,
• The failed projects, those that were never finished or never used.

Since 1994, the Standish Group holds statistics on all these three categories of projects. For the 1994-2009 period in the United States, the figures are those of the table 1 and figure 2 :

Figure 2 : Success rate of IT projects between 1994 and 2009 in the United States according to the Standish Group

Table 1 : Evolution in IT project success rate (in %) for the 1994-2009 period according to the Standish Group [3]

	1994	1996	1998	2000	2002	2004	2006	2009
Succeeded	16	27	26	28	34	29	35	32
Challenged	53	33	46	49	51	53	46	44
Failed	31	40	28	23	15	18	19	24

The stagnation of the success rate of IT projects since the year 2000 around 31,6% and the increase in the failure rate since 2002 shows the ineffectiveness, inadequacy or the lack of actual quality insurance method that are unable to insure the good completion of IT projects.
Those three observations, the increase of complexity in IT projects, the exponential increase of the error correction cost in the later stage of the project and the difficulty to finish a project according to the requirement and the original estimates shows the necessity of using effective quality insurance methods and procedure made to control quality from the beginning of the project and identify error before the software testing step by applying control and recording methods in the requirements and modeling stages.

2. Towards an improvement of quality insurance in IT software

In 2007, the spendings in IT quality of French companies represented 535 million Euros. Moreover, 53% of French companies consider that the quality insurance of their IT project will have an increasingly important impact on their business results. These two observations shows that we are no longer thinking about how much will cost an IT system, but rather how much will cost a defective system [4]. The quality insurance is an essential process to add to IT business development.

2.1. A definition of quality insurance

Quality insurance is: "A whole set of pre established and systematic actions that are required to give the necessary confidence in what a product or a service must satisfy to quality related requirements". It is a set of processes acting at each stage of an IT project and allowing each stakeholder to know at anytime the exact status of the project and its preceding states.
If the processes of the quality insurances comply with the needs of the V & V (Verification and Validation), are they enough to answer the increasing problem of IT project due to complexity and project size?

Quality insurance tools are, for instance, reviews, audits, cross readings, peer reviews, technical reviews, quality inspection or others. But theses tools, even if they identify some of the errors and omissions, they won't identify them early enough to save 80% of the budget of an IT development.
In order to solve the errors upstream, where they are less expensive to correct, it is necessary to implement technical add-ons, as soon as in the business needs analysis stage, to clarify the project requirement and help all the stakeholders follow their project with precision.

2.2. Verification and validation, a technical process of the quality insurance

When they are applied at each step of a project, verification and validation answers the problems of quality insurance from the inside of iterative project development steps.

• The verification: Its aim is to show that the activity has been well done, that it was compliant to the realization plan and that it didn't introduce any defects in the result. In order to achieve those 3 objectives, it needs to have access, at every step of the project, to executable tests for model and the system itself.
• The validation: Its aim is to show that the activity complied with its objectives. In order to succeed in this, we need to dispose of tests created from the requirements to execute them at each step of the project to verify that the product itself is compliant with its requirements.

The main advantage of the V & V is that it can be fully integrated in the software development lifecycle. And on the contrary of other quality insurance methods, it proposes technical solutions that helps every stakeholders of a project to better understand its development.

References:

1. http://www.bestpractices-si.fr/index.php? option=com_content&task=view&id=423&It emid=73
2. NIST, US Dept. of Commerce, The Economic Impacts of Inadequate Infrastructure for Software Testing, mai 2002.
3. http://www.projectsmart.co.uk/the-curious-case-of-the-chaos-report-2009.html
4. http://www.itrmanager.com/articles/70332/qualite-systeme-information.html