CHAPTER 10 - Software Standards

Introduction

This chapter describes the standards (military and international) that supplier use for the development of software, capability assessment, and software development measurement.

Objective Provides information related to military and international software standards, United States and international working groups, military standards status, and how to obtain standards.
Overview

Over the past 30 years, the ability to develop software has increased in complexity and the ability to improve the software development standards has also increased in complexity. In the 1970s and early 80s, various software development standards evolved such as DoD-STD-1679A, DoD-STD-7935A, Mil-STD-490, DoD-STD-2167A, and DoD-STD-2168.These documents were descendants from hardware standards that included software-related terminology.

 Today, various Military Standards and Specifications are being canceled. With the development of international standards, there seems to be some confusion between US Military Standards, International Organization for Standardization (ISO) Standards, and Institute of Electrical and Electronic Engineers (IEEE) Software Standards.  This chapter attempts to clarify what are “today’s” standards, what is their evolution, their current state, and purpose.

Standards Background

Various groups or committees develop all standards, and the development process of these standards is generally slow.  In the world of software development, technology can quickly out-pace the standard or process being developed.  Standards shall be maintained and updated or else they become outdated and obsolete.

U.S Military Standards

U. S. Military Standards are developed by organizations within DoD to direct or guide suppliers in the area of software development and are authored by industry suppliers. A committee guides these authors that consist of representatives from military and Industry organizations. These standards are legally enforced on military software suppliers and identify engineering and technical limitations for materials, processes, methods, designs, and engineering practices.

In June 1994, Secretary of Defense William J. Perry issued a memorandum “Specifications & Standards – A New Way of Doing Business." This memo outlined the need to reduce the requirements for MilSpecs and MilStds in our acquisition. Some of the MilSpecs and MilStds are misused, obsolete, redundant, or unnecessarily restrictive. Additionally, some of these documents have increased the price the Government pays for goods and services. Government and Industry shall make greater use of performance and commercial specifications, increase access to commercial advanced technologies, increase the industrial base upon which DoD depends, and help military suppliers to be more competitive in the marketplace. Today, this is known as “Acquisition Reform” – changing our acquisition management perspectives, roles, responsibilities, and goals.

ISO Standards

International Standards are created by committees of national representatives. The ultimate goal of these committees, when developing international standards, is to:

1.      Reduce the cost of international software engineering; 

2.      Improve communication between suppliers and buyers; 

3.      Improve the quality of delivered software and systems. 

The United States participates in the development of international standards through ISO/IEC JTC1 SC7 Technical Advisory Group (TAG). The JTC1 has a wide charter to develop standards for information technology. The SC7 is an international technical standards committee working on standards for software engineering, and is a subcommittee of JTC1.  

In the United States, there are 10 Working Groups (WG) that provide support to SC7.The function of these WGs is to develop guidance for the management techniques, standardization of supporting methods, and tools necessary for the development and testing of software. Members of these WGs represent the United States at International Plenary and Technical Working Group meetings. The purpose of attending International Meetings is to represent the United States in each SC7 Working Group and to establish United States position on SC7 standards development milestones and SC7 business plans.  

Participation on these working groups consists of TAG membership and Technical Experts (TE). Participation on these working groups is strictly voluntarily. There can be multiple TAG individuals representing a company but only one designated individual can vote on ballots. TEs have no voting authority but can participate in any work group they so desire. DCMA currently participates on WG 7, WG 10 and WG 13.  

SC7 Work Groups and Purpose

Working Group 2 – Systems Software Documentation
Purpose: Coordinate and develop standards for the documentation of software. 

Working Group 4 – Tools and Environments 
Purpose: Preparation of standards and technical reports for tools and CASE environment. 

Working Group 6 – Evaluations and Metrics 
Purpose: Preparation of standards and technical reports related to evaluation and metrics. 

Working Group 7 – Life Cycle Management 
Purpose: Preparation of standards and technical reports on life cycle management. 

Working Group 8 – Support of Life Cycle Processes 
Purpose: Preparation of standards and reports on support of life cycle processes. 

Working Group 9 – Software Integrity 
Purpose: Preparation of standards, technical reports, and guidance documents related to software integrity at the system and systems interface level. In this context, software integrity is defined as ensuring the containment of risk or confining the risk exposure in software. 

Working Group 10 – Software Process Assessment 
Purpose: Development of standards covering methods, practices and applications of process assessment in software product procurement, development, delivery, operation, evolution and related service support.

Working Group 11 – Software Engineering Data Definition and Representation 
Purpose: Develop standards and technical reports to define data used and produced by software engineering processes, establish representation for communications by both humans and machines, and define data interchange formats. 

Working Group 12- Functional Size Measurement 
Purpose: Establish set of practical standards for functional size measurement. Functional size measurement is a general term for methods of sizing software from an external viewpoint and encompasses methods such as Function Point Analysis. 

Working Group 13 – Software Measurement Process 

Purpose: Development of standards and technical reports to define and implement software measurement processes, frameworks, and guidance.
IEEE and EIA Software Standards

There are two accredited organizations collaborating with DoD on the creation of lifecycle standards for use in the United States. These organizations are Electronics Industries Association (EIA) and the Institute of Electrical and Electronic Engineers (IEEE). There are three standards objectives that have motivated this work:

  1. Should represent the best commercial practice; 

  2.   Suitable for application to the complex requirements of defense acquisition; 

  3. Should be compatible with those of the emerging global marketplace for software. 

Resulting products from the EIA and IEEE: 

  1. IEEE/EIA 12207: This is the strategic standard that addresses the three primary objectives described in the previous paragraph. This document is the United States version of the international standard ISO/IEC 12207 which provides a basis for organizational-wide adoption of software processes suitable for commercial and defense projects that serve both domestic and international customers; 

  2. EIA/IEEE J-STD16: This is the tactical standard that provides a continuing reference for organizations that have invested in software processes created under prior military standards. 
EIA/IEEE J-STD-016 J-STD-016 is the “demilitarized” version of Mil-STD-498 and the J-STD-016 exist as a single document and this document was considered the interim or “bridged” document to Mil-STD-498.It was intended that J-STD-016 will fill the role for organizations that continue to use processes that are a legacy of older standards and this document will allow them to reference those processes. The J-STD-016was developed to be applied in the following circumstances:
  1. Continuing projects that started under Mil-STD-498, DoD-2167A, etc; 

  2.  Enterprises that have put in place organizational processes based on Mil-STD-498, DoD-2167A, etc; 

  3. Projects that desire to continue using this structure. 

IEEE/EIA 12207 – Software Life Cycle Process, (widely known as US 12207) was approved as a standard in the United States in 1998.This standard was adapted from ISO/IEC 12207 that provided a framework of software lifecycle processes suitable for use in: 

  1. Acquisition; 

  2.   Supply; 

  3.  Development; 

  4. Operations of Software; 

  5. Maintenance. 

The following chart illustrates the software development lifecycle development based on past and current philosophy. 

Figure 10-1.Past and Current Development Philosophy.
US 12207 Layout The adoption of US 12207 has shifted the focus from compliance at the project level to the organizational level. The object is to have an organization develop its own set of processes and procedures compliant with the requirements of US 12207, which would be applied across the entire organization. 

US 12207 offers many advantages over past software lifecycle process standards such as: 

  1. Establishes a top-level process framework architecture of the software lifecycle; 

  2. Provides flexible approach to recording process and product data to be handled by CASE tools in contrast to more traditional reliance on paper documents; 

  3. Builds in quality – “in-process”; 

  4. Supports evolving technologies, independent of specific Languages, techniques, methods, models, and environments; 

  5. Fully compliant with the international version of the standard, permitting U.S. industries to develop a single set of organizational processes applicable to both global and domestic issues; 

  6. Information not documentation. 

 

Parts of US 12207 This standard is organized in three parts designated Part 0: 12207.0, Part 1: 12207.1, and Part 2: 12207.2.This may seem a bit confusing, but just remember that Part 0 is like a “sandwich” that wraps US material around the full text of ISO/IEC 12207. The difference in the U.S. version include:
  1. Annex that list errors discovered in the text of the international standard; 

  2. Set of process objectives and data objectives that assist in determining the intent of the process requirements specified in the standard as well as the data;  

  3. Replacement compliance clause that shifts emphasis toward compliance at the organizational level and requires documentation of the means of compliance. 

As previously described, US 12207 is broken down into three parts and the description of these three parts are as follows: 

PART 0 (Standard for Information Technology – Software Life Cycle Processes)

This part contains the body ISO/IEC 12207 in its original form and 6 US Annexes: 

Annex E (Informative) – Basic concepts of ISO/IEC 12207 that consist of four processes: Organizational, Primary, Supporting, and Special. The illustration below identifies these processes and their associated sub-processes: 

 

 

 

 

 

 

 

 

 

Figure 10-2.ISO/IEC 12207 Processes

Parts of US 12207   Organizational Life Cycle Processes: Establishes and implements an underlying structure of processes and personnel that provide the basis for the organization’s projects. Organizational processes include Management, Infrastructure, Training, and Improvement. 

Primary Life Cycle Processes: Processes that provides the framework to accomplish the Acquisition, Development, Operation, and Maintenance of software. 

Supporting Life Cycle Processes: Provides processes that are an integrated part of the software process and contribute to the success and quality of the software project. Supporting processes include Documentation, Configuration Management, Quality Assurance, Verification & Validation, Joint Review, Audit, and Problem Resolution.

Note: Informative is defined as sections that are considered information process activity and normative is considered sections that are required process activity. 

Annex F (Informative) – Provides compliance information related to situations, level, and criteria. 

Annex G (Normative) – Life cycle process objectives: describes the basic objectives to be considered in meeting the intent of each life cycle process defined in ISO/IEC 12207 

Annex H (Normative) – Life cycle data objectives: describe the basic principles to be considered in preparing data during the execution of the life cycle process of US 12207. 

Annex I (Informative) – Describes the relationships with other standards such as IEEE Std 1074, ISO/IEC 12207, J-STD-016, and ISO 9001 

Annex J (Normative) – Identifies and rectifies those clauses in ISO/IEC 12207 that have ambiguities that could not be removed prior to its publication.

PART 1 (Guide for Information Technology – Software Life Cycle Processes – Life Cycle Data)
Guidance document that provides recommendations that expands on the data objectives of Part 0.It also identifies the information items Part 0 requires or recommends to be produced as part of the life cycle process, activity or task along with identifying the Part 0 clause number citing each information item. 

Additionally, Part 1 provides content guidelines for each information item: 

  1. Generic – based on the kind of documentation; 

  2. Specific – for those information items that Part 0 provides specific requirements or recommendations. 

This document also provides references to other standards such as ISO/15504 (Software Process Assessment), ISO/IEC 15939 (Software Measurement), guides, and technical reports. 

PART 2 (Guide for Information Technology – Software Life Cycle Processes – Implementation Considerations)
Guidance document that provides recommendations on the implementation of US 12207 processes in the context of US best practices. The normative text of Part 0 is included in this document and this text is enclosed in boxes (attempt to make document user friendly). At appropriate intervals, guidance notes provide explanatory material and recommendations regarding implementation of the processes. 

AQAP-160 NATO Quality Requirements

NATO has published a document called the AQAP-160 NATO Quality Requirements for Software Throughout the Life Cycle. This publication provides for a common framework for software life cycle processes based upon ISO 12207, Software Life Cycle Processes. Once formally released, AQAP-160 publication will replace AQAP-150, NATO Quality Assurance Requirements for Software Development publication dated September 1997.

AQAP-160 continues to utilize ISO9001 (1994/2000), Quality Standards and incorporates ISO 12207.Section 3 and 4 of AQAP-160, provides tailoring instructions for ISO 12207 primary and support processes that would be applicable to the contract. The tailoring activities are to be completed prior to contract award. 

DOD-STD-2167A DoD-STD-2167A, known as the Defense System Software Development document was developed in the late 80s. The majority of the software suppliers used this document, along with Data Item Descriptions (DIDs) and DoD-STD-2168, as the contractual vehicles for software development.2167A established a strict set of requirements the software supplier had to follow. In the middle 90s, when the Perry Memo was released, 2167A, along with 2168, 7935A, and 1703 merged into a single document called Mil-STD-498.
Mil-STD-498 Mil-STD-498 was the first step to shift from military standards to commercial standards.  Mil-STD-498 provided a “bridge” from DoD-STD-2167A to J-STD-16.  In May 1998, Mil-STD-498 was canceled because this information is now contained in IEEE/EIA 12207.

ISO/IEC 15504 is a 9-part international standard that provides a framework for the assessment of software processes. Two parts are normative:

Part 2: Reference Model for Process and Process Capability. This part addresses a reference model for software processes and the process capability that forms the basis for software process assessment. 

Part 3: Performing an assessment. This part provides a framework for assessments and identifies the minimum requirements for performing an assessment in order to ensure consistency and repeatability of the ratings. The requirements help to ensure the assessment output is self-consistent, and provide evidence to substantiate the ratings and to verify compliance with the requirements. 

All the other parts are informative. 

This standard describes capability dimension, which is a 6-point ordinal scale that identifies the capability of any process (i.e., Incomplete through Optimizing). The measure of capability is based upon a set of Process Attributes (PA). These PAs are used to determine if a process has achieved a given capability. 

This assessment framework can be used by organizations involved in planning, managing, monitoring, controlling, and improving the acquisition, supply, development, operation, evolution and support of software. 

This standard provides a structured approach for the assessment of software processes for the following purposes: 

  1. By or on behalf of an organization with the objective of understanding the state of its own processes for process improvement; 

  2. By or on behalf of an organization with the objective of determining the suitability of its own processes for a particular requirement or class of requirements; 

  3. By or on behalf of one organization with the objective of determining the suitability of another organization’s processes for a particular contract or class of contracts. 

 The framework for process assessment: 

  1. Encourages self-assessment; 

  2. Takes into account the context in which the assessed processes operate; 

  3. Produces a set of process ratings (a process profile) rather than a pass/fail result; 

  4. Through the generic practices, addresses the adequacy of the management of the assessed processes; 

  5. Is appropriate across all application domains and sizes of organization.   

The sophistication and complexity required of a process is dependent upon its context. For instance, the planning required for a five person project team is much less than for a fifty person team. This context influences how a qualified assessor judges a practice when assessing its adequacy and influences the degree of comparability between process profiles. 

The process assessment framework is based on assessing a specific process instance. A process instance is a singular instantiation of a process that is uniquely identifiable and about which information can be gathered in a manner that provides repeatable ratings. Each process instance is characterized by a set of five process capability level ratings, each of which is an aggregation of the practice adequacy ratings that belong to that level. Hence the practice adequacy ratings are the foundation for the rating system. Practice adequacy is a rating of the extent to which a practice meets its purpose as defined in part 2 of this International Standard. 

The standard therefore, provides a rating framework that is as much an assessment of effectiveness as it is of conformance to the practice definition. From the ratings of process instances, a number of derived or average ratings can be determined that provide better insight into the capability of a process within an organizational unit as a whole. 
ISO/IEC 15939

ISO/IEC 15939 (Software Measurement Process). International standard that is currently being developed and that defines a software measurement process applicable to all software-related engineering and management disciplines. The software measurement process in this standard is described through a model that defines the activities of the measurement process, which are required to adequately specify what information is required. Additionally, it describes how the measures and analysis results are to be used and how to determine the validity of the analysis results within a project or organizational structure.

This standard is broken down into five chapters and has seven annexes: 

Chapter 1 – Scope. Describes: 
Purpose, Field of Application, Tailoring, Conformance, and Limitations. 

Chapter 2 – Definitions. 

Chapter 3 – Application of the Standard. Describes: Purpose and Outcomes of the Software Measurement Process, International Standard Overview, and Organization of the International Standard

  Chapter 4 – Description of Activities. Establish and Maintain Measurement Commitment, Plan Measurement, Perform Measurement, and Evaluate Measurement 

Chapter 5 – Informative References. Describes various international standards that are referenced in whole or part of in ISO/IEC 15939. 

Annex A – Tailoring Process. Describes the process fur customizing this International Standard for use by an organizational group. 

Annex B – Guidelines for Defining Metrics. 

Annex C – Example Criteria for Selecting Metrics 

Annex D – Example Criteria for Evaluating Analysis Results 

Annex E – Example Criteria for Evaluating the Performance of the Measurement Process 

Annex F – Example Contents of a Measurement Plan 

Annex G – Guidelines for reporting Analysis Results 

The following illustration describes the measurement process model. 

Measurement
Process Model

 

Figure 10-3. Process Model 15939

Software and Software Related Standards The following tables list some of the major software and software related standards that have evolved over the past 30 years and the standards current status (Canceled or Active).

Figure 10-4.Current and Canceled Standards

Software Standards Time Line

Figure 10-5. Standard Timeline

Timeline Update

ISO 9000 has been significantly revised to the following set format:

ISO 9000:2000
ISO 9001:2000 
ISO 9004:2000 

These ISO documents are in Final Draft International Standard (FDIS) status and they are scheduled to be released in CY 2001 as international standards. 

ISO/IEC 12207 is going through an amendment and this document is scheduled to be re-published in CY 2002. 

ISO/IEC15288 (System Life Cycle Process) is up to Committee Draft (CD) 3, and it should be published as an International Standard in 2003. 
Information on How to Obtain Standards

Some documents are available for free and some are a cost to Government employees. Some documents, such as J-STD-016, ISO/IEC 14958 (Product Evaluation) have not been adopted by the US Government, are not free to US Government employees.

IEEE/EIA 12207 (US 12207)
Defense Printing Service. Phone (215) 697-2179, FAX (215)697-1462. 

Effective October 1, 2001, Defense Printing Service will no longer provide US 12207 for free. This document will need to be obtained through IEEE or other sources.  DCMA HQ is currently working on an agreement between IEEE to provide specific international standards to DCMA personnel who require a need for these documents.   

J-STD-016
Order from IEEE or Global Engineering Documents at 800-854-7179,FAX 303-397-2740 
Cost as of 1 Nov 2000: $ 179.00 

Assist-Online Web Site

The ASSIST-Online web site is a robust, comprehensive web site providing access to current information associated with the military and federal specification and standards in the management of Defense Standardization Program (DSP). Managed by the DoD Single Stock Point (DoD SSP), Philadelphia, ASSIST-Online provides public access to standardization documents over the internet. Additionally, this web site provides many powerful reporting features and an exhaustive collection of both digital and warehouse documents. ASSIST is the official source of DoD specifications and standards.

ASSIST-Online provides direct, online access to over 88,000 digital documents in Adobe Portable Document Format (PDF) which may be downloaded to users’ local workstations. All ASSIST documents are now available to DCMA personnel free of charge. All you have to do is register at http://assist.daps.mil/. Select Registration, enter the required account information and within 48 hours you will receive your account name and password. 

Top

Return