Dependability Standards

Introduction

The major international standards bodies are the International Organization for Standardization (ISO) and the International Electrotechnical Commission (IEC). Examples of other industry and national standards bodies include the Institute of Electrical and Electronic Engineers (IEEE), the Society of Automotive Engineers International (SAE), the Canadian Standards Association (CSA) and the American National Standards Institute (ANSI).

When a new standard is created by one of these organizations, the process generally is initiated by an industry segment expressing a need for a new standard. Draft standards, known as working drafts, are created and evaluated by members. Revisions are made until members agree on the standard, at which point it is approved and made available. Standards are evaluated on a regular basis to decide whether there is a need for the standard to be revised, amended or withdrawn. Most standards bodies are made up of volunteers; members pay fees depending on their level of involvement. Copies of standards are available to members, or to non-members for a per-standard fee.

Standards are of course also updated. The rate of update depends on a wide variety of factors, including the industry, process or product targetted by the standard. Also, the rate of technological improvement has an effect on the rate of updates. Clearly an area with rapid changes and technological improvements will have greater need of current, up-to-date standards than an industry with little change. One example of this change is with the drive to cut costs through the use of COTS (Commercial Off The Shelf) equipment in military systems, as opposed to custom-designed products or assemblies. Traditionally, mil-spec parts have had a much more extensive set of standards with which they complied than COTS assemblies. It is important to have clearly defined standards for using and integrating COTS equipment into military equipment in a safe, cost-effective way.

Standards are sometimes merged with each other, depending on factors such as the degree of commonality in the standards, or an attempt to streamline multiple standards, or simply a re-evaluation of their applicability. As standards bodies increase the degree to which they are cooperating in the development of standards, this merging is more common. It is also becoming more common for standards bodies to accept standards created by other bodies and "rubber stamp" them. This tighter involvement is beneficial for all industries but it currently has some problems.

Problems and Resolutions

While standards are clearly useful, there can be gaps or ambiguities between standards issued by different standards bodies which cover similar topics. These differences often manifest themselves when the standards are from different countries. From a business point of view, these differences can increase the cost of product development, because of the necessity to have a product certified by multiple standards bodies. Additionally, ambiguities between standards language can increase the chances of contractual conflicts. In the best case, requirements for certification would be identical between two standards, or one standard would be a subset of the other. However, different standards sometimes have irregular overlap in their specifications, making it important to be aware of differences when designing products to be widely standards-compliant.

Recently, different standards bodies have made significant steps towards harmonization or unification of their standards. Prior to these efforts, different standards bodies had some complementary standards as well as some overlapping standards. However, it is important for the standards bodies not to lower or compromise existing standards in the process.

It is acknowledged by people associated with the development of standards that one of the greatest difficulties to further harmonization of standards is terminology differences between standards which cover the same, or similar topics [Benski96]. Fortunately, standards developers are aware of this problem and are working to mitigate it. Never-the-less, the fact remains that current standards often have contradictory language in them. For example, three standards (IEC IEV 191, ANSI/AIAA R-013, MIL-HDBK 338) all define "failure rate" in very different ways [Benski96]. The ramifications of this difference are significant, and this is only one example.

The only resolution of the consistency issues between standards is to be aware of terminology differences and possible confusion arrising from them. Different industries which may have similar standards may use different terminology; it is a virtual certainty that there will never be uniformity in terminology across all industries.

Compliance

The motivation to comply with various standards is often external to the development of a product. Generally, it is imposed by contractors or customers who insist that products adhere to certain standards. It may also be implicit market competition - purchasers may only buy products with certain specifications, or from companies that follow certain quality procedures. For example, within the European Union, governmental institutions are prohibited from trading with companies which do not follow ISO-9000 standards [Braa94]. A study of eight small companies about their ISO 9000 plans [McTeer] discovered that "In all cases the motivation was that the market-place was demanding or likely to demand an ISO 9000 series certificate of registration...The perception was that commercial forces were coercing them to gain such registration." Finally, there may be licensing or certification requirements (which are themselves part of a specification) which can only be fulfilled if certain standards have been met or followed.

Software Standards

Software presents a unique problem for dependability standards. It is acknowledged that it is practically impossible to write "perfect" software. The difficulty of generating useful values for RMA (reliability, maintainability and availability) makes the usefulness having standards that apply to the finished product debatable. For that reason, the utility of a process standard is much greater for software, since it is in the process of creating software that errors are introduced. There are virtually no errors in the "manufacturing" of software (CD-ROM duplication) and software does not deteriorate with age. If one accepts the premise that a standardized software creation process (or at least a systematic creation process) increases the reliability of software, then a process-based standard is a logical standard to implement.

Additionally, since it is so difficult to exhaustively test software (virtually impossible), the confidence in its operation based on a system test may be low. However, confidence in its operation can be much greater if there have been specific steps followed throughout its development, such as exception handling and code reviews, for example.

ISO 9000 Standards

The ISO 9000 standard is a series of standards detailing quality management processes in an organization [ISObusy99],[ISO12399]. It is one of the most recognized international standards, but does not directly deal with dependability. The different standards in the 9000 series cover different company procedures: ISO 9001 covers topics relevant to all aspects in the life cycle of a product, from design to service. ISO 9002 is similar but does not include the design and development phase. ISO 9003 simply covers procedures for testing and inspection. Emphasis is placed on repeatability, method and documentation. Those characteristics are themselves good, but do nothing to ensure the dependability of whatever the final product is. For that reason, ISO 9000 standards are never enough (nor were they meant to be) for an embedded system.

Relationships to other Topics

Conclusions

With today's rapid technological growth, standards in the engineering fields are changing. While this is necessary to ensure their continued applicability, it also means that embedded systems designers and users of embedded systems must keep themselves up to date with the latest standards. More standards make this challenging, but the collaboration between different standards bodies to unify their standards will make the job easier. Users of standards must be careful to understand terminology in the context of a standard and how it applies to their particular industry. For complex software systems, process-based standards may be necessary to increase confidence in the result in a way that product-based standards cannot.

References

[Benski96] Benski, Claudio, Schneider Electric, Grenoble. Dependability Standards: An International Perspective. 1996 Proceedings Annual Reliability and Maintainability Symposium, IEEE, 0-7803-3112-5/96, copyright 1996, pp 13-16.
Thsi is a report by the chairman of the IEC TC 56 on the state of dependability standards globally.

[IEC99] IEC Web site. http://www.iec.ch/gnote1-e.htm viewed 16 March, 1999.
This is an overview of the IEC's history and mission.

[Braa94] Braa, Kristin and Øgrim, Leikny. "Quality Assurance - an Assurance of Quality? Application of the ISO Standard in System Development." Proceedings of the Twenty-Seventh Annual Hawaii International Conference on System Sciences. 1994. Pp 842-851. 1060-3425/94 copyright 1994.
Good information about the ISO 9000 standard and its applicability to various situations.

[McTeer] McTeer, M M. and Dale, B. G. The attitudes of small companies to the ISO 9000 series. Proc Instn Mech Engrs Vol 210. B07694 pp 397-403.
Information resulting from a study of eight small companies and their attitutudes to the ISO 9000 standard.

[ISObusy99] ISO Web site. http://www.iso.ch/9000e/busy.htm viewed 18 March, 1999.
General information on the ISO 9000 standards from the ISO web site.

[ISO123] ISO Web site. http://www.iso.ch/9000e/123.htm viewed 18 March, 1999.
Differences between the parts of the ISO 9000 specification.