What is a standard?
A standard is an established norm or set of requirements, usually developed and approved by a recognized authority, that provides guidelines, rules, or definitions to ensure consistency, quality, and safety across processes, products, or services. Standards are used to harmonize technical criteria, ensure interoperability, and facilitate communication and trade.
Standards and Test Automation
Test automation has a dual relationship with standards:
Determining compliance to a standard can be the purpose of a testing process
There also exist standards that apply to the activity of test automation itself
More below.
Characteristics and Goals of Standardization
Consistency - Standards ensure that products, services, and processes are consistent and uniform. Uniformity allows for compatibility and interoperability across systems, to reduce variability and errors.
Quality - Standards help in maintaining a certain level of quality. By adhering to a standard, organizations can ensure that their products or services meet predefined criteria, which can increase consumer trust and satisfaction, and to meet government-mandated levels of quality and performance.
Safety - Many standards are designed to ensure the safety of products, services, or processes. Compliance with safety standards is often mandatory and helps in protecting consumers, workers, and the environment.
Interoperability - Standards enable different systems, devices, or components to work together effectively. For example, communication protocols like Bluetooth or USB are standardized, ensuring that devices from different manufacturers can connect and function together.
Efficiency - By following established standards, organizations can avoid reinventing the wheel, leading to more efficient processes and reduced costs. Standards can also streamline production and reduce the need for custom solutions.
Scope of Standards
International Standards are developed by international organizations like the International Organization for Standardization (ISO) or the International Electrotechnical Commission (IEC), these standards are recognized and used globally.
National Standards - Created by national bodies like the American National Standards Institute (ANSI) in the United States or British Standards Institution (BSI) in the UK, these standards are specific to a country.
Industry Standards - Developed by industry groups or consortia, these standards are specific to a particular industry, such as the Institute of Electrical and Electronics Engineers (IEEE) standards for electronics and computing.
Standards Creation
Creation and dissemination of standards can proceed “outward”, starting from standards bodies, like IEEE, ISO or ITU outward to manufacturers, integrators and installers, or from industry implementation “inward”:
De Jure Standards
A de jure standard is a standard that has been officially approved and formalized by a recognized standards organization or regulatory authority. The term "de jure" is Latin for "by law," indicating that these standards have been formally established through a legal or official process and are often mandated or required for compliance. Compliance with de jure standards is usually based on passing test suites or meeting key specifications.
De Facto Standards
These are standards that have not been formally approved by a recognized body but are widely accepted and used, often because of market dominance, like certain software platforms or file formats. Compliance with such de facto standards is usually based on comparison to a reference implementation.
Leading vs. Trailing Standards
Some standards lead by defining APIs, specifications and other parameters of interoperability. They are formally defined prior to implementation, as with De Jure standards. Other standards trail behind usage, by describing and formalizing existing implementations, like the Linux Standards Base.
Many standards are actually hybrids: they start life as De Jure standards, where vendors complement and expand them for utility and differentiation; those additions then find inclusion in subsequent revisions of the De Jure standard. A good example is HTML, which was originally narrowly defined as a markup language. Its implementation on actual web sites and in browsers led to expansion of the standard, and eventual conformance by web designers and browser developers to a common HTML definition. It was subsequently formalized by W3G by whatwg.org.
Conformance and Compliance
Both these terms describe adherence to the particulars of a standard: specifications, data types, APIs, etc. Some standards give these terms unique definitions, e.g., the POSIX Standard for portable operating system interface (IEEE/Open Group 1003.1-2017), defines conformance as strict adherence to all APIs and architectural definitions, and compliance as implementation to some defined subset for the standard.
Example Standards
There are myriad standards across the gamut of technical, industrial and scientific endeavors. A few familiar standards that impact most people’s lives and the practice of test automation are
ISO 9001 A standard for quality management systems.
IEEE 802.11 A standard for wireless networking (Wi-Fi).
HTML (Hypertext Markup Language) A standard for creating web pages.
SCPI (Standard Commands for Programmable Instruments) A standard for controlling test and measurement instruments.
Standards in Test Automation
Adhering to standards helps in creating robust, efficient, and maintainable test automation frameworks that can evolve with the software and hardware they are designed to test.
ISTQB Standards
International Software Testing Qualifications Board (ISTQB) provides a well-recognized standard for testing processes. ISTQB's Test Automation Engineer certification covers automation standards, methodologies, and practices.
IEEE Testing Standards
IEEE 829 (Standard for Software and System Test Documentation) Although primarily for test documentation, this standard can guide how automated tests are documented.
IEEE 1008 (Standard for Software Unit Testing) defines standards for unit testing, which is often automated.
IEEE 29119 (Software Testing Standard) provides a comprehensive approach to software testing, including guidelines on test automation. The international version, ISO/IEC/IEEE 29119 Software Testing , provides a comprehensive framework for testing, including guidance on test automation. It includes concepts, processes, and documentation requirements relevant to automated testing.
SCPI
A standard with particular relevance for OpenTAP and the ecosystem of instruments that it supports is SCPI (Standard Commands for Programmable Instruments), an interface for controlling programmable test and measurement devices.
Introduced in 1990 as an additional layer on top of IEEE 488.2-1987, SCPI provides a consistent set of commands for a wide range of instruments, such as oscilloscopes, multimeters, spectrum analyzers, and power supplies. Today SCPI is maintained and promoted by the IVI Foundation.
SCPI has become an essential part of instrument control in various industries, enabling efficient automation and integration of test and measurement systems. Absent SCPI, test automation software ends up being specific to vendors and/ to individual instruments, limiting interoperability and imposing technology silos on testing organizations.
OpenTAP SCPI capabilities not only offer support for a range of instruments from Keysight, but also provide a bridge between OpenTAP, a large ecosystem of legacy instruments and test equipment from Keysight industry peers.
Attributes of SCPI
Standardized Command Set – SCPI provides a set of ASCII-based instrument control commands, allowing users to control different types of instruments using the same or similar commands, simplifying the learning curve and reducing development time.
Text-Based Commands – SCPI commands are text-based and human-readable. For example, a command to measure voltage might look like MEAS:VOLT?, easing understanding and troubleshooting of scripts and programs that interact with instruments.
Hierarchy and Structure – SCPI commands are organized in a tree structure and grouped logically into subsystems, simplifying navigation and use. For instance, all commands related to voltage measurements might be grouped under a VOLT subsystem.
Compatibility – SCPI is backward-compatible with earlier instrument control standards, such as IEEE-488.2 (a.k.a. GPIB), allowing integration of older instruments into modern test systems with minimal changes to existing code.
Interface Agnostic – SCPI functions over diverse communication interfaces, including GPIB, USB, Ethernet, and RS-232. This flexibility allows SCPI to support a wide range of test and measurement environments.
Conclusion
Modern products and services are highly dependent upon the development and application of standards. Without the presence of and adherence to standards, software and systems could not interoperate – every piece of technology, down to basic nuts and bolts and nails used in construction, would be artisanal, raising costs and limiting utility.
Building and testing today’s complex products, and deploying services for and with them, would not be possible without standardization. If your organization defines its deliverables according to standardized specifications, don’t hesitate to support the foundations and .orgs behind those standards. Become a member, vote and help to define the next generation of the technologies at the core of your business.