ATCGen
Motivation for Development: DoD acquisition policy requires testing throughout the systems development process to ensure interoperability and conformance to standards. The Joint Interoperability Test Command (JITC) is the test organization responsible for certifying systems for joint interoperability. The JITC is striving to improve traditional standards conformance and interoperability testing methodologies in order to keep pace with the behavioral complexity of Future Combat Systems and others that are increasingly reliant on advanced information technology. Moreover, DoD has mandated use of M&S throughout the development life cycle, which requires testing methods that are themselves M&S-based. As a result, JITC is leveraging M&S efforts to modernize its testing processes with a view to automation and compliance to DoD M&S standards such as HLA (High Level Architecture).
Case in point: the Tactical Digital Information Link (TADIL)-J standard (MIL-STD-6016c) is intended to govern joint command and control system interoperability in Link-16 networks. The specification document is voluminous, with many interdependent elements, rendering its interpretation labor intensive and prone to error. Moreover, due to its size and complexity, the specification as a whole is potentially ambiguous and incomplete. As a consequence, traditional conformance testing procedures, developed from the document, do not fully assess the MIL-STD requirements and are inconsistent across military services and programs.
ATCGen Description: In 2003, the JITC started development of a "formalized" version of the MIL-STD 6016C rule sets with the objective to complete an unambiguous description of the specification, and at the same time, enable more automated test development. A contractor team under Northrop Grumman, that includes Arizona Center for Integrative M&S, led the effort to "formalize" the standard as a system specified by a DEVS which mandates the outcome of tests and sequences of tests. In the approach that evolved, a standards document (in this case MIL-STD 6016C) is analyzed to uncover relevant elements of the DEVS specification that then become the basis for semi-automated test case generation. In outline:
ATCGen analysts identify if-then rules from the document, casting them into XML, with associated condition and action variables.
Automated analysis of the variable dependencies enables visualization of rule firings and selection of potential test sequences. The test sequences are expressed in XML and stored in a repository for combinational reuse.
Test cases are represented as DEVS models that are semi-automatically generated from test sequences. DEVS hierarchical construction allows basic models from a small set of primitives to be coupled together in higher level models. An XML transformation automates the conversion of a set of test sequences into composite test packages that execute in the DEVSJAVA environment where they are tested against a stub representing the system under test (SUT).
Finally, a Test Driver that is based on the DEVS simulation protocol executes the test models against the SUT in a distributed simulation infrastructure based on the HLA.
Application of ATCGen: SIAP (Single Integrated Air Picture) refers to the goal of forming a unique shared understanding of the tactical air situation among participating combat systems. Developed by the Joint SIAP System Engineering Organization (JSSEO), the IABM supports an extension of TADIL-J developed in an HLA environment. Accordingly, it requires an HLA compliant infrastructure for testing. ATC-Gen will be one of the primary test vehicles for the first assessment of the IABM MIL-STD 6016c conformance to occur in fall of 2005. During integration testing, ATCGen has demonstrated the capability to assess the developmental version of the IABM (Time Box 24) and is already having a positive impact on IABM development. Extensive test case generation is underway for critical functional areas of the MIL-STD, such as correlation/de-correlation, that will substantially enhance the IABM's success as it is incorporated into DoD command and control platforms.
Future Extensions: ATCGen is leading an evolution of traditional test processes into a rigorous, automation-capable M&S-based acquisition-support methodology. As the DoD's Net-Centric Enterprise Services project matures, the HLA implementation can be augmented by web-services middleware to provide access to accredited test federations for conformance testing of a wide variety of proposed web services.