Editorial – ITEA Journal – September 2026

It has been our privilege to edit this special issue of the ITEA Journal highlighting work from this year’s Defense and Aerospace Test and Analysis (DATA) Workshop, or DATAWorks. DATAWorks is a multi-organization collaboration with the Director of Operational Test & Evaluation (DOT&E) within the Office of the Secretary of War, National Aeronautics and Space Administration (NASA), the Institute for Defense Analyses (IDA), and the Section on Statistics in Defense and National Security (SDNS) of the American Statistical Association (ASA). The tenth annual workshop was held from April 21-23, 2026 in Alexandria, Virginia, with the goal of helping the defense and aerospace communities strengthen their application of scientific approaches to test design and program evaluation. We personally look forward to DATAWorks every year as a unique forum where one can see presentations ranging from highly technical statistical methods development to hands-on applications on practical problems to emerging challenges in defense and aerospace testing. This issue describes several such works from the event.

The bulk of this issue is devoted to technical paper presented at DATAWorks 2026. The article topics heavily reflect the extent to which the field is evaluating the impact of artificial intelligence and machine learning (AI/ML), from assuring AI/ML-enabled systems to their applications to scientific techniques to their impact on great power dynamics.

Our first article, “Machine Learning-Based Cognitive Instrumentation for Runtime AI Assurance”, was written by Randall McCutcheon, Li Qiao, Heba El-Fiqi, and Matt Garratt from the University of New South Wales (UNSW) Canberra and Adam Olsen of the Center for Naval Analyses. It explores the concept of a machine learning (ML) based monitor to provide operational assurance of deployed ML systems. The findings provide preliminary evidence that this kind of cognitive instrumentation can serve as an additional layer of assurance for AI-enabled systems.

Continuing with the theme of assurance of AI/ML systems, our second article, from Tiffany A. Ceasor of the Technological Society of Applied Research (TSAR) and Borko Furht of Florida Atlantic University, presents a framework for full lifecycle validation, monitoring, and auditing of AI systems. Entitled “Compositional Assurance with the IRON COMPASS Framework: Integrating Formal Verification with Statistical Test and Evaluation for Mission-Critical AI Systems”, the paper describes a methodology that partitions AI system requirements into four assurance tiers, binds each tier to the strongest feasible verification method, places each at the appropriate lifecycle phase with post-deployment drift monitoring, and combines the resulting evidence into a single integrated audit record.

Transitioning from that high-level framework, our next paper, entitled “Breaking Agents: a Testing Framework for Red Teaming Large Language Model (LLM)-enabled Systems”, describes an approach to testing AI agents. By David Han of the University of Texas San Antonio, it proposes a formal T&E methodology that treats agent red teaming as systems testing with explicit, measurable failure oracles. The author defines security failures as invariant violations across three layers – narrative output, tool invocation, and downstream state change – and designs structured multi-turn adversarial campaigns that probe four manipulation pathways: direct prompting, indirect (retrieved-content) injection, tool-response manipulation, and memory poisoning.

Our fourth article treats LLMs not as agents but as models of knowledge. This paper, “Incorporating Large Language Models into the Bayesian Optimization Process for Engineering Design” by Sam Donald, Rounak Meyur, Tonya Martin, and Sumit Purohit of Pacific Northwest National Laboratory, studies whether large language models (LLMs) can be used to formulate prior knowledge to directly assist with Bayesian optimization (BO), a powerful tool for optimizing expensive functions with minimal evaluations. Using lift-to-drag ratio maximization as a case study, the authors evaluate two knowledge injection points: LLM-guided initialization, where space-filling schemes are replaced with domain-aware initial points, and LLM-augmented surrogates, which blend a Gaussian process with few-shot LLM predictions.

Continuing in the theme of testing agentic systems, our next paper, “Competence Estimation Impact in Multi Agent Systems” is by Francesca Reale McFadden of the Johns Hopkins University Applied Physics Laboratory. It describes how competence estimation, a method for estimating suitability of a model to a given task for a given input, can be integrated into coordination and reasoning in a multi agent systems. The approach transforms coordination in multi agent systems from static specialization based assignment to dynamic, input-dependent, competence based weighting and combination of agent responses.

Our sixth paper, “Deployment of Single Board Computer Mesh Networks for Remote Cyber Operations” by Ryan J. Monagle of the United States Military Academy, proposes a two-phase decision-support framework for designing and evaluating the placement of distributed commercial-off-the-shelf single board computers (SBCs) within a mesh network. The first phase uses integer programming to solve a set-cover problem and find efficient placement options, while the second phase applies a stochastic discrete time simulation to evaluate equally optimal solutions. This framework allows decision-makers to plan more effectively to integrate cyber operations into their missions and have confidence in the reliability of their devices.

Finally, we return to the theme of AI in our last paper, “Assessing Bio-AI/ML Convergent Threats: Using Historical Data for Future Trajectories” by Kathleen Bostick and Chenyi Ling of Applied Research Laboratory for Intelligence and Security (ARLIS) at the University of Maryland. This article looks at U.S. and Chinese leadership as AI/ML accelerate biotechnology innovation, creating a new convergent capability known as bio-AI/ML convergence with national-security implications. The authors conduct an assessment through a lifecycle data-collection approach applied to bibliometric and security-designation data from 2015–2026, retrieving publication output across four domains (gene editing, precision medicine, AI + biotechnology, ML + biotechnology), and find that, between 2024 and 2025, China surpassed the U.S. in combined biotech publication volume across three of four domains.

In addition to the feature DATAWorks articles, this special edition includes two other technical articles. One titled, “T&E in Practice: Applying the Defined Discipline”, was written by Dr. Ben Luther, who completed a PhD from the University of Adelaide. Luther’s paper addresses the gap between a definitional view of T&E and what the author perceives as the industry’s practical view of T&E. It points out observed points of confusion and argues for T&E to be framed as assurance of uncertainty rather than procedural test activity.

The other included article was written by Burak Oktenli, who holds an MBA and a Master of Professional Studies in Applied Intelligence from Georgetown University. The article is titled, “Bounded AI and a Layered Assured-PNT Architecture for Jamming-Resilient PNT.” Oktenli proposes a layered assured positioning, navigation, and timing (PNT) architecture that combines bounded artificial intelligence, integrity monitoring, and sensor fusion. In the proposed framework, authority boundaries are defined between machine recommendations, human decisions, and deterministic decisions.

It was a privilege to edit this special issue. We thank the authors for their timely contributions and edits, the reviewers for their careful assessment of submissions, and the ITEA Journal for giving us this opportunity and for their technical guidance along the way. We hope that you enjoyed the results.