2026 AI in T&E Forum

Author:  Misty Blowers, Ph.D., Datalytica

The rapid operational adoption of frontier generative and agentic AI systems, driven by the U.S. Department of Defense’s commercial-first strategy under the Chief Digital and Artificial Intelligence Office (CDAO), introduces a critical assurance challenge: how to evaluate the reliability and mission suitability of advanced AI capabilities deployed under constrained-access conditions. As commercially developed AI models are integrated into operational, intelligence, and enterprise workflows, traditional evaluation approaches—largely focused on nominal performance or static compliance testing—are insufficient to characterize system behavior under realistic operational stress and adversarial interaction.

This paper presents REVEAL, an interaction-driven AI assurance and red-teaming framework designed to assess AI-enabled and agentic systems without reliance on internal model access or proprietary details. The approach emphasizes structured interaction with deployed systems to characterize externally observable behavior under a range of adversarial and operational conditions, producing reusable behavioral artifacts that support comparative assessment and benchmarking. Rather than attempting exhaustive verification or internal inspection, the framework is designed to scale alongside commercial AI adoption and evolving tradecraft, while remaining compatible with bounded assurance techniques where formal reasoning is appropriate. Aligned with CDAO’s objective to accelerate AI adoption at the speed of operational necessity, this work outlines a practical assurance paradigm that enables early, repeatable evaluation of frontier AI systems while preserving intellectual property, supporting governance, and maintaining operational relevance.

Author: William Alexander Reed, Astrion

Modern defense systems increasingly rely on AI‑enabled architectures to interpret heterogeneous data, maintain situational awareness, and predict threat behavior across multiple physical regimes. Yet most Test and Evaluation (T&E) frameworks still treat atmospheric flight, orbital motion, re‑entry, and cis‑lunar dynamics as isolated domains, limiting the ability of AI systems to reason coherently across regime boundaries. This work presents UNIFIED, an AI‑enabled systems framework that uses a generalized inference engine to integrate multi‑regime dynamics, multi‑INT sensing, and maneuver‑centric uncertainty into a single predictive architecture for C5ISRT applications.

At the core of the framework is a regime‑agnostic inference engine that maintains a unified state and parameter representation for maneuvering objects. Instead of relying on separate models for each domain, the system uses a shared state structure augmented with higher‑order dynamics to capture thrust ramping, jerk‑limited maneuvers, and uncertain mass depletion. Latent physical parameters—such as drag, lift, reference area, specific impulse, thrust limits, and maneuver budget—are treated as uncertain variables with bounded priors. This allows the inference engine to reason jointly about platform capability, intent, and feasible future trajectories, even when observations are sparse or intermittent.

A multi‑layer sensing model provides a consistent measurement interface for EO/IR, radar, RF/SIGINT, and space‑based sensors. By mapping all sensor outputs into a common likelihood structure, the framework enables AI‑enabled systems to fuse heterogeneous data streams without regime‑specific logic. Regime‑dependent dynamics models—including atmospheric flight equations, orbital mechanics, re‑entry drag models, and CR3BP formulations—are encapsulated behind the inference engine, allowing AI algorithms to maintain continuity as objects transition between regimes.

To support T&E activities, the framework introduces a maneuver‑centric initialization strategy based on three canonical epochs (pre‑maneuver, maneuver, post‑maneuver). This structure enables evaluation of AI inference systems under uncertain maneuver timing, incomplete observability, and variable sensing conditions. A predictive reachable‑set module further supports threat envelope analysis, maneuver feasibility assessment, and cross‑regime intent estimation.

The primary contribution of this work is a non‑commercial, technically rigorous AI‑enabled systems framework that unifies sensing, dynamics, and inference for multi‑regime C5ISRT. By providing a consistent mathematical structure, a regime‑agnostic inference engine, and a maneuver‑aware initialization strategy, UNIFIED enables government, industry, and academic practitioners to conduct more realistic, repeatable, and analytically grounded Test and Evaluation of AI‑enabled sensing, tracking, and prediction systems.

Author: Eman Kawas, Independent Advisor

AI-enabled systems are increasingly integrated into safety-critical and security-critical mission environments. Yet, traditional verification, validation, and test and evaluation (T&E) approaches often struggle to produce decision-relevant assurance for complex and adaptive behaviors. In many programs, testing generates model performance metrics or system demonstrations but does not meaningfully constrain or inform the operational decisions that govern certification, authority, and deployment. For AI-enabled complex systems, assurance must evolve beyond retrospective validation toward faster learning cycles, shifting V&V left, building robust feedback loops, and establishing smarter decision gates so deployments are risk-aware, value-driven, and executed with “eyes wide open.”

This paper proposes a decision assurance framework for integrating and validating AI-enabled mission systems across the lifecycle. The approach focuses on aligning V&V and T&E evidence to the specific decisions that matter in complex systems: entry and exit criteria for autonomy, confidence thresholds for mission use, escalation conditions for human override, and governance boundaries for operational authority. The methodology combines three elements. First, decision-centered test design defines test objectives in terms of operational decisions rather than model outputs alone. Second, lifecycle confidence thresholds are established to quantify “sufficient confidence” relative to mission risk, uncertainty, and consequence. Third, digital engineering infrastructure, including high-fidelity digital twins, simulation environments, and learning-based operational scenarios, is used to accelerate coverage while preserving traceability between test evidence, system behavior, and mission context.

The paper further addresses the tension between speed and assurance in AI fielding. As programs push toward rapid iteration, decision assurance provides structured guardrails that enable decision velocity without eroding lifecycle confidence. Rather than treating testing as retrospective validation, the framework positions T&E as an active mechanism for shaping and governing AI-enabled capability insertion. The expected contribution is a practical, non-commercial structure for connecting AI T&E outcomes to operational authority and mission impact. By translating test evidence into decision confidence, the approach supports more rigorous integration, clearer certification pathways, and sustained decision advantage for the warfighter in increasingly complex AI-enabled systems.

Author: Mehran Irdmousa, MZI Aviation

Modern defense acquisition programs face unprecedented integration complexity. When multiple vendors simultaneously develop interdependent systems such as radars, communications networks, automation platforms, and command-and-control infrastructure. The number of potential interface conflicts grows exponentially. Traditional systems engineering verification methods, relying on manual document reviews that typically cover only 10-15% of artifacts, cannot scale to meet compressed timelines demanded by initiatives like the Department of Defense’s Combined Joint All-Domain Command and Control (CJADC2) or rapid acquisition authorities. This paper introduces the Systems Engineering Command Center (SECC), an AI-powered assurance agent that continuously validates requirements, architectures, and interface specifications across distributed vendor ecosystems before they inform digital twin simulations or production decisions. Unlike conventional requirements management tools that validate within a single source, SECC operates at the system-of-systems level, ingesting artifacts from multiple vendors, formats, and tools, including legacy PDFs, Model-based Systems Engineering (MBSE) models, Interface Control Documents (ICDs), and requirements databases while validating them collectively for consistency, completeness, and compliance.

Methodology:SECC integrates three core AI technologies: (1) vector databases enabling semantic search across requirement statements, (2) transformer-based language models for conflict detection and ambiguity identification using Retrieval-Augmented Generation (RAG), and (3) a Multi-Attribute Utility Theory (MAUT) scoring framework that computes system health based on issue severity and category. The architecture classifies findings across four categories: syntax, traceability, semantic conflicts, and cybersecurity gaps, with severity-weighted scoring that provides engineers with actionable, risk-prioritized insights through an interactive dashboard.

Validation and Results:SECC was developed and validated through a George Mason University SEOR graduate capstone project (Fall 2025) sponsored by MZI Aviation. The research team built a functional prototype and tested it against synthetic systems engineering artifact sets, aviation, drone, and smart home domains, each seeded with known inconsistencies across ConOps, requirements specifications, and design documents. Three LLM backends were evaluated (Llama 3.2, Qwen3, GPT-5 mini), with GPT-5 mini demonstrating superior detection of semantic conflicts and traceability gaps. Monte Carlo simulation modeling projected an 89% cost reduction compared with manual review processes. The prototype demonstrated cross-document conflict detection, automated severity classification, and real-time health scoring, capabilities directly applicable to defense programs managing hundreds of interface documents across distributed vendor teams.

Expected Contribution: SECC addresses a critical gap in digital engineering pipelines: ensuring that digital twins and simulations are built on verified, consistent data rather than propagating hidden conflicts at simulation speed. By positioning SECC as a validation gateway within the digital thread, defense programs can achieve higher confidence in integration outcomes and reduce costly late-stage discoveries. The methodology applies across DoD, NASA, and federal civilian agencies executing complex multi-vendor modernization efforts.

Author: Andrew Pollner, American Software Testing Qualifications Board (ASTQB)Generative AI is rapidly reshaping software engineering, and software testing is no exception. Tools now generate test conditions, test cases, and test data directly from user stories, requirements, APIs, and even source code. Organizations report dramatic productivity gains: faster draft creation, improved formatting consistency, and the ability to scale test artifact production across large backlogs. However, a critical question remains: does AI-generated testing inherently improve software quality — or merely accelerate the testing process? Generative AI operates through probabilistic pattern recognition rather than contextual reasoning or risk awareness. While it can efficiently produce plausible test cases, it does not inherently understand domain constraints, regulatory implications, architectural dependencies, or the nuanced tradeoffs between coverage depth and mission risk. As a result, AI-generated artifacts often emphasize straightforward “happy path” scenarios while underrepresenting boundary conditions, negative testing, state transitions, and non-functional risks unless explicitly prompted. This shift reframes the tester’s role. Instead of primarily creating test cases, testers increasingly review, evaluate, and refine AI-generated outputs.

They must:•Identify gaps in coverage•Detect ambiguity or logical inconsistency•Apply structured test design techniques•Align testing depth with risk exposure•Ensure traceability and meaningful coverage metricsIn this new paradigm, foundational testing knowledge becomes more—not less—important. Techniques such as boundary value analysis, equivalence partitioning, decision tables, state transition modeling, and risk-based testing are essential for both crafting effective prompts and validating AI results. Without these competencies, teams risk deploying superficially complete test suites that provide false confidence.

The central argument presented here is that AI does not replace testing expertise; it amplifies the need for it. Organizations that treat AI as a shortcut may experience increased velocity without corresponding improvements in quality. In contrast, organizations that combine generative AI with disciplined testing knowledge can significantly enhance both efficiency and effectiveness.

The future of testing is not “AI vs. testers.” It is AI-augmented testers who possess deep professional skills and can strategically govern intelligent tooling. The competitive advantage will belong to teams that understand how to integrate AI into a robust quality engineering framework rather than simply automating artifact creation.

Author: Duane Karns, PhD., Computation Physics and Synthetic Data Lead, US Department of Homeland Security

The mission of the Transportation Security Laboratory in the Department of Homeland Security plays a critical role in national aviation transportation security by enhancing and validating solutions for detecting and mitigating concealed threats. This mission guides the development of detection technologies to effectively meet the requirements of government stakeholders in support of the homeland security enterprise.

Vendors of detection technologies are increasingly utilizing artificial intelligence models to replace traditional engineered automated target detection algorithms. These automated target recognition artificial intelligence models are steadily more complex and require substantial amounts of data for training, testing, and certification. Meeting these data demands with real data alone is becoming more challenging, leading to the use of synthetic data to supplement real data. The ability to create synthetic data in support of training, testing and certification is essential to the Test and Evaluation mission at the Transportation Security Laboratory, and the mandates outlined in the Aviation and Transportation Security Act.

In the current age of big data, high performance computational tools are commercially available and necessary for organizations to gain valuable insight into their data of interest. Specifically, in this case of Test and Evaluation data generated by the Transportation Security Laboratory. Powerful hardware, optimized for processing huge volumes of information as well as enabling the generation of synthetic data and multi-physics modeling, are essential to advancing the state-of-the-art of Governmental Test and Evaluation techniques, and is essential for the Transportation Security Laboratory to meet the needs of its stakeholders.

The development of synthetic data for both X-ray CT and AIT systems is a priority for the Transportation Security Laboratory to meet the challenges of developing and testing artificial intelligence based automated target recognition models. This presentation describes the current work at the Transportation Security Laboratory to develop synthetic data for both X-ray CT and AIT systems. Synthetic data generation techniques include full physics models combined with threat insertion techniques and dual energy decomposition augmentation. Verification and validation methodologies will be discussed, as well. These will include the use of autoencoders for synthetic data verification, and the probability of detection and false alarms for validation.

Author: Muhammad F. Islam, Ph.D. MITRE and George Washington University; Co-Authors: Tomi Esho, Ph.D. and Jyotirmay Gadewadikar, PhD. MITRE

Modern systems engineering efforts often have to deliver detailed test and validation outcomes under strict timelines with limited resources. This can result in insufficient testing efforts, release delays, production issues or even critical failures after launch. This research proposes an AI-enabled framework that automates requirements-based test case generation while keeping human test engineers in the decision loop. At first, a set of requirements is provided to a large language model (LLM)-based AI framework and to test engineering subject matter experts, each of whom generates test cases independently. The resulting test cases are then evaluated blindly using a common set of criteria. The test cases are compared for traceability to requirements, reproducibility, objectivity of expected results, precision of test data, and robustness under boundary and non-standard conditions. This framework enables a comprehensive assessment of how AI-generated tests perform relative to human-developed tests. The proposed framework aims to enable test engineers performing rapid test and validation activities, while also providing empirical evidence on where it is effective and where it can be improved. This research seeks to reduce manual testing efforts, enable rapid delivery, and minimize failed test efforts for mission-critical systems in defense, aerospace, and other high-assurance domains.

Author: Steve Seiden, Acquired Data Solutions Inc. .

The Department of War (DOW) is increasingly evaluating artificial intelligence as an enabling capability within complex weapon, vehicle, and sustainment systems. In the field of vehicle maintenance, most technicians must perform manual visual inspection to identify and diagnose instances of wear, corrosion, or damage. Manually identifying issues requires both time and experience on the part of the maintainer, and diagnostic errors or repair delays may affect equipment availability and readiness. Augmenting historically manual inspections with AI-based computer vision represents a means to enhance the accuracy, reliability, and repeatability of vehicle wear analysis and damage detection, while increasing maintainer productivity by reducing the time required to identify and diagnose issues.

This abstract presents a technical approach for incorporating AI-enabled perception systems into military Test & Evaluation (T&E) environments, using an AI-based computer vision application for ground vehicle tread wear analysis as an illustrative example (through a Cooperative Research and Development Agreement (CRADA) with Devcon at Picatinny Arsenal). While the system discussed focuses on track pad analysis, the same AI-based computer vision approach may be applied to a wide range of visual assessment driven maintenance processes, including powertrain wear analysis, corrosion detection, and damage assessment.

This presentation will discuss a repeatable framework for developing, integrating, and evaluating AI-enabled computer vision systems within military T&E pipelines. By treating AI capabilities as measurable, instrumented components of a test architecture, the approach supports defensible assessment of autonomy-adjacent technologies, AI-assisted maintenance, and future human-machine teaming concepts. This framework provides a pathway for transitioning AI from experimental capability to testable, certifiable military systems

Audience Benefit: Attendees will gain practical insight into how AI-enabled computer vision can be structured, measured, and validated within existing military T&E processes. The presentation equips T&E professionals, engineers, and program stakeholders with a defensible framework for evaluating AI-assisted maintenance technologies, reducing risk in AI adoption, improving test repeatability, and accelerating the transition of AI capabilities from prototype to operationally relevant systems that directly impact readiness and sustainment outcomes.

Author:  Karen O’Brien, Modern Technology Solutions, Inc.

The classic DoW T&E paradigm (Operational Effectiveness-Suitability-Survivability-Safety) benefits from 40 years of formalism and refinement and has produced numerous specialized testing disciplines (e.g., the -ilities), governing regulations, and rigorous analysis procedures. T&E of AI-enabled systems is still new, and the laboratory of ideas is a constant source of new testing options, the majority of which focus on performance. The classic gap between measures of performance and measures of effectiveness is very much present in AI-enabled systems and the over emphasis on performance testing means we might miss the (effectiveness) forest for the (performance) trees. Stepping back from confusion matrix performance metrics reveals a much larger landscape of evaluation issues that derive from various policy sources — issues that are at risk of being overlooked. Issues like safety, transparency, robustnessBorrowing from the classic “integrated survivability onion” conceptual model that made T&E within Systems of Systems tractable, we propose a set of integrated and nested evaluation questions for AI-enabled systems that covers the full range of classic T&E considerations, plus a few that are unique to AI technologies within the military operational environment and implied by the DoW Responsible AI Guidance. All requirements for rigorous analytical and statistical techniques are preserved and new opportunities to apply test science are identified. We hope to prompt an exchange of ideas that moves the community toward filling significant T&E capability gaps – especially the gaps between performance and effectiveness – and advancing test activities across the nested hierarchy.

Author: Emily Mills, Ph.D., Deputy Director RDT&E, Design Interactive, and Co-Authors Dillon Gilbert and Lane Odom, et.al, Intuitive Research and Technology Corporation

Modern test and evaluation (T&E) environments face mounting challenges as autonomous, AI-enabled systems increase in complexity and operational risk. Traditional test processes cannot provide the timely, actionable insights needed to validate system performance and assure mission readiness before deployment. Despite mandates for Department of War organizations to adopt Model-Based Systems Engineering (MBSE), practitioners struggle to transform MBSE artifacts into operational T&E value. Models often remain static documentation rather than living, authoritative sources that drive simulation, test planning, and risk-informed decision-making. This disconnect between model creation and downstream utility limits digital engineering impact and leaves T&E communities unable to validate expected behaviors or provide early performance assurance across operational contexts.A new digital pipeline is presented which demonstrates the art of the possible with respect to MBSE models, digital twin assets, and virtual range operations. The framework transforms MBSE artifacts into executable simulation assets, allowing modelers, planners, analysts, and safety engineers to validate system designs, verify requirements, and assure operational readiness before committing physical resources. By integrating model-driven scenario construction with virtual range environments, the ecosystem provides repeatable validation workflows that identify performance gaps, assess mission-critical vulnerabilities, and assure compliance with safety and operational constraints throughout the testing lifecycle.The framework comprises several integrated components: (1) automated MBSE model ingestion that translates standardized system representations into validated simulation assets, (2) scenario development and orchestration tools that support both MBSE-driven and human authored scenario definitions, (3) a data integration and analytics layer capable of synchronizing geospatial, environmental, sensor, and test relevant data, (4) a simulation integration layer (e.g., connecting to AFSIM, JSE, Ansys), and (5) a visualization environment for executing, analyzing, and reviewing simulated and real test outcomes. Embedded analytics provide interpretable assessments of risk, coverage, and cost, while establishing a foundation for advanced capabilities such as AI-enabled anomaly detection, behavior deviation scoring, optimization, explainability indicators, and verification of decision pathways.The technical implementation addresses critical operational considerations including data provenance for validation traceability, data schema alignment for consistent performance metrics, automated verification workflows, and enterprise integration patterns. Advanced capabilities will enable AI-driven anomaly detection for validating edge-case behaviors, deviation scoring to assure alignment with expected performance envelopes, and explainability indicators to build trust.This framework demonstrates how MBSE can evolve from documentation to become the authoritative foundation for scalable, continuous validation and performance assurance. By connecting digital engineering artifacts directly to executable test environments, organizations gain earlier verification of system behaviors, quantifiable assurance evidence, and reduced risk in operational deployment. The open architecture foundation supports integration of AI-enabled validation tools while maintaining human-centered assurance workflows, positioning T&E communities to confidently validate and assure increasingly complex autonomous systems before mission-critical deployment.