AI in Test & Evaluation Forum – Agenda
Subject to change
Poster Paper Authors will be available throughout the day.
Tuesday, March 17
8:00 AM – 9:00 AM
Registration Open
9:00 AM – 9:30 AM
Opening Ceremony – Virtual presentation by Princess Anne High School NJROTC, Hampton VA
Welcome and Opening Remarks – Michael Barton, Ph.D., ITEA Fellow and Chairman of ITEA and Erwin Sabile, CTEP, ITEA Fellow and Vice-Chairman of ITEA
NEW Introducing the Audience!
9:30 AM – 10:00 AM
Sandeep (Sandy) Patel, Ph.D., AI/ML Enterprise Manager and Deputy Program Manager for KBR’s DIA and DOT&E/TETRA Contract
10:00 AM – 11:00 AM
Panel Discussion – Artificial Intelligence: An Industry Perspective
Moderated by Bryan Vandrovec, Chief Technologist, Autonomous and AI Systems, Booz Allen Hamilton
This panel examines how a Digital Proving Ground overcomes the limitations of traditional physical testing for complex AI-enabled systems. Industry experts will discuss leveraging a generative AI-powered knowledge assistant for automated test planning, high-fidelity test range reconstruction, physics-based digital twins to generate synthetic data, and interpretable runtime guardrails to assess machine reasoning. Together, these capabilities accelerate evaluation processes, realize significant cost savings, and establish the calibrated trust required for modern autonomous systems development.
Panelists
Judy Brown Stoer, Autonomy Test Team Lead, Weather Gage Technologies
Johannes Waldstein, Founder & CEO, PiLogic Inc.
Policarpio Soberanis, Ph.D. Synopsys Inc.
Nelson Santini, Senior Vice President, Edge Case Defense
11:30 AM – 12:45 PM
Poster Paper Authors Available
Lunch & Networking
1:45 AM – 2:45 PM
Audience Participation – Practical Strategies for Design and Execution of Test and Evaluation of AI Enabled Systems (AIES)
Moderated by Sara Jordan, Institute for Defense Analyses (IDA)
2:45 PM – 3:00 PM
ITEA & INCOSE
2:00 PM – 2:30 PM
James Sharp, Ph.D., Defense Science and Technology Laboratory (Dstl), Ministry of Defense, UK
2:30 PM – 2:45 PM – Poster Paper Authors Available
Break
2:45 PM – 3:45 PM
Panel Discussion – Testing and Evaluation Strategies with and for AI in Complex Systems
Moderated by John Frederick, Director, Innovation and Testing Strategies, Veracity Engineering
This panel will explore how T&E and organizational cultures must adapt to assess complex, nondeterministic AI and ML enabled safety and security critical systems. Panelists will discuss technical challenges, system characteristics, and metrics for integrating and testing AI, focusing on how verification and validation (V&V) evidence builds decision confidence, supports certification, and ensures operational suitability. The panel will examine the design and validation of data ontologies, decision support models, and data governance as essential enablers of AI in complex environments. The role of digital engineering, including the integral relationship between digital twins and AI/ML capabilities, will be highlighted. Finally, panelists will explore how AI and ML methods can enhance the effectiveness, efficiency, and coverage of V&V.
Panelists
Ian Levitt, Ph.D., Distinguished Board Member, National Aerospace Research & Technology Park
Eman Kawas, Independent Advisor, Decision Assurance using AI-Enabled Digital Twins
Antonios Kontsos, Ph.D., Henry M. Rowan Foundation Professor, Director of the Digital Engineering Hub
3:45 PM – 4:14 PM
To Be Announced
4:15 PM – 4:45 PM
Kerianne Hobbs, Ph.D., Senior Engineering Specialist, Vehicle Autonomy & System Trust, The Aerospace Corporation
Abstract: The rapid evolution of AI-enabled autonomy is reshaping operations across multiple domains. This talk presents an emerging integrated framework combining guardrails, watchdogs, live-virtual-constructive (LVC) testing, human-autonomy teaming (HAT), traditional processor/hardware-in-the-loop (PIL/HIL) methods, and unique approaches to test case generation to accelerate the responsible deployment of AI-enabled autonomy without sacrificing safety. As autonomous systems undertake critical decision-making, it is essential to establish clear behavioral boundaries, manage risk with structured representative testing at scale, and integrate human oversight with machine decision-making.
4:45 PM – 5:45 PM – Light Refreshments – Meet Poster Paper Authors
T&E Collaboration Social Hour, “Where Innovation Meets Opportunity.”
Wednesday, March 18
8:00 AM – 9:00 AM
Registration Open
9:00 AM – 9:15 AM
Opening Remarks – Day 2 – Program Chair, Erwin Sabile
9:15 AM – 10:15 AM
Amy E. Henninger, Ph.D., Senior Science Advisor for Advanced Computing, Science and Technology Directorate
U.S. Department of Homeland Security “Adversarial and Counter AI: Why it Matters Now”
10:15 AM – 11:30 AM
Panel Discussion – Overview: T&E Transformation
Moderated by: Daria Stafford, Technical Director, Director Operational Test and Evaluation
Digital Engineering (DE), Artificial Intelligence (AI), and Acquisition Transformation necessitate a fundamental rethink of our traditional processes. Led by Daria Stafford, Technical Director for DOT&E, this panel brings together leaders to explore how Test and Evaluation is shifting from a late-stage “final gate” to a continuous engine of discovery and learning. The panel will cover how T&E can provide decision advantage, the ability to make faster, better-informed choices in a digitally competitive world. The panel will discuss moving T&E from a series of discrete events at the end of the “V” to a mission-engineering continuum. The panel will discuss the potential for leveraging Digital Engineering, MBSE, spanning LVC environments, moving towards an authoritative data environment that validates complex AI-enabled systems earlier in the lifecycle. However, while DE and MBSE offer a path toward more agile validation, the panel will also address the significant technical and cultural hurdles of creating truly useful digital test environments. This includes the difficult work of integrating authoritative data sources with operational test characteristics, such as representative users and realistic combat environments, to ensure digital models provide a high-fidelity reflection of the battlefield. The discussion will also tackle the unique challenges of AI-enabled systems. Panelists will share insights into the use of guardrails and Bayesian models to build confidence in AI performance across the acquisition lifecycle. Finally, the panel will discuss policy and guidance updates that align with these technology advances. Ultimately, this session challenges T&E professionals to refine their role in ensuring a lethal, effective, and AI-ready force through more integrated, data-driven outcomes.
Panelists
Laura Freeman, Ph.D., Deputy Director, Virginia Tech National Security Institute, Assistant Dean for Research, College of Science, and ITEA Fellow
Dr. Robert “Riddle” Houston, Director of T&E, Chief Digital and Artificial Intelligence Office (CDAO)
To Be Announced, Military Service Representative
11:30 AM – 12:00 PM
Networking – Poster Paper Authors
12:00 PM – 1:00 PM
Casual Lunch
1:00 PM – 1:30 PM
Jeremy Werner, Ph.D., Defense Tech Architect & Ambassador, Cadence Design Systems, Crossing the Valley of Death: Shifting Left using AI and Hardware-Accurate Digital Twins to Accelerate Acquisition
1:30 PM – 1:50 PM
NEW! Special Software Presentation
This presentation introduces the prototype AI-enabled Test & Evaluation Module (ATEM), a core component of Booz Allen’s Digital Proving Ground (DPG). ATEM is engineered to meet the challenges of testing complex, adaptive autonomous systems, including expansive state spaces and emergent behaviors. The session outlines ATEM’s generative AI-driven workflow, which draws on mission engineering documents, system under test (SUT) artifacts, and a curated T&E knowledge base to generate test scenarios, statistically rigorous run matrices, test strategy recommendations, and draft Test & Evaluation Master Plans (TEMPs). It also highlights ATEM’s integration with Shield AI’s Hivemind and Forge test service, enabling automated translation of experiment designs into machine-readable test scripts for high-fidelity virtual simulation and performance evaluation. Together, ATEM and the DPG provide a scalable, traceable, and repeatable pipeline for the continuous test and assurance of AI-enabled systems.
1:50 PM – 3:30 PM
Technical Track – Academic & Government Voices at the Forefront of AI T&E
Moderated by Erwin Sabile
Steve Robert Crews II, PhD., Georgia Tech Research Institute
A Digital Twin Maturity Model for Digital Engineering and Test of AI-Enabled Space Systems
Dr. Rachel Brower-Sinning, Carnegie Mellon Software Engineering Institute
Using MLTE to Support Integrated T&E for ML-Enabled Systems
Kelli Esser, PhD., Chief Strategy Officer, Virginia Tech National Security Institute (VTNSI)
A Mission-Centric Approach to AI T&E: Extending Mission Engineering for AI-Enabled Systems
Josef B. Schaff, DSc., Chief Scientist, Cyber Dominance Group (A4J) Non-Kinetic Warfare Branch, Johns Hopkins Applied Physics Lab
3:30 PM – 3:45 PM – Poster Paper Authors Available
Break
3:45 PM – 4:15 PM
Matt Maroofi, Senior Director of Product Development, Shield AI
4:15 PM – 4:45 PM
To Be Announced
4:45 PM – 5:00 PM
Closing Remarks – Erwin Sabile