Members have access to all the archived issues of the ITEA Journal of Test and Evaluation as a benefit of membership. As a non-member, you have the opportunity to access the archived abstracts (below). In 2023, we implemented a few changes in the format and access to the Journal and welcome you to read the most recent issue. We hope you will join your colleagues and peers and become an ITEA member.
We invite you to read the archived journals from 2023.
To close out 2022 we have two new additions for our Workforce of the Future Column. The first article comes from a Navreeta Singh, Princeton University, who interned with Dr. Jeremy Werner at Director Operational Test and Evaluation (DOT&E). In their article, “Uncertainty Analysis Demonstration: A Missile Case Study” they highlight that validation is a key component of modeling and simulation (M&S) and role uncertainty quantification plays in the validation using sample missile test. The second student article comes from a team at the Virginia Tech National Security Institute. In her internship Cassidy Honea, Virginia Tech work with researchers at VTNSI to develop the article, “Synthetic Aperture Radar (SAR): Utilizing Radon Image Transformations to Enhance the Detection of Ship Wakes.” Mr. Erik Higgins, Mr. Daniel Sobien, and Dr. Justin Kauffman worked with Ms. Honea to conduct the research where they show using simulated SAR that Radon Transformation improves the probability of ship wake detection.
The first technical article on “Chaos Engineering” is authored by Ms. Jenn Bergstrom, Fellow at Parsons.
In the second technical article Mr. Michael Leite, Test Area Manager, R&D Programs at Department of Homeland Security, Science and Technology Directorate discusses, “Testing of Research and Development Projects.”
Our final technical article comes from Dr. William “Data” Bryant at Modern Technology Solutions, Inc. and discusses, “Measuring the Measurers: Using Test to Validate Cyber Risk Assessments.”
Enjoy the Issue!
This month’s issue starts off with another great Conversations with Experts column in which J. Michael Barton, Ph.D., interviews Malcolm Tutty, Ph.D. Dr. Tutty has served in the Royal Australian Air Force, Public Service and Industry in a multitude of test, operations, engineering, staff, project management and command roles. Dr. Tutty shares his experiences in and out of T&E. He also calls back to numerous quotes that I am sure will keep ITEA readers engaged. A quick line of advice that he slips in, “Enjoy your service, make friends everywhere, and I hope you have successful careers in and out of the T&E space along with having as much fun as I have had with many of y’all.”
In our Workforce of the Future column this month we have contributions from undergraduate and graduate student-led research at the University of Memphis. The first article, “Testing the Control Software in Autonomous Cyber-Physical Systems,” by Bhavana Garimilla and Mark L. Gillenson, Ph.D., examines how to test software in autonomous cyber-physical systems. The authors highlight the need for ensuring safety and the critical role of testing software extensively when it incorporates neural networks. In the next article, “Managing the Security of Data in the Cloud,” Venkata Akshara Motati and Mark L. Gillenson, Ph.D., discuss the security of cloud-based data storage. They provide recommendations that merit consideration before moving data to the cloud. In our third article, “Simulation and Testing of AI/Autonomous Systems,” Faizan Zafar and Eddie Jacobs, Ph.D., report on state-of-the-art practices and conduct simulations using the Mississippi State University’s Autonomous Vehicle Simulation (MAVS) library.
Our first technical article comes from Nick Jones and Wayne Adams at the Scientific Test and Analysis Techniques Center of Excellence (STAT COE), Air Force Institute of Technology. Their article entitled, “Model Selection and Use of Empiricism in Digital Engineering,” addresses the important role that modeling plays in test and evaluation. The article outlines a process based in STAT by which a digital engineering test program should leverage the digital engineering (DE) environment by first utilizing existing resources on the digital thread and then first-principles and empirical data to build new models when needed.
In our next article Donald H. Costello III, Ph.D., from the United States Naval Academy and Huan Xu, Ph.D., from the University of Maryland discuss “Using a Run Time Assurance Approach for Certifying Autonomy Within Naval Aviation.” The article illustrates how to use run time assurance as the framework for a safety of flight certification of autonomous behavior. They leverage an aerial refueling task as a use case for the framework for certification.
In our final article a research team from Virginia Tech including Laura Freeman, Ph.D.; Justin Kauffman, Ph.D.; Erin Lanus, Ph.D.; Daniel Sobien; and Tyler Cody, Ph.D., summarizes some best practices they have adopted in testing Artificial Intelligence Enabled Systems (AIES). As the lead author on the article entitled, “Best Practices for Addressing New Challenges in Testing and Evaluating Artificial Intelligence Enabled Systems,” I would like to highlight that these practices might prove useful to others as they embark on developing test programs for AIES, but they are surely not complete and will need to be updated as our understanding of testing AIES evolves.
Enjoy the Issue!
In this month’s Conversations with Experts column J. Michael Barton, Ph.D., interviews Eileen Bjorkman, Ph.D., Executive Director at the Air Force Test Center. In the article, “Lofty Goals, A Celebration of Flying,” Dr. Bjorkman discusses her career in the US Air Force, the decision to pursue a Ph.D., and her career in Test and Evaluation (T&E). In her advice to people interested in T&E careers, Dr. Bjorkman recommends that individuals “try out different types of jobs early on and don’t be afraid to try something outside your area of expertise.” She highlights how technologies change and to think of your academic degree as just a starting point.
In our Workforce of the Future column with three articles written by students that participated in the High Performance Computing Modernization Program Internship Program:
In our first technical article Alexander Hillman, a major in the US Air Force and a Ph.D. candidate in aeronautical and astronautical engineering at the Massachusetts Institute of Technology, discusses how testers are uniquely equipped to lead new data strategy efforts. Hillman’s article, “Leveraging a Tester’s Experience to Build a Successful Enterprise Data Strategy,” describes the years of experience, knowledge, and the challenges that testers bring to the data strategy discussion. He outlines the critical elements of data strategies and provides astute observations on how Department of Defense processes and protections can inhibit progress in achieving strategic objectives.
In our next technical article Everett H. Oliver, Ph.D., and Shahram Sarkani, Ph.D., P.E., and Thomas A. Mazzuchi, D.Sc., discuss the value of systems thinking and model-based systems engineering in assessing the resilience of critical infrastructure sectors. In their article, “Assessing the Resilience of Critical Infrastructure Systems and Supply Chains Against Cascading Disruptions,” they provide a case study to demonstrate the approach leveraging the historical example of Superstorm Sandy and the food supply chain.
In our final technical article Frank Cruz from the US Air Force, Applied Spectrum Technology Research Office (ASTRO), at Edwards AFB discusses, “Optimizing Antenna Placement Using Modeling and Simulation.” Mr. Cruz’s paper showcases how modeling and simulation can be used to enable trade-space exploration in order to optimize hardware design. He highlights the potential cost savings when compared to the cost of chamber testing.
Enjoy the issue!