December 2012 – Cultivating the T&E Workforce.

Test and evaluation are professions not academic disciplines and as such we can’t merely recruit more as needed. We recruit engineers, physicists, computer scientists, mathematicians, chemists and other degreed professionals and train them in test and evaluation. As technology changes and systems and instrumentation become more complex, test and evaluation (T&E) professionals need to continue formal education as well as improve T&E expertise. In addition, we need to consistently attract young people to the disciplines of science, technology, engineering, and mathematics. Cultivating the T&E workforce requires asking the question: what should the T&E professional’s background consist of today and what should it be tomorrow? We need to prepare the future workforce for T&E, and prepare T&E for them. This issue features articles from students in high school and our service academies, from faculty, from T&E leaders, and from our T&E professional society on certifying the T&E workforce.

Dr. Robert McGrath, Director of the Georgia Tech Research Institute, describes workforce challenges from an academic perspective in his Guest Editorial: encourage the current outstanding T&E force to stay as long as possible, recruit adequately prepared new hires from other parts of the government and industry, and hire new STEM graduates in the right academic mix to support T&E of today and the future. Dr. Catherine Warner, Science Advisor to the Director, Operational Test and Evaluation, uses Inside the Beltway to encourage early and continuous engagement between the testing and requirements community to better craft requirements that are mission oriented, realistic, and testable. In Historical Perspectives, Arnold Air Force Base Historian David Hiebert traces the 60 year history of what was originally the Air Engineering Development Center, one of the country’s premiere developmental testing complexes.

The articles begin with a special student section arising from the ITEA Annual Symposium Academia Day paper competition, held in Huntington Beach, CA. First place winner, Bradley Matheus, senior at California Academy of Mathematics and Science in Carson, CA, outlines the benefits of numerically controlled machines and 3-D printing for rapid prototyping. Second place winner, Zaki Molvi, senior at Troy High School in Fullerton, California, provides a future professional’s perspective on increasing the number of graduating scientists and engineers through a combination of STEM education, specialized courses and hands-on training during undergraduate years. Finally, third place winner Sara Pak, senior at Diamond Bar High School in Diamond Bar, CA, reinforces the necessity of test and evaluation for future product development, to ensure to consumers that new products are safe and suitable.

Robert Arnold, Chief Technologist of the 96th Test Wing, Eglin Air Force Base, Dr. Eileen Bjorkman, Chief Technologist of the 412th Test Wing, Edwards Air Force Base, and Dr. Edward Kraft, Chief Technologist of the Arnold Engineering Development Complex, Arnold Air Force Base, present the Air Force Test Center’s new human capital strategy which emphasizes investment in technical competence on par with test infrastructure improvements and sustainment. James Gaidry, ITEA Executive Director, rolls out the new ITEA Test and Evaluation Certification Credential and reviews how it came about and its benefits. Dr. Michael Kendra of the Air Force Office of Scientific Research explains workforce development in two components, bringing new science and engineering personnel into T&E, and advancing the professional development of the existing T&E workforce through training, coursework, certification, and advanced degree awards.

Tony Stout et al., speaking for the Joint Interoperability Test Command, encourage shifting focus from managing people and projects to measuring the gap between current skills and future needs and using this knowledge to drive training, hiring, work allocation, contracting and human resource management decisions. Thomas Simms, et al. describe the Deputy Assistant Secretary of Defense for Developmental Test and Evaluation roadmap for workforce development. Dr. Raymond Hill, Professor of Operations Research with the Air Force Institute of Technology, defends using specialty training, for example in statistics and design of experiments, as a supplement to formal undergraduate and graduate education, not a substitute for it.

Second Lieutenant Luke Grant presents his senior engineering project from the United States Military Academy where he used computational fluid dynamics and high performance computing to study the physics associated with fluid flow over gas turbine engine blades. Dr. Mehdi Ghoreyshi et al. from the United States Air Force Academy compare reduced order modeling with full field equations modeling of fighter aerodynamics and verify its accuracy while improving computational speeds by several orders of magnitude. Daniel Carlson and Erich Brownlow of the 412th Electronics Warfare Group, Edwards Air Force Base, apply Bayesian techniques to combine simulation data with limited flight test data to get a better understanding of aircraft performance. The issue concludes with an article by Dr. Paul Fortier, et al. of the University of Massachusetts-Dartmouth, who offer microsystems for test and evaluation applications requiring reduced power, weight and space, and for new capabilities that could not exist otherwise.

Finally, the Publications Committee thanks one of its own, Dr. Steven ‘‘Flash’’ Gordon, who adopted this issue and took a leadership role in defining, shaping and populating it. Thank you, Flash, excellent job.

SEPTEMBER 2012 – Drowning in Data, Thirsty for Information.

Dr. Ashton Carter coined the phrase, ‘‘Doing more without more,’’ while he was Under Secretary of Defense for Acquisition, Technology and Logistics in a June 2010 address on better buying power and better value in weapon system acquisition. It was repeated several times and finally published as DoD guidance in September 2010 and as an implementation directive in November 2010. It has become the norm in test and evaluation (T&E), not least due to shrinking budgets and accelerating technology development which put ever-increasing pressure on product development: shorter schedules, fewer personnel, less access to facilities. Extreme environments and complex systems add additional demands. To provide the requisite test capabilities, the T&E community must be agile and responsive as well as innovative. The automobile and aircraft industries have long spread liability and gained benefit from seeking standard parts production from independent manufacturers, or specific systems development parceled to risk sharing partners. Strategic partnering takes such forms as outsourcing, reuse, and collaboration; common test and training infrastructure; integrating developmental and operational testing; shared facilities among government, industry, and academia. Cooperation comes with its own issues: some loss of control, policy or statutory impediments, protection of proprietary rights, and conflicting goals of leadership. This issue addresses all forms of partnering, allowing testers to do more without having more to accomplish their task.

In his Guest Editorial Major General David Eichhorn, Commander of the Air Force Operational Test and Evaluation Center, Kirtland Air Force Base, encourages us to step back and think inside the box, to reconsider what we have and how we can incorporate it in new ways to maximize our output, the epitome of doing more without more. In Historical Perspectives, ITEA Historian Dr. James Welshans provides an example from Public Health to illustrate the strength and benefit to the country of strategic partnering.

The contributed articles open with Herdlick et al. who describe the development of systems of systems test capability through use of virtual environments, and operator in the loop simulation of the system of systems. They address the critical element of building confidence in the simulation (verification and validation) through federation pedigree and operator experience. Dr. Paul Fortier and colleagues examine microsystems, their background, development, and emergence in weapon systems. They are developing an approach for assessing the future of microsystems technology and its value for the test and evaluation community.

In an article from international collaboration, McKee and Tutty take a comprehensive look at the role of testing and how it needs to adapt to accommodate joint and coalition testing of networked systems of systems, and how experimentation, testing and training need to be integrated more closely and need to share capabilities. Holmes et al. present sensors developed for measuring blast effects to support the Army’s Rapid Equipping Force. They also describe an impact test method to simulate the ballistic impact that can occur in combat and use this test method for assessing the sensors. Finally, Remund and Kitto continue the theme of increased scientific rigor in testing at the Air Force Flight Test Center. They use the analogy of court proceedings to illustrate the application of power and confidence for improving efficiency and reducing the cost of test and evaluation.

JUNE 2012 – Drowning in Data, Thirsty for Information

Digital technology and Moore’s law provide us with the ability to acquire, create, and store data at unprecedented rates and volumes. Literature searches that would have taken days or weeks in the age of library card catalogs can be accomplished in seconds over the internet. Data, or more generally information, has become big business in addition to being the business of test and evaluation. Yet technological solutions also come with their own problems and the morass of data has not led to a commensurate growth of knowledge or ability to exploit the data. This issue examines the plethora of data, which is growing exponentially, and the ever critical demand to extract meaning and value.

Eileen Bjorkman, Technical Adviser at the Air Force Flight Test Center (AFFTC), Edwards Air Force Base, echoes the consistent AFFTC theme of scientific rigor in test and evaluation in her Guest Editorial. She tempers the message with one important proviso, that ‘‘…scientific rigor in testing should be driven by the science of the system under test and the test environment, not the statistics used to help plan the test and analyze the data.’’ In Historical Perspectives Dr. James McCracken, Founder, Chief Scientist and Vice President of The Design Knowledge Company, presents a vivid life or death example of the solution hidden in amaze of data.

Dr. Lauren Mailman et al. outline the importance of and a scalable and extensible approach for automating steps in the test data process. They also outline requirements for the underlying data management system. Dae Hong et al. report progress by the Department of Defense Target Control Steering Group in achieving the goal of ‘‘… the operation of any target on any test range …’’ Interoperability is being approached through a collection of standardized interfaces and results from two demonstrations are presented. Jonathan Nolan et al. describe best practices developed for the Terminal High Altitude Area Defense (THAAD) missile defense test and lessons learned from the initial operational test and  evaluation of a system that produced 15 terabytes of data.

Frank Alvidrez explains the growth of test complexity resulting from networked weapon systems and an approach using model based systems engineering and related tools to systematize and simplify testing. M. R. Srikrishnan and Dr. Ramesh Chandran Nayar, Anna University Chennai  in Madras, India describe fundamental examinations of structural characteristics of yarns to understand the ultimate performance of military products (uniforms, body armor, helmets, etc.) made from them. Of particular note is that the cumulative effect of strain in processing the yarn impacts product quality. Finally, Andrew Brock of The Defence Science and Technology Organisation, Edinburgh, South Australia notes the absence of experience in operational testing of service oriented architectures and emphasizes the importance of incorporating the requirements of all end users.

The topic of data is vast, encompassing acquisition, storage, archiving, access, validation, exploitation, and visualization; data as a service; cloud computing; metadata syntax and semantics; data and sensor fusion; data preservation; distributed and non-relational databases, among other topics. The purpose of test and evaluation is risk reduction and to support decision making, both accomplished with data, the core of our business. We read about the Department of Defense priority in Data to Decisions, about the need to address Big Data, and we see bold evidence of the commercial value of data manifested in Google, Yahoo and elsewhere. As the Publications Committee prepared this issue, however, we became painfully aware of how little the importance of data in our business is reflected in investments to improve the tools, processes, and interoperability among test organizations and even within them.

MARCH 2012 – T&E at the Speed of Need

Dr. Steven Hutchison, Department of Defense Principal Deputy, Developmental Test and Evaluation, conceived the phrase “T&E at the Speed of Need” when he was Test and Evaluation (T&E) Executive at the Defense Information Systems Agency (DISA). The speed of need is the time between definition of a user need and initial operation of the capability. The demands of war have shortened the timeline on requirements for military systems. Rapid acquisition and rapid fielding initiatives arising from urgent operational needs have created an entire industry in the defense community. Commensurately hastened is the pace at which transportation security and border protection measures need to be deployed. We see information technology (IT), especially software, change with a frequency of months not years. T&E must be responsive to the acquisition timelines. The Federal Aviation Administration, Border Patrol, law enforcement and many other organizations have adapted to the changing speed of need. This issue takes a candid look at agile software development processes, defense IT acquisition reform, rapid acquisition and fielding, reconfigurable test capability, testing on demand, reuse and other ideas for streamlining the T&E process in support of accelerating deployment of new products, services and capabilities.

Brian Simmons, Executive Technical Director and Deputy to the Commander, US Army Test and Evaluation Command, uses the Guest Editorial to illustrate how T&E assists rapid delivery to ourwarfighters, and the inroads made that allow testers to review in-theater  erformance once initial deployment has occurred. Mr. Simmons then issues the imperative that we need to be prepared to take rapid acquisition lessons learned to our programs of record, because going faster is a reality for T&E. Dr. James Welshans, ITEA Historian and chair of the ITEA History Committee, documents in Historical Perspectives the speed of need epitomized in Britain’s response to the German V-1 attacks, defending  against an entirely new kind of weapon.

Army Major General Heidi Brown, Director for Test for the Missile Defense Agency (MDA), uses the platform of an Invited Article to describe  the rate of ballistic missile threat development and proliferation and evolving threats, and the MDA response in the form of a test as we fight approach to developing and fielding the ballistic missile defense system. Randy Herrin and Neil Barrett of the DISA Joint Interoperability Test Command present the DISA solution to speed of need, embodied in implementing agile software development; attracting and retaining staff; and fostering collaboration among all major DoD T&E organizations. Agile is intrinsically collaborative, involving all stakeholders in its  processes. Jana Gallatin, also of the Joint Interoperability Test Command, presents agile processes to help integrate vendor and government testing.

Charles Wentz shares lessons learned from joint urgent operational needs programs and recommendations for improved integration and testing. Alan Jenkins of the Naval Air Systems Command advocates for a closer relationship between program management and test and evaluation teams for the benefit of both. Dr. John Colombi of the Air Force Institute of Technology and Captain Christopher Cobb of the Air Force Distributed Missions Operations Center review the current state of practice in the live-virtual-constructive test and training community. Dr. Haydee Cuevas et al. make the case for a multidisciplinary approach in the test and evaluation of operator performance of military systems. The multidisciplinary approach offers an effective and timely means for integrating technological innovations, addressing complexity and advancing the understanding of and improving human performance. Daniel Carlson and P. Travis Millet of the Air Force 412th Test Engineering Group at Edwards Air Force Base present the problem of efficient use of test resources as manifested through optimal planning of test points in a flight test program. They propose using a genetic algorithm and treating the problem as a constrained optimization, rapidly surveying a large parameter space and and producing a near-optimum ordered set of test points.