18th Engineering Workshop Tutorials (2018)

 T&E in Support of Prototyping and Experimentation

January 16 - Pre-Workshop Tutorials
January 17-18 - Workshop Plenary and Technical Track Sessions
Sheraton Uptown ~ Albuquerque, NM

Hosted by the ITEA Roadrunner, White Sands, Huachuca, and Valley of the Sun Chapters

 The 18th Engineering Workshop will be co-located with the Directed Energy (DE) T&E Conference,
which is sponsored by the Directed Energy Professional Society (DEPS) - Information link below.

2018 DEPS Directed Energy T&E Conference

Some portions of the DE T&E Conference will be available to any Conference registrant,
others will require that attendees be employees of the U.S. Department of Defense or its contractors.

Please go here for more information.


 NOTE: Tutorials require a separate fee from the Symposium.

  • Single 4-hour Tutorial - $205
  • Two 4-hour Tutorials - $385 (use discount code "TWO-Tutorials" at check out)

Register for ITEA Tutorials HERE

Register for DEPS tutorials - DEPS Tutorial registration



Critical Thinking in Test Design and Analysis - ITEA

Instructor: Mark Kiemele, PhD - Air Academy Associates
Course Length: 4 hours/ Open, Public Release, Distribution Limitation A

This tutorial will cover the need for critical thinking as well as a high-level view of a variety of data analytic tools that can be used to enhance critical thinking. In a data-driven economy, industry and government leaders rely increasingly on skilled professionals who can see the significance in data and use data analytic techniques to properly collect data, solve problems, create new opportunities, and shape change.  Data science can be defined as the art and science of solving problems and shaping decisions through the precise collection and analysis of data.  This tutorial is intended for executives, leaders, managers, and practitioners who need to know how their critical thinking can be impacted by such things as Big Data, Predictive Analytics, Design of Experiments (DOE) and other tools in the Data Science toolkit.   Even if one never designs a test or evaluates its results, this tutorial participant will be able to explain the uniqueness of DOE and why big data and predictive analytics are needed to generate the analytical capability every organization needs.

TENA and JMETC for Distributed Systems Engineering - ITEA

Instructor: Gene Hudgins - KBRWyle, TENA / JMETC
Course Length: 4 hours/ Open, Public Release, Distribution Limitation A

Together, TENA and JMETC enable interoperability among ranges, facilities, and simulations in a timely and cost-efficient manner. TENA provides for real-time system interoperability, as well as interfacing existing range assets, C4ISR systems, and simulations; fostering reuse of range assets and future software systems. JMETC is a distributed, LVC capability which uses a hybrid network architecture; the JMETC Secret Network (JSN), based on the SDREN, is used for secret testing and the JMETC Multiple Independent Levels of Security (MILS) Network (JMN) is the T&E enterprise network solution for all classifications and cyber testing. JMETC provides readily available connectivity to the Services' distributed test and training capabilities and simulations, as well as industry resources. This tutorial will address the current impact of TENA and JMETC on the Test and Training community; as well as their expected future benefits to the range community and the Warfighter.

T&E in Support of Prototyping and Experimentation - ITEA

Instructor: Albert A. Sciarretta - CNS Technologies, Inc.
Course Length: 4 hours/ Open, Public Release, Distribution Limitation A

As one of the principal developers of a prototyping and experimentation course at the National Defense University (NDU), this tutorial address definitions of prototypes and experiments, a design of experiment approach for designing and executing an experiment, and a “case study” that addresses designing an experiment for counter-small unmanned aircraft systems (i.e., counter systems for weaponized hobby drones).

  1. A 45-minute block of instruction that includes: 1) discussion about the importance of prototyping and experimentation; 2) definitions for conceptual, developmental, and operational prototypes; and 3) definitions for live, virtual, and constructive experiments
  2. A 3-hour and 15 minute block of instruction covering a design of experiments (DOE) approach for designing an experiment.  This instruction will walk through a process that: 1) identifies the military capability/problem that requires experimentation, 2) using the capability/problem, defines the primary objective and optional secondary objectives for the experiment; 3) given the objective and sub-objectives (if defined), generates “issues” that need to be addressed; 4) for each individual issue, identifies hypotheses that need to be proved/disproved; 5) for each hypothesis, identifies metrics that are need to prove/disprove each hypothesis; 6) for each metric, identify the data that needs to be collected for each metric; 7) given the data needs; develops the scenario/vignettes that will maximize the generation of the needed data; 8) again given the data needs, identifies approaches (both instrumentation and non-instrumentation) for collecting the data.  [Note: Given the brevity of the course, the identification of issues, hypotheses, metrics, and data needs will not be all inclusive; examples of each will be used to generate student discussion.].  Finally, at the end of this block of instruction, there will be a short discussion of what should be in an experiment plan as well as a short discussion of some considerations for executing the experiment plan.

Introduction to the DoD Test & Evaluation Process - DEPS

Instructor: Sean Ross, U.S. Air Force
Course Length: 4 hours/ Open, Public Release, Distribution Limitation A

This course discusses the fundamentals of testing DoD systems, with an emphasis on directed energy systems. Topics include:

  • Overview of Defense Acquisition System
  • Where do we start? - Good requirements
  • The role of the Systems Engineering process
  • Test Planning - when and what do we do it?
    • Test Development Strategy
    • Test & Evaluation Master Plan
    • Role of Test Lead manager
  • Types of testing/when started/finished
    • Developmental Testing
    • Initial Operational T&E
    • Live Fire T&E
  • Test execution and reporting
  • Directed energy testing issues
    • Instrumentation
    • Safety
    • Training
    • Best practices and lessons learned
    • Some important test parameters of laser systems

Intended Audience: While the primary audience for the course remains DoD staff, DEPS shares the course broadly, not as a blueprint, but in a spirit of collegiality and an interest in contributing to others' efforts and continuing our collective dialogue about Directed Energy. As a basic introduction to T&E, it is suitable for personnel in other technical acquisition management and program management positions who want to understand more about T&E and the critical role it plays in DE system acquisition.


Cyber Operational Vulnerability Assessments; Or How I Learned to Stop Worrying and Love the Six-Phase Cybersecurity T&E Process


Many Service acquisition and Test and Evaluation (T&E) programs find it difficult and confusing to negotiate the policies and processes in order to develop their requirements and strategy for cybersecurity T&E. The Cyber Operational Vulnerability Assessment (COVA) is one way to identify credible vulnerabilities and develop actionable requirements that can be used to design efficient T&E. 
The COVA was developed and implemented by Electronic Warfare Associates Inc, (EWA) as a joint effort by industry partners within the Cybersecurity community and is a structured, mission-based approach to analyzing the risk of cyber threat vulnerabilities. It is a scalable and interactive event designed using the foundation of the highly successful Cyber Table Top Wargame, also spearheaded by EWA.  The COVA is also intended as an aid in developing and understanding cybersecurity requirements, as well as identifying vulnerabilities and adversarial capabilities. This information can then be used to develop and/or refine a strategy and plan for cybersecurity T&E. In short, this tutorial will introduce the COVA as a low-cost and proven method to start planning for, or to update, a strategy for cybersecurity T&E.  This tutorial will present attendees with an detailed overview of the COVA process and methodologies, as well as highlight successful uses.

Who should attend this tutorial:   It is intended for use by Acquisition Program Management Offices, Chief Developmental Testers, and also Operational Test and Evaluation (OT&E) organizations and programs tasked with developing strategies for and executing cybersecurity testing for DoD ACAT programs.  But anyone in the Acquisition or T&E who is tasked with planning or executing Cybersecurity T&E will benefit.

T&E of HPM Systems - DEPS

Instructor: Jeff Schleher, American Systems
Course Length: 4 hours/ Unclassified, Limited Distribution C

The short course is an introduction to the testing of high power radio frequency weapons. The course contains a short introduction to the fundamentals of these high-power microwave (HPM) weapons to establish a common vocabulary. The focus of the course is on test practices as they apply to HPM weapons for U.S. system testing and susceptibility testing as specified in MIL STD 464C. HPM instrumentation, test measures, and analysis make up a section of the course. Various presentations address military applications of HPM weapons and how fielded systems have been tested. Substantial new HPM test equipment has been recently made available to military test ranges and developmental laboratories. The course will address the availability, location, and procedures for these test capabilities as well as the process used to determine what test equipment was needed. A portion of the course will address the specialized requirements of these systems for safety considerations and permitting required for testing to proceed.

Topics include:

  • HPM System Fundamentals
  • Testing Fundamentals as Applied to HPM Systems
  • Test and Evaluation HPM Environments
  • Sources for HPM Testing and Where to Find Them
  • HPM Test Locations
  • HPM Instrumentation and Setup
  • HPM Safety and Spectrum Management

Intended Audience: This course is appropriate for managers and analysts requiring an overview of HPM weapons and how they are tested in addition to new HPM field test engineers. A background in Science and Engineering at least to the technician level is required to understand the technical portion of the course.

 T & E of HEL Systems - DEPS

Instructor: Mr. Larry McKee, American Systems
Course Length: 4 hours/ Unclassified, Limited Distribution C

An introduction to fundamental considerations for the test and evaluation (T&E) of High Energy Laser (HEL) weapon systems. Students will be given an overview of the various distinct types of HEL testing, including example test concepts/configurations, considerations for test instrumentation, and key testing issues, such as safety and environmental concerns. Topics include:

  • HEL employment/testing challenges
  • Types of HEL testing
  • Lethality phenomenology testing
    • HEL effects
    • HEL lethality testing types and test diagnostics
  • System output testing
  • System performance testing
    • HEL examples of static ground testing, dynamic OT, OT/Live
    • HEL test measures
  • Instrumentation Considerations
    • HEL instrumentation summary
    • Instrumentation protection
    • Non-intrusive measurements
  • HEL testing considerations
    • HEL testing issues
    • HEL testing safety
    • Test planning tools

Intended Audience: T&E engineers who may be responsible for planning, supporting, and/or executing range tests that involve HEL weapon systems.

International Test and Evaluation Association (ITEA) | E-mail: info@itea.org | www.itea.org © 2015 | All Rights Reserved.
4400 Fair Lakes Court, Suite 104, Fairfax, VA 22033-3899 | Phone: 703-631-6220 | Fax: 703-631-6221
See ITEA on Facebook Visit ITEA on LinkIn Web Graphic Design of Warrenton, VA - Hosting and Construction by Moe Technologies (MoeTec)