DECEMBER 2023 I Volume 44, Issue 4
Workforce Development in Test and Evaluation
DECEMBER 2023 I Volume 44, Issue 4
Volume 44 I Issue 4
The field of Test and Evaluation (T&E) falls behind in efforts to recruit personnel into the profession. Little has been done to draw recent graduates and students towards careers in T&E. If the trend of narrow recruitment is allowed to continue, the T&E community’s effectiveness at advancing testing and evaluation methods will be jeopardized. Potential ramifications are technical performance risks not being adequately assessed. Several courses of action about T&E workforce development have been proposed, such as creating a T&E specific degree program along with academic resources for universities, furthering scholarship opportunities for students pursuing T&E, and making room for internship positions at T&E organizations, but these initiatives often lack funding and other resources to be effective. Resources must be intentionally devoted to raising awareness of the T&E field and the vast opportunities within it, so the community can grow to be better able to innovate and share ideas about the advancement of T&E.
Keywords: Bias, capabilities, innovation, recruiting, Test and Evaluation, workforce development.
Test and Evaluation (T&E) is essential to all technological development, without it the systems of engineering process is incomplete. To create functional systems, emerging technologies must go through rigorous and specific testing procedures to ensure that the system performs within the requested capabilities suitable for investment. This test and evaluation require professionals trained to design, conduct, and analyze experiments on various technological systems. Having experienced and specialized T&E personnel is integral to any engineering effort.
Diversity of ideas is critical for innovation to take place in the Science, Technology, Engineering, and Mathematics (STEM) field and fostering this diversity improves the ability to innovate. Current demographics show trends not aligned with the potential workforce population. Demographic data pulled from the general population of federal employees shows that the average age of a government employee is 47.5 in 2017 (U.S. Office of Personnel Management 2017). This average already diverges from industry age demographics in the private sector, but T&E fields show even higher age averages. This mean age further increases when narrowed to the T&E field in DHS. This discrepancy presents a concern about the livelihood of the T&E field and its workforce as a gap in the workforce is bound to catch up to the field as experienced professionals will retire down the line. (Dumay & Rooney, 2011) While the DHS has initiatives for connected culture and data-driven diversity as outlined in their FY2018 DHS Inclusive Diversity Annual Report, the organization is not finished yet. Department-wide rollout of the Inclusive Diversity Dialogues Program and the Veterans/Women Veterans Recruitment Strategy are two ways diversity is promoted in the organization, but the field of Test and Evaluation still lags (FY2018 DHS Inclusive Diversity annual report – homeland security 2018).
Such a disparity is rooted in the lack of T&E awareness. Academia neglects to highlight T&E as a career option, so students lack the awareness and training to enter the workforce. To encourage the development of the T&E workforce towards newer generations, early outreach is essential to recruit and retain. Changing the landscape of T&E recruiting requires change in the relationships between the T&E professional workforce and universities and their student population.
Many professionals in the field of T&E fell into their careers. After conversations with T&E professionals in both DHS and DoD, it becomes clear that T&E is picked up by those who happen to stumble into the opportunity rather than an intentional career path. Oftentimes, individuals learn after the fact what T&E practices are. Very few students intend to enter the T&E field immediately following graduation, rather, as testing opportunities come up, they are accepted, and the candidate may have to learn after the fact what T&E practices are.
Without a T&E degree plan, hiring in T&E becomes more about skills and previous experience, rather than the formal education and depending on the role and organization, the desired background for a T&E professional varies. DHS’ T&E workforce often comes from the DoD. The federal government has an initiative to recruit and retain veterans outside of more than just their military careers. There are two paths veterans can take to get these federal jobs. Firstly, the Veterans’ Recruitment Appointment (VRA) allows DHS and other agencies to appoint eligible veterans without competition. (FedsHireVets n.d.)Secondly is a veteran’s preference in the normal hiring process. This is a points-based system that is dependent on active-duty service, receipt of a campaign badge, receipt of a Purple Heart, or a service-connected disability. (FedsHireVets n.d.) 30% or More Disabled Veteran authority and Veterans Employment Opportunities Act of 1998 (VEOA) are two additional policies that DHS specifically outlines on their website. While DoD provides a highly skilled pool of professionals, a concern of diversity of backgrounds becomes critical. When decision-making sectors of government are filled by similarly experienced individuals, biases are more likely to affect the outcomes of government efforts.
The Oak Ridge Institute for Science and Education (ORISE) sponsors the Homeland Security Professional Opportunities for student Workforce to Experience Research (HS POWER) program to introduce undergraduate and graduate students to the research areas and mission of DHS. The various components of DHS can differ greatly in their approach to homeland security, so HS POWER interns have opportunities to see many different sides of research. The Office of Test and Evaluation within DHS’ Science and Technology Directorate opened two intern positions for the summer (ORISE n.d.).
Coming into this internship a question that hung in the air was “what is T&E?” To a student their experience may be simply limited to preplanned experiments in a chemistry or physics lab, but what about collecting the data and the design of the experiment itself? Our experience with T&E was like many professionals who enter the field, minimal and without intention. Within the standard engineering curriculum at both of our universities, a test and evaluation module or course is not included nor an option. Many students lack exposure and experience in the T&E field prior to graduation, but there are ways students can be involved in T&E before entering the workforce.
So, what early outreach is being done for the T&E community?
The International Test and Evaluation Association (ITEA) is a prime example of how academia and the T&E field can work together. (International Test and Evaluation n.d.) ITEA drives outreach efforts in several ways and has many different initiatives taking place. They publish the “Workforce of the Future” column in issues of the ITEA journal devoted to showcasing the voices of students participating in the T&E community. This creates opportunities for students to get involved with the organization on a deeper level. ITEA offers scholarship monies to chapters that go to students who work on T&E related topics in university. ITEA also sponsors and hosts events such as panels or expos to engage faculty and students. Partnering with DoD organizations being one of them alongside the DoD funding scholarships. ITEA recognizes a need for a dedicated scholarship program but has yet to be accomplished as ITEA funding is based entirely off member dues. With low dues, funding is limited.
The Systems Testing Excellence Program is a research initiative directed by Dr. Mark Gillenson at the University of Memphis under FedEx Institute of Technology with support from the Fogelman College of Business and Economics. This program is tasked with building up research and curricular competencies to position the university as a national and internationally recognized leader in the science of systems testing. This program has a focus on testing research, a world class testing training and education center, and is creating a hub for testing by providing an international forum for learning and disseminating testing knowledge. (The University of Memphis n.d.)
Educational opportunities are available for those interested in entering T&E. The DHS offers a certification program that works for DHS specifically, but also applies outside of the organization in basic principles, processes, etc. These certifications are offered through the Office of Acquisition Workforce. Defense Acquisition University (DAU) offers a certificate in T&E as well (Defense Acquisition University n.d.). Georgia Tech also offers a professional certification program for Test and Evaluation (Georgia Tech Professional Education n.d.). These programs provide opportunities to learn practical “real world” skills to be used within the field. It is available to complete at any point within an individual’s career but is heavily encouraged as education advancement. While certification programs provide extra support to professionals within the field, they do little to draw attention from those who aren’t already aware of what T&E has to offer. Virginia Tech is in the works of implementing a Defense Technology Master’s degree with this certificate in T&E being a requirement. Despite this, no formal degree plans are offered for the field of Test and Evaluation at any university.
Virginia Tech and the University of Memphis seem to be anomalies in the sense that they are two of the very few universities with a high interest in T&E supported by an organization or initiative. They both just so happened to also be the universities we are studying at.
Understanding how DHS and DoD recruit for their T&E, can provide context on what areas can be improved to expand outreach for the industry.
The primary applicant is typically recruited via Indeed but are redirected to the DoD T&E websites where postings are publicly made. Indeed works well for contractors, but the federal government must use USA jobs. The process is very similar to any hiring process but will have a few extra steps such as the ability to obtain security clearances and background checks (Understanding the federal hiring process 2023). Recruitment through LinkedIn and local universities (local to the testing centers themselves) are also key hubs for new hires as recruitment officers can make direct contact with those interested.
The DoD has internship opportunities starting at the high school level. Defense Interns being one of these programs, where interns can rotate company to company every three months to be exposed to the variety of STEM work under the DoD. Events such as air shows, county fairs, and job fairs are also ways for the DoD to recruit. Being able to connect with the public and showcase new and /\upcoming technology gets people interested in areas that they may not know of. Events like Modern Day Marine, Figure 2, and the Association of the United States Army Annual Meeting and Exposition, Figure 1, is a great example of how the DoD connects.
Figure 1: The Polaris Industries Defense exhibit booth displays helicopter-mobile combat vehicles used by U.S. Special Operations during the Association of the United States Army annual meeting and exposition at the Washington Convention Center October 15, 2014
Figure 2: A man demonstrates the use of a robot by Roboteam during the Association of the United States Army (AUSA) Annual Meeting and Exposition in Washington, DC on October 14, 2014
A general way to get into federal work would be the Pathways Program. The Pathways Program is comprised of three civil service programs that offer federal internships and employment opportunities open for high school students, college students, and recent graduates (Pathways programs – careers 2023).
The Office of Academic Engagement (OAE) builds and sustains relationships with the academic community to support the DHS mission (U.S. Department of Homeland Security 2022). OAE’s “Come to the U.S. DHS” diversity initiative to create a Memorandums of Understanding (MOU) to pursue partnerships to fill diversity gaps in federal employment across faiths, races, genders, ages, and geographical locations. This partnership creates a platform to engage with universities, provide internship opportunities, temporary assignments, and pathways to permanent federal hire (U.S. Department of Homeland Security 2022).
The Office of University Programs (OUP) allows access to academia to conduct research. Centers of Excellence (COE) are research led by universities to combat challenges faced by DHS. COEs work to complement DHS research and development programs and provide outcomes useful to federal, state, and local governments, private sector, and international partners (U.S. Department of Homeland Security 2023)
To develop the future workforce, action needs to be taken at the university/organization level and individual level. At the university level, certificates, degrees, research, and outreach are all points of interest. At the individual/organization level, internships and general awareness are key. The T&E community has a need for outreach that informs young professionals and students about the field to grow the pool of employees and expand the workforce as the T&E field grows.
Universities expanding their opportunities with degrees and certification is key for T&E growth. ITEA works to raise awareness of T&E within high schools and universities, but more expansive actions such as the creation of courses and degree programs would expose students to concrete opportunities in T&E and encourage them towards T&E as a viable career option. The challenge that comes with this though is the lack of Accreditation Board for Engineering Technology (ABET) accreditation for a potential T&E degree. Accreditation is a way to assure a collegiate program has met standards essential for entry into the workforce. Without accreditation, universities are not motivated to create a T&E degree (ABET n.d.).
A dedicated scholarship for students interested in T&E does not exist. Scholarships are an effective way to encourage and retain students in the field. ITEA is associated with some scholarships, but a dedicated scholarship program does not exist. ITEA’s scholarships are administered by local chapters and are often limited due to funding as is the case for many organizations (Academic scholarships 2023).
For the individual interested in entering the T&E field, certification can be beneficial. Certification provides a way to demonstrate competency and credibility on the topic. The more interest a program receives, the more likely options will expand; so, certification benefits more than just the individual but the industry. While certifications from universities such as Georgia Tech and Defense Acquisition University may not be an option for everyone, there are alternatives in the meantime. Statistics, material science, and design of experiments are all options T&E professionals discuss using within the field. (MIT Professional Education n.d.)
Internships are amazing opportunities to get introduced to the T&E field, gain work experience and valuable training, and network with professionals. Offered through universities, organizations, or directly from a business, internships allow students or recent graduates to get involved in T&E. Internships open the path to employment and offer important opportunities to network with professionals across multiple fields within the organization providing long lasting relationships that will serve long past the initial conversations had.
For those who may doubt the value in the field, T&E expands past the DHS, the DoD and other government work. All technological development requires testing to ensure the creation of a reliable product and T&E is essential for businesses wanting to commercialize a product that holds up to their standard and listed capabilities.
Research and development require verification and validation, which is something only proper T&E can provide. Efficient and reliable testing procedures combined with consistent and thorough evaluation methods allow accurate assessments of the functionality of systems so that the best product can be attained. Without dependable T&E experts, verification of technology would not be possible, and technological progress would be halted.
Increasing awareness of the T&E career field serves only to benefit the community. Expanding early recruitment efforts to academia will contribute to a diversity of experience, background, and ideas within T&E. Creating T&E specific academic resources and degree programs will incentivize students to become knowledgeable about the field before entering the career space. In addition, funding scholarships for students seeking to specifically enter T&E would serve to push students to research and consider related careers.
“This research was performed under an appointment to the U.S. Department of Homeland Security HS-POWER Program administered by the ORISE for the U.S. Department of Homeland Security (DHS). ORISE is managed by Oak Ridge Associated Universities (ORAU). All opinions expressed in this paper are the author’s and do not necessarily reflect the policies and views of DHS, ORISE, or ORAU.”
We want to extend a huge thank you to our two mentors, Adam Martin and Wayne Dumais, who kept us busy this summer as we kept them busy with questions. Thank you to the Test and Evaluation team at DHS who willingly spent the time with us to teach what they have been studying and working on for years. Thank you to Mr. James Wells for welcoming us to his area of work. Thank you to Laura Freeman for letting us pick your brain and offering your support.
ABET. n.d. Why ABET Accreditation Matters. Accessed 2023. https://www.abet.org/accreditation/what-is-accreditation/why-abet-accreditation-matters/.
Chron. 2014. Crazy stuff at military trade show. Accessed 2023. https://www.chron.com/news/slideshow/Crazy-stuff-at-military-trade-show-95620/photo-7002661.php.
Defense Acquisition University. n.d. Test and Evaluation. Accessed 2023. https://www.dau.edu/functional-areas/test-and-evaluation.
Dumay, J., & Rooney, J. (2011). Dealing with an ageing workforce: Current and future implications. Journal of Human Resource Costing & Accounting, 15(3), 174–195. https://doi.org/10.1108/14013381111178578
FedsHireVets. n.d. Job Seekers – Special Hiring Authorities. Accessed 2023. https://www.fedshirevets.gov/job-seekers/veterans/special-hiring-authorities/.
—. n.d. Job Seekers – Veterans’ Preference. Accessed 2023. https://www.fedshirevets.gov/job-seekers/veterans/veterans-preference/.
FY2018 DHS Inclusive Diversity annual report – homeland security. (2018). https://www.dhs.gov/sites/default/files/publications/20_0611_chco_fy18-dhs-inclusive-diversity-annual-report_0.pdf
Georgia Tech Professional Education. n.d. Test and Evaluation Certificate. Accessed 2023. https://pe.gatech.edu/certificates/test-evaluation-certificate.
International Test and Evaluation. n.d. About ITEA. Accessed 2023. https://itea.org/about-itea/.
John Dumay, Jim Rooney. 2011. “Dealing with an ageing workforce: current and future implications.” Journal of Human Resource Costing & Accounting.
ITEA. (2023, March 24). Academic scholarships. International Test and Evaluation Association | Advancing the field of Test and Evaluation worldwide in government, industry and academia. https://itea.org/about-itea/scholarships-2/
MIT Professional Education. n.d. Design and Analysis of Experiments. Accessed 2023. https://professional.mit.edu/course-catalog/design-and-analysis-experiments.
ORISE. n.d. Homeland Security Professional Opportunities for Student Workforce to Experience Research. Accessed 2023. https://orise.orau.gov/dhseducation/hs-power/default.html.
The University of Memphis. n.d. Systems Testing Excellence Program – Welcome. Accessed 2023. https://www.memphis.edu/step/.
Understanding the federal hiring process. DOL. (2023). https://www.dol.gov/general/jobs/understanding-the-federal-hiring-process
U.S. Department of Homeland Security. 2022. Academic Engagement. Accessed 2023. https://www.dhs.gov/topics/academic-engagement.
—. 2022. Outreach. Accessed 2023. https://www.dhs.gov/academicoutreach.
—. 2023. Welcome to the Centers of Excellence. Accessed 2023. https://www.dhs.gov/science-and-technology/centers-excellence.
U.S. Department of State. (2023, August 15). Pathways programs – careers. U.S. Department of State. https://careers.state.gov/interns-fellows/pathways-programs/
U.S. Office of Personnel Management. 2017. Full-Time Permanent Age Distributions. https://www.opm.gov/policy-data-oversight/data-analysis-documentation/federal-employment-reports/reports-publications/full-time-permanent-age-distributions/.
Camille Holley, a senior Mechanical Engineer student at the University of Memphis interning with the Department of Homeland Security/ORISE. She works as a STEM/NASA Ambassador at the West TN STEM Hub, assists with research as a data analyst with Dr. Ranganathan Gopalakrishnan’s team studying Dusty Plasmas and Powder Dispersion, and is a cadet at the University of Memphis Army ROTC Tiger Battalion preparing to commission as an Active-Duty Infantry Officer May 2024.
Natalie Heinrich, a sophomore in Chemical Engineering at Virginia Tech interning with the Office of Test and Evaluation within the US Department of Homeland Security. She is a member of Galipatia, an engineering-centered LLC, and participates in events encouraging diversity in STEM fields.