Defining T&E as a Discipline

Human interaction with the physical world has long been characterised by iterative experimentation, reflecting an inherent curiosity to observe, adjust, and learn from system behaviour. In its simplest form, this manifests as trial and error applied to discrete artefacts, where the human operator constitutes the primary interface. As technology evolved toward integrated systems comprising multiple interacting components with internal interfaces, behaviour could no longer be understood solely through observation of individual elements. The increasing integration of components to be systems necessitated a more structured approach to understanding system behaviour. Systems engineering emerged in the mid-twentieth century in response to this challenge, with origin stories centred around wartime programs and the subsequent development of large-scale, complicated systems. Early developments toward managing integration recognised that system performance is determined not solely by the characteristics of individual components, but by their interaction across defined interfaces. Bell Telephone Laboratories formalised its methods to optimise components to be systems, coining the term systems engineering in the 1940s [1], with practices maturing in technology-driven industries through the Cold War and space race.

Within this context, Test and Evaluation (T&E) emerged as the disciplined means by which uncertainty about system attributes and behaviour can be empirically assessed. Early reference to the term test and evaluation starts to appear in the mid-twentieth century [2]¹, corresponding with the increasing need for structured feedback toward the optimisation of components in the development of systems. The use of T&E within systems engineering provides a bounded context within which to establish a stable, domain-agnostic definition that can avoid conflation with broader and unbounded uses of the term test. In addressing this definitional gap, this paper adopts a systems engineering lens to position T&E not as an independent construct, nor as ownership of the colloquial notion of testing, but as an integral systems engineering function concerned with the rigorous generation and interpretation of evidence about system attributes and behaviour.

¹ A search of the Adelaide University library identifies early use of the term “Test and Evaluation” in D. Israel’s contribution to the Association for Computing Machinery’s 1956 Conference, later published in the ACM Journal in 1957.

Both systems engineering and T&E represent comparatively recent developments within engineering, with less than a century of formalisation. The relative immaturity of these disciplines is reflected in the continued evolution of their underlying theory, particularly as contemporary systems exhibit increasing complexity, with dynamic configurations and emergent behaviour [3] that cannot be resolved through informal trial-and-error approaches alone.

This paper proposes a conceptual contribution to the discipline of T&E, framed through a systems engineering lens. It does not report new primary research, nor does it propose a new method, standard, or process. Instead, the paper clarifies the foundational concepts that underpin T&E. It provides a coherent abstraction of the underlying theory to establish the fundamentals of the discipline, informed by a literature review and empirical observation of practice. It examines how T&E functions as a systems engineering discipline, independent of delivery frameworks, organisational structures, or governance arrangements, to distil its scientific core. This is done to distinguish it from adjacent but conceptually distinct activities, to articulate a complete and coherent discipline. The intent is to bring coherence to the profession across the domains that employ T&E, so that a body of knowledge can accumulate rather than be a collection of disparate, bespoke applications of similar activity.

The need for clarification is further reinforced by the increasing complexity of contemporary systems [4]. As systems exhibit tighter coupling, dynamic configurations, and emergent behaviour, their attributes cannot be inferred from component-level analysis alone [5]. In the presence of complexity, T&E plays a critical role in generating evidence of system behaviour that may not be determined analytically because of emergence. However, the perceived value of T&E is eroded by the persistence of context-specific practice. The absence of a coherent theoretical foundation occludes the consist foundation upon which the T&E function is performed, highlighting the importance of establishing a definition of T&E that is independent of application context, to be grounded in first principles.

This paper is scoped to the narrow and immediate objective of bringing conceptual clarity to T&E as a coherent and complete scientific discipline within a systems engineering context. It seeks to establish a domain agnostic grounding, concerned with a definition of T&E that is independent of its tools, methods, system lifecycle phase, and timeframes of the project being supported; to be common across technologies and domains. Questions of enterprise governance and delineation from adjacent management frameworks, are important, though have been set aside for subsequent work. Similarly, the positive impact of a coherent and complete discipline upon practitioners remains for a subsequent paper.

The term T&E is applied across a wide range of domains, resulting in a correspondingly broad and heterogeneous body of literature. To examine how T&E is defined at a conceptual level, a structured search was conducted using the Adelaide University library online catalogue, which federates major academic databases including Scopus, Web of Science, and ProQuest. The search was constrained to peer-reviewed journal articles published between 2015 and 2026, in English, with books, book chapters, and conference proceedings excluded to maintain consistency of academic standard. Articles that applied T&E to a specific technology were excluded in favour of those addressing T&E in a conceptual, multi-disciplinary manner, consistent with the objective of establishing a technology and domain-agnostic definition.

Two search terms were employed: ‘Test and Evaluation’ and ‘Systems Engineering Test and Evaluation’, yielding more than 600,000 and 78,000 results respectively. To establish a tractable body of literature for analysis, the top 50 results from each search were screened by abstract for relevance, producing a set of 10 peer-reviewed articles addressing T&E at a conceptual level. This sampling approach, as illustrated in Figure 1, is intended to capture the most prominent contemporary formulations of T&E as presented in the academic literature. The resulting corpus is not exhaustive; rather, its purpose is to characterise how T&E is presently defined, enabling comparison of definitional structure and identification of areas of convergence relevant to the development of a coherent, theoretically grounded definition.

Figure 1. The filtering implemented through the systematic search
to generate the corpus of literature.

Papers addressing T&E in artificial intelligence and digital engineering were retained on the basis that they reflect contemporary, application-independent instantiations of T&E across diverse system contexts. The integration of digital technologies spans domains, being applied within systems that are subject to T&E, and therefore does not preclude the development of a domain-agnostic definition of the discipline.

In addition to the peer-reviewed literature, a small number of authoritative professional and governmental sources were included to reflect the definition of T&E as it is applied in practice. These sources, including defence policy and professional society guidance, are not peer-reviewed in the academic sense. However, they are widely adopted within the practitioner community and therefore provide an important reference to how T&E is defined in operational practice. Inclusion of these sources enables a comprehensive characterisation of how the discipline is presently defined across both research and application.

The definitions collated in Table 1 exhibit a degree of surface-level convergence, most commonly framing T&E in terms of activities that generate information to support decision making regarding system performance, capability, or risk. This consistency however, is limited to identifying outputs and does not extend to a coherent characterisation of the underlying discipline. Across the literature, T&E is variously described as a set of activities or a methodological approach, sometimes tied to a system’s lifecycle stage or project phase. Each formulation emphasises different aspects of practice without reconciling them into a unified conceptual basis. As a consequence, even an apparent agreement on purpose conceals a deeper dilemma in the definition: alignment on what T&E produces does not establish what T&E is. Accordingly, convergence at the level of outputs does not constitute the definition of the discipline, but rather reflects common objectives across applications. These objectives could even be satisfied outside of T&E. A definition of a discipline must specify its nature and conceptual boundaries, not describe its outputs.

The characterisation of outputs as a proxy for the definition of T&E, and fragmentation in those descriptions, reflects an absence of a clear ontological grounding for T&E as a discipline. Ontology concerns the nature, scope, and boundaries of what T&E is, whereas epistemology concerns how T&E generates knowledge about system behaviour. Across the literature, T&E is predominantly described in epistemic terms, by reference to the information T&E produces or the decisions it supports, without establishing an ontological definition of the discipline itself. As a result, T&E is characterised through its methods and outputs rather than its underlying nature, reinforcing its treatment as a set of application-dependent practices rather than a coherent discipline.

The absence of a coherent definition for T&E has substantive consequences for both theory and practice. Although widely practised and consistently associated with generating information to support decision making, existing definitions of T&E remain fragmented, derived from application contexts and characterised by outputs rather than an ontological basis. Without such a foundation, the discipline lacks a stable conceptual grounding from which to derive first principles, constraining theoretical development and similarly fragmenting research to context-specific fields that limit comparability; preventing the aggregation of findings into generalisable theory. The absence of an ontological definition also results in variability in practice, with T&E interpreted in alignment with local objectives, governance structures, or system characteristics, reinforcing its treatment as an activity defined by application rather than as a discipline grounded in complete and coherent theory. Accordingly, there is a need to articulate T&E in domain-agnostic terms, to establish a definition independent of application while bounded within the systems engineering context; this need forms the basis for the conceptual contribution of this paper.

Table 1. The definitions of T&E across the reviewed academic and professional literature.

The preceding analysis of the literature demonstrates that while T&E lacks a coherent, conceptually grounded definition, its diverse formulations exhibit a consistent underlying mechanism. Across the literature, T&E is described in epistemic terms, concerned with generating information to support decision making, yet these descriptions implicitly rely on structured empirical observation, controlled variation, and evaluation against expectation. This common structure reflects the application of the scientific method, providing a unifying basis beneath otherwise fragmented definitions. Recognising this consistency allows T&E to be defined not by its activities or contexts of application, but by the disciplined mechanism through which it generates knowledge about system behaviour. Building on this observation, this section formalises T&E as an application of the scientific method within a systems engineering context, from which a domain-agnostic definition of the discipline is derived, together with its scope and boundaries.

Across the literature and observed practice, T&E consistently exhibits the elements of the scientific method. A baseline expectation of system performance is established, forming an explicit hypothesis regarding how an integrated system is expected to behave under defined conditions. Controlled variation is then introduced, whether through changes to system configuration or operating environment, to observe the effect on attributes of interest. Objective data are collected as evidence of system response, and these observations are evaluated against the baseline expectation to determine whether the hypothesis is supported or refuted. This sequence – hypothesis, controlled variation, observation, and evaluation – is present irrespective of the tools employed or the context of application, and it constitutes the underlying structure through which T&E generates knowledge about system behaviour.

While the literature describes T&E in terms of the information it produces or the decisions it supports, these descriptions are manifestations of a consistent epistemological process: the scientific method. This method is applied across the diverse forms of T&E irrespective of context, and therefore constitutes its defining characteristic. Accordingly, the scientific method provides the ontological basis of T&E, defining the nature of the discipline rather than merely describing its activities, as a discipline is characterised by the underlying principles that give rise to its outputs.

Given that the defining characteristic of T&E is the disciplined application of the scientific method to assess system behaviour, T&E may therefore be defined as:

Test & Evaluation is the deliberate collection of objective evidence, using the scientific method, to assess baselined attributes of an integrated system in order to inform decisions.

Clarifying the terms within this definition to ensure consistency of interpretation across domains:

Objective evidence refers to data obtained through controlled observation relative to a defined expectation. This does not imply exhaustiveness or absolute certainty, but sufficiency relative to the decision being supported. Evidence is therefore generated at a level of fidelity commensurate with the uncertainty associated with the attribute under assessment and the consequence of that uncertainty upon the decision.

Baselined attributes refers to system characteristics for which an explicit expectation has been defined, forming the hypothesis against which observations are evaluated. These attributes may relate to performance, effectiveness, suitability, or other system attributes, but must be sufficiently specified to enable comparison between expected and observed properties. Without a defined baseline, the scientific method underlying T&E cannot be applied; where no expectation exists, the activity constitutes exploration rather than evaluation.

Integrated system constrains the scope of T&E to apply to systems that comprise of components, having a system boundary with the surrounding environment and interfaces. This distinguishes T&E from component-level investigation, which may be scientifically rigorous but does not address an integrated system.

Together, these elements ensure that T&E is interpreted as a disciplined application of the scientific method to assess defined system attributes, with the scope, depth, and fidelity of the evidence generated being governed by the assurance requirement.

4.1 Selectivity

The definition of T&E as a disciplined application of the scientific method implies that its application is inherently selective. Within this framework, T&E cannot be conducted as a comprehensive examination of all possible system attributes and behaviours, but as a targeted evaluation of attributes for which uncertainty exists, where that uncertainty is material to a decision related to the subject system.

Complementing this, the scientific structure established in Section 3 requires that a hypothesis be explicitly defined and that evidence be generated to assess that hypothesis under controlled conditions. This necessitates the deliberate selection of attributes and operating conditions as variables of interest, rather than an exhaustive exploration of the system. Accordingly, T&E is invoked in response to specific questions regarding system behaviour, with the scope of evaluation constrained by the need to generate evidence sufficient to resolve those questions.

This selectivity is not a limitation of T&E, but a defining characteristic of the discipline. The requirement to establish a hypothesis and generate evidence relative to that hypothesis precludes unfocused or exploratory activity being considered as T&E; where no defined expectation exists, the activity constitutes investigation or experimentation rather than evaluation. The scope, depth, and fidelity of T&E are determined by the assurance required in the corresponding decision to be informed, ensuring that evidence generation is proportionate to the uncertainty being addressed.

4.2.Various Tools

The definition of T&E as a disciplined application of the scientific method distinguishes the discipline from the tools employed in its execution. In practice, T&E utilises a range of tools, including test, inspection, demonstration, modelling & simulation, and analysis; each of which contributes to the generation of evidence. Though these tools are often conflated with T&E itself, they are instruments within the discipline rather than defining characteristics.

Test is one such tool, characterised by the deliberate application of controlled variation to observe system response under defined conditions. Inspection and demonstration similarly produce data relevant to system attributes and behaviour but differ respectively in the level of manipulation in the variables, and the scope over which variation is introduced. Modelling & simulation may also contribute evidence, either as a substitute for, or complement to, physical observation. This is provided the resulting data can be evaluated against a defined expectation with sufficient fidelity to support the decision.

What unifies these tools is not their form, but their role within the scientific structure of T&E. Each serves as a means of generating empirical evidence that can be evaluated against a baseline expectation in order to reduce uncertainty to inform a decision. It is this integration of tools within a disciplined hypothesis-driven framework that characterises T&E, rather than any individual activity or method. Accordingly, no single tool constitutes T&E; rather, the discipline encompasses the selection and application of appropriate tools to generate evidence consistent with the scientific method.

4.3.Fundamental Forms of T&E

The selectivity inherent in T&E, together with its basis in the scientific method, constrains how variation is introduced in the evaluation of system attributes or behaviour. As established in Section 3, T&E requires the controlled variation of a dependent variable within the scientific method to observe the effect on attributes of interest (independent variable), with all other variables held sufficiently constant to enable causal inference. Within this structure, variation can be introduced in only two fundamentally distinct ways: by varying the system itself, or by varying the environment in which the system is operating.

Where variation is introduced within the system boundary, through changes to configuration, design parameters, or integration state, the dependent variable lies within the system. This form of T&E is concerned with how system attributes respond to controlled changes in the system itself, supporting decisions relating to design, integration, and configuration.

Alternatively, where variation is introduced through the operating environment, user interaction, or employment context, the system configuration is held sufficiently stable and the dependent variable lies outside the system boundary. In this case, T&E is concerned with how the system performs when subjected to variation in its external conditions, informing decisions relating to capability, suitability, and use.

These two forms correspond to Developmental Test and Evaluation (DT&E) and Operational Test and Evaluation (OT&E), respectively. These are not defined by organisational constructs, lifecycle phases, or programmatic boundaries, but by the location of the dependent variable in the application of the scientific method. This distinction is both precise and exhaustive. Any instance of T&E can either vary the system or its environment; no other locus of variation exists.

Figure 2 illustrates this relationship, showing that T&E is conducted by varying either the system or its operating environment while holding all other factors sufficiently constant to support evaluation. All other labels applied in practice, such as preview testing, acceptance testing, live-fire testing, or follow-on operational test, are derivative constructs that reflect organisational, contractual, resource or scheduling considerations. While these distinctions may be meaningful within specific programs, they do not alter the underlying conceptual basis of the discipline.

Figure 2. T&E varies either the system or its operating context,
while holding all else equal, to be either DT&E or OT&E.

The definition of T&E established in this work addresses the deficiency identified in the literature by providing a coherent, domain-agnostic conceptual basis to define the discipline, grounded in first principles. By defining T&E in terms of the disciplined application of the scientific method, rather than as an aggregation of activities or outputs, the paper resolves the fragmentation observed across academic and professional sources. Descriptions of T&E as process, activity, or methodology are reconciled within a single framework by recognising them as context-specific manifestations of a common underlying concept.

This framing has implications for both theory and practice. Theoretically, it establishes a foundation upon which a cumulative body of knowledge may be developed, as studies framed within this definition share a consistent conceptual basis. This enables comparison and aggregation of findings across domains, addressing a key limitation identified in Section 2. Practically, it shifts the focus of T&E from the execution of activities to the formulation and assessment of explicit hypotheses. This repositions T&E as a question-driven discipline, in which the design of evaluation is governed by the uncertainty to be resolved in support of the required decision.

The distinction between T&E and its tools clarifies the role of activities such as testing, inspection, and modelling as tools within a broader scientific method. This supports more deliberate selection and integration of tools, aligned with the assurance requirement rather than adherence to prescribed procedures. Similarly, the identification of selectivity and the two fundamental forms of T&E provides a consistent structure for practice, independent of lifecycle models, organisational constructs, or programmatic labels. These constructs allow practitioners to reason about T&E in terms of first principles rather than derived classifications.

The systems engineering context provides a natural boundary for the discipline, constraining its application to the evaluation of integrated system behaviour relative to defined expectations. Within this boundary, the scientific framing of T&E offers a basis for internal coherence, enabling alignment with related domains such as systems theory and decision science without conflating their roles. The resulting definition therefore supports both conceptual clarity and practical application, reinforcing T&E as a coherent discipline rather than a context-dependent, bespoke collection of activities.

In practice, T&E is conducted within project environments, where additional requirements arising from organisational, contractual, political, marketing, budgetary, scheduling, test asset availability or lifecycle considerations are imposed upon its execution. These factors introduce overlays that influence how and when T&E is conducted, but they do not alter the underlying conceptual basis of the discipline. The fragmentation observed in practice reflects variation in these external constraints, rather than any inconsistency in the discipline itself. The definition of T&E, together with the identification of its fundamental forms, provides a clear distinction between disciplinary requirements and those imposed by project context. This separation enables practitioners to address discipline and programmatic demands independently, ensuring that evidence generation remains consistent with the discipline while accommodating the realities of project delivery. In this way, T&E is properly understood as a professional discipline with its own coherent conceptual integrity, rather than as an externally mandated or administratively defined activity.

This paper addresses a fundamental deficiency in the definition of T&E identified across academic and professional literature and practice: the absence of a theoretically complete, coherent, and conceptually grounded basis for the discipline. While T&E is widely practised and consistently associated with the generation of information to support decision making, existing descriptions remain fragmented and context-dependent, lacking a stable ontological foundation. By identifying the consistent application of the scientific method as the unifying mechanism across diverse formulations, the paper establishes a definition of T&E as a discipline grounded in first principles. This enables T&E to be defined in domain-agnostic terms as the deliberate collection of objective evidence, using the scientific method, to assess baselined attributes of an integrated system in order to inform decisions related to the system. This definition distinguishes the discipline from the tools it employs, and constrains its scope to the evaluation of integrated system attributes to provide a coherent foundation for both theory and practice.

The definition of the discipline reconciles previously fragmented interpretations by locating activities, tools, and organisational constructs as context-dependent manifestations of a consistent concept. The identification of selectivity as an intrinsic property of T&E, and the derivation of its two fundamental forms, further provides a structured basis for understanding its application independent of project or organisational constructs, lifecycle models or programmatic labels. The persistence of variation in practice arises from the influence of external project and organisational constraints that impose overlays on the execution of T&E. This variation reflects differences in these contextual factors, rather than any inconsistency in the discipline itself. The separation of disciplinary requirements from project-specific constraints enables T&E to be applied with greater clarity and consistency, while accommodating the realities of project delivery. In this way, T&E remains a conceptually grounded discipline, even as its implementation is shaped by external demands.

The contribution of this work lies in establishing T&E as a discipline, rather than as an aggregation of activities. It establishes a foundation for the accumulation of knowledge across domains, supports the consistent application of T&E across increasingly complex socio-technical systems, while providing a basis for debate within the profession. Future work may extend this foundation through integration with systems theory and decision science, and through examination of its implications for governance, assurance, and risk management in practice. In doing so, further theoretical development may also consider the role of human judgement in shaping the application of T&E, recognising that the underlying socio-technical systems require interpretation, decision-making, and responsibility, as inherently human concerns.

The author gratefully acknowledges the reviewers at the Journal of Test & Evaluation for their constructive feedback. Dr Barry Elsey (Adelaide University) provided informal review and helpful suggestions that contributed to refinement of the argument. Large language models were used to assist with editorial tasks, including grammar and language refinement. Generative tools were used to refine and render figures based on the author’s original sketches. All conceptual content, analysis, and interpretations remain the responsibility of the author.

This research received no external funding.

The author declares no conflict of interest.

All data supporting the findings of this study are contained within the article; the reference list provides comprehensive coverage of relevant sources.

1. INCOSE, Systems Engineering Handbook. 5 ed, ed. D.D. Walden. 2023, San Diego CA: John Wiley & Sons, Incorporated.
2. Israel, D.R., Simulation Techniques for the Test and Evaluation of Real-Time Computer Programs.Journal of the ACM, 1957. 4(3): p. 354-361.
3. Luther, B.S. Assuring 2121. Central Blue, 2021.
4. Luther, B.S., I. Gunawan, and N. Nguyen, Identifying effective risk management frameworks for complex socio-technical systems.Safety Science, 2023. 158.
5. Leveson, N., Engineering a safer world: systems thinking applied to safety. Engineering systems. 2012, Cambridge, Mass: The MIT Press.
6. Weiss, B.A., et al., Test and Evaluation of Autonomous Systems: Developing the Educational Landscape.Computer (Long Beach, Calif.), 2025. 58(1): p. 121-125.
7. Karimoddini, A., et al., Automatic Test and Evaluation of Autonomous Systems.IEEE access, 2022. 10: p. 72227-72238.
8. Joiner, K.F. and S. Reay Atkinson, Australia’s future submarine: shaping early adaptive designs through test and evaluation.Australian journal of multi-disciplinary engineering, 2016. 12(1): p. 3-26.
9. Freeman, L., Test and Evaluation for Artificial Intelligence.Insight (International Council on Systems Engineering), 2020. 23(1): p. 27-30.
10. Flanigan, D. and K. Robinson, Exploring the Test and Evaluation Space using Model Based Conceptual Design (MBCD) Techniques.Insight (International Council on Systems Engineering), 2026. 29(2): p. 13-17.
11. Warfield, J.D. and S.E. Roberts, Comparison of Test Sizing Approaches for Initial and Follow-On Evaluation of Strategic Weapon Systems.Quality engineering, 2015. 27(2): p. 230-252.
12. Bjorkman, E.A., S. Sarkani, and T.A. Mazzuchi, Using model-based systems engineering as a framework for improving test and evaluation activities.Systems engineering, 2013. 16(3): p. 346-362.
13. Torkjazi, M. and A.K. Raz, A Systems Engineering Methodology for System of Autonomous Systems: Test and Evaluation.Systems engineering, 2026. 29(3): p. 425-443.
14. Garcia, M.R., J. Castillo, and A. Shirin, Test and Evaluation of AI/ML Enhanced Digital Twin.Systems (Basel), 2025. 13(8): p. 656.
15. Defence T&E Strategy, C.I. Department of Defence, Test and Evaluation Branch, Editor. 24 August 2021, Commonwealth of Australia: Canberra.
16. Cortes, L.A., Best practices for highly effective test design; Part 1 – Beginners’ guide to mapping the T&E strategy, S.T.a.A.T.S.C.o. Excellence, Editor. 2014, Office of the Under Secretary of War, Research and Engineering: Washington DC.
17. Test and Evaluation Enterprise Guidebook, U.D.o. Defense, Editor. 2022, Office of the Under Secretary of War, Research and Engineering: Washington DC.
18. Collins, C. and K. Senechal, Test and Evaluation as a Continuum.Journal of T&E, 2023. 44(1).