Developing Winning Proposals through the Lens of Test and Evaluation

It is a good assumption that most DOD contractors approach the Request for Proposal (RFP) process required by the US Government in the same manner. The branch/agency releases the request online at SAM.gov, the viable contractor goes through the process of creating program content based on past program construct – design approach, following the traditional Systems Engineering VEE from user needs to validation. The proposal then builds in this classic sense, capturing legacy performance in the same fashion and tailors/scales to the new program content. Historically, the results have proven less efficient and timely. Late findings, delays in delivery for validation to meet the user’s timely need and, of course, cost overruns. Given the legacy Joint Capabilities Integration and Development System (JCIDS) requirements for acquisition, proposals are constructed in a way that does not lend itself to the rapid fielding of systems.

In August of 2025, the Department of War (DoW) initiated a change to the acquisition process, implying new evaluation criteria for future selection (Obis 2025). This opportunity forces contractors to rethink the approach of development from the implementation of self-directed Internal Research and Development (IRAD) to the commitment of capital funds for production readiness. Changing the approach, the contract base must look for many areas to improve performance. Implementation of the proposed approach in this paper will provide an enabler for contractors to meet the expectations of the DoW. The details, developed independently of the US Government, have been churning in the minds of contractors for several years (Honea 2020). The language from DoW regarding the delivery of systems faster drives the industry to look for new ways of getting through systems development. It is expected that making these methods a key point in proposal language will be widely accepted as the right kind of change the DoW is looking for from industry. Additionally, contractors needing to pivot to the demand from the customer can quickly recognize the benefits of the approach by tailoring and implementing the suggestions herein.

In the interim, until the authorities in the US Government approve the new process language, defense contractors can still overcome the past delivery challenges by taking advantage of focusing on the content of the IV&V phase of the Systems Development Life Cycle (SDLC). Rather than look at how to go faster, more efficiently, or pulling work left it is the authors’ experience that by applying a Test and Evaluation (T&E) focus to the program construct, an offeror can, with a priori knowledge, provide a proposal that is not only compliant but one that is structurally robust and has a higher likelihood of success.

Literature Review

A search on Litmaps® narrowed down to the engineering and manufacturing fields, using the search terms: test-driven development, test architecture, test strategy and architecture-based testing, through the period 2004 to present yields several results in the space, but these do not directly address test strategy & architecture as applied to proposal development and execution for the DoW acquisition frameworks covered in this paper Figure 1.

Figure 1: Litmaps Results for Test Strategy & Architecture-Based Testing

Burak & Tekinerdogan (Burak and Tekinerdogan 2018) provide a literature review and systematic foundation for Model-Driven Architecture Based Testing (MDABT), illustrating how model transformations can bridge the gap between system design and verification. Their review highlights shift toward automated test artifact generation across software-centric systems, where platform-independent architectural models serve as the “single source of truth” for deriving test cases and scripts. This model-based approach is further explored in the test architecture related literature described below.

Zimmerman (2019) and Paulisch (2016) each wrote extensively about formalizing the Test Architect as a role and the discipline of Software Test Architecture within large-scale software-intensive industrial environments such as those at Siemens AG (Zimmerer 2019). Their early focus was on defining a model-driven and risk-based Test Architecture encompassing the infrastructure, design and strategy required to make testing more systematic and repeatable. They proposed a “Two-Architect” model with a Test Architect and Software Architect collaborating early in the product development lifecycle, and the establishment of formal training and deployment of Siemens Test Architects across Siemens’ global business units (Paulisch and Zimmerer 2016).

Manas (Manas and Guise 2013) defines the Role of the Test Architect and Test Strategy and Architecture as applied to Systems Engineering and product development as applied at Raytheon Missile Systems. Jaramillo et al. build on the Raytheon Test Strategy and Architecture process to further establish a framework for Environmental Qualification test strategy development and strategic requirements verification testing to mitigate “design escapes” (Jaramillo, et al. 2020). Sivertson addresses the economic challenges of aerospace and defense manufacturing by proposing a Socratic approach to test architecture, highlighting a shift toward risk-based verification in the product development lifecycle. The core argument is that cost optimization in complex systems is achieved not only through increased automation but through applying intellectual rigor and challenging the necessity of all manufacturing tests within the lifecycle (Sivertson 2020). Maksi et. al (Maksi, et al. 2021) contribute to the field of industrial test architecture by providing a quantitative framework for hardware standardization. The authors apply clustering algorithms to historical test requirements to demonstrate that a Common Factory Test Platform (CFTP) can satisfy most verification needs across highly diverse product lines.

While Siemens and Raytheon both utilize the “Test Architect” role to manage system complexity developing model-based and risk-based approaches to product development testing, their frameworks differ significantly in their primary objectives and focus areas: Siemens focuses on architectural parity between test and software architecture in software-intensive industrial systems, while Raytheon focuses on Mission Assurance and early integration within a broader Systems Engineering (SE) framework and applied to diverse programs across the aerospace & defense industry.

The remaining results from the Litmaps® search focus primarily on software focused systems testing and architecture and not specifically on aerospace and defense products and acquisition (Lam 2004), (Mottu, et al. 2019), (Tekinerdogan, et al. 2016), (Thomas, et al. 2008), (Keum, Kang and Kim 2013).

DoD Context

Department of Defense (DoD) materiel development programs follow robust processes. Codified as DoD Instruction 5000.85 (University 2022) DoD acquisition professionals employ an adaptive acquisition framework in accordance with (IAW) the category/type of acquisition program, Figure 2, (OUSD(R&E) 2024) Each program is further categorized by Acquisition Category (ACAT) with further refinement. The distinction of each ACAT being the total acquisition dollar value, as well as the Milestone Decision Authority (MDA), including a designation of special interest. This special interest designation may include the following: technological complexity, congressional interest, a large commitment of resources, or the program is critical to the achievement of a capability or set of capabilities, part of a system of systems, or a joint program.

Figure 2: DODI 5000.85 Adaptive Acquisition Framework

Major capability acquisitions follow the widespread acquisition model proceeding from the decision to acquire a material solution, technical developmental phases through deployment, sustainment and disposal, Figure 3. In reviewing DoDI 5000.85, one becomes informed of the complexities in each of these stages and the justification to commit funds to each evolution. As such, Requests for Proposals (RFP) at each milestone are dictated by entrance criteria for the acquisition community and accordingly pass these requirements to the contractor providing the recurring and non-recurring efforts. RFP solicitations describe their requirements, anticipated terms and conditions that will apply to the contract, information required in the offeror’s proposal, and the criteria used to evaluate the competitive proposal and their relative importance. Federal Acquisition Regulations (FAR) Subpart 15.2. FAR Subpart 15.403-4, (Government, Acquisition.gov 2025) defines the threshold for obtaining certified cost or pricing data for any procurement contract for prime contractors as $2M. This implies proposal compliance with the Truth in Negotiations ACT (TINA), (Government, US Code House Title 10 – Armed Forces 1995) and public contracts (Government, Title 41 – Public Contracts 2011). The offeror is required to justify pricing through quantifiable evidence of past work, or an explanation if the effort contains novel/uncommon efforts. Additionally, risk attributable to program cost, schedule, and technical execution is regarded as the assessment of the proposal. Failing to meet the basic requirements is common enough that the FAR also defines the clause for rejection of bids, (USG, FAR – Subpart 14.404 – Rejection of Bids 2025).This codified approach dovetails directly into (by design) the acquisition models described above.

Figure 3: Major Capability Acquisition Model

Generally, major acquisition programs proposals contain the classic waterfall approach with agile software acquisition being an exception. For simplicity of argument, the construction of the program schedule takes on series of activities by the integrated product team leaders, planners, and program management. Sequential activities associated with dependencies relate to a series of handoff constraints. The most familiar being: start to finish, finish to finish, start to start. This is not to say the waterfall is the root cause of delays, but the waterfall mindset has, as one of many factors, contributed to contractors experiencing less than successful execution to the baseline plan, (Berteau, et al. 2010).

The use of the waterfall is partly an effect of convenience due to the System Development Life Cycle (SDLC) and the Systems Engineering (SE) VEE construct, Figure 4. The SDLC allows a logical progression of a success-oriented expectation for program execution. However, without shoring up the SE decomposition and allocation of capabilities to functions on the left-hand side of the VEE, for instance, creating multiple “loopbacks”, many technical discoveries/risks are realized late, causing delays and cost increases. The uncertainty of performance compliance, technical risk and technical debt pushed to later in the schedule results in a long extension of the Integration, Verification, and Validation (IV&V) phase of the program. It is acknowledged that there are many other factors that contribute to these issues, including a program execution risk posture of: technical uncertainty, supply chain limitations, political changes, and the most insidious – the unknown unknowns.

Figure 4: Systems Engineering VEE

The results of execution by the contractor are openly presented by the DoD through the Contractor Performance Assessment Report (USG, CPARS.GOV 2025). Paraphrasing the CPAR site, performance evaluations in the CPAR contain both government and contractor comments that provide a balanced view of performance, allowing source selection officials to look beyond contractor references. This assessment is used in consideration of contractor selection for future programs.

Current proposals rely on the historical performance of programs. The process of collecting cost and schedule actuals to build a basis of estimate from past “like-in-kind” programs provides a method of estimating expectations of execution success, and to substantiate the offeror’s cost proposal. This implies that both current and future offers are based on actuals IAW the SDLC approach. During proposal development, the budget/delivery timing constraints are assumed, and programs face challenges using learning curves, (Barber 2011) price-to-win, or technical justification based on leverage from the maturity of technologies being integrated into the new system. These are incorporated in the cost volume as Ground Rules and Assumptions (GR&As) as well as the Risks and Opportunities (R&Os) list, where the assumptions tend to be the driving cause of later challenges.

The following approach accepts elements of the program actuals from other Integrated Product Teams (IPTs), including supplier management, program office and the Life cycle engineering to capture total ownership costs. However, it deviates in the GR&As to the extent that an “if-then-else” approach is taken using early cycles of learning to correct assumptions, reduce technical risks and provide the ability to quickly anchor simulations that are driving performance requirements. This approach presents itself as the ability to pivot at critical stages early in the program with data-driven program decisions. It allows the technical teams to address uncertainties/risks through early requirement allocation verification and initial functional / performance characterization, thereby informing the likelihood of maintaining cost and schedule goals.

Implementing a historical Integrated Test and Evaluation Plan (ITEP) approach to IV&V is seen as a standard process IAW the SDLC (DAU 2005). Additionally, the DoD acquisition process requires the contractor to provide a resource-loaded Analysis of Alternatives (AoA) effort on the left-hand side (LHS) of the SE VEE. If T&E is truly the driver, then it should be utilized to ensure optimization of the program’s Test Strategy and Architecture to reduce test cycle times and improve execution efficiencies (Honea 2020) with earlier test implementation on the LHS of the SE VEE. Optimization is an intentional effort to minimize gaps and/or redundancy in the T&E program. The TS&A development process addresses the formal content of all test elements of the program, fully detailing them to allow execution of all tests (including some simulation environments). Given the commonality of many of the DoD program test elements, this means that past performance (manpower efforts), technical data packages (test equipment), test configurations (unit or system under test) and support infrastructure (facilities) are clearly defined for later proposal submittal with full actuals justification.

Understanding of the acquisition selection process enables quicker development of an optimized TS&A. As part of the source selection process, acquisition agencies assess the best value of the source along a continuum, balancing contract performance risk against program acquisition type and requirement definition maturity. The best value is determined via a combination of one or more source selection approaches, including but not limited to: tradeoff analysis process and lowest price technically acceptable source selection process (USG, Subpart 15.3 – Source Selection 2025). The trade-off process may be used when it is in the interest of the Government to award a contract to someone other than the lowest-priced offeror or the highest technically rated offeror. In this case, the RFP must clearly state the relative importance of the evaluation factors, and the relationship between the combined non-cost/price related factors to cost (more important than, equal, or significantly less important than). In the case of the Lowest price technically acceptable source selection process, best value is expected from the selection of the technically acceptable offer with the lower evaluated price. Understanding how best value is assessed is critical in developing the right-sized Test and Evaluation strategy that balances contract performance risk and the cost of the T&E effort.

Early development of a Knowledge Point (KP) Plan tied to critical technical and programmatic risks can aid in earlier maturation of the program win strategy, and development of an executable T&E schedule with an acceptable risk posture. Knowledge Points (KP) are critical questions and technical knowledge of the system that drive decisions at a program level and burn down uncertainty and risk over time. In the KP plan, KPs are decomposed down to the required data, events and resources, to answer these critical questions (Honea 2020) An example of KP decomposition is shown in Figure 5. Example KPs may include:

1. Can the system operate in the relevant environments?
2. Does the test environment have enough fidelity to predict system performance?
3. Is the system interoperable with other platforms in the System of Systems?
4. Can the system self-detect and report faults?
5. Can the system be produced at the desired rate?

During the proposal phase, the technical leadership team should generate top-level KPs to help inform the test program definition. The initial set of KPs may be generated based on several factors, including:

1. Key system capabilities and requirements specified in the RFP
2. System quality attributes identified through stakeholder engagements, RFP documentation, and technical baseline development, ex. security, producibility, interoperability, usability, performance. (Bass, Clements and Kazman 2022)
3. Technology insertion of Internal Research and Development (IRAD) or Cooperative Research and Development Agreement (CRADA) systems.
4. Average Unit Procurement Cost (AUPC) targets, when cost is a key evaluation criterion.

KPs provide a path to system maturation of low TRL (U.S. GAO 2020) system components by pulling cycles of learning to the left in the development cycle, as well as help define what “done” looks like for the program.

Figure 5: Knowledge Point Decomposition

The development of the program TS&A leverages an understanding of DoD customer needs and evaluation processes, such as the Integrated Decision Support Key (IDSK) Figure 6. The integrated decision support key (IDSK) (Office of the Director of OT&E 2024) is part of the DoD Test and Evaluation Master Plan (TEMP) and Strategy and is used to inform decision making for a program across the life cycle, increase T&E efficiencies and optimize the evaluation of technical requirements and DoD system operational effectiveness.

Figure 6: Integrated Decision Support Key (IDSK)

This optimization occurs through the following four activities showing alignment between the IDSK and the program test strategy:

1. Identification of the right qualified T&E data from Contractor Test & Evaluation (CT&E) and other test-dependent agencies that support program and acquisition decisions.
2. Maximizing the use of T&E data and results collected in support of previous acquisition or program decisions
   1. accounting for additional opportunities to collect test data and results during future programs to optimize planning in support of the current acquisition decision.
3. Enabling early and frequent coordination of stakeholders across CT&E and other test-dependent agenciesto plan and execute integrated T&E events to meet their independent objectives.
   1. Incorporating operational realism early in the test program is critical to improving the probability of identifying problems early when redesigns are more economically and technically feasible.
4. Implementing advanced statistical inference methods, digital engineering, digital tools, and digital technologies, as appropriate, to maximize the knowledge gained thru test and analysis activities (modelling and simulation results, live test data, etc).

Proposal T&E plans should not be developed in a vacuum. To move faster, early considerations of what can be T&E data and processes leveraged from prior programs or deferred to a future stage can aid in the development of an executable T&E strategy. For the contractor, ensuring the TS&A includes relevant test views and plans tied to operational objectives to ensure the product meets operational needs, within budget and scope constraints. The program test strategy is the single source of truth for documenting these views.

The Test Strategy and Architecture (TS&A) process is a Raytheon standard that provides for quantification of the necessary data to implement a data-driven development process beyond simply incorporating all test and evaluation details per the traditional approach to the RFP response. For a test program, the strategy is very similar to the KP decomposition process: create an overarching strategy for the program, then decompose to individual strategies aligning to the lifecycle of the program (user operations, development, and production testing). The TS&A process enables engineers to provide clarity to expectations, address risks and uncertainties, create a plan that aligns with the program’s visions and goals, integrate all elements to achieve those goals, and actively manage the strategy as the program evolves. The TS&A process is more than creating an ITEP – more cohesive and more enabling in maintaining the goal of delivering a time-critical capability to the customer without solely focusing on development, but spanning the entirety of the system’s life cycle testing ​(Honea 2020)​.

The “Strategy” of TS&A refers to creating the understanding at the program management level of how T&E intends to contribute to achieving its vision by “ways and means.” These refer to the approach (tactics) for reaching the goals with a broad brush, simple, and clear descriptions and a meaningful outcome. A life example of an early career strategy is that of becoming an engineer. The vision is “to become an engineer.” The strategy continues “… by going to an accredited college of engineering, taking engineering classes, and graduating with a degree in an engineering discipline.”

In this context, the Test Strategy is how the entire program treats test and evaluation content to get to the final design. Test Strategies are highly influenced by the System Under Design (SUD). This includes the program’s Concept of Operations (CONOPS), Capability Definition Document (CDD), and fielding sustainment and support operations. Additional content included in the Test Strategy can be explicitly or implicitly defined in the contractual/program requirements – the Statement of Objectives, Statement of Work, contract definitions and partnering agreements.

As an iterative process, the strategy is created from the source content defined above and refined as the data content is provided/assessed by the stakeholder. The litmus test for the final data content is “how does it contribute to necessary programmatic decisions?” Understanding these two critical pieces (stakeholders and their data needs) constructs to organize the details must be created that evaluate overlapping needs – or gaps – to information that informs the program at the right time. For Raytheon’s Systems Engineering & Test Capabilities organization (SE&TC), two frameworks align the teams. The first creates a basis of all test strategies and architectures across the entire lifecycle of a system – the Three Pillar Framework, and a second aligns the data content and association of stakeholders’ work product contents – the Zachman Framework™.

The Three-Pillar Framework provides a cohesive set of fully integrated tests (data products) and the test content that crosses each pillar (common content and maturation). The operational (user) test defines the expected data needed to ensure systems’ readiness in any phase of the lifecycle, from fielding and maintenance to sustainment and decertification. During the development phase, the bulk of testing follows the traditional design, characterization, verification, qualification, integration, and validation of the system. Finally, production (or product assembly) testing shifts the focus to verifying that the product is built right to the final technical data package of the system.

The Zachman Framework™ is an Enterprise Architecture Framework, which provides a formal and highly structured way of viewing and defining an enterprise. The framework uses a journalist’s ontology – who, what, when, why, how, ​(Zachman 2008)​. Evaluation of the interrogatives for each stakeholder reveals the output from the test architecture and defines the time-phased maturation of these artifacts at the right stages of the program, from proposal to operational testing. Between these frameworks, all test content is optimized to stakeholder needs and time-phased to meet program execution timelines.

With a cohesive set of test strategies and required frameworks, the next step for the Test Architect is to develop the architecture. In general, system architecture is a unifying structure defined in terms of system elements, interfaces, constraints and behaviors that allows one to realize the strategy ​(Maier and Rechtin 2009)​. There are many standard architectures, but based on past evaluations, ​(Manas and Wilson, Test Perspectives for Architecture 2014)​, Raytheon has determined that the most applicable architecture for a T&E program is the System Architecture. The System Architecture provides the structure and elements to create the viewpoints, such as those described in the United Architecture Framework ​(OMG 2022)​ and artifacts to satisfy key stakeholders of the T&E program across the organization (the “data” in the Zachman™ process). Information and work content then flow from this construct and can be accounted for in terms of need and purpose.

With a cursory understanding of the construct and content of a TS&A, one can note that uniqueness compared to traditional integrated test and evaluation plan development. The process by which the TS&A process addresses critical decision points as well as incorporates the data-driven making process at the outset of the program allows it to be both the narrative and content of the technical volume proposal submission. The team at Raytheon utilized the well-established Raytheon Enterprise Architecting Process (REAP) to develop the TS&A process, maintaining the consistency between the Systems Engineering standards and the SE&TC desired process. REAP fully defines the architecture development, description, evolution, and assessment required to finalize a robust enterprise architecture. It is a closed-loop process that drives reflective reviews as the process proceeds, thereby ensuring a robust architecture to support the refined strategy as new details are evaluated/discovered.

The result of the TS&A process – creating the strategy, aligning within a framework, detailing the architecture, and identifying the required data at the phase (KP plan) – is the exact process that creates the proposal’s test and evaluation content. It also supports the system engineering function as well as the hardware and/or software design organizations, even supply chain and logistics, Figure 7. The benefit of applying the TS&A process is that the offeror is provided with an inherently repeatable, scalable, and flexible method for a test construct that can be quickly leveraged to other future proposal submissions.

Figure 7: Test Strategy & Architecture Process

While T&E may comprise 15% of a program lifecycle cost (Honea 2020), there are several challenges related to the remaining costs that may not be directly addressed by a comprehensive TS&A. These items include technical uncertainty, supply chain limitations, political changes, and other unknown unknowns, that may impact the risk profile, execution to the baseline plan and determination of the price-to-win (PTW). In determining PTW, teams should consider tradeoffs between several factors including: understanding of proposal requirements and evaluation criteria for the acquisition, competitive analysis, and understanding of the offeror’s technology, manufacturing and business capabilities to identify risks and develop an executable solution.

DoD contractors have a focused intent to deliver timely and affordable technical solutions that address War Fighters’ needs. From the request for proposal response through the validated systems, the timeline relies on the accuracy of informed decision making starting before a single contract is let. By improving the completeness and accuracy of the offer, the contractor has a higher likelihood of winning the contract and a greater chance of successfully executing the plan to cost, schedule, and technical performance.

Applying the TS&A process early facilitates data driven decision making early on the left-hand side of the SE VEE, improving the likelihood of success and timeliness on the right-hand side. It brings the program mission into focus through delivery of tangible knowledge and capabilities at earlier phases of the program development cycle. The process actively minimizes gaps and redundancies by clearly defining content with traceability to the customer’s objectives.

Barber, Elen. 2011. APPLICATION OF LEARNING CURVE THEORY TO SYSTEMS ACQUISITION. 10. DAU, Elen Barber, “APPLICATION OF LEARNINBusiness, Cost Estimating and Financial Management Department, Defense Acquisition University.

Bass, Len, Paul Clements, and Rick Kazman. 2022. Software Architecture in Practice, 4th Edition. Addison-Wesley.

Berteau, David, Joachim Hofbauer, Gregory Sanders, and Guy Ben Ari. 2010. “Cost and Time Overruns in Major Defense Acquisition Programs.” Seventh Annual Acquisition Research Symposium. Monterey: NPS Archive:Calhoun.

Burak, Uzun, and Bedir Tekinerdogan. 2018. “Model-Driven Architecture Based Testing: A Systematic Literature Review.” Information and Software Technology 102: 30–48.

DAU. 2005. Test and Evaluation Management Guide Fifth Edition. Fort Belvoir: DSU Press.

Government, US. 2025. “Acquisition.gov.” FAR Subpart 15.2- Solicitation and Receipt of Proposals and Information. January 17. Accessed March 12, 2025. https://www.acquisition.gov/far/subpart-15.2.

—. 2011. Title 41 – Public Contracts. January 4. Accessed January 20, 2026. https://uscode.house.gov/statviewer.htm?volume=124&page=3677.

—. 1995. US Code House Title 10 – Armed Forces. January 4. Accessed February 12, 2025. https://uscode.house.gov/view.xhtml?req=50+usc&f=treesort&num=806.

Honea, Gary A. 2020. “Test Strategy and Architecture – The Groundwork for a Successful Program.” The ITEA Journal of Test and Evaluation 244-251.

Jaramillo, Phillip, Eric Jauregui, Marco Rascon, and Charles Adams. 2020. “Fundamental Principles, Processes, and Roles of Environmental Qualification Test Strategy for Complex Engineered Systems.” Journal of Information Technology & Software Engineering 1–5.

Keum, Changsup, Sungwon Kang, and Myungchul Kim. 2013. “Architecture-based testing of service-oriented applications in distributed systems.” Information and Software Technology 55 (7): 1212-1223. https://www.sciencedirect.com/science/article/pii/S095058491300013X.

Lam, Hua. 2004. “New design-to-test software strategies accelerate time-to-market.” IEEE/CPMT/SEMI 29th International Electronics Manufacturing Technology Symposium (IEEE Cat. No.04CH37585). San Jose: IEEE. 140-143. doi:10.1109/IEMT.2004.1321646.

Maier, Mark W, and Eberhardt Rechtin. 2009. The Art of Systems Architecting – 3rd Edition. New York: CRC Press. Accessed 09 10, 2019.

Maksi, L, S Berryman, A Brio, A Burkhardt, S Elder, and S Ferkau. 2021. Advanced Aspects of Engineering Research, May 7: 38–49.

Manas, Joe, and Beth Wilson. 2014. “Test Perspectives for Architecture.” 17th Annual Systems Engineering Conference. Springfield.

Manas, Joe, and Louisa Guise. 2013. “Systems Engineering for Test: Implementation of Test Strategy & Architecture.” 2013 IEEE International Systems Conference (SysCon). Orlando: IEEE. 597–602.

Mottu, J., P. Andre, M. Coutant, and T. Le Berre. 2019. “Shall We Test Service-Based Models or Generated Code?” ACM/IEEE 22nd International Conference on Model Driven Engineering Languages and Systems Companion (MODELS-C). Munich: IEEE. 493-502. doi:10.1109/MODELS-C.2019.00078.

Obis, Anastasia. 2025. Federal News Network. August 25. Accessed January 5, 2026. https://federalnewsnetwork.com/defense-news/2025/08/dod-dismantles-decades-old-jcids-in-joint-requirements-process-overhaul/.

Office of the Director of OT&E. 2024. “DOD INSTRUCTION 5000.98 OPERATIONAL TEST AND EVALUATION AND LIVE FIRE TEST AND EVALUATION.” Defense Acquisition University. December 9. Accessed January 20, 2026. https://www.esd.whs.mil/Portals/54/Documents/DD/issuances/dodi/500098p.PDF?ver=InJ_BDOOVcUHyurlNv_FAg%3d%3d.

OMG. 2022. “Information technology – Object Management Group Unified Architecture Framework (UAF) Part 1: Domain Metamodel (DMM), 1.1.” Object Management Group. April 3. Accessed January 20, 2026. https://www.omg.org/spec/UAF/ISO/19540-1/PDF.

OUSD(R&E). 2024. “Engineering of Defense Systems Guidebook.” Acquisition Guidebook. February. Accessed February 20, 2025. https://www.cto.mil/wp-content/uploads/2024/10/Eng-Def-Sys-Change2-7October2024-v3.pdf.

Paulisch, Frances, and Peter Zimmerer. 2016. “Collaboration of Software Architect and Test Architect Helps to Systematically Bridge Product Lifecycle Gap.” Proceedings of the 1st International Workshop on Bringing Architectural Design Thinking Into Developers’ Daily Activities (Bridge 2016). New York: ACM. 2–5.

Sivertson, Lisa. 2020. “A Socratic Approach to Optimizing Aerospace Manufacturing Costs.” International Journal of Systems Engineering 4 4 (1): 1–9.

Tekinerdogan, Bedir, Nour Ali, John Grundy, Ivan Mistrik, and Richard Soley. 2016. “Quality concerns in large-scale and complex software-intensive systems.” Chap. 1 in Software Quality Assurance, 1-17. Morgan Kaufmann. doi:10.1016/B978-0-12-802301-3.00001-6.

Thomas, F., J. Delatour, F. Terrier, and S. Gerard. 2008. “Towards a Framework for Explicit Platform-Based Transformations.” 11th IEEE International Symposium on Object and Component-Oriented Real-Time Distributed Computing (ISORC). Orlando: IEEE. 211-218. doi:10.1109/ISORC.2008.64.

U.S. GAO. 2020. “Technology Readiness Assessment Guide: Best Practices for Evaluating the Readiness of Technology.” U.S. Government Accountability Office Page. January. Accessed January 20, 2026. https://www.gao.gov/products/gao-20-48g.

University, Defense Acquisition. 2022. “DAU Instruction Link.” Instruction – Link – DoDI 5000.85 Major Capability Acquisition (MCA). August 2022.

Accessed February 7, 2025. https://www.dau.edu/cop/rqmt/documents/instruction-link-dodi-500085-major-capability-acquisition-mca.

USG. 2025. CPARS.GOV. November 3. Accessed January 20, 2026. https://www.cpars.gov/cparsweb/home.

—. 2025. FAR – Subpart 14.404 – Rejection of Bids. October 10. Accessed January 20, 2026. https://www.acquisition.gov/far/part-14#FAR_14_404.

—. 2025. Subpart 15.3 – Source Selection. October 1. Accessed December 10, 2025. https://www.acquisition.gov/far/subpart-15.3?searchTerms=Part%2015.3.
Zachman, John. 2008. Concise Definition of The Zachman Framework(TM). Zachman International.

Zimmerer, Peter. 2019. “Test Architects at Siemens: Motivation, Achievements, and Experiences.” International Workshop on Software Test Architecture. Xi’an: InSTA.