Australia’s Pentagon Wars Moment

The ‘Pentagon Wars’ is a period in the US Defense history named after an autobiographical book by Colonel Burton [1] and the movie it inspired starring Kelsey Grammer, Cary Elwes, Viola Davis, and John C. McGinley [2]. Colonel Burton describes a reform movement spanning about eight years ending in the mid-1980s, where there were significant conflicts in how Defense capabilities were acquired. The reform focused on the need for timely and realistic test and evaluation (T&E) to support all decision-making. An independent test agency known as the Director of Operational T&E (DOTE) was formed under the Office of Secretary of Defense (OSD) to help Congress regulate T&E in the Services, with a Presidentially-appointed Director, independent reports to Congress and oversight provisions, all codified in Title 10 Congressional Law [https://www.dote.osd.mil/About/Background/]. Australia has no equivalent to the independent test reform of the US DoD despite recommendations by the Australian National Audit Office (ANAO) starting in 2002 [3], and serious concerns by the ANAO again in 2011 [4] and 2015 [5] and the Australian Parliament in 2012 [6], 2016 [7] and 2023 [8]. An independent bill to legislate a Defence Capability Assurance Agency (DCAA) with comparable regulatory powers to the US DOTE has been under Parliamentary consideration for the last two years [9].

An Australian MRH-90 helicopter crash in 2023 killed four crew members during low-level, over-water formation exercises for special forces operations [10]:

Four Australian army aircrew members who were aboard a helicopter that crashed into the sea on Friday have now been declared dead, according to Australian Defence Minister Richard Marles on Monday. Efforts to rescue Captain Danniel Lyon, Lieutenant Maxwell Nugent, Warrant Officer Class Two Joseph Laycock and Corporal Alexander Naggs have now turned to an effort to recover their bodies. …The mission was a part of large-scale military drills involving the United States, known as Talisman Sabre. The drills are held every other year, alternating between the two countries, and also often involve other allied forces. …Friday’s crash is the second time this year the Australian Army has been forced to ditch an MRH-90 Taipan into the sea. Two injuries resulted from the previous incident in March, which was put down to engine failure. In 2019 the Australian government announced it would retire its entire fleet of Airbus-manufactured Taipans years ahead of schedule, describing the Taipans as a “project of concern for the last decade.”

A formal inquiry by the Inspector General of the Australian Defence Force (IGADF) heard testimony throughout 2024 of considerable disagreement amongst the test community about the circumstances for approval of the Helmet-Mounted Display Sight (HMSD) used on the fatal mission. Developmental T&E staff had vigorously opposed the use of that sight in these flight conditions due to the risk of disorientation leading to commanded flight into terrain. For example, testimony to the IGADF from the lead developmental test pilot concerning qualifying tests in 2019-20 states [11]:

We told our aircrew via the Standards Manual to operate in the full range of missions and role environments. That constitutes a lawful order. So we’re commanding our subordinate pilots to take unairworthy systems into an area where we know they’re unsafe. That’s what I would call reckless. (pp. 4047-4048)

Witnesses have alluded to pressure to accept the sights because they were only tested after purchase. Numerous US Title 10 regulations are likely to have provided opportunities to resolve the conflicts in this testimony. The early evidence from this Australian helicopter crash is diverse and challenging to interpret, however, coupled with the current DCAA Bill, Australia has finally had its inescapable ‘Pentagon Wars.’

The article first refreshes what was the US Pentagon Wars period and how this period reformed T&E (operational and live fire) as an instrument for informed decision-making in the US DoD. Second, it briefly outlines the different path that Australia’s acquisition undertook in its use of T&E and where many reviews over the last 25 years highlight the lost opportunity of such a divergence from Australia’s main Defence supplier and ally. Third, it covers enough parts of the lengthy Helicopter Crash Inquiry to illuminate the conflicts within T&E informing the accident. The intent here is not to ascribe blame, as the Inquiry is ongoing, but rather to highlight there is an immediate and important measure already before the Australian Parliament to address such conflict. The article discusses if, in the context of the Helicopter Crash Inquiry and the DCAA Bill, how Australia might have had its sovereign episode of Pentagon Wars. It concludes by arguing Australia might avoid such sovereign disasters and wastage by recognizing sooner the benefits of aligning all Australian acquisition test processes to a much larger, more advanced and more experienced US DoD. This recommendation applies across new fields like cybersecurity, AI-enabled systems, directed energy, space, and hypersonics.

The Pentagon Wars is a period in the US Defense history named after an autobiographical book by Colonel Burton [1] and the movie it inspired starring Kelsey Grammer, Cary Elwes, Viola Davis, and John C. McGinley [2].Colonel Burton describes a reform movement spanning about eight years ending in the mid-1980s, where there was significant conflict in the way Defense capabilities were acquired. One of his personal experiences was testing on the Bradley Fighting Vehicle where he was assigned as an Air Force Officer to a predominately Land and Maritime Program. Such cross-service measures were experimented with to help ensure independence and lessen the effect of careerism. Extraordinary measures are documented in the book and film to avoid significant and realistic testing in case it derailed a program that had become ‘too big to fail’ or what was later characterized as a megaproject [12]. The often-dangerous ramifications of this internecine political warfare for soldiers who might use the vehicle make for comedic relief in the movie. An example from the Movie and Book was a test firing of a penetration round into the aluminium armour to determine toxicity of the vapours caused. This scene is depicted in Figure 1 from the Movie [2] and in Colonel Burton’s words in the following quote [1].

Figure 1: Loading of Sheep into A Bradley Fighting Vehicle for a Vaporifics Test as depicted in the movie Pentagon Wars [2]

To make matters worse, the Army refused to conduct tests on the Bradley to see what would happen when it was hit by enemy fire—any kind of fire, even small arms. By the time of the 1980 decision to start production, no vulnerability tests of any kind had been conducted against a Bradley vehicle. A few small arms tests against pieces of Bradley armor had been performed, and the armor plates had failed the tests. This was one of the main issues debated at the production decision meeting. … As the Bradleys began rolling off the assembly line, no one had any idea what would happen in combat to the thousands of 18-year-old kids who would ride in these vehicles. The tragedy was compounded because no one seemed to care. …The British experiments showed that the aluminum armor caused more casualties than steel from burns, flash blindness, shrapnel wounds, blast overpressures, and toxic gases. …

The aluminum vaporifics experiments began in November 1983. BRL [Ballistics Research Laboratory] constructed a large metal box about the size of the Bradley troop compartment. The box was instrumented to measure all the terrible things that would happen inside. When a shaped-charge warhead penetrates armor, the metal particles vaporize and form a large fireball that fills the troop compartment, hence the term vaporifics. This fireball is larger and hotter and lasts about twice as long with aluminum as with steel armor. …By the spring of 1984, the vaporifics tests had moved into their second phase, which involved taking measurements inside an actual Bradley vehicle. But the vehicle was empty! I could not talk BRL into putting live ammunition and fuel inside to determine if the fireball and hot aluminum particles would start an ammunition or fuel fire. Vitali had agreed in writing to test for fuel fires, but he refused when it was time for the tests.

Now, the games really began. The tests suddenly began to lose their objectivity and realism. A series of events occurred that caused me to doubt that we would ever get the real answers. Without my knowledge or approval, Rumanian weapons with considerably smaller warheads were substituted for Soviet weapons. …Large doses of toxic gases were detected, larger than anticipated. BRL stopped taking measurements. The Army Surgeon General placed pigs and sheep inside the vehicle to test the effects of the choking fumes following a hit. Over my objection, the animals were sacrificed within minutes after each test. No tests for flash blindness were conducted, even though we had agreed there would be. No one was permitted to observe the autopsies, and no animals were allowed to live long enough to determine if complications developed from breathing the terrible fumes. The Surgeon General’s action officer reported that no serious aftereffects were noticed, but I did not believe him. I had conducted a literature search on all similar experiments on other vehicles in the past, and I knew that many animals had died a week after the tests as the result of complications in the respiratory tract. The Surgeon General’s officer was an admitted Bradley advocate and an extremely ambitious young man who was not going to produce any data that endangered the Bradley program. (Ch. 8)

The movie has been shown to a generation of aspiring senior officers in Staff Colleges. The usual lesson is the negativity of excessive competition and narrow focus, such as on one program to the exclusion of the overall Service good. Of course, the fundamental message is to be rational and thus led by testing. However, much of the deeper conflict portrayed in Pentagon Wars is about the loss of objectivity when developmental testing by a defensive acquisition program is not balanced objectively by broader operational testing. The distinction between these test types is described in the following quote from Colonel Burton’s book [1], because as the research article will later show it is at the heart of the conflict in Australia’s Helicopter Crash Inquiry.

There are basically two types of tests in the life of a weapons system, developmental tests and operational tests. They are distinct and separate, conducted by different agencies, and serve different purposes. Developmental tests are highly controlled, engineering-oriented tests designed to determine whether a new weapon meets technical requirements and formal contract specifications. The tests are conducted by the development community itself and are controlled by the program manager of the weapons system being developed. Program managers are rewarded for moving a new weapon through the developmental stage and into production. They are not rewarded for contributing to the demise of their programs. Therefore, developmental tests are usually oriented toward success. Operational tests are designed to stress the weapon in as realistic an operational environment as possible in order to determine its combat effectiveness. Operational tests are conducted by a testing agency within the service that is independent of the development community and the program manager. These tests are usually performed under combat like situations and are less engineering oriented than developmental tests. Operational tests usually occur near the end of the developmental phase, and the weapons are representative of those that will come off the production line and go into the inventory.

Colonel Burton’s book goes into much more depth on the battle within the US Congress to establish legislation to create an independent regulator for operational test oversight. This establishment is covered in the following quote [1] because it will help explain later Australia’s difficulties following the US to its current DCAA legislation.

In the spring and fall of 1982, Senator Pryor introduced legislation to create a director of operational testing as a coequal with the secretary’s chief developer. The new director would report directly to the secretary of defense on the adequacy of testing, as well as on the implications of test results related to production decisions. More importantly, the director also would report directly to Congress on these matters; the director’s views would not be edited by the secretary or anyone else. In other words, the director would work for two bosses, the secretary of defense and the Congress—a unique arrangement. Naturally, Senator Pryor’s proposed legislation was not well received in the Pentagon. If the chief tester could speak freely and often to the secretary and Congress, the real basis for decisions on new weapons would become apparent to all. The acquisition community could not afford to have this happen, so it fought the legislation tooth and nail. …

On 24 September 1983, the bill passed the Senate with a 91-5 vote. The companion House bill, as an amendment to the Department of Defense Fiscal 1984 Authorization Bill, passed by a unanimous voice vote. The fight was over. The reformers had won their first major victory. The chief tester would be independent of the chief developer and would have guaranteed access to all test data required from the services (rather than having to fight for them constantly). The operating budget of the chief tester would be determined by Congress …. The chief tester would … report test results directly to the secretary of defense and Congress simultaneously, without anyone editing them. (Ch. 7)

Australia’s T&E processes were audited by the ANAO in 2001-2002, where based on concerns within several major projects such as the Collins Class Submarines they observed [3]:

The ANAO considers that one practical option for strengthening Defence’s OT&E and for promoting a unified approach to T&E would be to establish an office in the Owner Support Executive, similar in concept to the US DOT&E. It would be responsible for common standards for, and independent oversight of, OT&E policy and processes, and assist in strategic management of OT&E. (p. 48)

The ANAO’s second major recommendation was therefore “… with a view to improving the strategic management of operational test and evaluation (OT&E), Defence assess the costs and benefits of establishing, in its Owner Support Executive, an office responsible for common standards for, and independent oversight of, OT&E.” The Defence response dismissed the recommendation [3] appealing as follows to the lesser scope of Australia’s future anticipated acquisition development when compared to the US:

Defence strongly disagrees with the ANAO’s suggestion that Defence establish an independent T&E office similar in concept to the US T&E. … This recommendation mimics the US model without consideration of Australian circumstances. With few exceptions, the Australian approach is geared to acquiring and integrating weapons and platforms developed by the US and other countries that have already completed rigorous T&E including OT&E. Defence incorporates the outcome of this testing in its OT&E programs where applicable. It then carries out the additional quantity of testing required to confirm that the Australian implementation meets agreed operational and support requirements for formal introduction into service. The amount of OT&E required is typically small by comparison with that required by the US. The higher levels of risk and cost involved in US Defense have justified the establishment of a large and very expensive independent organisation to carry out operational testing and evaluation. The scale of procurement in Australia would not warrant a similar relatively high level of investment.

The predilection described above for Australian acquisition to rely on commercial and military off-the-shelf (COTS & MOTS) with limited real testing and independent oversight would plague the next decade of acquisitions in Australia leading to repeated ANAO reports in 2011 and 2015 and Parliamentary Reports in 2012 and 2016 [4-7]. The ANAO returned to maritime acquisitions in 2011 finding as follows concerning T&E of COTS and MOTS acquisitions [4]:

When the acquisition involves military off‐the‐shelf (MOTS) or commercial off‐the‐shelf (COTS) equipment that needs no further development or integration into other systems, Defence may choose to rely on the vendor’s systems engineering processes. … However, the experience of JP 2070 Lightweight Torpedo Replacement project identifies that claims surrounding the development status of a product offered as off‐the‐shelf require verification to confirm that the product being offered is actually off‐the‐shelf. Additionally, where claims about the development status are verified, the method of integration with other platforms or systems also requires close consideration as this may introduce developmental risk to a project. … The acquisition of off‐the‐shelf equipment still needs to be accompanied by sufficient and appropriate test and evaluation data suitable for the Services to assess the equipment’s fitness for service, and its risk to personnel, public safety, and the environment. In all cases, it is fundamental that senior responsible officials managing the definition, acquisition and acceptance phases have clear visibility of the key decisions, assumptions and test and evaluation data that led the vendor to categorise the equipment as off‐the‐shelf. (p. 48, p. 148, p. 152)

In their 2011 report the ANAO continued to appeal to US acquisition guidance for closer cooperation in testing (i.e., pages 25, 69, 123 & 135). They found fundamental problems with the development and approval of basic T&E master plans across more than half of all maritime projects which would be a breach of several basic US T&E regulations. A major bipartisan Senate Inquiry into Australian Defence Procurement (2012) concluded the following regarding the T&E of COTS and MOTS capabilities [6]:

Proponents of OTS acquisition have highlighted that the selection of developmental products is a source of added complexity and greater uncertainty to an acquisition project, thereby increasing the risk of problems emerging during the procurement process. The committee observes that not only has this view led to the current preference for OTS acquisition expressed by many in government, Defence and the media, it may well have given rise to the increasing practice of manufacturers claiming that products are OTS when in fact they turn out to be developmental. Witnesses have presented numerous cases whereby the expectation that a procurement activity is OTS has led Defence to believe that a product is more mature or an outcome more predictable than experience (or an experienced review) would indicate. The conspiracy of optimism, referred to by a number of witnesses, appears to have led Defence to undervalue the role that developmental test and evaluation can play in the early stages of the acquisition cycle to identify and analyse risk in a quantifiable and defensible manner.(Ch. 12)

Whether the program optimism referred to is intentionally self-serving, as the Pentagon Wars [1, 2] would suggest, or naïve through poor processes, is largely immaterial to the case for independent operational T&E oversight of such decisions. The 2012 Senate Report concluded rather damningly the following [6]:

The committee finds that the current management structure in Defence has produced an organisation that lacks a robust risk regime: an organisation where its personnel are insensitive or unresponsive to risk, where no one owns risk and is incapable of learning lessons from past mistakes. … Throughout this report, the committee has referred to numerous instances of non-compliance with policy or guidelines. The gate review examples cited by the ANAO throw into sharp relief, how genuine, sound reforms can be rendered useless by a management structure that cannot or will not exert authority. … The challenge for Defence is to change an organisational structure with entrenched attitudes that despite repeated reforms:

• cannot learn lessons from past mistakes;
• still resorts to changes to process rather than implementing genuine organisational reforms designed to clarify responsibilities and make individuals accountable for their decisions and performance; …
• fails to understand and appreciate the importance of contestability and simply cannot, or refuses to, comprehend the meaning of ‘independent advice’; …
• undervalues technical advice and has serious shortcomings in technical analysis, critical to engineering based projects; particularly its downgrading of the importance of T&E; (Ch. 15)

That Senate Inquiry sought fundamental strengthening of policy, competence and oversight of T&E in Defence acquisition decisions but stopped short of a US-style external regulatory directorate for T&E.

The ANAO again audited Defence T&E in 2015 with particular concern about the slow progress of meeting Senate Inquiry recommendations for T&E competency. They chose to focus the audit on “T&E for Australia’s evolving amphibious deployment and sustainment capability—specifically: development and acceptance T&E for two Canberra Class Landing Helicopter Docks (LHDs); and operational T&E for the two LHDs, 47 MRH90 helicopters, and 12 LHD Landing Craft.” In a prescient warning for the MRH-90 incidents that followed, they found the following three conclusions that are relevant to this research article:

1) “The key lesson from the MRH90 acquisition and other recent OTS acquisitions, such as Land 121 and Land 125, is that OTS equipment should be subjected to T&E sufficient to allow the mitigation of cost, schedule and capability risks, including its ability to interface with the Fundamental Inputs to Capability.” (p. 43).

2) “In the case of the first Landing Helicopter Dock (LHD), HMAS [Her Majesty’s Australian Ship] Canberra, key management decisions were usefully informed by Defence’s T&E, which identified numerous defects and deficiencies for resolution. Defence decided, on balance, to accept HMAS Canberra on the understanding that the deficiencies would be addressed during the ship’s operational phase. In doing so, the Chief of Navy accepted greater risks than would have been the case had System Acceptance been based on more complete objective quality evidence of compliance with contracted specifications, and had Initial Materiel Release been based on less qualified findings by Defence’s regulators concerning compliance with technical, operational and safety management system requirements.”

3) “… this performance audit has highlighted the inherent challenges in Defence’s entity-level management and conduct of T&E, which remains distributed across 12 internal organisations. Scope remains to improve key aspects of Defence’s administration of T&E, specifically:

• assuring consistency in the conduct of T&E and compliance with whole of entity T&E requirements;
• establishing entity-level performance measures for T&E; and
• T&E personnel competency management.”

On 3 March 2016, the Joint Committee for Parliamentary Accounts and Audit (JCPAA) held a special hearing and subsequent report [7] concerning the 2015 ANAO report on T&E [5] and another audit report where T&E was of particular concern being Army’s hardened heavy vehicles [13]. Clearly Parliament was increasing its oversight of Defence decision making with an increasing frustration about T&E competence and oversight.

The second of the ANAO audit findings listed above concerning the Canberra Class ships is of significance to the later testimony from the Helicopter Crash Inquiry. There was a growing focus in Australia and the US around 2014-15 on the tendency for projects to proceed to operational testing regardless of whether developmental test findings had been resolved. The US DOTE report to Congress in 2014 provided a survey, shown in Figure 2, to illustrate this tendency.

Figure 2: 2014 US DOTE Annual Report to Congress – DOTE Activity and Oversight – Survey results of problem discovery during OT&E [14]

In the US the DOTE oversight of this phenomenon, particularly the forward look at the 2015 programs (Figure 2, right-hand side), led to several large programs like the Joint Strike Fighter and Littoral Combat Ship focusing some assembling OT&E resources into resolving DT&E deficiencies (i.e., see DOTE Annual Reports for FY 2015). One further difference between Australian and US acquisition that dramatically shapes T&E is when production is approved by Government. The US acquisition process has an additional Congressional approval known as Milestone C when compared to Australia’s Second Pass approval. The Title 10 Section 4171 (formerly 2399) requirement to perform an operational assessment before moving beyond Low-Rate Initial Production (LRIP) constrains overinvestment and some of the ‘jobs in States’ that can politicise acquisition before objectively deciding operational merit. Australia generally provides Defence its approval to enter a development and production contract at a Second Pass approval, without needing to return for production approval after an operational assessment. In the author’s experience, this freedom can be abused, whereby project offices focus on purchasing the COTS or MOTS equipment before conducting early operational assessment. The decision is usually not informed by whether prior certification was for the same Configuration, Role (i.e., mission) and Environment (CRE). Much of the risk of OTS acquisitions are in different use, such as special forces missions rather than conventional missions, different environments such as the unique wet-hot (B2) climate of Australia’s North (STANAG 2895), and configuration differences forced by factors like national release constraints, different power and spectrum, or simply production differences over the time since foreign certification and the first Australian testing.

After 2015, Australian Defence underwent a major organizational reform based upon a First Principles Review (FPR) [15]. While the FPR was well-intentioned to reform Defence acquisition in areas highlighted earlier like contestability, it was silent on who, or how, T&E would be managed. Two agencies with central control of T&E were disbanded and reformed during the FPR-implementation, leaving the Services to individually protect, and in some instances resurrect their T&E (i.e., Army). Remnants of the central Australian Defence T&E Office moved to the Vice Chief of Defence Group. The first reconstruction of formal T&E policy in the new organisation was not issued for another four years. During those years central policy authority for Defence T&E was carried by individual Services, including the period of the contentious MRH-90 HMSD testing in 2019. In 2020, the current Defence T&E Manual (DTEM) [16] was issued alongside a new T&E Strategy [17] with similar objective themes to a T&E Roadmap of 12 years earlier [18]. While the Government continued to work with Defence on the T&E strategy, further ANAO reports on Defence projects led many Australian Parliamentarians determinedly towards a legislative solution. That legislative solution is covered in the next section and has distinct similarities to Colonel Burton’s Chapter Seven descriptions from forty years earlier, albeit without the TV appearances he describes [1].

A legislative solution to Australia’s lack of independent oversight for its Defence Capability decisions was tabled by Senator Fawcett in Australia’s Senate on 10 May 2023 as a private member’s bill [8]. Senator Fawcett was a former Army flight test pilot and Commanding Officer of the Royal Australian Air Force’s Aircraft Research and Development Unit. He had a significant personal role in T&E of helicopter acquisitions before retiring and was a member of the Senate Inquiry into Defence Procurement in 2012 [6]. The bill was referred to a Senate Inquiry which summarized the intent of the bill as follows in its 2023 report [8]:

1.13      A central component of the bill is the establishment of an independent statutory body, the Defence Capability Assurance Agency (DCAA), charged with responsibility for assessing the complex risk associated with materiel procurement and sustainment. Risk assessment would include but not be limited to technical risks pertaining to performance and certification. The explanatory memorandum states that this would ensure a high degree of confidence in the veracity and completeness of information relied upon to make risk-informed decisions.

The Senate Inquiry Report was divided on party lines [8] yet the Bill was referred by the Australian Senate to the House of Representatives and read a first time on 12 February 2024. Some key points from the Senate Inquiry Report follows:

1.8       Defence has itself acknowledged that it has ‘sometimes failed’ to live up to its commitment to ensuring adequate T&E is performed in the context of capability acquisition and sustainment decisions.

1.9       The perspectives of Australia’s AUKUS partners, the United States of America (U.S.) and the United Kingdom of Great Britain and Northern Ireland (U.K.), on capability assurance have evolved differently to those in Australia, with public accountability resulting in independent oversight of capability assurance mechanisms still lacking in Australia.

1.10     The explanatory memorandum states that the bill seeks to ‘address the root cause of past failures’ and ensure that Defence acquisition and sustainment decisions are well-informed by T&E based on four core principles:

Independence. The risk identification function must be independent so that assessment is made without bias or influence (intended or unintended). Independence also ensures that the assessor of risk has a voice (NB not a veto) that is heard at each decision-making level of the capability life cycle.

Task-specific competence. Government must ensure staff with the right skills are employed to identify and manage risk. Competence is a matrix of qualifications and experience that are directly relevant to the task at hand.

Transparency. Previous inquiries highlight that risk assessors working within Defence face various barriers (individual or organisational) that influence whether decision-makers actually consider their assessments. Given the costs and national security implications, the taxpayer deserves to know that decisions are being made on the basis of accurate understanding of risk.

Accountability. The DCAA will be underpinned by an audited and enforceable requirement that Defence engages the Agency to evaluate risk across the capability life cycle. DCAA reports are to be specifically included in briefs provided to project managers, assurance bodies, Defence Investment Committee and the National Security Committee of Cabinet.

The DCAA Bill remains in the backlog of legislation to be considered by this Parliament and even if the Parliament is disbanded, has the prospect of being tabled again in future parliaments. Now that the background of Pentagon Wars and Australia’s T&E has been covered, the research article returns to the recent testimony of T&E staff at the Helicopter Crash Inquiry to illustrate common themes.

The review of the testimony for this research was focused by the news article of 18 October 2024 [19], titled ‘Retired Australian Defence Force test pilot apologises to families of airmen killed in Taipan helicopter at inquiry in Brisbane.’ Four main testimonies from just five days (7-9 August, 17-18 October) were reviewed from more than thirty days of hearings held throughout the year [11]. As such, a major limitation of this research article is it cannot be used to infer any definitive cause for the accident, rather it can only reasonably infer some significant disagreement by T&E staff responsible for aspects of the airworthiness and service release testing of the helicopter, mainly the HMSD. The testimony reviewed was by the three qualified test pilots who tested the system in its developmental T&E, Major David Lamb, Major Ian Wilson (retired) and Lieutenant Colonel (LTCOL) Brendan Reinhardt, plus the Head of the Defence Aviation Safety Authority (DASA), Air Commodore (AIRCDRE) Joseph Medved. All four of the testifying staff had over thirty years of experience in their fields.

Significantly, this research did not review the testimony of those responsible for the Operational Evaluation (OPEVAL) criticized extensively by the Developmental Test pilots; their testimony is either still not available or they are not testifying.

Developmental testing of the TopOwl HMSD sight and a new symbology known as version 5.10 were flight tested as a major change in two flights between seventh and eleventh of June 2019. The change introduced a ‘distance to go’ marker in the HMSD that was previously only available in the Primary Flight Display. As a change to the HMSD is a critical flight system, the testing was deemed a Category Two flight test, requiring a flight test qualified unit and personnel. The flight test was flown by Army Aviation T&E Section (AATES), with the lead test pilot being Major Wilson and the Flight Test Engineer being Captain Jordan Zahra. The results were peer reviewed by Major David Lamb as another test pilot qualified on the MRH-90 helicopter and by the Commanding Officer of the section, LTCOL Reinhardt. The flight tests were conducted in daytime only and they quickly disclosed a misleading attitude issue with the HMSD whereby if the pilot looked straight ahead, correct pitch and roll would be displayed but if the pilot looked off-axis, false pitch and roll would be displayed. Flight testing did not proceed to night testing as there was a risk of causing accidents and AATES deemed that the incorrect display required the acquisition agency to obtain an explanation from the manufacturer. A report was briefed and sent assessing the change as ‘unacceptable’ which is the risk rating with the highest likelihood to impact safety or capability.

Testimony about actions after the developmental testing covered interactions with the acquisition project and the standards office of the Army Aviation Headquarters. These actions include discussions about foreign certifications that lacked detail, confirmation that the system conforms to its technical specification but that the specification allows incorrect information, and questioning by external parties of the seriousness of AATES’s developmental flight test finding. In an attempt to further investigate the matter AATES facilitated a special flight permit from DASA to allow an operational evaluation to occur involving different pilots and more flights, restricting  the minimum illumination to be present during the operational testing. After the OPEVAL, a service release occurred with no flight restrictions but with two cautions in the event of an Unusual Attitude (UA); effectively overwriting the AATES concerns and preventing AATES from any further involvement in the decision. The first caution required pilots to look directly ahead if they become disoriented before deciding what evasive maneuver, if any, to perform. It is described in LTCOL Reinhardt’s examination as follows [11]:

“So there are two elements to that warning. They probably should be broken out separately. We have discussed initially that they are saying that if you – okay, a UA is an unusual attitude. So you have mis-assessed the aircraft state, and it has extremes of pitch or 30 roll, or excessive airspeed, or rates of descent or climb, right. It’s not where it’s meant to be. It is unusual. And if you do not recover from an unusual attitude, you’re going to lose control of the aircraft, or fly into the ground before you lose control of the aircraft. So this is an emergency that we train for, and there’s drills and that. So they’re saying here, I guess, that if you – you could enter a UA because of ambiguous symbology information. And they’re saying that if you do identify you’re in an unusual attitude, you are to discipline yourself coolly and calmly in this unusual situation to look forward and then apply the correct – look at the correct attitude information now, and recover from that.” (p. 2036)

The second caution is a little more complex and is again explained by LTCOL Reinhardt and his legal examiner:

There is a second part to the warning which talks about, alternatively, if you can’t discipline yourself, and you’re confused and you don’t know what’s going on, you could use the AFCS go-around mode for an automated unusual attitude recovery. So I have written another document, which is – I sent my AATES MRH-90 flight test pilots to go out and determine if this AFCS go-around function could recover from a UA. This was completely within the flight envelope of the aircraft, and they were appropriately qualified and trained, being experienced QFIs, to do this. So they put the aircraft in a number of unusual attitudes, and pushed the go-around button. So the go-around button is designed when you’re on approach, if you muck up the approach to a hover landing, you can press go-around, and it will automatically apply power and put the aircraft in a climb attitude, and safely fly you away from a mishandled landing. So we put them – or they put the aircraft in representative speeds and attitudes, and then pressed the go-around button. This is probably not where the aircraft is designed to be – this is not where the go-around button is designed to be used, and we experienced the go-around function could not correct the aircraft attitude and we got into situations where we were approaching vortex ring states. We had excessive rates of descent. And we did not agree with that recommendation at all, and I articulated that to Avn Command.

MAJ CHAPMAN [Lawyer]: So in a low-level environment, would that shortfall be more exacerbated?

LTCOL REINHARDT: Yes. You could – if you were flying – so lift aircraft can fly down to 50 feet. We generated high rates of descent that, you know, would’ve meant impact with the ground. You’re already confused, you’re already in an unusual attitude, and in some flight parameters where the go-around function was not designed to be used, pushing that button just made things worse.

MS McMURDO: [Board Member] So the crash we’re investigating, we know the helicopter was number 3 in a four-helicopter formation, doing a left bank turn in showery conditions at night, possible long-term fatigue involved, stress. Do all those factors mean that it would become more difficult for a pilot to heed that warning?

LTCOL REINHARDT: Yes. (pp. 2036-2037)

As seen in the excerpt of the transcripts, the AATES formally objected to the service release. The opposition was not only about the inadequacy of the cautions but how all the restrictions that he used to get the flight permit for the OPEVAL were not imposed on the subsequent release. An example of such a restriction was on the minimum illumination for safe HMSD use.

Much of the cross-examination of the three test pilots centered on the impact of flying formation in low visibility where the pilots must monitor one helicopter ahead and off-axis (i.e. staggered formation) where the HMSD displayed incorrect orientation. In these dark conditions, witnesses noted looking directly ahead for correct orientation would risk losing formation and terminating the mission. Some cross examination sought to understand if there was a regulatory obligation to return to developmental flight test to establish the safe operating limits in the presence of the incorrect display of critical flight information. Testimony concerning the limitations of the development flight test established that the report approvers were focused on getting acquisition clarification in the strongest terms concerning the incorrect display before proceeding; that is, to influence redesign rather than focus operational testing or service release. However, it appears that in a MOTS and COTS acquisition such redesign was not an option and what would have been the ‘due test process’ in US T&E regulations, was not followed in Australia. Two good summaries of LTCOL Reinhardt’s opposition to the service release are as follows:

MAJ CHAPMAN: So just to clarify, you were not formally invited to respond to the OPEVAL report. You initiated your own response out of a concern for safety. You sent that response and you received no formal response back from the addressees?

LTCOL REINHARDT: I don’t believe so. (p. 2039) …

AVM [Air Vice Marshal, Board Member] HARLAND: So to follow on from that, would you characterise that as unusual, that an aircraft would fly around with a mismatch between attitude displays?

LTCOL REINHARDT: Yes. (p. 2055)

Major Wilson also formally opposed the service release of the HMSD, including bypassing the chain of command and seeking intervention from another Service. Such actions illustrate significant cultural tension around what is appropriate T&E, illustrated earlier in the introduction where Major Wilson supported his use of the adjective ‘reckless.’ One other aspect of Major Wilson’s testimony under cross examination illustrated the tension again on a related aspect, that of safely extracting the pilot from an underwater helicopter crash. It is offered here to illustrate that the T&E tensions were on multiple aspects.

LCDR GRACIE: Yes, thank you. You talked about your own experience in a loss of situational awareness. I won’t go back over that. But my understanding is that you have also participated in some AATES testing with respect to the egress from the pilot seat during a rollover, or underwater rollover?

MR WILSON: Yes.

LCDR GRACIE: What can you say about what AATES did there in relation to the sort of processes that you’ve undertaken with the TopOwl? Was there a full test report?

MR WILSON: Yes, there was.

MR WILSON: Okay. The underwater egress had not been tested with the TopOwl system. That gave rise to a number of – or across the aircrew community, a number of different ideas about what was the best way to egress the aircraft. AATES was given the task to assess all these ideas. To do that, we had to go and use the – there’s a full-scale cockpit lockup which we use in Townsville. That’s lowered on a crane into a pool, rotated upside down. As the test body myself, I had the full aircrew ensemble on with ballistic protection, survival vest, a TopOwl simulator that we had commissioned ourselves which is printed up and attached to the aircraft but, importantly, using the two cables. We tested all the proposed methods of egress. As I was in the water upside down, we’d start taking the helmet off and we would truncate testing at the point where we identified there was a significant snag hazard. The safety team would remove the snag hazard and then I would egress, and we would go and repeat the next sequence. … I egressed the aircraft successful, was swimming to the surface and one of the cables became snagged. … I kept struggling and eventually the snag broke free and I got to the surface. I discussed that then with the test engineer, or we decided that was probably not a good thing. The next serial was to test the inclusion of a fusible link in the cables between the TopOwl and how they’re connected on to the aircraft. … We went back in and tested that as a proof of concept because we didn’t actually have that breakaway link. What we achieved then was what I would call a system which was demonstratable. It was repeatable and, most importantly, it would be repeatable by others as a means of egressing, reliably egressing the aircraft without implying a snag hazard. We put the report together. … I made that point to highlight that we need systems in place to, as I said, allow a reliable and repeatable means of getting out of the aircraft underwater. The report was tabled, it stood, and, to my knowledge, no treatments have been put in place by Defence until –at all. (pp. 4029-4030)

In addition to this developmental egress testing not realising changes in configuration, later testimony focused on the fact that aircrew training for such egress used an unrepresentative caving helmet and no TopOwl sight. When asked later by counsel for the deceased pilot, “If they survived the crash and were conscious, was there a reduced chance of them being able to safely egress that aircraft?”, Major Wilson replied “Yes” (pp. 4031-32).

One last testimony of relevance is from Air Commodore Joseph Medved who is the Head of the DASA. Even though LTCOL Reinhardt was not the service release authority for the revised HMSD (only the developmental test authority), some of the cross examination of him concerned the responsibility of Army’s Aviation Command to appropriately assess risks to the Workplace Health and Safety (WHS) Act. The Inquiry clearly sought to understand what were legislative responsibilities in this key decision. Air Commodore Medved clarifies this responsibility in Australia as follows:

I’ll provide some context – and it’s clearly annotated in my statement – but DASA is not a legislative regulator. …However, we do model ourselves on Australian government advice to government regulators that are legislated … So in civil aviation there’s a Civil Aviation Act. And under international conventions for aviation there’s a separation of civil aviation from state-based aviation. And, in Australian context, that’s Military Aviation. So we model ourselves on civil practices, standards and recommended practices but tailored, clearly, within a military context. Defence is subject to the WHS Act and because the WHS Act was never designed to account for Aviation safety, because it does not apply in civil aviation at all, there is no applicable code of practice, for example, for Aviation in the WHS Act. So given that Defence is subject to WHS Act we have set up the framework, the Defence Aviation Safety Framework, to amplify our obligations under the WHS Act. (p. 2244, p. 2254)

The importance of the above tortuous legislative approach to aircraft safety is germane to whether the T&E underpinning such decisions should at least have a legislated regulatory basis, per the currently proposed DCAA Bill. This and the other implications for T&E in Australia will now be discussed.

Many of the Pentagon Wars themes are evident in the testimony concerning conduct of the MRH-90 HMSD testing. Much of the Pentagon Wars accounts concern limiting development test so as to proceed to operational testing, often due to excess expectations about milestones. In the MRH-90 testing, there was a significant ADF commitment to build flight test competency and experience in the test pilots tasked which meant developmental testing quickly recognized an inappropriate display and vigorously opposed it proceeding beyond limited operational evaluation. Examples of Defense programs overriding such T&E objections were noted consistently in Pentagon Wars accounts, in Australian audits and inquiries since then, and as a principal reason for the pending DCAA Bill.

In a prescient warning to the MRH-90 T&E testimony, Senator Shoebridge said the following in his speech to the Senate last year commending the DCAA Bill for another year of consideration (Hansard, 7 December 2023, p. 6799):

When you look at the sorry tale of Defence procurement over the last two or three decades at least—a chunk of that lies in the coalition’s basket of responsibility and another chunk of it lies in Labor’s basket of responsibility—what we’ve seen has been a kind of collective miasma in the parliament when it comes to Defence procurement. It has been a case of not looking, not checking and letting Defence just grind its way through, as it does. We had some pretty disturbing evidence in the course of this inquiry about what’s actually going on at the moment, particularly on test and evaluation. I’ll sort of summarise the broad nature of that evidence. There are elements within Defence that have clear expertise and responsibility for test and evaluation. They are staffed by people who know their stuff and, when they are called upon, they provide competent, independent advice. Many of them are understaffed and face morale pressures. They are facing structural challenges. But there are parts within the Defence establishment that have skills and competencies that you’d want to draw upon in this.

But we heard time and time again that, when they came up with an inconvenient conclusion, Defence found three, four, five or six ways to go around their conclusion and to just drive through the project that they had already decided on. No system has integrity when the system is so opaque and so riddled with political, promotional and vested interests that, regardless of what the independent subject matter experts say, projects seem to get a life of their own and get delivered—or not delivered, as the case may be.

Senator Shoebridge then covers an example provided as televised testimony to the Senate Inquiry on the DCAA [8], concerning a different helicopter acquisition from 2014. That example also dealt with attempts to avoid the most basic of flight test by a test pilot with over 30 years and 4000 flying hours of helicopter test experience. The battle for adequate T&E is in sharp relief when safety is concerned, however, from that example adequate T&E is equally needed to achieve sound decisions for value for money and to avoid waste. Well-planned test activities build consensus and get necessary design changes and capability happening in ways that paper-based argument usually does not. Just like the sporting arena though, you have to have respect for the umpire and rules. Therein, Australian Defence is at a distinct disadvantage compared to its main allies.

Senator Shoebridge summarises the theme of this research article nicely by concluding his speech, once again, on how Australia could do better by following the US lead on T&E.

It’s a demonstration of the lack of independence and what’s wrong with the system. I’ve got to tell you, that evidence is a tiny part of what the committee actually heard. We heard evidence in camera [not televised] that puts that in the shade. Nobody, I would have thought, could have fairly sat through that evidence and come to the conclusion that the current pathway for ADF procurement is anything like ‘on track’. … The issue is trying to come up with an independent, credible procurement process which involves independent tests and evaluation so that people serving in the Defence Force have equipment that meets the needs of our Defence Force and our national security without bankrupting taxpayers in the process. …

What is extra frustrating about it is that the bill largely seeks to replicate what’s happening in the United States and, to a degree, the United Kingdom. It’s not like this was just created in a thought bubble of Senator Fawcett’s mind. It’s based on existing international practice where independent agencies—who aren’t perfect—in other jurisdictions add a huge amount of transparency and accountability to their procurement. What’s so special about Australia that we don’t need an independent agency doing it?

It is important to caution that the examples used were for flight test, where there is substantial history in providing highly qualified test pilots with year-long postgraduate qualifications in test. Many reviews, like the Senate Inquiry into Defence Procurement [6] have decried the lack of equivalent qualification for the T&E of ships, vehicles and some weapons. Modern capabilities are as complicated to test as aircraft were half a century ago when aircraft developers first committed to that year-long T&E education standard, and yet almost no non-flight-test equivalents exist in the US and none in Australia. The US T&E regulators have competency standards they can use to implement such change (e.g., with the US Defence Acquisition University, etc.), however, in Australia such provisions depend, at least independently, on the proposed DCAA Bill.

Like the US, Australia faces some impressive technological and operational challenges where legislating for public safety will challenge Defence to operate safely. In his testimony, Air Commodore Medved outlined that the current Defence approach to deal with how safety is established and maintained for aircraft is an internal process, different to civil aviation safety. It appears Australian Defence are taking a similar approach with safety of military space systems [20] and they have also avoided inclusion in the Home Affairs cybersecurity requirements for critical infrastructure [21]. In contrast to these technologies Australia passed legislation on 10 October 2024  for an external regulator for nuclear propulsion safety [22]. Further, there appears to be close cooperation with the US and UK on hypersonic testing [23] where US funding invokes Congressional oversight and their T&E regulation. There are many areas of technology where Australian Defence’s reputation for self-governance of T&E may be out of touch with its ambitions, such as robotic autonomous systems, space systems, AI-enabled systems, and directed-energy weapons. As a minimum, regulation of the T&E that informs such decision-making per the DCAA Bill makes efficient and eminent sense. Such T&E regulation is part of the fabric of assurance the US has used now for forty years to assure its Congress and public that Defense can be trusted to be safe and reliable with technology.

The US Defense Pentagon Wars period of forty years ago illustrates difficulties securing sound and respected T&E for acquisition programs, and explains the US Congress’s creation of an independent operational and live fire T&E regulatory and oversight office. That office has helped redress such behaviours in the US. However, Australian Defence has repeatedly opposed and resisted such a change in Australia, even in the face of strong auditing and inquiry pressure concerning the inadequate importance and effectiveness of T&E. Despite Defence opposition, the Australian Parliament has been considering a bill to create a Defence Capability Assurance Agency that would legislate a fundamentally equivalent regulation and oversight of Australian Defence T&E. This research article reports a limited review of testimony by T&E staff to the MRH-90 Helicopter Crash Inquiry in Australia to understand what parallels there were to Pentagon Wars. The conflict surrounding T&E reported to the Inquiry is substantial and dishearteningly similar to the US experience forty years ago. The advice of highly qualified test pilots was obviated. Moreover, decades of Australian audit reports and inquiries show such conflict around T&E was entirely predictable and frequently common. The current Bill to legislate some independent regulation and oversight in Australian Defence capability is ready to implement fundamental change to address such conflict. The measures in that Bill will provide greater assurance across a wide range of new technologies like cybersecurity, space, hypersonics and directed energy weapons to ensure at least the T&E underpinning internal Defence safety processes are independently accounted and reported. Furthermore, that Bill provides the mechanisms in Australia to address competency of T&E staff beyond the traditional one-year top-end of flight test into the equally complicated weapon systems that now pervade all domains.

[1]        J. G. Burton, The Pentagon wars: Reformers challenge the old guard. Naval Institute Press, 2014.

[2]        R. Benjamin, “The Pentagon Wars,” ed. United States: HBO, 1998, p. 1 h 43 min.

[3]        (2002). ANAO Audit Report No. 30: Test and Evaluation of Major Defence Equipment Acquisitions.

[4]        (2011). Audit Report No.57: Acceptance into Service of Navy Capability.

[5]        (2015). ANAO Report No 9: Test and Evaluation of Major Defence Equipment Acquisitions.

[6]        Australian Senate, “Senate inquiry into defence procurement,” Canberra: Australian Parliament House, 2012.

[7]        (2016). Report 456: Defence Major Equipment Procurement and Evaluation, and Great Barrier Reef Regulation. Available: https://www.aph.gov.au/Parliamentary_Business/Committees/Joint/Public_Accounts_and_Audit/

[8]        (2023). Senate Inquiry Report: Defence Capability Assurance and Oversight Bill 2023. Available: https://www.aph.gov.au/Parliamentary_Business/Committees/Senate/Foreign_Affairs_Defence_and_Trade/CapabilityAssuranceBill23/Report

[9]        Defence Capability Assurance and Oversight Bill 2024, P. N. 47, 2024.

[10]      A. Watson and M. Tawfeeq, “Four Australian army aircrew dead after helicopter crash during joint wargames with US,” in CNN, ed. Online: CNN, 2023.

[11]      “INSPECTOR-GENERAL AUSTRALIAN DEFENCE FORCE INQUIRY INTO THE CRASH OF A MRH-90 TAIPAN HELICOPTER IN WATERS NEAR LINDEMAN ISLAND ON 28 JULY 2023,” in Inspector-General of the Australian Defence Force, The Brisbane Convention & Exhibition Centre, Plaza Level Room P11, Merivale Street, South Brisbane QLD 4101 ed. Online: Australian Government, Defence, 2024.

[12]      B. Flyvbjerg, N. Bruzelius, and W. Rothengatter, Megaprojects and risk: An anatomy of ambition. Cambridge University Press, 2003.

[13]      (2015). Report No.52 2014–15, ANAO Report No.52 2014–15: Performance Audit Australian Defence Force’s Medium and Heavy Vehicle Fleet Replacement (Land 121 Phase 3B). Available: https://www.anao.gov.au/work/performance-audit/australian-defence-forces-medium-and-heavy-vehicle-fleet-replacement-land-121

[14]      (2015). DOT&E 2014 Annual Report to Congress. Available: https://www.dote.osd.mil/Annual-Reports/2014-Annual-Report/

[15]      D. Peever, R. Hill, P. Leahy, J. McDowell, and L. Tanner, “First principles review: Creating one defence,” Commonwealth of Australia: Canberra, 2015.

[16]      (2021, December). Defence Test and Evaluation Manual (DTEM).

[17]      (2021). Defence Test and Evaluation (T&E) Strategy. Available: https://www.defence.gov.au/about/strategic-planning/defence-test-and-evaluation-strategy

[18]      (2008). Defence Test and Evaluation Roadmap.

[19]      L. Gwynn and L. Lavelle, “Retired Australian Defence Force test pilot apologises to families of airmen killed in Taipan helicopter at inquiry in Brisbane,” in ABC News, ed. Online: Australian Broadcast Commission, 2024.

[20]      Anonymous, “Inaugural satellite safety regulations to be released in 2025,” in Defence Connect, ed. Online: Momentum Media, 2024.

[21]      (2024). Introduction of landmark Cyber Security Legislation Package. Available: https://www.homeaffairs.gov.au/news-media/archive/article?itemId=1247

[22]      (2024). Australian Naval Nuclear Power Safety Bill 2024. Available: https://www.aph.gov.au/Parliamentary_Business/Bills_Legislation/Bills_Search_Results/Result?bId=r7104

[23]      (2024). AUKUS Partners Sign Landmark Hypersonics Agreement. Available: https://www.defense.gov/News/Releases/Release/Article/3966986/aukus-partners-sign-landmark-hypersonics-agreement/