There is a persistent misunderstanding about what DOT&E (Director, Operational Test and Evaluation), the Pentagon’s weapon and system test group, is and does. As incredulous and unbelievable as it seems, there are actually some who view DOT&E as an obstacle or impediment to fielding weapon systems. This can only stem from a complete failure to understand what DOT&E is and does. I’ll now make clear what they do.
Let’s start with what DOT&E does not do.
- They do not approve or disapprove projects, weapons, or systems. If they did, the F-35 and LCS would have been cancelled long ago!
- They do not set criteria or specifications for weapon systems.
- They do not decide when a system can be fielded. The Navy routinely fields systems that have not been completely or successfully evaluated by DOT&E.
Now, what do they do?
At its simplest, DOT&E takes specifications that are established and provided by the Navy and implements test programs to verify that the system under consideration meets those specifications.
This is important enough to say again. DOT&E takes specifications that are established and provided by the Navy and simply verifies that the system meets those specifications.
Some people believe that DOT&E mandates unnecessary tests and becomes an obstacle to progress. Nothing could be further from the truth.
How does DOT&E determine what tests are needed and how many? DOT&E testing, as in industry in general, is based on statistics and experimental design. There is an entire field of mathematics devoted to figuring out how to set up experiments to obtain the maximum amount of data from the minimum amount of test runs. I won’t even attempt to describe how this is done. Suffice it to say that this is pure mathematics. There is no personal opinion involved. The experimental design does not care what the designer thinks of the system being tested. In other words, DOT&E’s opinon, if they even had one about a given system, does not enter into the design of the experiment and has no influence on the type or number of tests required. See reference (1) for a brief discussion of statistical experimental design.
The biggest obstacle to rapid assessments is the Navy. The Navy routinely declines to make ships, aircraft, and systems available for timely testing. Consider the Navy’s habit of putting off shock testing. If the Navy would simply perform the shock testing with the first of class at the earliest opportunity, the testing would be over and the ship could get on with its service life and deployments. Instead, for reasons unfathomable to me, the Navy spends more time fighting with DOT&E and trying to delay testing than the time it would take to simply do the testing. The only explanation I can come up with for the Navy’s reluctance to perform shock testing is that recent ship designs are so structurally weak and poorly designed that the testing will reveal significant weaknesses and problems which will, rightly, make the Navy look bad, so they try to avoid the testing.
Further, the Navy routinely fails to fund the necessary threat surrogates that would conclusively verify weapon and system performance. The Navy will spend several billion dollars on buying a ship but refuses to spend a hundred thousand dollars or a million dollars on simple, realistic threat surrogates to determine whether that ship can actually perform its duties effectively. I think, by now, you can reason out the why of the Navy's reluctance to perform realistic testing.
If the Navy had their way, no testing would have been done on the LCS or its modules and we’ve seen how that turned out. DOT&E is the only thing standing between badly flawed weapons and sailor’s lives.
The Navy’s adversarial relationship with DOT&E is inexplicable. The Navy should be the strongest supporter of DOT&E since DOT&E is the group that ensures that the Navy will get what it wants in the way of weapon and system performance. Indeed, the Navy’s history of being cheated and shorted by fraudulent manufacturer’s claims should be all the evidence needed for the Navy to be DOT&E’s most enthusiastic supporter. Sadly, this is not the case. The Navy’s reluctance to fund proper testing and perform said testing borders on criminal negligence and incompetence. The next time we enter high end combat we’re going to look fondly back on the faulty torpedo episode of WWII compared to the failings we’ll find in today’s systems due to the Navy’s unwillingness to embrace proper testing. Many sailors will pay with their lives for the Navy’s gross negligence.
Brief Background: DOT&E was established by an act of Congress in 1983. The organization reports directly to the Secretary of Defense which keeps it out of the chain of command of the services, thus ensuring its independence. The current Director, Dr. J. Michael Gilmore has served since 2009. Dr. Gilmore and his group have been nothing short of spectacular in their performance and are the saving grace of a badly flawed military acquisition process. Countless service members undoubtedly owe their lives to DOT&E.