Showing posts with label Testing. Show all posts
Showing posts with label Testing. Show all posts

Sunday, March 30, 2025

Hypersonic Intercept … Well, Not Really

ComNavOps never ceases to be amazed at the deceptive spin (I’ll refrain from using the word fraud, in this case) put out by manufacturers, the Navy, and complicit ‘news’ sources.  As you know, the ability of defensive systems to intercept hypersonic attacking missiles is questionable.  Well here’s a headline from a Naval News website article that sounds like a piece of great news:
 
Aegis Combat System Demonstrates System’s Capability to Counter Hypersonic Threats[1]
 
A Burke class destroyer, USS Pinckney (DDG-91) conducted a successful intercept of a hypersonic missile.  Well, that certainly sounds like good news.  Aegis performed a successful intercept of a hypersonic missile.  Great!
 
However, as we read a bit further into the article, we note the following: 
The USS Pinckney (DDG 91) successfully completed Flight Test Other 40 (FTX-40), also known as Stellar Banshee, using Lockheed Martin’s Aegis Combat System to detect, track and perform an engagement against a live advanced hypersonic Medium Range Ballistic Missile (MRBM) target using a simulated SM-6 Block IAU.[1][emphasis added]

Wait, what now?  The intercept used a simulated SM-6 defensive missile????  So, in reality, all the destroyer’s Aegis system did was track the hypersonic target.  It didn’t engage.  No actual intercept occurred.
 
Well, that changes the tone of the article and essentially refutes the headline, doesn’t it?
 
So, what did the test actually accomplish?  I don’t know the test objectives but it certainly didn’t demonstrate a successful intercept.  At best, it demonstrated the ability to track a hypersonic target which we already knew we could do.  At worst, it was a purely theoretical, software exercise that proved nothing.
 
The main thing all of this demonstrates is the need for us to be very careful and diligent in our reading of articles.  Take nothing for granted.  Assume whatever you’re reading is deceptive and make sure you really understand what you’re reading.
 
Congratulations Lockheed and Navy.  You theoretically shot down a target drone with a theoretical missile.  Theoretically … good job.
 
Congratulations Naval News website.  You managed to parrot a Lockheed press release without adding any analysis or value whatsoever.  You’re a credit to news reporters everywhere.
 
 
 
_________________________________
 
[1]Naval News website, “Aegis Combat System Demonstrates System’s Capability to Counter Hypersonic Threats”, Carter Johnston, 25-Mar-2025, Lockheed Martin Press Release
https://www.navalnews.com/naval-news/2025/03/u-s-navy-downs-maneuvering-hypersonic-missile-in-sm-6-block-iau-test/

Monday, May 6, 2024

US Weapon Failures in Ukraine

How often have you heard someone say, sure, previous weapons of this *fill in the blank” type weren’t successful but this new generation will be because it has new and improved *fill in the blank* technology?  Plus, the manufacturer claims *fill in the blank*.  And, how often has ComNavOps flatly stated that all systems will significantly underperform in actual combat regardless of the new and improved, miracle technology they’re built on?  Most of you have your own favorite, pet weapon/sensor system that you’re convinced will be a world beater … but you’re wrong.
 
Well now, courtesy of Ukraine and our giveaway weapons transfers, we have some actual combat performance to look at for various weapons.  The Daily Caller website has a fascinating article about several US weapons that have been provided to Ukraine and have been found to significantly underperform compared to claims.  Here they are with quotes from the article: [1]
 
 
Abrams Tanks 
Ukraine has withdrawn some of the U.S. M1A1 Abrams main battle tanks from the front lines after Russian drones destroyed five of the heavy Western tanks it fought for months to obtain …
 
Droves of Russian surveillance and hunter-killer drones have been able to detect and pursue the tanks more quickly than expected.
 
Excalibur Artillery Shells 
Excalibur precision artillery rounds initially had a 70% efficiency rate hitting targets when first used in Ukraine. However, after 6 weeks, efficiency declined to only 6% as the Russians adapted their electronic warfare systems to counter it.
 
Guided Multiple-Launch Rocket System (GMLRS)
 
Russian electronic warfare has successfully redirected the Guided Multiple-Launch Rocket System (GMLRS), another long-range precision munition off its planned course, according to reports.
 
 
Ground Launched Small Diameter Bomb 
It didn’t work for multiple reasons, including [electromagnetic interference] environment, including just really the dirt and doing it on ground …

Switchblade 300 Drone 
The first weapon that showed itself vulnerable to Russian electronic warfare was the Switchblade 300 …
 
According to the article, the common thread in most of these is that Russian electronic warfare (EW), which includes GPS spoofers, has proven effective at disrupting the weapons guidance by redirecting them away from their intended targets.
 
To be fair, the military has belatedly recognized, to some degree, that earlier GPS guided weapons are, indeed, vulnerable to EW and have been incorporating multiple guidance methods in more recent weapons.  Ominously though, this suggests that the majority of our current weapon stocks, being older, are of questionable effectiveness in the face of peer enemy EW and other countermeasures.
 
This demands that we begin instantly testing all our weapons under realistic EW conditions to see which work and which don’t (probably all of them!).  The military, however, steadfastly refuses to conduct rigorous, realistic testing which is why the Ukraine findings are coming as a surprise to the military.
 
Equally disturbing should be the realization that each of these weapons was pronounced a miracle for the ages and passed all tests with flying colors … … … and yet they’ve failed in combat.  This demonstrates the uselessness and simplicity of our so-called testing.
 
Now, this doesn’t mean that these weapons can’t be fixed and improved.  Most probably can … now that we know what’s wrong with them.  However, what about the rest of our weapons that haven’t been tested in combat or under realistic conditions?  What’s wrong with them?  It’s guaranteed that they have problems but we just don’t know what, yet.  I guess China will let us know when we use them and they fail.  Of course, that will too late but, at least, we’ll finally know.
 
 
 
_____________________________
 
[1]Daily Caller website, “The US Sent Billions In Military Aid To Ukraine. Many Weapons Are Massively Underperforming”, Micaela Burrow, 3-May-2024,
https://dailycaller.com/2024/05/03/ukraine-aid-weapons-russia-tactics/

Tuesday, April 9, 2024

LRASM Testing

How many new weapon systems work perfectly (or even moderately well) in their first combat test?  The answer, of course, is none.  That’s why it’s so important to conduct as realistic testing as possible of new weapon systems and to test according to the way you anticipate using the weapon.
 
For example, the Navy’s ‘new’ AGM-158C Long Range Anti-Ship Missile (LRASM) will be used in massed attacks of hundreds of missiles when attacking a Chinese surface or carrier group.  Anything less will be an unsuccessful waste of time, resources, and opportunities.  Therefore, we’ve undoubtedly been conducting tests using many dozens of missiles to prove out the concept and uncover the hidden problems – because, of course, there are always hidden problems.

LRASM


The LRASM program began in 2009 so now, 15 years later, the weapon must be pretty thoroughly tested and debugged … right?  Well, let’s check in on the latest test.
 
The U.S. Navy in partnership with Lockheed Martin successfully conducted a historic Long-Range Anti-Ship Missile (LRASM) flight test with four missiles simultaneously in flight.

Four missiles?  Four?  Four??????  4????????  Fifteen years into the program?!  We’re going to use these in volleys of many dozens or hundreds.  Fifteen years later we’re now proudly testing four missiles … apparently for the first time ever?
 
What happens when we launch seventy or a hundred missiles simultaneously?  Will they co-exist in the same airspace or will they collide as they conduct their individual maneuvers and respond to variations in wind speed, direction, turbulence, and engine wash from the surrounding missiles?  Can we apply mid-course guidance updates to that many missiles in the same small airspace or will the signals get lost, interfered, or mixed up?  Can the weapon’s sensors function with that many other missiles around?  How big of a radar return will that many missiles generate?  These are supposed to be somewhat stealthy missiles but will that many missiles just provide an easy detection for enemy sensors or can the aggregate remain stealthy despite being a giant ‘ball’ of missiles?
 
Four????  There are always unanticipated problems.  Four????
 
So, how does the Navy think the test went?  I’ll bet it was flawless.
 
During the 12th Integrated Test Event (ITE-12), the U.S. Navy was able to demonstrate the weapon’s inherent high-end lethality from mission planning through kill chain integration and its effects on the target. All mission objectives were met … [1]

When are all mission objectives ever not met according to Navy and contractor announcements?
 
The LRASM program has been in existence for 15 years and this is the 12th test???  Assuming each previous test involved just one missile, as suggested by the breathless excitement of this announcement about multiple missiles, that means we’ve tested just 11 missiles in 15 years and, likely, the majority of those tests didn’t involve actual missile performance but were things like captive carry tests, pylon separation tests, telemetry/comms tests, and so forth.  How many actual, end to end tests have been conducted?  I can’t find any data but it’s probably just a few.
 
So, to repeat, how many new systems fail their first combat test?  All of them plus, now, this one, for sure.
 
 
 
________________________________
 
[1]Naval News website, “Lockheed Martin Conducts Test With 4 LRASM In Flight”, Naval News staff, 3-Apr-2024,
https://www.navalnews.com/naval-news/2024/04/lockheed-martin-conducts-test-with-4-lrasm-in-flight/

Wednesday, January 17, 2024

Do You Believe? – Follow Up

You may recall that we recently noted and expressed disbelief about a Navy announcement related to the Large Unmanned Surface Vessel (LUSV).  Supposedly, the manufacturer/Navy completed a 30 day, uninterrupted run of the diesel engines as a demonstration of reliability.  ComNavOps expressed doubt.[1]
 
Well, here’s a bit of related news from Captain Scot Searles, the program manager of the Unmanned Maritime Systems Program Office, which starts the process of backtracking and qualifying the previous announcement: 
“It [the test engine] doesn’t have to be on for 30 days. It can start and stop, but nobody can touch it”[2]
My interpretation of that statement is that the vendor failed the test and the Navy opted for verbal gymnastics to turn it into a success.
 
As it turns out, potential engine vendors don’t actually have to be tested by the Navy. 
… vendors don’t necessarily have to stick to equipment tested by the government. They can also do their own 30-day testing on various pieces of equipment. However they must inform the program office ahead of testing and coordinate the testing with them.[2]
So, a manufacturer can test their own equipment.  I can’t see anything suspicious about that.  In fact, I’m going to make a daring and bold prediction (don’t try this at home;  I’m a trained analyst and predictor).  Every piece of manufacturer tested equipment will …      wait for it … … PASS!
 
So, to sum up, the engines don’t actually have to run non-stop for 30 days and manufacturers can test their own equipment.  Hmm …

 
LUSV Concept Image


It is telling that the Navy’s history of being veracity-challenged is so severe that my default position is disbelief about anything they say.  Sad, but that’s the position the Navy has earned for itself.
 
 
 
___________________________

Thursday, December 21, 2023

Do You Believe the Navy?

The Navy claims to have successfully completed a 30-day test of a diesel generator in connection with the large unmanned surface vessel (LUSV) program. 
The generator test was mandated by a congressional requirement inserted in the 2021 National Defense Authorization Act, directing the Navy to achieve the 720-hour test milestone before the Large USV could proceed into formal development. The congressionally directed testing included 100 hours of pre-testing before the 720-hour demonstration phase commenced. During the demonstration phase, no human intervention and no preventative or corrective maintenance on the equipment was allowed.[1]

Note that it was Congress that mandated the testing not the Navy.  Well done, Congress.
 
The question is, do you believe the Navy?  Given the nearly endless list of deceptions and outright lies we’ve documented on this blog, do you believe the Navy?
 
I don’t.
 


 
____________________________
 
[1]Naval News website, “U.S. Navy Completes Large USV Testing Milestone”, Staff, 21-Dec-2023,
https://www.navalnews.com/naval-news/2023/12/u-s-navy-completes-large-usv-testing-milestone/

Thursday, January 5, 2023

LCS and Ford Self-Defense Untested

The LCS program problems are many and, generally, well known.  Still, there are aspects that are surprising even for this troubled program.  One such issue is the Rolling Airframe Missile (RAM) anti-air defense weapon for the Freedom class LCS.  The issue is that the RAM system has not been tested on the Freedom class and the Navy has decided not to conduct any tests.

 

Both LCS seaframes have limited anti-ship missile self-defense capability.  The Navy has not fully tested these combat systems and the Navy does not plan to conduct further air warfare operational testing of Freedom seaframes 1 through 15 in their current combat system  configuration. The Navy has accepted the risk of continued operation with a combat system that is not operationally tested. (1, p.190)

 

No tests – that’s worse than the WWII torpedo fiasco!

 

This may also partially explain why the Navy is trying to retire the first fifteen or so Freedom class LCS.

 

The DOT&E testing also mentions a vital issue that ComNavOps has harped on and that is performance of radars and missiles in the face of debris-filled skies during attempted intercepts.  To briefly summarize the problem, many people feel that ships can simply vomit forth dozens of missiles, simultaneously, at any single target.  The reality, however, is that only two missiles at a time can be effectively utilized because the explosions from the initial intercepts will fill the skies with radar-reflecting debris and hot (IR) sources and subsequent intercepts will be blinded.  DOT&E notes this problem in one of their LCS recommendations:

 

Correct the SSDS scheduling function to preclude interference with the RAM infrared guidance capability stemming from prior intercepts and warhead detonations. [emphasis added] (1, p.209)

 

The refusal by the Navy to conduct tests of the anti-air system is also seen in the USS Ford self-defense testing.  Significant problems were found with the ‘Slick’ 32 electronic warfare (EW) system, the SPY-3 radar, and the Cooperative Engagement Capability (CEC).

 

The Navy conducted developmental and operational tests on the Self-Defense Test Ship (SDTS) that revealed combat system deficiencies and limitations associated with the SLQ-32(V)6 electronic warfare system, the SPY-3 Multi-Function Radar (MFR), and the Cooperative Engagement Capability (CEC). These deficiencies and limitations reduce the overall self-defense capability of the ship. The Navy has conducted only one of the four planned CVN 78 SDTS operational test events and has not resourced [funded] the remaining testing. [emphasis added] (2, p.123)

 

In June 2019, the Navy conducted one of the four planned CVN 78 operational tests planned for FY19 on the SDTS.  However, the remaining three tests are unlikely to be conducted in accordance with the DOT&E-approved CVN 78 data collection plan, the DOT&E-approved Capstone Enterprise Air Warfare Ship Self-Defense TEMP, and the DOT&E-approved SSDS TEMP. The Navy canceled one test event because they did not incorporate software changes required to conduct the test on the SDTS and the event was not resourced [funded]. The Navy delayed another test event due to poor SLQ-32(V)6 performance in developmental testing. The final, most challenging test event planned for 2QFY20 is not currently funded. The Navy may have to cancel the remaining delayed/unfunded events if they are not conducted before the MFR is removed from the SDTS; this removal is currently planned for the end of 2QFY20. If the Navy does not conduct all of the remaining events, testing will not be adequate to assess the operational effectiveness of the CVN 78 combat system. 

 

So, the Navy’s solution to finding problems is to stop testing.  Yeah, that sounds about right.

This is telling us that the Navy knows the ship self-defense systems have serious problems but, for reasons unknown, refuses to further test the systems.  Presumably, the reason is that the Navy knows that the system problems are serious and, at the moment, unsolvable, and further testing would only reveal more problems that the Navy already knows about.  In other words, from the Navy’s perspective, why spend money testing when you already know the results will be bad?  I have no problem with this scenario if the Navy is working to correct the problems.  Unfortunately, there is no evidence of that they are doing so.  In fact, the total cessation of funding suggests that the Navy is simply going to accept the problems and make no effort to fix them.

 

In addition to halting physical tests, the Navy has also halted its modeling efforts which were supposed to have taken the place of physical testing.

 

The Navy’s refusal to conduct anti-air tests also highlights the recent post about Intellectual Property rights and the problems the Navy has encountered in trying to acquire them (see, “The Navy and Intellectual Property”).  In an irresponsible attempt to cut costs, the Navy is attempting to shift from physical testing to modeling and simulation (M&S) but, in order to do so, they must have detailed design and engineering data and companies are frequently refusing to provide that.


The Navy has neither resourced nor conducted any air warfare test events against anti-ship cruise missile surrogates planned as part of the DOT&E-approved Enterprise Air Warfare Ship Self-Defense Test and Evaluation Master Plan (TEMP) or the LCS TEMP. The Navy’s Program Executive Office for Integrated Warfare Systems halted all work to develop a Probability of Raid Annihilation (PRA) M&S [Modeling and Simulation] suite of the combat systems in FY15 and has not yet restarted the effort. [emphasis added] (2, p.141)

 

DOT&E noted in its 2017 annual report that the Navy has halted funding for attempts to model the LCS combat system due to system component data rights being unavailable:

 

The Navy halted all work to develop a PRA M&S suite of LCS combat systems in FY15 because some combat system element models (e.g., radars) were not available. (1, p.190)

 

DOT&E noted in its 2019 annual report that the Navy has halted funding for attempts to model the Ford combat system due to system component data rights being unavailable:

 

The Navy has not resourced [funded] combat system testing on the lead ship or the modeling and simulation (M&S) required to support evaluation of the ship’s Probability of Raid Annihilation (PRA) requirement. (2, p.124)

 

 

Conclusion

 

The decision to drop physical testing in favor of modeling is purely a budget driven attempt to save money.  However, moving to a modeling approach to testing has run into the issue of intellectual property rights.  The Navy’s conflict with industry about intellectual property rights is complex and challenging.  Both sides have very valid concerns.  The Navy needs the data rights in order to conduct modeling and simulation but industry, quite rightly, has no desire to give away its hard-earned competitive advantages which is exactly what would happen.  Of course, if the Navy would show an ounce of wisdom, they’d return to actual performance testing and the data rights issue would be largely moot.

 

The result of all this is that Navy leadership is knowingly sending combat systems to the fleet untested.  That’s an obvious recipe for disaster when combat comes.  This violates the trust of the sailors under their command and constitutes dereliction of duty.

  

 

 

________________________________ 

(1)Director, Operational Test and Evaluation, FY2017 Annual Report

(2)Director, Operational Test and Evaluation, FY2019 Annual Report


Tuesday, July 5, 2022

The Navy and Intellectual Property

One of the problems the Navy has run into in recent years is intellectual property rights. Understandably, manufacturers don't want to give up their proprietary data. Equally understandable, the Navy wants exactly that data so that they can model system performance as they contemplate system integration and modifications.


It's an impasse, currently, and has resulted in the Navy abandoning some promising systems.

The seriousness of the problem is magnified by the Navy’s [unwise] attempts to substitute modeling for actual tests in order to save money.  DOT&E has documented this problem multiple times across multiple weapon system programs in their various annual reports.

 

Consider some of the implications of this issue.  What about the manufacturer who was asked to provide proprietary data for the 32 Zumwalts.  Hmm … a contract for 32 ships might be worth giving up the data.  Unfortunately, the production run was reduced from 32 to 3.  Is it still worth it to the manufacturer to potentially give up their hard won institutional knowledge that gives them a competitive advantage for just 3 ships? 


The Navy is erratic, to say the least. Those 20 FFG may become 40 or the Navy may decide to go whole hog on unmanned and cut the FFG at 8. Who knows? The point is that the manufacturer is being asked to give away their proprietary information for the hope of a substantial sale that history strongly suggests is unlikely to occur.

Further, the Navy's ability to protect proprietary data is highly suspect with network infiltrations happening all the time in addition to whatever unethical and improper actions the Navy might opt to take that gives away the manufacturer’s data.

If I were a manufacturer, I'd be very reluctant to give away my hard-earned industrial advantage for a shaky deal with the US Navy.  Some companies might consider it worthwhile but, apparently, many are deciding it's not worth it.

 

Consider the following specific example from a DOT&E report.

 

In FY15, DOT&E learned that the Program Executive Office for Integrated Warfare Systems (PEO IWS) stopped work on the PRA [ed. Probability of Raid Annihilation] Test Bed for the Freedom variant because a high-fidelity model of the ship’s AN/SPS-75 radar was not being developed. Development of an acceptable radar model requires intellectual property rights that the Navy does not hold … the Navy has also been unable to develop a high-fidelity model of that ship’s AN/SPS-77 radar for the same reason.[1, p.226]

 

Instead of conducting realistic, live tests, the Navy wants to model and simulate self-defense capabilities for the LCS.  That’s a very poor approach.  Modeling and simulation are fine but as an adjunct to live fire testing, not as a replacement for it.  That aside, in order to construct a valid model the Navy needs all the data on the radars.  The radar manufacturers, however, refuse to provide the proprietary data.  This leaves the Navy unable to program realistic models and simulations. 

 

It is easy to see, then, that the Navy’s choice of future equipment (of all types) is limited to those for which they can obtain proprietary data.  This eliminates many fine pieces of equipment from manufacturers who are unwilling to expose their data.

  

 

 

________________________________ 

[1]Director, Operational Test & Evaluation, “FY 2015 Annual Report”, Jan 2016


Monday, June 6, 2022

Why Test?

The Navy abhors testing.  The only thing the Navy hates more than testing is realistic testing (and Congressional oversight but that’s for another post).

 

To everyone but the Navy, testing seems like an obvious and excellent idea;  the more the better and the more realistic the better.  So, if it’s that obvious and that good an idea, why is the Navy so resistant to testing?  Well, from the Navy’s warped perspective we get the following reasons:

 

 

Cost – Every test costs money and the Navy has so skewed the budget to shiny new ships that they’ve allocated almost nothing for testing and they begrudge every penny that goes to testing.  Testing, even for a good program, always reveals problems and problems require more money to correct and re-test.  This process eats into the budget and reduces funding for more shiny new ships. 

 

Time – Tests take time and the Navy wants to push weapon programs down the acquisition path as fast as possible so that no one has the opportunity to question a program and so that they can get on to the next acquisition program as quickly as possible before anyone realizes what’s wrong with the last one.

 

Congress – The Navy is deathly afraid of (and resents!) Congressional oversight.  Every test opens the possibility for Congressional oversight and questions and oversight potentially threatens funding.  The Navy would much rather push on with a defective product than stop and give up the funding.

 

Embarrassment – Every negative test is viewed (incorrectly) by the Navy as embarrassment for the institution of the Navy and the individuals responsible for the problems/failures.

 

 

So, now we know the Navy’s reasons for wanting to avoid testing.  Why then, from their warped perspective, would the Navy want to test and test realistically?  Is there any reason, from the Navy’s perspective, to conduct testing?  The answer is, yes … or at least it should be.  Let’s take a quick look at the reasons that ought to motivate the Navy to want to conduct testing.

 

 

Combat Effectiveness - Testing increases combat effectiveness by identifying the strong and weak points of each weapon system.  A military will be more combat effective if it has a clear understanding of what works, what doesn’t, and under what circumstances.

 

Tactics and Doctrine - Testing informs tactics and doctrine.  Tactics depend on weapon capabilities and you can’t know what the weapon system capabilities are without extensive and realistic testing.  If you want effective tactics you need thorough testing.  This was graphically demonstrated by the British in the Falklands.  Their anti-air defensive tactics turned out to be wrong because they didn’t test their Sea Dart missile system enough to understand that it was not effective against low level targets and it cost them dearly.

 

Survivability - Testing informs survivability by identifying shortcomings in shock hardening, electronic emissions (EMCON), and Electromagnetic Pulse (EMP) hardening.  In addition, ships and aircraft should be tested for weapon penetration.

 

Design - Testing feeds back into the design process.  This is the old ‘test a little, learn a lot’ philosophy that feeds test results back into design evolution before a program advances too far along a flawed path.

 

Credibility - Testing increases the Navy’s credibility either by demonstrating that the Navy knows how to manage a program and effectively deal with the inevitable problems or by demonstrating that the Navy knows when to cut their losses and terminate a poor performing program.  Either way, it makes Congress more likely to trust (and fund!) the Navy.  This is, arguably, the biggest reason why the Navy should embrace testing.  Note, this is not the best reason, just the biggest in terms of furthering the Navy’s goal of bigger budgets.



Conclusion

 

So, if we have plenty of compelling reasons why the Navy should want to conduct testing, why aren’t they?  Well, sadly, the reason is linked to what the Navy’s purpose is.  All of us outside the Navy understand that the Navy’s purpose is to control the seas in war and peace.  Unfortunately, Navy leadership believes their purpose is to protect and expand their budget slice by building new ships.  Once we understand the fundamentally warped view that the Navy has, we are able to understand – but not approve of – the Navy’s attitude towards testing.  In the Navy’s view, testing does not further their quest for expanded budgets.  Therefore, they don’t want to do it for all the reasons cited above.  So, the Navy ought to enthusiastically embrace testing as the shining path to greater combat effectiveness but their twisted concept of their purpose pushes them down a diametrically opposed path.

 

By failing to embrace testing, Navy leadership is abandoning their true purpose and betraying the trust of the sailors of the Navy and the citizens of our country.


Friday, December 31, 2021

LCS RAM Untested

The LCS program problems are many and, generally, well known.  Still, there are aspects that are surprising even for this troubled program.  One such issue is the Rolling Airframe Missile (RAM) anti-air defense weapon for the Freedom class LCS.  The issue is that the RAM system has not been tested on the Freedom class and the Navy has decided not to conduct any tests.  No tests.  None.

 

The Navy has not fully tested these combat systems and the Navy does not plan to conduct further air warfare operational testing of Freedom seaframes 1 through 15 in their current combat system configuration. The Navy has accepted the risk of continued operation with a combat system that is not operationally tested. (1, p.190)

 

The Navy has neither resourced nor conducted any air warfare test events against anti-ship cruise missile surrogates planned as part of the DOT&E-approved Enterprise Air Warfare Ship Self-Defense Test and Evaluation Master Plan (TEMP) or the LCS TEMP. The Navy’s Program Executive Office for Integrated Warfare Systems halted all work to develop a Probability of Raid Annihilation (PRA) M&S suite of the combat systems in FY15 and has not yet restarted the effort. (2, p.141)

 

RAM Launcher


Yes, the Rolling Airframe Missile itself has been tested (is that a true statement?  I’m not sure) but the launcher, combat system software that controls the launcher, the sensors that have to integrate with the combat system and launcher, and possible sensor interference has not been tested.  That’s a lot of potential problems.  For example, prior to the Navy canceling testing, DOT&E noted the need to,

 

Correct the SSDS [Ship Self Defense System software] scheduling function to preclude interference with the RAM infrared guidance capability stemming from prior intercepts and warhead detonations. (1, p.209)

 

The Navy is knowingly sending Freedom variants into combat with untested point defense missile systems.  This ranks right up there with the WWII torpedo fiasco.  The only saving grace is that no one, not even the Navy, believes that the LCS is an actual combat asset.

 

On the plus side, the Navy appears to have standardized on the SeaRAM instead of the RAM so if they’ll quickly follow through and retrofit SeaRAM onto the Freedom variant then the testing halt is acceptable.  If not, it’s simply criminal.

 

SeaRAM Launcher


This does, however, raise a few questions/issues:

 

  • Who initially thought having two different point defense systems (RAM and SeaRAM) within the same general class of ship was a good idea?  Someone should be fired for that.
  • Who thought a point defense system without a self-contained targeting sensor (radar) was a good idea?  Someone should be fired for that.
  • This is what happens when you allow two variants of the same ship to be built using completely different equipment.  Who thought that was a good idea?  Someone should be fired for that.

 

The larger point is that the Navy has been exhibiting a marked tendency to truncate or eliminate testing over the last decade or so.  For example, the Navy delayed the shock testing of the LCS as long as they could and attempted to eliminate the shock testing of the Ford.  Numerous other examples of the Navy skipping testing are documented throughout the DOT&E annual reports.  This is what happens when there is no watchdog.  The Navy cannot be trusted to conduct thorough testing or, indeed, any testing at all.  This is what makes the recent decision to classify, eliminate, or significantly reduce public DOT&E reporting so concerning.

 

 

 

 

___________________________________

 

(1)Director, Operational Test and Evaluation, FY2017 Annual Report

(2)Director, Operational Test and Evaluation, FY2019 Annual Report


Wednesday, December 8, 2021

The Demise of DOT&E

The most valuable organization in the US military complex is the office of Director, Operational Test and Evaluation (DOT&E).  As documented countless times in this blog, DOT&E is the only thing preventing the military from engaging in wholesale fraud when it comes to weapons development testing.  For their part, the Navy has done everything they can to skirt, avoid, bypass, and ignore DOT&E test requirements and has been publicly slapped down multiple times for it.  A good example is the Navy’s attempt to avoid shock testing on the Ford which was thwarted by DOT&E.

 

Unfortunately, someone has gotten to DOT&E and appears to have neutered them.

 

But next year, the Pentagon’s office of the Director of Operational Test and Evaluation will issue two different versions of the 2021 report: An unclassified version that will be available to the public, and a version with “controlled unclassified information” (CUI) that will only be made available to lawmakers and those inside the department, according to Raymond O’Toole, acting head of the DOT&E office. [1]

 

 

As a reminder, DOT&E is required by law to release a public version of its annual report.

 

DOT&E is required by law to submit an unclassified version of the report every year. [1]

 

Why is the Director taking this action?

 

“I thought it very important to provide Congress and the Secretary the test evaluation details that shouldn’t wind up in our adversaries’ hands, hence the new CUI version of the annual report,” he said at an event hosted Monday by the National Defense Industrial Association. [1]

 

No one wants to make sensitive information available to the enemy but the dual reports are not necessary.  DOT&E already routinely issues classified sub-reports, as warranted.

 

The problem is that the negative information will, undoubtedly, be deleted from the public version and pushed to the ‘controlled’ version where we will not be able to see it.

 

Mandy Smithberger, director of defense information for the Project On Government Oversight, raised concerns that most technical details could be pushed to the CUI version, leaving the unclassified version a shell with little relevant insight. [1]

 

“DOT&E already has the ability to include classified reporting, and I worry that this will end up [resulting] in a watered down version of the report that makes it more difficult for the press and the public to make sure that we are buying weapon systems that are effective and safe,” she said in an email to Breaking Defense. [1]

 

More frightening is that, under this new system, the military gets to decide what is ‘controlled’ and what is not.

 

O’Toole also stated that the decision about what constitutes “controlled unclassified information” ultimately rests with the services, not DOT&E itself. [1]

 

Do you really think the military is going to allow negative results to appear in the public version?  I think we can anticipate, with 100% certainty, that the services will use a very broad definition of ‘controlled’ to keep unflattering information from going public.

 

Someone has gotten to the Director and applied pressure to coerce him into hiding weapon failures.  There is no other explanation.  As someone who has read every DOT&E report issued, I can assure you that the information in the DOT&E reports contains nothing that could remotely be considered classified or sensitive.  The reports contain almost no actual data and only vague descriptions of the issues that arise during testing.

 

Beyond that, our enemies have active and robust cyber espionage programs and know far more about our weapon programs than what appears in the DOT&E reports.

 

The pubic exposés of the various program failures are the only means the tax-paying public has to evaluate whether their tax dollars are being well spent.

 

DOT&E reports are often the only negative reports ever seen as the Navy/military trots out an endless string of spokesmen to sing the praises of the various programs.  According to Navy spokesmen, there has never been a weapon system or test that did not far exceed our wildest hopes … and yet we have the LCS, Ford, Zumwalt, and so many other abject failures.  Well, you didn’t hear about those failures from the Navy.  Only DOT&E gave you the straight truth.

 

The Director has been co-opted by the Pentagon and is doing a disservice to the American public by watering down the public version of the annual report.  This is only going to make an already badly broken weapon development, testing, and acquisition process even worse … if that’s even imaginable.


You'll recall that INSURV reports got classified when they were demonstrating too many negative results, and now this.


DOT&E was the shining light in military systems development and testing and now that light has gone out.

 

 

 

 

____________________________________

 

[1]Breaking Defense, “Pentagon tester to restrict info on weapons programs, raising transparency concerns ”, Valerie Insinna, 6-Dec-2021,

https://breakingdefense.com/2021/12/pentagon-tester-to-restrict-info-on-weapons-programs-raising-transparency-concerns/


Monday, May 20, 2019

Threat Surrogate Status

We’ve often discussed the Navy’s myopic focus on new, shiny, sexy ships like carriers at the expense of logistic vessels, drydocks, depot maintenance, shipyard improvements, testing facilities and equipment, etc. – you know, all the things that actually make a navy function and contribute to readiness. 

One of the specific problem areas is threat surrogates.  While we’d like to test Aegis against a Chinese or Russian missile, we usually don’t have any available to us.  Thus, we have to use threat surrogates – another, similar, asset that can mimic the real threat to a sufficient degree as to allow meaningful and useful testing. 

This seems simple enough.  If we don’t have a particular Chinese missile, we should be able to modify and mock up some existing missiles to simulate the Chinese missile’s flight profile, on-board ECM, speed, terminal maneuvers, seeker/sensor radiations, etc.  While the one-off, hand modified nature of such a threat surrogate will result in a high cost compared to the cost of the base production missile, the cost is virtually free compared to the Navy’s overall budget or the cost of the acquisition program it is intended to be used in.

For example, if we want to develop and deploy a new submarine sonar, a program that would cost billions to research, develop, and install, the cost of a diesel sub surrogate – a modified torpedo, in essence – would likely be a few million dollars which is round off error in the math of the billion(s) dollar sonar program.  We’d have to be idiots to try to scrimp and forego acquiring the needed threat surrogates to thoroughly test our systems just to save some fraction of a fraction of a percent of the cost of the parent program, right?

Well, sadly and unbelievably, that’s exactly what the Navy is doing.  They’re routinely refusing to acquire the threat surrogates needed to test our shiny new systems.  Here’s some quotes on the subject from the 2017 DOT&E Annual Report.  I’ve included the page numbers with the quotes so you can read them yourself because I’m pretty sure you’re not going to believe the degree of stupidity these shortcomings demonstrate.

What’s one of the major threats to the Navy?  Why it’s shallow water non-nuclear submarines (SSK), of course!  Obviously we must have threat surrogates for those, right?  Wrong.

No assessment can be made against operationally relevant midget and coastal diesel submarine threats because the Navy does not have any test surrogates that accurately represent these platforms. (p. 152)


Continuing with submarines and sonar, the undersea warfare combat software needs torpedo threat surrogates to test against.  We’ve got hundreds (thousands?) of torpedoes.  How hard could it be to tweak some into enemy torpedo threat surrogates for something as important as testing the main undersea combat control software?  I’m sure we must have torpedo threat surrogates, right?  Wrong.

A representative threat torpedo surrogate is needed to adequately assess future AN/SQQ-89A(V)15 variants.  (p. 152)


Along the same lines, the LCS ASW module has various pieces of equipment intended to detect and defeat submarines and torpedoes.  We must have torpedo threat surrogates for use in testing the LCS ASW module, right?  Wrong.

Current test surrogates have significant limitations representing threat torpedoes.  (p. 191)


Aegis is the foundation of our entire naval defense capability.  We don’t even need to ask whether the Navy has provided for an Aegis surrogate test bed ship, right?  Wrong.

To adequately assess the Probability of Raid Annihilation requirement for the self-defense mission for Flight III DDG 51 destroyers/ACB-20, the Navy must provide … an Aegis-equipped Self-Defense Test Ship (SDTS) where the ship’s full self-defense kill chain can be tested.  (p. 139)


Our close range air defense is the Rolling Airframe Missile (RAM).  I’ll bet we must have an adequate anti-ship missile surrogate because I know we have drones, including a supersonic one, right?  Wrong.

The Navy has not completed the following previous recommendations:

Develop a Multi-Stage Supersonic Target adequate for use in a phase of RAM Block 2 FOT&E.  (p. 209)

Develop an improved steerable antenna system for its ASCM surrogates. (p. 209)


The America class LHA uses the standard Ship Self-Defense System (SSDS) installed on all amphibious ships.  The SSDS is the standard for all amphibious ships and has been around for quite some time so we must have fully developed threat surrogates, right?  Wrong.

… the Navy has not resolved the following previous recommendations related to LHA 6 ship self-defense:

Develop an open-loop seeker subsonic ASCM surrogate target for ship self-defense combat system operational tests.  (p. 212)

Develop an adequate Multi-Stage Supersonic Target (MSST) and electronic warfare target surrogates for operational testing.  (p. 213)


You’ll recall that the Navy fielded a torpedo defense system on carriers in response to an Urgent Operational Need (UON) requirement.  If any system would have adequate threat surrogates it would have to be a carrier protection system developed for a UON, right?  Wrong.

The Navy has not accredited the surrogate torpedo targets used for testing as representative of any real-world threat torpedo.  (p. 225)


I can go on listing the threat surrogate deficiencies but you get the idea.  For reasons that totally elude me, the Navy steadfastly, adamantly, stubbornly, and stupidly refuses to fund and acquire the requisite threat surrogates to allow adequate testing.  There’s simply no excuse.  Compared to the cost of the parent acquisition programs, the surrogates are free.  This is just inexplicable stupidity on a scale that defies belief (geez, I seem to say that in a lot of posts, don’t I?).

Monday, November 19, 2018

Secrecy Is The Enemy Of Readiness


Just as perfect is the enemy of good, sometimes secrecy is the enemy of readiness.


Let's look at a few examples.


F-22

The F-22 is artificially limited in its performance during training exercises out of fear of revealing too much about its capabilities.

F-22 pilots may be restricted from flying the F-22 the way they would fly it in combat -- due to security concerns about exposing the F-22's unique capabilities," the report said. "These restrictions limit the value of the exercises and can result in pilots developing bad habits, according to Air Force officials. (1)

On a related note, this may be an explanation for those occasional stories about foreign aircraft/pilots who claim to have defeated F-22s – along with the desire by the American military not to embarrass foreign militaries during training exercises.

For many years, the Air Force refused to commit the F-22 to operational missions – missions which would have yielded vast amounts of actual performance data.  F-22 IOC was declared in 2005 but the first combat sortie did not occur until 2014 in Syria.


Virginia

Secrecy has crippled the ability of the Navy to evaluate the Virginia class submarines.

Because Navy security rules prevent the ability to collect useful operational test data from Virginia when conducting exercises with foreign ASW capable platforms, the Navy finished IOT&E without testing the Virginia class submarine against one of its primary threats, the foreign diesel electric submarine (SSK). (2)

Refusing to test against a primary threat because of secrecy concerns is insane.


B-2 Bomber

For many years, the Air Force refused to commit its B-2 bombers to operational missions – missions which would have yielded vast amounts of actual performance data.  The first B-2 was delivered in 1993 but the first mission did not occur until 1999 in Kosovo.


Electronic Warfare

The US military has many electronic warfare (EW) platforms and the Russian’s Ukrainian and Syrian involvements would seem to offer an excellent opportunity for some real world EW testing but, so far, we seem to be withholding much of our EW capability although it has been noted that EC-130 aircraft have been ‘disrupted’ over Syria.  EW is an area that is very difficult to find definitive information on so, to be fair, it’s difficult to determine the exact extent to which our EW is, or is not, being actively employed.  However, circumstantial evidence strongly suggests that we are greatly throttling back our EW other than monitoring Russian performance.


Conclusion

While there is a strong argument to be made for secrecy, if it is taken to the extreme where we don’t even know how our systems perform then it’s been taken too far.

Another key point is that there are very few real secrets anymore.  Cyber hacking by Russia, NKorea, Iran, and China has been so successful that attempting to preserve ‘secrecy’ is probably pointless.  The benefits of actual testing now far outweigh the dubious preservation of secrecy.

As an example, when the F-117 was developed the Air Force kept it under tight wraps, refusing to even admit its existence until it was used in Desert Storm.  At the time, this was probably appropriate and effective.  Today, however, the Chinese are probably seeing our F-35 technical data in near real time!  Thanks to the Internet and proliferation of networks, the days of physical isolation of a secret platform being able to ensure secrecy are long gone.

We need to start vigorous, real world testing and find out what works and what doesn’t.  Secrecy is no longer possible.



_________________________________________

(1)Military.com website, “Air Force Missing Out on Opportunities to Employ F-22, Report Finds”, Oriana Pawlyk, 20-Jul-2018,

(2)DOT&E Annual Report 2011, p. 176

Friday, July 13, 2018

ESSM Test Firing

The Navy is excited about the first live fire test of an Evolved Sea Sparrow Missile (ESSM) Block II which just recently occurred.  The Block II incorporates an active guidance seeker in addition to the legacy semi-active seeker of the previous version.  In the test, the ESSM Blk II successfully intercepted a BQM-74E drone target.  The BQM-74E is a subsonic drone with a max speed of around 500 kts at sea level.

Okay, so what’s noteworthy about this?  The noteworthy part is the extent of the test program, or lack thereof.  From a USNI News article,

“The recent test follows two June 2017 Controlled Test Vehicle flight tests to prove the missile’s ability to launch. Four additional live fire tests will follow, ahead of starting production of the Block 2 missile.” (1)

So, a grand total of five live fire tests will be conducted prior to beginning production.  Does that really sound sufficient?  Will five tests really prove out the reliability of the missile, the performance of the seeker under the hundreds of possible scenarios, the effect of an ECM/decoy environment on the seeker, the effectiveness of the seeker against supersonic missiles, the stability of the seeker in response to launch and Mach 4+ maneuvering stresses, etc.?

The simple lack of testing against a supersonic target would seem reason enough to delay production, wouldn’t it?

The Navy, in their zeal to enter production, is glossing over critical testing.  We’re putting essentially untested weapons into the fleet.  People are potentially going to get killed expecting this thing to work, only to be tragically proven wrong.

Come on, now, ComNavOps, you say, the ESSM has been around for quite a while and this is just a seeker head upgrade – no big deal and no big test program is needed.  Once a missile has been in service for a while we don’t really need repeats of extensive testing – there’s nothing else to go wrong and nothing else to find out.

Well, there’s a few things wrong with that.  First, while I don’t have the data, I’m pretty sure the original testing program was far from extensive.  For example, I don’t recall off the top of my head that the ESSM has ever been tested against a supersonic target drone.  Just as importantly, I know there hasn’t been enough testing to demonstrate overall missile reliability.

To illustrate the reliability concern, consider the Standard missile which has been around forever and, supposedly, thoroughly tested.  Guess what?  They keep blowing up!

  • A Standard SM-2 Block IIIa blew up upon launch from a German frigate on 21-Jun-2018. (2)


In both cases, it appears that the rocket motor exploded.  Apparently, testing was insufficient to detect the rocket flaw and the missile was released into the fleet where it is now putting ships and personnel at risk.

Considering how few live fire tests are performed each year, even these couple of explosive failures are disturbing.

The point of this post is not the reliability of the Standard SM-2 Block IIIa missile.  I have no problem with failures.  That’s how you find problems and fix them.  The point is that without extensive testing, these kinds of problems can’t be found.  Five test launches is not enough to detect whatever problems are lurking in the ESSM – and they’re there - we just haven’t tested enough to see them.

This reluctance to test is typical of the Navy’s constant, on-going battle with DOT&E.  DOT&E wants to conduct proper testing and the Navy constantly wants to skip testing and rush weapons into production.  Recall that DOT&E had to go around the Navy to get the Navy to conduct shock tests on the LCS.  Remember the result of those tests?  Sure enough, the LCS failed and the tests had to be conducted at reduced explosive levels and the final tests had to be dropped due to the certainty of excessive, possibly fatal, damage.  And yet, the Navy keeps wanting to rush systems into production.

Everyone except the Navy is all too aware of the now legendary problem with the WWII torpedoes that the Navy refused to properly test.  Everyone except the Navy recognizes the wisdom of extensive and realistic testing.  Everyone except the Navy understands that it is far better to find problems in peacetime than during combat.

The Navy needs to put the ESSM through rigorous testing under realistic conditions and against supersonic target drones using evasive maneuvers and ECM. 

The Navy got people killed by not taking the time to train and certify personnel on the recent Burke collisions and now they’re failing to take the time to properly test the ESSM.  More people will die someday.  Why can’t the Navy learn its lessons?  Why do people have to die because the Navy won’t take the time to properly train and test?  CNO Richardson, this is directly on you.



___________________________________________

(1)USNI News website, “Evolved Seasparrow Missile Block 2 Successfully Intercepts Aerial Target in First Live Fire Test”, Megan Eckstein, 6-July-2018,

(2)USNI News website, “Missile Explodes During German Frigate Training Exercise; Incident Similar to 2015 U.S. Navy Explosion”, Sam LaGrone, 27-Jun-2018,