Pages

▼

Wednesday, July 17, 2024

F-35 Software Problems Continue

We’ve previously noted that software has become the leading cause of schedule delays and cost overruns and we cited the F-35 program’s attempt to implement the Technology Refresh (TR-3) leading to Block 4 upgrades (see, “F-35 Software Case Study” and “The Heartbreak of Software”) without which the F-35 cannot achieve full combat capability.
 
About a year ago, the Pentagon put a freeze on deliveries of new F-35s pending fixes for the TR-3 implementation.  F-35s have been piling up in warehouses awaiting a resolution of the software issues.
 
Honestly, the freeze on deliveries has been more symbolic than effective since the Pentagon has continued payments for the new aircraft with just a $7M withholding per aircraft.[1]  That means that Lockheed has still been getting around 91% of the contract price for aircraft that don’t meet spec and can’t be delivered.  That’s not a bad deal if you can get it!
 
Financial aspects aside, I’m not sure any of us fully appreciate just how badly broken the software side of things are in the F-35 program.  The Pentagon has just caved to various pressures and announced that the delivery freeze has been lifted despite the software problem remaining unresolved.  Hmm …
 
Bowing to operational demands, the Pentagon has lifted a year-long freeze on accepting new F-35 stealth fighters — even though the problem that prompted the standstill has not been fully resolved.[1]
 
Persistent problems with TR-3 prompted officials to eventually capitulate to an interim software fix …[1]
 
Read this next quote slowly and carefully and fully grasp the meaning and implications.
 
… jets will be delivered with interim software that facilitates training, but a second software drop that enables combat capabilities likely won’t be available for at least another year.[1]

That’s right.  We’re delivering training jets but not fully combat capable jets.  We’re decades into this program, have built a thousand aircraft, and still don’t have a fully combat capable aircraft.  Someone should face a firing squad for this.
 
The purpose of this post is not to simply beat on the F-35 program.  Instead, I’d like to highlight a couple of points that this development hammers home for us.
 
Software – As we’ve noted, software has become the main obstacle to successful programs.  We’ve already stated that we need to change the way we treat software and make it its own program instead of just an afterthought for the hardware.  This latest incident just hammers that point home.  Our frontline combat aircraft, the F-35, is not yet fully combat capable due to continuing software problems because the software was treated as an afterthought. 
 
For acquisition programs, our focus has to be software, software, software.
 
SOFTWARE, SOFTWARE, SOFTWARE !
 
Phased Delivery – The incompetent, lazy method of running an acquisition program is to use some sort of incomplete, so-called ‘phased’ delivery where the product is delivered incomplete, to be finished over subsequent years of development.  Of course, this never works out.  The F-35, for example, has yet to achieve full combat capability and we’ve built some thousand aircraft, none of them fully combat capable.  That’s criminal and leads to endless upgrades which add to the cost of the aircraft.  That $80M aircraft becomes a $100M+ aircraft after all the upgrades are included. 
 
Further, by all accounts, we’ve produced hundreds of F-35 orphans which, due to concurrency, will never be brought up to standard and are, essentially, throwaways.  Again, that’s criminal.  This half-assed F-35 production program just hammers home the folly of any kind of phased delivery program.
 
We have to adopt a philosophy of the product, the whole product, and nothing but the whole product or don’t bother building it.
 
Contracts – The final point that this incident hammers home is the unmitigated stupidity of a contract that does not specify and demand delivery of a fully capable product in order to get paid a single penny.  You wouldn’t make a partial payment on an incomplete automobile, would you?  So why are we paying for incomplete aircraft (or ships or anything)?  Again, we need to bring back firing squads on a regular basis.
 
 
Inescapable Conclusion
 
Our acquisition program methodologies are badly broken and our military leadership is 100% complicit to the point of dereliction of duty and criminal fraud.  We have got to start learning lessons from our endless string of failures.  Congress needs to start firing admirals.  Accountability is the only way people as stupid as our military leaders will ever learn a lesson.
 
 
 
_____________________________
 
[1]Breaking Defense, “F-35 deliveries to resume next week, despite incomplete upgrade”, Michael Marrow and Valerie Insinna, 11-Jul-2024,
https://breakingdefense.com/2024/07/f-35-deliveries-to-resume-next-week-despite-incomplete-upgrade/

at

28 comments:

  1. AnonymousJuly 18, 2024 at 1:14 AM

    Sorry CNO I disagree with you on one thing : being paid only when you deliver the final product isn't realistic for something that takes years to design and test like a big defence program (true it shouldn't be 20+ years). There has to be staged payments, with some sort of intermediate validation of course. Any other solution is unmanageable from a cashflow point of view.

    ReplyDelete
    Replies
    1. ComNavOpsJuly 18, 2024 at 6:25 AM

      "Sorry CNO I disagree with you on one thing"

      No need to apologize. This is how you learn.

      "disagree with you on one thing : being paid only when you deliver the final product isn't realistic"

      You're not actually disagreeing with me about anything. You're just being pedantically literal. There is nothing wrong with paying for an occasional, logical intermediate step.

      For example, the design should be (and often is) a separate step and a complete project in and of itself - delivered and paid for when complete, reviewed, and verified.

      A nuclear reactor is probably a separate contract.

      Certain long lead items might warrant a separate contract to be paid ONLY when the items are in the hands of the builder and have been inspected and accepted by the Navy.

      Partial payments during construction should be done rarely and never in significant amounts that would encourage the builder to reach that point and then lose incentive.

      You wouldn't pay an automobile manufacturer for long lead items, would you? You wouldn't pay for an intermediate level of completion, would you? Why would you think it's valid to make partial payments on a ship or plane? As I said, there are a few stages that may warrant breaking into separate contracts or intermediate payments but not for the general construction, as a rule. If the builder wants his payment, he can work to speed up construction and minimize QA problems. That's called incentive.

      We get in trouble when we fail to insist on delivery of the fully completed, fully functional product before payment. That's insane. Whether it's the final aircraft, ship, or equipment or it's an [unwise] intermediate design, equipment, or stage, we have to withhold payment until that 'product' is actually delivered and is fully functional. You wouldn't buy a car that is partially built and only partially functional so, again, why would you do that for a aircraft or ship?

      We've had a badly broken approach to acquisition for so long that you've come to believe our broken system is correct. It's not! You've fallen into an incorrect paradigm.

      Don't worry. You'll learn. I'm here to help you!

      Delete
    2. AnonymousJuly 21, 2024 at 2:54 AM

      Progressive payments are a pain but they do serve a purpose. If the government does not make progress payments then a contractor has to rely on private finance to fund its operations (usually a bank). That just puts the price of a product up and you can buy less. Simple truth, taxpayers money is cheap, bank money isn't. The problem is that governments tend to reward. failure when they should walk away.

      Delete
    3. ComNavOpsJuly 21, 2024 at 7:08 AM

      "If the government does not make progress payments then a contractor has to rely on private finance to fund its operations (usually a bank)."

      From a quick Wiki search, LM net profit for 2023 was $6.9 billion. Cash from operations in 2023 was $7.9 billion. Free cash flow in 2023 was $6.2 billion. The annual gross profit for 2023 was $8.479 billion.

      LM could simply rely on PROFITS to fund the day to day expenditures that occur before profits start coming in from the delivered product.

      LM has no need for progressive, partial payments in order to meet operating expenses or order parts. If they would deliver the aircraft quickly and flawlessly, they'd be paid LONG before any cash flow problems could develop. We've become so accustomed to excusing poor manufacturer performance and propping the manufacturers up that we've come to think that's normal. IT'S NOT!!!!!

      Delete
  2. AnonymousJuly 18, 2024 at 5:10 AM

    F-35 TR-3 upgrade is a base step for future Block 4 upgrade. F-35 TR-2 are delivering and combat ready.

    Question is - should Air Force order F-35 TR-3 before its development is completed?

    The whole Block 4 upgrade was pushed in a hassle, include new radar (APG-85), new IRST, new electronic war suite, new engine (solve electricity and cooling problem),... etc. One big factor for these upgrade is J-20. After serios delays in almost all parts, even if Block 4 succeeds in years later, you cannot image how far J-20 have been upgraded,

    F-35 starts with a strategic blunder thinking US would fight regional powers (China deemed as one regional power) but now need to face a superpower.

    ReplyDelete
    Replies
    1. ComNavOpsJuly 18, 2024 at 6:17 AM

      "F-35 TR-2 are delivering and combat ready."

      As you know, Blocks 1,2, and 3 were initial capabilities with Block 3 providing basic combat capability. Block 4 is the full combat capability. Many of the planned Block 4 capabilities have now been pushed into the indefinitely-delayed-to-some-future-date category which means they'll never happen.

      Blocks 1A and 1B - initial pilot training and multi-level security
      Block 2A - improved training capabilities
      Block 2B – basic air-to-air combat capability; basic air-to-ground combat capability
      Block 3i – Block 2B plus new hardware to support USAF IOC
      Block 3F - full flight envelope and baseline combat capabilities; began 2018 and completed 2023
      Block 4 – full weapons (17 new weapons) and ESM capabilities; pending; requires Technology Refresh 3 (TR-3)

      Delete
    2. AnonymousJuly 18, 2024 at 2:51 PM

      TR-3 only paves the way for Block 4. There are many new technological challenges and new products to be developed. Projected deliver date of Block 4 keeps postponing. Latest is that Block 4 implementation will not be in one go but introduced one by one base on part and software development, just like current TR-3.

      Delete
  3. AnonymousJuly 18, 2024 at 9:28 AM

    ComNavOps last time I've read your Long Range Carrier Fighter - F-22 / F-15 Hybrid. I'd like to read your posts about warship, aircraft designs. I think you should concept ideal designs AEW plane for aircraft carrier, ideal aerial tanker and EW plane. Also it's time to replace the SH-60 Seahawks and to concept new ideal helo for search and rescue missions. The plane should also sit down on the water in the ocean and to be afloat there.

    ReplyDelete
    Replies
    1. ComNavOpsJuly 18, 2024 at 9:54 AM

      "you should concept ideal designs AEW plane"

      I've already done this in multiple posts. Here's one to get you started. It's for a passive version of the E-2 Hawkeye:

      "Passive Hawkeye"

      I've also discussed a B-2/21 version of an E-2 Hawkeye. Take a look at the archives and you'll find these and more.

      I've also extensively discussed tanker aircraft and we've even had guest posts on this. Again, browse the archives.

      I've not addressed an EW aircraft too much and I'll give that some thought. The main problem is that there is so very little in the public domain about the EW capabilities beyond very general descriptions so it's difficult to conceptualize an EW aircraft.

      Enjoy the archives!

      Delete
    2. G2milJuly 19, 2024 at 7:23 PM

      The USA recently developed exactly what the Navy needs called the 350ER King Air and our SpecOps folks have used them for over a decade. Just called it the S-2 HawkEar and use C-2s airframes and engines now produced for the E-2D line with the 350ER tech. Our allies have seen the light.

      https://www.thedefensepost.com/2022/04/25/canada-king-air-spy-planes/

      And since Marines are looking for unique missions, this is perfect for them too, so they can man them and provide dets to carriers when desired. Establish a couple Marine S-2 squadrons on each coast and procure four fewer F-35 squadrons.

      Delete
    3. AnonymousJuly 20, 2024 at 7:12 AM

      Thank you, CNO. What is the future of carrier strike aircraft? Is it worth to design A-12 Avenger attack aircraft or refuse from carrier strike aircraft at all?

      Delete
    4. ComNavOpsJuly 20, 2024 at 1:52 PM

      "What is the future of carrier strike aircraft?"

      I've addressed this with several blog posts and many comments. The role of the modern carrier is to protect and escort the surface ship Tomahawk shooters and to provide local air superiority in support of operations. Strike is not a primary carrier aircraft role. Carriers should have, at most, a single squadron of strike aircraft and, given today's shrunken air wings, probably not even that.

      Delete
  4. JjabatieJuly 18, 2024 at 12:15 PM

    This is akin to the Constellation- beginning construction before the plans are done. I often mention that a lack of urgency is a big problem. Seems like thats the case here as well. If we have a thousand combat-incapable aircraft, that needs fixed YESTERDAY. How many software engineers are currently working in this?? 50, 100, 500??? Sounds like maybe we need 1000, working 50+ hour weeks. Whether the govt or the contractors pays for this is another topic almost. The fact that we got here is inexcusable, but since we are, we need to make maximum efforts to correct it (and of course avoid it in the future). Same urgency needs to be applied elsewhere also....

    ReplyDelete
    Replies
    1. ComNavOpsJuly 18, 2024 at 12:28 PM

      Just to be clear, the Block 3x aircraft do have a minimal, basic combat capability. However, all the weapons, sensors, and electronic warfare capabilities that were supposed to make the F-35 special are in the several year delayed Block 4. Many of those capabilities have now been deferred to the next generation aircraft and will not appear in the F-35.

      Delete
    2. JjabatieJuly 18, 2024 at 12:54 PM

      Understood. Although for the $$$, shouldn't we ne getting that "special" level of aircraft?? Its a bit cringe to think that F-35 will be around for decades and never be 'complete'!! Deferring capabilities to a future aircraft is a sad statement in how bad this truly is!!
      On another related note- just read the Congressional Report on the Ford program- and the fine print tells us that AAG and EMALS are still not performing to spec. True numbers are still classified, but "no significant improvement" says enough. The weapons elevators seem to work now, but the crew is unable to fix them, and is still reliant on contractors to do so.
      Our procurement is such a failure...

      Delete
    3. Chinese GordonJuly 19, 2024 at 9:02 AM

      Jja:
      old It joke
      What is an IBM man year?
      730 people trying to finish a project before lunch.
      see the book "The Mythical Man -month" for why adding too many people will turn any software project into a F35 grade mess.

      Delete
  5. AnonymousJuly 18, 2024 at 1:32 PM

    This snippet is from an article in" Air & Space Magazine" it is dated 2023 but still may be relevent today .
    "As a fleet, the F-35 still isn’t meeting the reliability and maintainability standards set in the Operational Requirements Document, the report noted. "
    " The main drivers of critical failures were “troubleshooting (including software stability), attaching hardware (including nutplates), wires/tubes/ducts/ fiber optics, throttle grip, aircraft memory device, [low observability] repair, standby flight display, refueling door, position light,” the report noted "
    My take : Not sure if many of the problems have been resolved but one wonders when this jet will meet it's full
    Not sure when this fighter jet will meet it's full potential.
    " DOT&E’s recommendations regarding RMA mirrored those of the fiscal 2022 report. It urged continued maintenance system improvements and more investment in spares, “especially for F135 engines, to reduce down-time for aircraft waiting [for] spare parts by developing alternate sources of repair, including organic repair.”

    ReplyDelete
  6. LonfoJuly 19, 2024 at 1:38 AM

    The fact that they delivered a bunch of planes with nearly useless software will never not be amusing to me.

    ReplyDelete
  7. AnonymousJuly 19, 2024 at 7:13 PM

    LM is not lonely, Boeing is worse. Just google web on KC-46 oil tanker. Of course, its civilian product - 737-Max were even fatal.

    ReplyDelete
  8. AnonymousJuly 19, 2024 at 9:07 PM

    Honestly the system is broken. The F-16 wasn't really feature complete either until we got to the F-16C Block 30/32 and Block 40/42 aircraft. Even the Block 50/52 aircraft, the final versions accepted by the Air Force, still lack the full promised suite of features - for example, that base hump in the tail stab is supposed to house an onboard self protection jammer, but that jammer was abandoned before it was fully developed, so F-16s have spent their entire lives carrying ECM pods, which eats into your range, drag, and weapons carrying ability.

    ReplyDelete
  9. AnonymousJuly 20, 2024 at 4:56 PM

    Slightly OT, but interesting to read that it looks as though the Air Force is getting ready to can the NGAD Project.
    If you’re going to can it I guess now’s the best time.

    ReplyDelete
    Replies
    1. Chinese GordonJuly 21, 2024 at 11:26 AM

      The current NGAD is estimated @$300m each,
      so stopping and rethinking seems like a good idea.
      {source, AW&ST}

      Delete
  10. NICOJuly 20, 2024 at 5:58 PM

    I amicably disagree with this notion that F16 wasn't great either out of the box or not a good example of a good program, lets look at some history:

    A. YF16 first flight: 2 FEB 1974. First operational F16s to HILL AFB in Jan 1979. First strike mission by Israeli Air Force over nuke reactor in Iraq: 7 June 1981. Israeli Air Force turkey shoot over Bekaa Valley: 9 June 1982. So it took 7 to 8 years for the F16 to do what it was supposed to do FOR REAL COMBAT.

    Let's remember that the origin of the F16 was LIGHT FIGHTER, DAY ONLY, NICE WEATHER, that was it's job, it was designed to do it, turned everything at the time immediately obsolete and it did it in less than 8 years.

    B. Let's look at our friendly F35: X35 first flight: 24 Oct 2000. F35 first flight: 15 Dec 2006. First operational "combat ready" Block 2 HILL AFB: 24 OCT 2015.....so it took 15 years or CLOSE TO TWICE AS LONG as the F16 to have the first go to war BLOCK 2.


    I think the F16 program did it a lot better than the F35 program: the way General Dynamics did a great job on the best airframe/engine combo for max performance, fly by wire, good ergonomics, nothing special radar BUT it works and worried about all the fancy complicated stuff like night flying or bad weather capability, more precision weapons, better radar,etc on the future upgrades.

    Most of the Block 15s were later upgraded so how many "bad" or "training" early F16s were there? Not that many compared to probably 100s of early Block 1,2, 2A F35s that can't be upgraded as already too old or just not worth the expense....

    Now, realistically, could LMT have done the same with the F35? Probably not, the common fighter for 3 services pretty much meant the program was going to be a giant mess. I guess you could have structured it to do the USAF first, Navy second and USMC third but not sure DoD would have been ok with that. In fact, the opposite happened! Software wise, Im not enough of an expert to say it could have been done at a better incremental rate but it does make one wonder why it has taken so long to get all these capabilities in the jet.....

    Last thought, what makes the F16 so great is that we have to remember the USAF that existed at the time, let's not just compare the F16 to F35 of today: look at what the USAF was flying at the time, late 1970s to 1980s: A7, A10s were coming up, F15s, tons of F4s, couple remaining F105s from Vietnam, F106 ANG, plenty of F111s and around the corner EF111s, plenty of F4G Wild Weasels and top secret F117 was around the corner too....it was a completely different air force with quality and QUANTITY. The first F16s didn't have to be all special capability because we had other jets that could do the job at the time, it was only necessary to give more capability as we replaced older air-frames.....

    USAF doesn't have that luxury today, F35 has to eventual work!

    ReplyDelete
    Replies
    1. ComNavOpsJuly 21, 2024 at 6:53 AM

      "I amicably disagree"

      I love this. This is how to disagree and discuss instead of trying to win an argument. Well done!

      "Software wise"

      Got a post coming on exactly this and it will describe how we should have handled the software and how we should handle all software in the future.

      Delete
    2. AnonymousJuly 21, 2024 at 7:18 PM

      Kinematically the F-16 was a good plane from the start, but I'd disagree that it turned everything obsolete when it was introduced. Sure, the original concept was LIGHT FIGHTER, DAY ONLY, NICE WEATHER, and that's fine for fighting the last war in Vietnam. That's not so great in Europe where the weather is worse than Vietnam, with less daylight hours, and you only have two Sidewinders on the wing - meanwhile, the Russians are concurrently introducing the MiG-29 as their F-16 equivalent, which comes with radar-guided missiles and a bigger payload out of the box, plus helmet-mounted sights to cue their high off-boresight dogfight missiles (something we only fielded in the early 2000s).

      Once you get into the merge, the F-16 is a hell of a dogfighter, no question about that - it's surviving to get to the merge that's the problem for early F-16s.

      Delete
  11. Polish GhostJuly 20, 2024 at 8:33 PM

    Block 4 requires TR-3 (which is about replacing older hardware like microprocessors with newer one).

    In other words: F-35 computer hardware has become obsolete before the project has even reached full combat capability.

    Things like that usually used to occur in countries like India before (just check out Arjun MK1 main battle tank ), but not in the US ...

    ReplyDelete
    Replies
    1. AnonymousJuly 21, 2024 at 3:27 AM

      They used to claim APG-81 radar has life time beyond an F-35. Now, they scramble to have APG-85 to replace it. There aren't even simple description on what improvements from APG-81. They then say F-35 electricity supply and cooling capacity are insufficient. Likely, while they designed F-35, they didn't think that there is need to drastically upgrade its avionics.

      TR-3 hardware and software are only a base for Block 4 upgrade. Block 4 upgrade are still far from now.

      Of course, it is a disgrace that the software team cannot finish their works while there are no hardware problem.

      Delete
  12. CDR ChipJuly 21, 2024 at 7:51 PM

    "Our acquisition program methodologies are badly broken and our military leadership is 100% complicit to the point of dereliction of duty and criminal fraud. We have got to start learning lessons from our endless string of failures. Congress needs to start firing admirals. Accountability is the only way people as stupid as our military leaders will ever learn a lesson."

    Having run a few software installation projects in my day, I agree 110%.

    ReplyDelete

Comments will be moderated for posts older than 7 days in order to reduce spam.