Friday, April 12, 2024

F-35 Software Case Study

We’ve come to recognize that software has become the major stumbling block in weapon systems development, even more so than construction and physical performance issues (see, “The Heartbreak of Software”).
 
As you know, the F-35 was delivered in a non-combat capable condition due to software limitations.  The full-combat capable software was planned to be delivered in Block increments as listed below.
 
Blocks 1A and 1B - initial pilot training and multi-level security
Block 2A - improved training capabilities
Block 2B – basic air-to-air combat capability;  basic air-to-ground combat capability
Block 3i – Block 2B plus new hardware to support USAF IOC
Block 3F - full flight envelope and baseline combat capabilities; began 2018 and completed 2023
Block 4 – full weapons (17 new weapons) and ESM capabilities;  pending;  requires Technology Refresh 3 (TR-3)
 
Block 4 is the full combat capability version.  See Forbes[2] for a good discussion of the Block 4 upgrade but note the 2022 time of the report.  Today, Block 4 is still pending despite many of its features having been deleted and pushed into some nebulous future date land where they will languish forever and never get implemented.  Thus, even the dumbed down version of Block 4 cannot be delivered in a timely manner, being years overdue, already, and still years in the future.
 
We now have yet another example of major software problems in the software-cursed F-35.  Technology Refresh 3 (TR-3) upgrade which is required to implement the dumbed down Block 4 and make the F-35 fully combat capable has encountered major software problems resulting in the military halting acceptance of new aircraft.
 
Since July 2023, the Pentagon has refused to accept newly built F-35s due to software woes with the TR-3 upgrade, which has slipped numerous times past its original fielding date expected for April 2023.[1]
 
Problems with an upgrade installed on some Lockheed Martin F-35 Joint Strike Fighters rolling off the production line have now disrupted plans to incorporate those upgrades on existing aircraft, and the F-35 Joint Program Office does not have a date for when those jets will get the much-anticipated retrofits.[1]
 
The F-35 program “was scheduled to begin TR-3 [Technology Refresh 3] retrofits in April 2024 with the intent to modify 149 aircraft over the subsequent 12-month period,” JPO spokesperson Russ Goemaere told Breaking Defense. But now, “[t]he Program is working closely with F-35 customers to establish a new start date for those modifications based on a number of factors including software and supply chains.[1]
 
TR-3 — which features a more powerful processor, greater memory and a panoramic cockpit display that collectively enable a suite of new capabilities known as Block 4 … [1]
TR-3 began being installed in new production aircraft with Lot 15.
 
 
The military’s solution? 
“Very capable TR-2 jets will continue to fly operational missions while awaiting the start of TR-3 retrofits.”[1]
I’m sorry but no.  F-35s with TR-2/Block 3x are not ‘very capable’.  They lack most of the weapons and sensor capability required for full combat capability.
 
To provide some perspective, the F-35 has been in production since 2008-9.  That’s 16 years and we still don’t have full combat-capable aircraft due to software delays.  Just as we’ve begun retiring LCSes without them ever having had fully functional modules installed, we may see F-35s retire without ever having been fully combat capable.
 
We have got to start recognizing that software is now the primary obstacle in weapon system development and we need to modify how we approach program management and software development.
 
 
 
_____________________________________
 
[1]Breaking Defense, “Pentagon delays F-35 retrofits amid upgrade woes”, Michael Marrow, 4-Apr-2024,
https://breakingdefense.com/2024/04/pentagon-delays-f-35-retrofits-amid-upgrade-woes/
 
[2]Forbes, “Inside Block 4—The Mostly Secret Plan For Making The F-35 Fighter Even More Lethal”, Loren Thompson, 14-Nov-2022,
https://www.forbes.com/sites/lorenthompson/2022/11/14/inside-block-4-the-mostly-secret-plan-for-making-the-f-35-fighter-even-more-lethal/?sh=3a0dc3184423
Posted by at
Labels: ,

13 comments:

  1. Trondude 5952April 12, 2024 at 1:16 PM

    When do you think L-M will discover software as a service model of business?

    ReplyDelete
  2. Austin VernonApril 12, 2024 at 1:26 PM

    Part of the problem is internal at the primes - it is hard to be great at software. Part is that they are asking too much of the software.

    A big part of the solution is already what you suggest with aircraft development - more incremental improvements instead of clean sheet design and focus on one role. Focusing on one role simplifies the software problem and incrementalism allows reuse of proven code.

    ReplyDelete
  3. AnonymousApril 12, 2024 at 2:35 PM

    TR-3 is not just software upgrade, it includes hardware upgrade also. TR-3 pave base for the Block 4 as higher computation capabilities required.

    The Block 4 includes lots of hardware upgrade, for instance, AN?APG-85 radar, new EOTS, ... etc., aim to close current technological gaps with China's J-20. Unfortunately, delays after delays ... Ah!

    Other than claim it is best, very little information given on AN/APG-85 AESA radar. Given China has advanced into DAR, it is interesting to know if AN/APG-85 is DAR or not.

    Air Force now heavily focuses on NGAD as funding has been increasing while F-35 procurements cutting down. Strangely, Navy slows down F/A-XX development.

    ReplyDelete
  4. deariemeApril 14, 2024 at 4:04 AM

    I've followed the tale of the shortcomings of the F-35 for years. A couple of times I've remarked on British blogs on the unwisdom of our buying some of the wretched things. The response has been odd - apparently the fact that the Royal Navy flies them cleanses them of all sin.

    ReplyDelete
  5. ComNavOpsApril 14, 2024 at 10:45 AM

    "apparently the fact that the Royal Navy flies them cleanses them of all sin."

    Could we rotate all ours through the RN to have their flaws removed?

    ReplyDelete
  6. CDR ChipApril 14, 2024 at 6:34 PM

    Anybody who has ever worked on a software project knows that getting computer jocks to wrap up any project is damned near impossible. They always want to add one more bell or whistle.

    I remember a project early in my career where the project leader asked everyone to attend a meeting and bring a legal pad and a pencil. He started the meeting by going around and breaking everybody's pencil. The message was clear--take what we have and wrap it up. Don't add any more bells and whistles. Maybe whoever is running these projects needs to break some pencils.

    ReplyDelete
  7. John GaltApril 16, 2024 at 4:58 AM

    When you have undisciplined and unsophistciated Customers and Engineering Leads you get the SW teams that are described above. Anyone that tolerates the "I'll tell you the design after it is coded" gets what they deserve. Unfortunately SW has sold and ingrained the culture of rapidly producing immature versions as a measure of progress or genius. No one holds SW teams accountable for versions that do not work like they do the HW teams. Becuase it is only labor that is wasted (and what else would they be doing) and not capital funds (in the case of a poorly designed circuit board) (oh and we have to show capital expenditures on the books), there is no feedback to impove the quality of SW products. Make inital product development SW labor a capital expenditure and you will see management (or at least the CFOs) wake up and pay attention.

    ReplyDelete
    Replies
    1. ComNavOpsApril 16, 2024 at 6:39 AM

      I have a somewhat different take on software. I think it's treated as just another component, laying on a shelf. "We'll buy a fuselage, some electronics, an engine, and some software."

      As we've repeatedly seen, buying non-existent hardware leads to runaway costs and major schedule delays. Identically, buying non-existent software leads to runaway costs and major schedule delays - even more so than hardware, history suggests.

      We need to stop viewing software as just another component and start viewing it as a major acquisition project of its own. The military should require prototype demonstrations as they [supposedly] do with hardware. If it doesn't exist, don't buy it. As a side benefit, perhaps this would encourage/force manufacturers to:

      - write highly object/class oriented software than can be easily reused/repurposed
      - simplify software and eliminate the tons of garbage functions that are currently included in every piece of software but will never be used in combat. This is analogous to word processing software. Initially, those programs allowed the user to type and print. Simple, functional. Now, word processing software contains a million functions that the average user never uses. They just cause code bloat.

      Delete
    2. John GaltApril 17, 2024 at 4:51 AM

      After 40 yrs of developing Mil and Comm systems that increasingly rely on SW to work, I find the lack of discipline in SW Dev to be the major problem. I have worked on many programs at different companies and the lack of use of design patterns, code review and test tools. This is becuase the CFOs don;t want ot spend the money and the coders don't want anything reviewing their work. Developers still want to use C for all application development because it lets them do any thing at any time.

      To your points, any Marketeer that pitches, and any Customer that believes SW is an off the shelf component is really naive. Just as in HW; First you look for a team that has experience in the problem domain, Second you carefully evaluate if the off the shelf claims are valid, Third if it is a new development you want the experienced folks to give you realistic estimates for development time and cost.

      Separating SW development from the overall system development is IMO an even bigger opportunity for disaster. We have too few experienced users to provide good input to system developers now, we can never find enough to support separate HW and SW development. And letting undisicplined SW teams develop on thier own would be an unmitigated disaster.

      I am intrigued and support by your suggestion for prototyping. That might put some discipline into the SW Teams. Discipline and accountability are the key issues to solving the SW problem. Red Minasi's book the SW Conspiracy to get a now historical but still current view of some issues with SW quality.

      Delete
    3. ComNavOpsApril 17, 2024 at 8:23 AM

      "any Customer that believes SW is an off the shelf component is really naive."

      That would be the military and that's why so many (all?) of our acquisition programs having been failures.

      "Separating SW development from the overall system development is IMO an even bigger opportunity for disaster."

      This is betraying your lack of experience as a programmer. Object/class programming is a technique whereby the specifics don't matter (the specifics are just parameters to be passed to/from the object). For example, I can software model an elephant and a tank using the same software object. When I want to use it, I just supply the specific parameters such as speed, weight, armor (metal on a tank and thickness of skin on an elephant), lethality (gun or tusk), sensors (eye or IR), and so on. Thus, generic software can be written that covers any specific.

      Of course, assuming the software company has any relevant experience (a defense company), they'll write object/class code that focuses on military applications and not waste time writing objects for non-military actions like hibernation or reproduction.

      Separating software from hardware is actually a good thing in that it forces the programmer to write very good code since there are no specifics. The code must be good enough to be applicable to any set of specifics.

      It's kind of like building a ship with standard interfaces for modular payloads. Modularity, as the Navy envisioned it, is a failed concept but standard interfaces are not.

      "prototyping"

      Once a company has developed a non-specific, object/class software program prototype, the military can evaluate it using whatever specifics (ship, aircraft, missile, whatever) they're currently interested in. The program is generic so it doesn't matter. I've done this in real life albeit mostly non-military applications (did a few low level military applications, all successful).

      Delete
    4. John GaltApril 18, 2024 at 3:38 AM

      I did IV&V on the DDG-1000 and saw first hand the debacle that resulting from folks thinking Object Oriented Programming would solve all problems. From just that one datapoint (not to mention 40 yrs of other examples) excuse me if I do not believe the hype of Object Oriented. I have managed Development efforts since OO came out in the 1990s. Since it has been out so long and is such an easy and obvious fix, why do ALL systems (Commercial and Military) continue to have failures? Ask Alaska Airlines about SW upgrade impact on operations.

      You mistook my comment about separating the SW development. Objects cannot be developed independent of knowledge of how they are expected to work. A simple real world example from the dot bomb era. A web builder looked at the zip code (probably in Calif where Zips start with 9) and made the input field a number type. They at least realized it had to be 5 digits long. Guess what in NE you could not enter your Zip Code to register becuase it starts with a 0 and leading zeroes are stripped off. 10% of their market was unable to register with them. Opps good object structure, lousy understanding what data will come in.

      Delete
    5. ComNavOpsApril 18, 2024 at 6:09 AM

      You seem to have had some very bad encounters with some very poor programming people. That's unfortunate. You've apparently never seen what good programmers can do when guided by knowledgeable people.

      You also seem to think that I'm suggesting that object programming is a miracle that allows barely literate people to program a trip to the moon. Good programmers carefully study their subject to understand what objects to construct and what actions the subject requires. Much of that study comes in the form of consulting with, and listening to, subject matter experts.

      Your example of zip codes illustrates the poor quality of the programmers you've encountered and the value of object programming. What should have happened is that the parameter (the zip) should have been passed as a string of indeterminate length and any characters and then parsed out. Thus, the input becomes generic and can represent ANY string of ANY length. That's flexible programming that's foolproof. It can, with no changes, accept zip codes, phone numbers, names, passwords, etc., all from a single object/action. THAT'S what I mean by generic programming that can, with minimal modifications, be applied to ANY weapon system. That's what allows the program to be separated from the specific subject.

      I see a real world example of the failure to implement this approach every time a new sensor or weapon is added to the Navy's inventory. It always has to be individually integrated into the existing combat control software at great cost and great delay. If the combat control software were programmed generically, it wouldn't matter what weapon or sensor was added. The only thing that would change is the specific parameters that get passed.

      Of course, I'm grossly simplifying this discussion but you understand that.

      I too have encountered some stunningly bad programs and programmers but they're instantly recognizable and I've avoided them like the plague. The Navy, on the other hand, seems to seek them out.

      If you're not yet retired, I hope you have the opportunity to work with good programmers at some point. Good luck!

      Delete
    6. AnonymousApril 18, 2024 at 12:19 PM

      Maybe you have selected a bad example, but you describes a really bad programming practice regarding usability, security and more.

      In my experience some of the issues that plagues software development in general are:

      Customers that don't know what they really need and demand unnecessary functionalities that waste efforts forgetting what is really useful.

      Lack of frankness on the side of sales management that hide real times and costs selling software development like something easy and effortless on client side, when in fact requires almost the same effort on the client side as in the development side.

      Lack of discipline from the part of project management enforcing adoption of proven methodologies and methods of development.

      Inadequate mix of senior and junior developers. If most are seniors probably became bored doing easy parts and adding unnecessary features. If most are juniors probably seniors will be overworked and juniors will wander headless.

      There are more but I think this is enough for now.

      JM

      Delete

Comments will be moderated for posts older than 7 days in order to reduce spam.

Subscribe to: Post Comments (Atom)