As we’ve seen and discussed, software has become the major
obstacle to successful programs and the F-35 is probably the poster child for
this. The F-35 logistics and maintenance
program, ALIS, was supposed to have been a miraculous piece of software that
would do … well … everything plus several things we haven’t even thought of
yet. As it turned out, it couldn’t run a
toaster correctly and may be the biggest military software flop in
history. In addition, the vaunted Block
4 software that enables full combat capability from the aircraft is years
overdue with no implementation date in sight and many of the features have been
permanently deferred to a non-existent ‘future’ date or deferred to the next
aircraft program. Those features will
never be part of the F-35. Even worse,
because of the hugely delayed software and the resulting concurrency, we’ve
produced hundreds of orphan F-35s that are so out of date that they will never
be upgraded. They’re essentially garbage
throwaways at $100M each.
Could all this have been avoided? I don’t see how. After all, this is how aircraft development
and acquisition programs go, right? I
mean, everyone knows that, right?
Well, let’s now take a look at how the software should have
been handled and how all of this could have been avoided. Wait … is that really possible? Could all of this have been avoided? Yes, it could, quite simply, as a matter of
fact, and now we’ll see how.
Requirements – To begin, the software was insanely
over spec’ed. Only a lunatic – or the US
military – would have attempted to cram so many features into the software. We’ve talked about this at length. The software had far too many requirements
that had nothing to do with combat (looking at you ALIS) and nothing to do with
realistic combat (looking at remote guidance handoffs and similar technology
for the sake of technology features).
The military is obsessed with technology for its own sake. Our guiding principle should be K.I.S.S. (Keep
It Simple, Stupid). One could also refer
to this as K.I.S.A. (Keep It Simple, Admiral) since ‘admiral’ is a synonym for
‘stupid’. The two words are
interchangeable.
By reducing the software requirements to just direct and
realistic combat features we could have cut the scope, cost, and time to
produce the software in half, or whatever actual large proportion. So, half our job is done before we even
begin!
Think of this as the Boyd approach to software: not a single line of code that doesn’t directly
enable realistic combat. Programmer Boyd
would be proud of us!
Of course, even with drastically pared down requirements, some
significant amount of software is still required and here’s how it should have
been handled.
We start by recognizing three absolutely critical and
obvious conditions:
1. Software is
platform agnostic.
2. Software must be
its own, stand-alone program, separate from the hardware program.
3. Hardware development cannot proceed until
the software is finalized.
Platform Agnostic Software - This is the key to software
development. We didn’t need to have an actual F-35 aircraft in order to develop
the software. The software can be
developed in laboratory simulations or on surrogate aircraft if an actual aircraft
is needed for some reason that I can’t fathom.
We could have loaded the software on a Piper Cub, if necessary. This would have allowed us to completely
develop the software without every spending a penny on the physical aircraft. The physical aircraft is the last thing we need. It’s the final piece of the puzzle not the
first. The F-35 attempted to do the
reverse. They built the aircraft first
and then remembered they had to add some software. Utterly predictably, that approach failed
miserably.
Stand-Alone Program Management – Software comes
before hardware and must be its own, stand-alone project from a program
management perspective. Software cannot
be an afterthought, add-on to the hardware as it was with the F-35. The only way to effectively manage the
software and avoid having it become an afterthought is to make it its own
program with its own, clearly and rigidly defined requirements, schedule, and
milestones. If the software fails to
meet its milestones, you don’t screw around or extend the program; you kill it, learn lessons, fire and
court-martial the managers, and move on.
Failure, then, becomes a positive in the sense that it serves as a
warning – a severe warning – to the next project and manager to maintain tight
control on the project and to be realistic regarding capabilities. No more promising the moon and delivering
nothing. Fire and court-martial a few
people and the next ones fall in line.
Start Point - Software is the hardware enabler. Without the software, the hardware is just a very
expensive paperweight. In other words,
there is no point even buying the first hardware rivet until the software is
complete. Thus, the completion of the
software is what controls and triggers the start of hardware development. This doesn’t mean partial completion with
some nebulous phased delivery in the ephemeral future; it means just what it says: complete.
Finished. Nothing left to
do. The contract is fully
satisfied. ‘I’s’ are dotted and ‘T’s’
are crossed. Last line of code is
written. Had we followed this with the
F-35, we would not yet have spent a single penny on the F-35 aircraft,
itself. What a savings! We would not now have hundreds of orphans and
wasted billions of dollars and have an entire fleet of only partially combat
capable aircraft. We might have even
recognized, decades ago, that the software was not viable and halted the
program while we could have still easily gotten out from under it. We could have learned lessons and moved on to
a second, wiser attempt (who am I kidding?
we don’t learn lessons … but, I digress).
Conclusion
Nothing that I’ve described is anything other than common
sense and none of it requires any act of Congress. It could be implemented tomorrow by nothing
more than a command from the Chief of Naval Operations or Secretary of the Navy. Some of you will protest some aspect or
another but that’s just you being trapped in your paradigm and unable to see
outside it.
Had we done this with the F-35, do you see the very early
off-ramp that would have presented itself when it quickly became obvious that
the software would not, and could not, be delivered in any useful time
frame? For just a hundred million
dollars or so of initial programming, it would have become clear that we did
not have a viable software and the program would have terminated with the
hardware portion never having begun. How
many billions of dollars would we have saved?
Forgive me for being thick. But don't you need a physical plane or at least a computer generated one so you can write the software on how to fly it? It would be interesting to know how Airbus / Boeing etc do software / hardware integration on new aircraft. At what point the go from computer screen to physical prototype etc. Clive F
ReplyDeleteNo. Previous aircraft lay the foundation and simulations and wind tunnel tests get most of the rest. Sure, eventually you have to try it out on a prototype/test aircraft.
DeleteThink about it. When that first prototype makes its maiden flight, what is it using for flight control software? The pilot isn't sitting in the cockpit writing it as he takes off. It's already been completely written. Test flights may result in tweaking some of the parameters but that's it. The software was already done.
LockMart has the Catbird a 737 with F-35 hardware kit
Deleteinstalled for software/hardware development.
The plane first flew about a month after the F-35 first flight, a bit late. Good news Tech Refresh 2 Block 3 has been on the Catbird since 2014, bad news, 9 years and TR2 isn't ready for prime time. LockMart should have used Windows for Warplanes. The F-35 is fine example of today's software trying to run on a decade old computer,
slow and crunchy performance results. Not what you
want at 600 knots.
"Think of this as the Boyd approach to software: not a single line of code that doesn’t directly enable realistic combat."
ReplyDelete"Directly enable" may be doing a lot of work here. The software does need to be able to check its own integrity, both at start-up, and while the plane is flying. Most computers don't get shot at, and electronic attacks on aircraft are also a real thing.
You also need to at least try to keep the plane flyable when it is damaged, although that might well be done by dropping down to a much simpler set of control laws.
The Catbird also makes a special problem obvious. You either need to upgrade aircraft computers regularly, or accept that they will be using hardware that is very old and slow by current standards. The difficulty with upgrading is the approval standards for military computers. Those are quite onerous and expensive for manufacturers, and the USAF and USN tend to want to stick with familiar hardware. Then they demand wonderful new software. And the salesmen, with their commissions in mind and no personal obligation to deliver, promise the impossible. The customers should know better to believe them. But they don't.
Before Boeing got ruined by GE alumni, and had captured the FAA, it knew how to build complex HW and SW systems, i.e. commercial airliners. Furthermore time is money and they usually delivered on-time. The 777 is a good case to look at. I believe it was certified faster than any other airliner. It also used a strong typed language called Ada, even having one sub start over, See the link here: http://archive.adaic.com/projects/atwork/boeing.html
ReplyDeleteIf you stay focused on only necesary requirements, use tried and true manufacturing methods, and have a highly skilled workforce you can make good products on time and make a ton of money.
"In addition, the vaunted Block 4 software that enables full combat capability"
ReplyDeleteBlock 4 upgrade includes many new Hardwares, for instance, APG-85 radar, new IRTS, new engine able to extract more electricity, ....
Hardware is basis of software which builds upon hardware. Many of these new hardware don't even have firm projected dates.
We will never know (top secret and garbage) where all this code goes BUT it would be interesting to have a break down of what requires all this code, radar, ECM, flight controls, weapons,etc.... what is so software intensive and why?
ReplyDeleteAs a curious detail, just today, image was released of a new pylon on a Ukrainian Mig29 carrying 2 SDBs.....Im sure they aren't as well integrated as on a F16 or F35 BUT how can we get them on a MIG! so fast and seem to take forever to get them integrated on a US platform?!? How and who did all that software?
Last, concerning weapons, why does it take so long? I can understand a brand new weapon BUT some US weapons have been around for ever, sure, AIM9B isn't the AIM9X in terms of software requirements BUT it's a not a new weapon, how come it's not more PLUG and PLAY? it seems like we have to reinvent the wheel or write the software each time on a new jet like we have never seen or heard of a SIDEWINDER before?!?
"what is so software intensive and why?"
DeleteYou ask a good question and I have no answer. However, I would point out the number of functions that are purely software and offer no significant combat enhancement. For example, the 360 degree sensor fusion with real time helmet virtual reality requires incredible amounts of software to work (it doesn't work, yet!) and offers no real combat advantage. If the pilot wants to know what's to the left of him, all he has to do is turn his head slightly and look! He doesn't need a sensor fusion, 3D, holographic, magic helmet. That's technology for the sake of technology. That's an example of the kind of thing we waste software resource on that offers no significant return on investment effort.
" reinvent the wheel or write the software each time on a new jet like we have never seen or heard of a SIDEWINDER before?!?"
DeleteI'm pretty sure the software to launch a Sidewinder is not the problem. The problem is the added capabilities like being able to take a guidance handoff from a cowboy in Montana and then hand off to a Boy Scout in the Arctic just so we can guide a missile that the launch platform could have just as well guided on its own. Again, tech for the sake of tech.
ALIS software works well in Australia.
ReplyDeleteIt was advertised as "soldier proof". It should be as it is its design goals.
It isn't.
In Australia it had to be certified before it was rolled out. Many people with doctorates tested it.
When rolled out they had reach back to the experts.
So you have to be trained to use ALIS. It is not soldier proof.
Admiral’ is a synonym for ‘stupid’. - That is very funny :)
ReplyDeletehttps://nationalsecurityjournal.org/f-a-xx-fighter-is-in-trouble-what-does-that-mean-for-navy-aircraft-carriers/ ( By Robert Farley ) "But now the Navy is hesitating to invest development funds in the F/A-XX. Beset by a bewildering array of priorities and concerned about recent, massive procurement failures, the Navy quietly cut back on F/A-XX investment in its latest budget request. " My take: We see in this article that the Navy will discontinue the purchase of the Super Hornet. No mention in this article about the F35 software problems . The massively expensive Ford class carriers have major issues for resolution.
ReplyDeleteSo the that Navy has a problem with the F35 and the aging F18 superhornet fleet. )
" There is no question that the Navy faces a difficult financial future. The Columbia class ballistic missile submarines, replacement of the long-serving Ohios, have taken and will continue to take a gigantic bite out of the service’s budget. Other surface and submarine procurement projects also have huge budgets (and sometimes huge budget overruns).
"There is no question that the Navy faces a difficult financial future."
DeleteThis is utterly false. The Navy's financial difficulties are PURELY their own fault. If they wouldn't spend money stupidly (Ford, LCS, Zumwalt, LAW, AFSB/MLP, etc.) they'd have unimaginable gobs of money for whatever they wanted. Simply reverting to Nimitzes instead of Fords would save around $8B per carrier!!!!!.
"There is no question that the Navy faces a difficult financial future." This quote is from Robert Farley's article.
DeleteThere have been many spending travestys with the Navy's programs ! And now the F35 is not up to full potential. ( I am just a layman who likes your blog and wonders if the Navy will ever change course in the right direction ! )
"wonders if the Navy will ever change course in the right direction"
DeleteHonestly, like most (all?) individuals and organizations, they'll change only when forced to. What can force them to change? A severe defeat as occurred at Pearl Harbor and in the naval battles of Guadalcanal. It is tragic but people will have to die in large numbers in order for change to occur.
If one looks at the very poor performanceof the MK14 torpedo, in WW2, up until a good part of 1943; it took leadership to force the fixes needed for much improvment. It took Admirals Lockwood, Nimitz , King & others to force change. The BUORD did not see a problem with the torpedo, until they were forced to say we have a problem.. Compare this with Navy leadership today.
Delete"It took Admirals Lockwood, Nimitz , King & others to force change"
DeleteNo, that's not quite right. Those worthies did not force change before the war though they had every opportunity. They forced change only after repeated defeats and lost subs.
"Compare this with Navy leadership today."
It seems identical. We, too, have developed a peacetime mentality and personnel, as happened prior to WWII, and only significant defeats will force change.
The British are unhappy with their F-35s and some call for pulling out of the program and moving on.
ReplyDeletehttps://www.yahoo.com/news/why-britain-f-35s-could-110000357.html
I never believed for one second that UK was going to buy the full complement of 138!!! F35 fan club kept telling us not to worry about the UK order,yeah, right, I would be surprised if they order more than 24 in the next batch and that's it, orders over.
DeleteIn the 80's through the 90's, DOD had a systems agnostic programming language (Ada) that was used for developing systems across the board. It has the usual problems of something developed by government committee; complicated, unreliable (at first), and the compiler for it had a reputation for being difficult to use. Nonetheless it was used in the F-14 and F-22, and air traffic control systems world wide among other things. Its purpose was to be a foundation for systems agnostic software you described.
ReplyDeleteBut future commonality on software died due to a move toward COTS and those easily available systems used code that was often propriety, and the nail was driven in by concurrency allowing software to not only be propriety, but now also safeguarded by the excuse of security.
One of Israel's conditions for buying the F-35 was being able to do some of their own programing. Internet rumors are that the Israeli F-35 availability rate is significantly higher.
If Israel can demand that for a sale why can't the US?
"future commonality on software died due to a move toward COTS"
DeleteSo, you seem to be suggesting that commercial off the shelf products may have been a case of 'penny wise and pound foolish' ? That's interesting to contemplate and, if true, offers lessons for us in the future as we embrace commercial cloud systems, AI, and many other things.