Thursday, March 10, 2016

Credibility - Not All Reports Are Created Equal

ComNavOps happened to read a comment posted on another blog in which the reader accused critics of the F-35 of only giving credence to other critics and ignoring positive reports even from presumably knowledgeable sources like pilots who had actually flown the aircraft.  It made me reflect that, yes, there is a natural human tendency to do that.  We form an opinion and then tend to cherry pick the data and reports that support that opinion.

The thought made me reflect on my own tendencies and, to be honest, I do that the same as anyone else, to a degree, although I try very hard to stay objective and give equal weight to the good and bad reports about any particular subject.  Anyway, that would have been the end of the matter – a mental reminder to myself to stay objective – except that I then happened to read two articles in the most recent issue of Proceedings which illustrated and reminded me why I don’t always give equal weight to both good and bad reports.  It’s because not all reports are created equal.  Not all reports are objective.  Not all reports are fact and logic based.  All reports have an agenda.  Recognizing that, I weight the reports according to the degree of objectivity evidenced in the report, the degree of bias revealed, the agenda of the author, etc.  Based on that assessment, I may accept the report at face value, I may reject the report utterly, or, more likely, I may weight the various points and factor them into my own opinion accordingly.

Here’s a simple example.  Early on, when criticism of the LCS was mounting and the Navy was desperate for some good PR to rebut the critics, they ran the ship’s Captains out and had them do interviews, articles, and reports.  Unsurprisingly, those interviews and reports were over-the-top positive, providing the impression that the LCS was exactly the miracle war machine the Navy claimed it was.  Well, by that point it was obvious to all that the LCS had serious problems and lots of them.  Thus, the Captain’s interviews and reports were clearly not objective – there was no hint of even the slightest negative - , they were clearly heavily biased, and the Captains and the Navy had an obvious agenda which precluded objectivity.  Therefore, I read the reports but largely dismissed them – not because I refused to listen to opinions that ran counter to my own but because the reports were not credible even though the source, the ship’s Captain, should have been nearly the ultimate authority on the subject.

Here’s another example near and dear to us.  The current Proceedings has an article by Col. Matthew Kelly, USMC, who claims to have flown “hundreds of hours” in the F-35.  There should be no more authoritative expert on the subject, right?  Well, the article spends several pages extolling the virtures (almost magical, according to his descriptions) of the aircraft and offers a single, vague sentence about unspecified hardware and software updates that are coming.  That’s the extent of the negatives in the article.  Given the litany of problems with this aircraft, that alone suggests a severe lack of objectivity by the author.  The author’s affiliation, the USMC, suggests an agenda and bias – the USMC declared IOC in what was clearly a public relations stunt after a hugely unsuccessful trial.  Thus, the author’s credibility and objectivity are seriously questionable.

The author describes how the F-35 will operate in teams of four to sweep the skies clear of enemy planes.  No credit is given to enemy capabilities, whatsoever.  The F-35 has stealth, data linking, off-boresight targeting, etc. which will utterly overwhelm the enemy.  Absolutely no attempt is made to explain how the enemy’s stealth, data links, and (superior?) off-boresight targeting will factor into this. 

In short, the article reads like a sales brochure. 

So, what did I take away from this article?  Not a thing.  The Marines trotted out a pilot to do a PR sales piece.  I rejected the article not because it differed from my own opinion but because it wasn’t credible. 

The second Proceedings article was about the Zumwalt and was written by the ship’s current Captain, James Kirk.  Again, the article reads like a sales brochure.  The ship apparently has no flaws except for an almost insignificant restriction in the heavy seas operating envelope.  Other than that, it’s clear from the article that a single Zumwalt can win any war by itself (so why did we build three, I wonder?).  The only question is whether the ship will need to leave dock to do it or if it can win the war from its home port.

While the article offers a few interesting factoids, it is clearly a PR piece and, again, I reject it as a basis for influencing my opinion.

Now, recognize that this works both ways.  I’ve read reports on the F-35 and LCS that were nothing but criticism and those are no more credible than the reverse.  A report that would have me believe that those platforms have not a single positive, redeeming characteristic is just as clearly biased and unbelievable as the reverse.

So, to respond to the original complaint by that far removed, unknown commenter, the reason why positive reports are often summarily dismissed is because they lack credibility.  I would also remind the commenter that supporters cherry pick their data as much or more than critics!


  1. I really was a supporter of the F35, LCS, etc when the projects were announced and were moving along the first couple of years. There's no doubt that we have to replace our older military systems eventually or try some new concepts, I slowly changed my mind and became "against" them when it was OBVIOUS to all to see that these projects were running into trouble, were late, over budget or had lost their significance BUT we were still getting positive news from manufacturers and military, that's when they lost me. You can't retain your credibility when in the face of facts and reality, you still maintain everything's fine!

    1. Spot on.

      I am dismayed that Managers cannot say: "We learned a lot on this project, namely not to try that again. Now we move on."

      But that is something that LEADERS say NOT managers.

    2. You always have to look at the intent of the source whenever you have a report.

  2. I've wavered on the F-35. I'll admit to wanting it to work in my heart of hearts. The problem is every time I start to lean that way, then DOT & E comes out with some report that is hard to justify as 'bias' (Like them being unable to keep the bay doors closed at speed for more than 10 minutes).

    The reporting on the F-35 to me is one of cover up and subterfuge. 'Its meeting targets (that we moved). Its at IOC (but can't do anything)!

    I thought the LCS was neat originally. And then it all got werid. NLOS failed. The modules failed. And the reporting got all sorts of insane. 'It will dominate the littoral battlespace!'. What does that even mean? And how can any 3000 ton ship do it? I listened to a midrats where Bob Work went on about how it was bigger because we couldn't count on having foreign bases. Then I read in ID that its range isn't an issue because we'll be deploying it in flotillas in foreign bases.

    I read that its okay it doesn't have good range or a committed ASW suite because it won't be doing that in the blue water. Then I read it will be.

    I read that its okay its taken so long because 'its expirimental' yet we are going into serial production without *any* of its concepts proven.

    The list goes on and on; and gives me the impression that A) They don't know what they want it to do and B) they just throw out claims to keep support. I won't even get into the price.

    I honestly think there is a group of people who may or may not like the ship, but are so tied to the modularity concept they can't admit defeat.

    Anyway, that's one of the reasons I come back here. I may not always agree with you CNO but I think you do a very good job of balancing the reports (and occasionally ripping the bad ones to shreds, which is fun).

    What worries me is that the Navy has gone so head over heals to support questionable weapons. They are burning their integrity so to keep $ flowing from Congress because 'they have to'. But what will happen when the fleet can't do what the President wants it to do? After Congress has been convinced to fork over the money?

    The Navy will look like fools. And they'll be hard pressed to get more cash in the future.

    1. "I may not always agree with you CNO "

      I'm impressed that you admit that you're sometimes wrong!

    2. CNO, I hear its the beginning of wisdom. *grin*

  3. No sane active duty officer dare utters anything negative about anything their service is buying. Retirees may speak out, but then check who they are now working for. And when you see those expensive ads for weapons in Proceedings, that is a form or bribery to keep the editors in line. That's why they give that money, not because they think a reader might want to buy a nuclear attack sub.

    This reminded me of the V-22 when G2mil once wrote:

    "I took note that "Aviation Week" had run several negative letters about the V-22 in recent weeks. They depend on the defense industry for advertising money, so they rarely mention anything negative about any major program. I knew they had read my recent article, so maybe this is their way of getting the word out. It must have angered Bell-Boeing, so it looks like they cut a deal. The 1-02-06 "Aviation Week" has a very unusual 16 full-page advertisement about the great V-22, it must have cost Bell-Boeing a million dollars, oh I mean American taxpayers. What is strange is that it is written as an article, but according to very small print at the bottom of the first page, it notes this is a special marketing supplement.

    That section quotes the head of Marine Aviation, General Michael Hough, about what he told all Marines about the second operational evaluation: Hough stated: "And we have to do things in such a way that we can show unequivocally that this capability is safe." As a result, Hough should be fired and possibly arrested for criminal conspiracy. The experts at IDA told him the V-22 was not safe, which is why he put out the word that "we have to do things in such a way." He should have stated that we will have to see if the contractor has built a safe aircraft that meets the performance they promised. However, he is quoted several times saying about what "we" needed to prove. He sounds likes someone working for Bell-Boeing because they are the ones who needed to prove the V-22 to the Marine Corps. (His three predecessors went to work for V-22 contractors shortly after retirement.) So Marines followed orders and did things "in such a way" like skipping tests and having a loyal "friend" at DOT&E falsify the results."

    1. Anon, those IDA views about V-22 were done back in 2003.
      There is an anti V22 website
      When you look at some of its conclusions, maybe they were wrong too?
      "The bottom line is that all of the these concerns argue that V-22 is likely to have a larger accident rate when operating within a combat or hostile environment than any conventional helicopter simply because the aircraft has many more ways of getting into trouble than does a conventional helicopter."
      Hows that V-22 accident rate these days?

    2. And for the commandants of the marine corps retiring and going to work for V-22 contractors.
      Well the first aviation CMC General Amos was appointed a director and is now Chairman of Lord Corp. yes Id never heard of it either.

      "LORD Corporation is a diversified technology and manufacturing company developing highly reliable adhesives, coatings, motion management devices, and sensing technologies that significantly reduce risk and improve product performance."

      What exactly they make for the V-22 isnt clear, but it probably applies to all helicopters.

    3. We don't know the mishap rate because they now restrict access to mishap data, and only acknowledge mishaps that become public. There is strong evidence that a dozen V-22s that were seriously damaged and quietly scrapped with no mishap investigation. Congress asked questions and..

      This indicated that as of June 3, 2009, 105 MV-22 "Osprey" had been delivered to the Marine Corps since 1988. Only 47 are combat deployable; 29 are not usable (includes experimental, damaged, destroyed, or otherwise unflyable); 27 are useable only for pilot training; and two paid for but not delivered. Of the 47 combat deployable, only 22 were mission capable on June 3, 2009. According to a June 19, 2009 internal memorandum from the committee staff:

      "The Committee staff encountered major difficulties in attempting to determine the answer to what might ordinarily seem like an obvious question: How many Ospreys does the Marine Corps have and what is their flight status? The Defense Department seemed to have serious difficulty in assembling this information.

      ...on May 28, 2009, the Committee received a partial document production of four small binders, and on June 2, 2009, staff met with Marine Corps officers who provided additional documents that filled some of the information gaps remaining from the initial production regarding inventory. During the June 2 meeting, staff again requested internal memoranda concerning the operational status of the MV-22 fleet and received a second production, a single binder, on June 5, 2009. 'We do not believe that the total of five binders produced to the Committee represents the full universe of responsive records."

      But they have limited the V-22 problems by keeping it off dirt LZs and using CH-53Es for more missions. And payloads are limited to just 8000 lbs, check the data in the 2015 Marine Aviation plan, not the 15,000 to 20,000 promised. So the V-22 is the size of a CH-53E but lifts only as much as an MH-60! And it costs twice as much and has a 50% readiness rate. This is why no other nation is buying tiltotors, although they arm twisted Japan to buy 17, but that is still not final.

    4. Anon.... your information has my blood boiling...but why hasn't any decent, non military based paper done anything? The NY Times? The Washington Post? This sounds like flat out corruption. And its not like those news orgs have any love for the Military, as far as I know.

    5. Jim Whall, you are entirely right and that's another problem we are facing in this country. We don't have reporters anymore, they have been fired or all ran for cover, we have "commentators", it's not the same thing!!! Nobody except for a few bloggers like COMNAVOPS,etc are even talking about this, once in awhile you'll see an a hard hitting article but they are too easy for the big defense boys to shoot down or never get any airtime on the "big" media waves....there's no accountability!!!!NONE!

    6. "Anon.... your information has my blood boiling..."

      Really? That's what got you fired up? You do recall the post documenting the doctored operational trial of the F-35B by the Marines and how they declared IOC despite a 50% unavailability during the test and numerous other covered up shortcomings? This is a pattern by the Marines.

    7. I think the next big eye opener will occur in a couple of years, when after much dicking around, someone will have to stand in front of a mic and announced that the Ford's launch and arresting system does not work (but will say lacks reliability). Then what? Will anyone dare ask Congress for $5 billion rip it apart and put in steam cats?

    8. Jim, you want to boil your blood more, read this list of V-22s no longer in service due to reported and unreported mishaps.

    9. Agree with ANON, with the new Ford carrier, how will someone explain away a carrier that can't launch jets?

      Will USN still go for it and risk a couple of jets crashing in the water before they pull the plug? Then which admiral goes on stage in front of national media and explain that EMALS doesn't work?!?

    10. CNO, honestly its been just a drumbeat of things that hit a tipping point with me. Maybe I'm slow. Maybe I just want to assume that there's well intention people making bad decisions. But... *sigh*.

      Its not just the V-22. Its all of it. Osprey, F-35B, Ford, Zumwalt, LCS...

      and people defending it all! Good Lord! We're moving into an arena where 'Our Navy is X times larger than...' won't mean a whole lot because its operational ability will be so limited.

      its a fine line for me though. Blood boiling can go to despair pretty quick when I realize how deep the problem is.

    11. Its impossible to respond when "Anon' doesnt respond when his falsehoods are revealed and just moves on to 'hidden secrets' and 'suppressed data'.

      Its not as though there isnt enough incompetency and waste of money out there, as CNO regularly shows - in the correct way.

    12. Ztev, I assume you're taking exception to the MV-22 altered data accusation and the g2mil data? If so, I've carefully read the g2mil site's data and I find the information compelling and, as far as I can tell, accurate. Other reports strongly suggest that the Marines have been, and still are, engaged in systematic under reporting of mishap incidents. This probably also ties in with the DOT&E's documentation of the Marine's farcical F-35 operating trial which lead to IOC.

      If you have data to the contrary, please share it.

  4. In 2003 the V-22 Nacelles were being, or had just been, FINALLY redesigned and the PM was an AIR FORCE LtCol.

    The accident rate has been low SINCE the redesign, but the point of this blog article is how trust has been lost by service reports.

    It took 34 Marines Killed and 2 crashes to get the redesign and the Vortex Ring effect to be investigated. In the Meantime the Commandant flies in one with is Wife to reassure folks.

    1. Now we are being told the vortex ring effect in V22 is actually less than other helicopters.
      have you moved on from 13 years ago ?

    2. I haven't seen that. Do you have a reference or link?

  5. I dont think the F35 is particularly bad, I think its pretty normal, I think whats changed is the ease with which we can read about it, understand it, and most importantly, talk about it.

    The lamentable SA80

    Its fine to say "only 20% of the code has been written", it sounds really bad, but, as someone who doesnt write code for jet fighters for a living, but does design spreadsheets one, 95% of the job is getting the customer to specify WTF it is they actually want, that 20% written could be 95%, or 1% of the actual work completed.

    I was reading something about the radar, which apparently fails every 5 hours and overheats on the ground, arghhhhhh, crisis, except the most modern F15 radars fail every 15 hours, and the overheating issue is worked around by leaving the bomb bay doors open on the ground

    The F35 isnt some wonder weapon that will win wars from the ground, I very much doubt its going to get clubbed like a baby seal either.
    Concurrency has certainly been a disaster, and your 5 year rule would have solved many of the problems.

    1. "I dont think the F35 is particularly bad, I think its pretty normal, I think whats changed is the ease with which we can read about it, understand it, and most importantly, talk about it."

      The availability of both information and our ability to discuss it has certainly increased many fold over the last couple of decades and that puts a much brighter spotlight on normal developmental hiccups. Flaws are magnified.

      On the other hand, the F-35 has been in development for 20 years and that is unprecedented and unacceptable. That is a badly flawed program with or without a spotlight!

  6. TrT: As I said earlier. In my heart of hearts I'd love for the F-35 to work... but the issues that keep cropping up (like needing to keep the weapons bays open during a large portion of the flight envelope) are a night mare 15(!) years into development.

    As stated in an earlier post, by this time the Tomcat was in the fleet and we were working on upgrades. Not only that, it had shot down planes!

    The F-35 at this stage isn't really combat ready, and likely won't be for a few years yet.

    This process and plane are flawed; and by the time we get it to the point of fighting, I'm not sure it will have the edge its supposed to.

    1. I dont so much disagree, but, I lay the blame differently.

      The F35 development cycle has been slow, its been even slower if you look at the programs that led to the F35 competition.
      But no ones been cracking the whip and moving things along.

      At every step of the way, every party has dragged their feet, because they have all wanted to avoid spending any money.

      The Eurofighter started in 83, flew a demonstrator in 86, was finalised in 94, went operational in 2005 and as of 2016 it is still awaiting funding for Meteor, its main air to air weapon, thrust vectoring, conformal fuel tanks, aesa radar, ecm/eccm upgrades ect ect ect

      Nothings ever simple.

    2. Eurofighter was held by its political masters mostly plus german re-unification threw a spanner in the germans spending plans.
      if it was me I would have used the new engine to make the Tornado a proper air to air fighter first and made the Typhoon more focused on its multi role capacity

    3. "At every step of the way, every party has dragged their feet, because they have all wanted to avoid spending any money."

      I don't see it as an unwillingness to spend money. Heck, the military is gung ho to spend. If they could, they'd accelerate the F-35 buy rate! So, quite the opposite - money is the least of the problems (politically, that's another story, of course). The issue is simple technology failings. The sensor fusion has struggled badly causing delays. The helmet has been a disaster and caused delays. The airframe has had problems causing delays (cracked bulkheads, arrestor hooks, etc.). The maintenance software is years behind schedule. And so on.

      Of course, the manufacturing companies benefit from delays because the military keeps pouring money into fixes and remanufacturing so they're happy to take their time but it's not because of a reluctance to spend money - they're spending the government's money (and mine!) not theirs.

      I think the military is pressuring the manufacturer and "cracking the whip" as hard as they can. Look at the Marine's fairy tale IOC episode. They're bending over backwards (and fraudulently) to speed up the acquisition because they're terrified that Congress is eventually going to get fed up and start cutting the program. Unfortuantely, there are just too many tech problems to overcome in the desired time frame. This is all due to initiating production with non-existent technology.

  7. The technical psychological term for the tendency to believe information that agrees with our current opinion and dismiss info that disagrees is "confirmation Bias".

  8. A Navy Warfare Doctrine challenge...

    food for thought

  9. "We don't have reporters anymore, they have been fired or all ran for cover, we have "commentators"

    This is very true. I'd love for a huge expose on all of this to happen.

  10. "We don't have reporters anymore, they have been fired or all ran for cover, we have "commentators"

    This is very true. I'd love for a huge expose on all of this to happen.

    1. Who's gonna write it?

      "We don't have reporters anymore, they have been fired or all ran for cover, we have "commentators"

  11. Nice little article. It is an f'ing mine field.

    I cannot believe the USSR and NATO produced this level of dis-information during the entire cold war.

    Attempting to extract fact or even determine ( as i tend to go for now ) just the level of bias. Is a full time job.

    And this before we even face up to our own preferences.

    Lets all just agree its a rich tapestry... lol

  12. Oh btw. Have you checked out BREAKING DEFENCE today. Some disturbing slides shown recently by Mr Bogdan relating to F35 and B21. One or two points on there really had me worried about F35s true focus.

    1. Half of the AF's items are demonstrably false and the other half are questionable. This is just outright lying.

    2. Just confirms a long held view that the F-35 is the AF 'short range bomber'


Comments will be moderated for posts older than 7 days in order to reduce spam.