Credibility - Not All Reports Are Created Equal

ComNavOps happened to read a comment posted on another blog in which the reader accused critics of the F-35 of only giving credence to other critics and ignoring positive reports even from presumably knowledgeable sources like pilots who had actually flown the aircraft.  It made me reflect that, yes, there is a natural human tendency to do that.  We form an opinion and then tend to cherry pick the data and reports that support that opinion.

The thought made me reflect on my own tendencies and, to be honest, I do that the same as anyone else, to a degree, although I try very hard to stay objective and give equal weight to the good and bad reports about any particular subject.  Anyway, that would have been the end of the matter – a mental reminder to myself to stay objective – except that I then happened to read two articles in the most recent issue of Proceedings which illustrated and reminded me why I don’t always give equal weight to both good and bad reports.  It’s because not all reports are created equal.  Not all reports are objective.  Not all reports are fact and logic based.  All reports have an agenda.  Recognizing that, I weight the reports according to the degree of objectivity evidenced in the report, the degree of bias revealed, the agenda of the author, etc.  Based on that assessment, I may accept the report at face value, I may reject the report utterly, or, more likely, I may weight the various points and factor them into my own opinion accordingly.

Here’s a simple example.  Early on, when criticism of the LCS was mounting and the Navy was desperate for some good PR to rebut the critics, they ran the ship’s Captains out and had them do interviews, articles, and reports.  Unsurprisingly, those interviews and reports were over-the-top positive, providing the impression that the LCS was exactly the miracle war machine the Navy claimed it was.  Well, by that point it was obvious to all that the LCS had serious problems and lots of them.  Thus, the Captain’s interviews and reports were clearly not objective – there was no hint of even the slightest negative - , they were clearly heavily biased, and the Captains and the Navy had an obvious agenda which precluded objectivity.  Therefore, I read the reports but largely dismissed them – not because I refused to listen to opinions that ran counter to my own but because the reports were not credible even though the source, the ship’s Captain, should have been nearly the ultimate authority on the subject.

Here’s another example near and dear to us.  The current Proceedings has an article by Col. Matthew Kelly, USMC, who claims to have flown “hundreds of hours” in the F-35.  There should be no more authoritative expert on the subject, right?  Well, the article spends several pages extolling the virtures (almost magical, according to his descriptions) of the aircraft and offers a single, vague sentence about unspecified hardware and software updates that are coming.  That’s the extent of the negatives in the article.  Given the litany of problems with this aircraft, that alone suggests a severe lack of objectivity by the author.  The author’s affiliation, the USMC, suggests an agenda and bias – the USMC declared IOC in what was clearly a public relations stunt after a hugely unsuccessful trial.  Thus, the author’s credibility and objectivity are seriously questionable.

The author describes how the F-35 will operate in teams of four to sweep the skies clear of enemy planes.  No credit is given to enemy capabilities, whatsoever.  The F-35 has stealth, data linking, off-boresight targeting, etc. which will utterly overwhelm the enemy.  Absolutely no attempt is made to explain how the enemy’s stealth, data links, and (superior?) off-boresight targeting will factor into this. 

In short, the article reads like a sales brochure. 

So, what did I take away from this article?  Not a thing.  The Marines trotted out a pilot to do a PR sales piece.  I rejected the article not because it differed from my own opinion but because it wasn’t credible. 

The second Proceedings article was about the Zumwalt and was written by the ship’s current Captain, James Kirk.  Again, the article reads like a sales brochure.  The ship apparently has no flaws except for an almost insignificant restriction in the heavy seas operating envelope.  Other than that, it’s clear from the article that a single Zumwalt can win any war by itself (so why did we build three, I wonder?).  The only question is whether the ship will need to leave dock to do it or if it can win the war from its home port.

While the article offers a few interesting factoids, it is clearly a PR piece and, again, I reject it as a basis for influencing my opinion.

Now, recognize that this works both ways.  I’ve read reports on the F-35 and LCS that were nothing but criticism and those are no more credible than the reverse.  A report that would have me believe that those platforms have not a single positive, redeeming characteristic is just as clearly biased and unbelievable as the reverse.

So, to respond to the original complaint by that far removed, unknown commenter, the reason why positive reports are often summarily dismissed is because they lack credibility.  I would also remind the commenter that supporters cherry pick their data as much or more than critics!