I was stunned by the overwhelming advantage that the test
conditions gave the AI. For example, the
AI was given perfect 360 degree situational awareness and perfect knowledge of
not only its own performance but that of the human pilot’s aircraft which is
utterly unrealistic. Another example is
that the AI merely had to have the target in a conical field of view to count
as a gun hit – not actually place the piper on the target. And so on.
The results proved absolutely nothing. I’m not going to go any further with this
because the linked video does a much better and more complete job of analysis
than I can. All I’ll say is that you owe
it to yourself to watch the video. It
will completely change whatever impression you had about the results.
I went into it thinking that I’d see some pretty impressive
AI and I came away from it thinking the event was almost a waste of time for
all parties and was tantamount to the usual staged war games and exercises that
the military is so famous for.
For those who can see the imbedded video, below, check it out for yourself. If you can't see the video, follow the link and see for yourself.
Link: https://www.youtube.com/watch?v=ziCQqmEllZo
Link: https://www.youtube.com/watch?v=ziCQqmEllZo
Same here, was very disappointed, was expecting lot more. I almost feel DoD is getting shafted again by industry, IMO, what was the difference between this AI and just a regular flight sim game that I can get at Best Buy for $50 bucks?!? Some are saying this was spectacular, sorry, not seeing it, I played video games in the 80s!, yes, I'm old, that the computer put up a better and more interesting fight....
ReplyDeleteYeah, the media overly hyped the competition. The machine-learning AIs all lost to the one that was programmed by a human.
ReplyDeleteThis is basically the equivalent of an air-to-air missile in terms of technology where you spend time programming in how the computer handles different situations to give it optimum performance. Similar techniques are used to program computer opponents in flight sims.
"Artificial Intelligence" still has a long way to go since it clearly lost to the computer programmed by a human.
"the event was almost a waste of time for all parties"
ReplyDeleteExcept for those who profited off it, yes.
I stand corrected!
DeleteProbably the best test would be to take some drones, add some air to air weapons and watch them try to kill each other over the desert. Even without human pilots in the loop this should allow a good evaluation of the capabilities of the software. If you can use cheap enough drones, it need not be too pricey.
ReplyDeleteWe have boneyards full of actual aircraft that could be fitted out for such battles.
DeleteI kept noticing this trend with the military about how they are treating DARPA projects. I believe that historically (i might be wrong), DARPA programs are either basic research, foundational technologies and proof-of-concepts. Their research isn't aimed at making actual systems and weapons, they cooperate with larger defense companies and made it a reality. However the military decided to use whatever DARPA researched and hail it as the latest weapons that will be widely adopted without regards of how they are designed for (a laboratory condition). They use these demonstrations (historically they are as top secret as it gets and are only supposedly only shown to top generals and senators) as a publicity stunt and a promotional tool. Unfortunately for them, real military observers and analysts can see right through their PowerPoints and praising words. What a shameful display in my opinion.
ReplyDeleteI am somewhat reminded of video game AI like the "superhuman" who sometimes blow themselves up trying to shoot through walls, like Soldier of Fortune 2 review below:
ReplyDelete"Which brings us to the game's single biggest flaw - the AI. Put simply, every scruffy looking rebel soldier you come up against is in fact Superman wearing a really cunning disguise. He can see through walls (and will sometimes attempt to fire through them, with often hilarious results), poke arms and legs through solid objects, and consistently land a grenade at your feet from fifty feet away, over a hill, without being able to see you. "
https://www.eurogamer.net/articles/r_sof2
And of course below:
https://tvtropes.org/pmwiki/pmwiki.php/Main/TheComputerIsACheatingBastard
I'm sure had it actually been "physical" the AI may have/would crashed a long time ago (you can't noclip cheat in real-life)
https://en.wikipedia.org/wiki/Noclip_mode
Of course to experienced video game vets this is a somewhat well known common problem (at least from what I known, if you don't, I apologize and now you know) when the AI behaves in ways the player can't, as show by links as below:
https://www.pcmag.com/opinions/unfair-advantage-dont-expect-ai-to-play-like-a-human
There is a miss concept among many that AI means computers think as human.
ReplyDeleteNO! Today's tech is far far away from that.
Basically, AI are written by human being. An AI runs on computers won't out smart than the one writes it. What a computer can outrun human are its consistency, fast response, precision memory, etc.
Basically, the so called AI fight human was done on an F-16 simulator. Practically, you can say an AI software wins a video game over people.
In real combats, many so called self-initiate weapons are piratically AI. For instance, if sensors detect a missile approach, it automatically fires a missile to intercept it. Like many technologies, in some cases, AI are great tools but not all cases. How good an AI depends on the one who write it (mainly algorithm behind it).
After watching and re-watching ComNavOps's linked youtube video, I think that the AI (not to mentioned the human pilot, Banger, was relatively inexperienced. Heck use a real experienced DCS player as suggested, into the sim and they may make the AI crash like 128th TIE Interceptor Squadron crashing into the surface of Gerrard V (in the video game)
ReplyDeletehttps://starwars.fandom.com/wiki/128th_TIE_Interceptor_Squadron )
Also the AI would have shot itself down had it actually been physical like below doing that "really bad BFM" face-shots:
https://www.popularmechanics.com/military/aviation/a27078461/dutch-f-16-flew-into-its-own-gunfire/
As already typed there is no "noclip" cheat in real-life no matter how "in the cone" it is ,like the poor dutch F-16.
https://arstechnica.com/tech-policy/2019/04/dutch-f-16-takes-cannon-fire-from-itself/
Of course as in the video "we trust what works" and what what will likely work is the the AI will run into its own fired ammo in real life and likely shoot itself down (or damage itself)as plenty of real-life examples like below shows:
" Sept. 21, 1956, test pilot Tom Attridge began a shallow dive in his F11F. As he did, he fired two short bursts from the aircraft's four 20mm cannons, and thought nothing of it – until he got to the end of his dive, and the bursts began to shoot up his aircraft. He started at 20,000 feet and then went into a Mach 1 dive as he fired. He accelerated with afterburner and at 13,000 feet, fired to empty. "
(like the stuff the AI f-16 did in that video for example as explained in the "training rule thing" with all those face shots, unlike a skilled human pilot)
https://www.wearethemighty.com/history/f11f-shot-itself-down
And what's to prevent below if it really had been "physical"(using two pieces of tape no less) since there was no collision logic...
https://www.technologyreview.com/2020/02/19/868188/hackers-can-trick-a-tesla-into-accelerating-by-50-miles-per-hour/
Thought somewhat the same thing, if a "face shot" is so difficult a shot and not really a guarantee for a human or AI, wouldn't you just let the AI go for face shots, let the AI F16 run out of ammo then shoot him down? Its tricky scary but if the AI stupidly wastes its ammo, why not???
ReplyDeleteI noted that the human never made any adjustments. He just rode the exact same flight/fight profile to five straight defeats. Didn't exactly make humanity proud, did he?
DeleteThe human pilot seemed completely lost in the simulator - like he had no idea how to operate it or what it could do or not do. Those giant, wide circles that he was flying, for example - he seemed to have no idea of his speed, g's, or enemy position; I can't believe he'd do that in real combat. If he would then he's a horrible pilot. It just seemed like it was the first time he'd seen a simulator (and it was the first time he'd seen this one!) and he was just flying around trying to figure out how to do it.
If an enemy, real or AI, had 'shot' you in the face three or four times in a row, wouldn't you change tactics or, at the very least, fire back, even if it thought it was a low probability shot … cause, clearly, in this simulator, it WASN'T A LOW PROBABILITY SHOT - so take advantage of it.
I really hate to say it but the pilot could have been my pet cat for all the intelligence and adaptability he exhibited.
So, what this spectacle demonstrated was that an AI, custom programmed for this simulator, and with all the advantages of perfect awareness, is capable of beating a pilot who was lost in his first time in a low quality simulator and made no adjustments as time went on.
I really wonder if the pilot had "orders" not to do anything fancy or change tactics. As you said, he pretty much flew in a circle the whole time, never tried anything different or try to go much into the vertical plane....
Delete"The human pilot seemed completely lost in the simulator - like he had no idea how to operate it or what it could do or not do."
DeleteDid he even have experience with the simulator before?
Alternatively, NICO's explanation makes a lot of sense as well.
In the video, the YouTuber did mention about his doubts of the pilot competent in the simulator due to the same reasons listed before. It's highly possible that he was brought to be used as a test case scenario without any prior training with the simulator.
DeleteFood for thought:
ReplyDeleteWhat if how the human pilot flew in the simulation is how pilots are taught how to fly in combat nowadays?
Yikes! I sure would like to believe that's not true but …
DeleteFor the last several years our non-deployed pilots have barely been getting enough flight hours to stay flight certified let alone combat proficient.