4 Comments
User's avatar
Jürgen Boß's avatar

Adversarial cross-inspection is essentially how the modern world makes humans safer.

AIs, who know all of human history and theory (evolutionary game theory, optimistic tit-for-tat as evolutionary stable strategy), will find it obvious, plausible and technologically easier than inspecting irrational humans.

Treating humans nicely can then become a valuable pathway for "honest signaling".

Steven Vincent's avatar

Ramez, the line I keep coming back to is that concentration of power is the real risk, not any single model. That's the center of what we call POND: personal, on-device, nodal, networked. My version of your argument is that centralized vertical AI inevitably gets used to project power, and a "safe" singleton is still a singleton. I also agree an open/closed mix is inevitable and open can't realistically be banned. One caution I'd add: plural isn't automatically secure. A model on your own hardware still needs real defenses, so your point about arming defenders matters even more at the edge. We're working on this as a Human-Machine Intelligence (HMI) ecosystem, with personal nodes and many small models in niche roles. If it's useful: https://www.pond.is/p/pond-a-human-machine-intelligence

Anomaloid's avatar

As they say, the best defense against a bad guy with a gun is a good guy with a gun.

Oleg  Alexandrov's avatar

We have only two choices here: existential angst, pause, and/or tight control on one hand, vs treating this as any other tech, as Arvind Narayanan claims, and focus on well-undrstood common-sense approaches.

Using AI will help us find failures, which will result in fixes. AI will explore the problem space and find flaws in its architecture and in other AI. All this calls for a very dynamic competitive, ideally reasonably decentralized market.