Discussion about this post

User's avatar
Jürgen Boß's avatar

Adversarial cross-inspection is essentially how the modern world makes humans safer.

AIs, who know all of human history and theory (evolutionary game theory, optimistic tit-for-tat as evolutionary stable strategy), will find it obvious, plausible and technologically easier than inspecting irrational humans.

Treating humans nicely can then become a valuable pathway for "honest signaling".

Steven Vincent's avatar

Ramez, the line I keep coming back to is that concentration of power is the real risk, not any single model. That's the center of what we call POND: personal, on-device, nodal, networked. My version of your argument is that centralized vertical AI inevitably gets used to project power, and a "safe" singleton is still a singleton. I also agree an open/closed mix is inevitable and open can't realistically be banned. One caution I'd add: plural isn't automatically secure. A model on your own hardware still needs real defenses, so your point about arming defenders matters even more at the edge. We're working on this as a Human-Machine Intelligence (HMI) ecosystem, with personal nodes and many small models in niche roles. If it's useful: https://www.pond.is/p/pond-a-human-machine-intelligence

2 more comments...

Ready for more?