> Nothing will be able to stop you from pulling the plug.
Who is "you"? And what do you envision is the "plug" to pull? And when does an intelligence become a superintelligence - do "we" know when to pull the "plug"?
For example, a superintelligence may be born in a datacenter. Would you expect politicians are aligned with shutting down a datacenter (privately owned) in which they may have heavy stake? What if critical systems are also running on the same infrastructure, will it be easy to cherry pick the superintelligence nodes to shutoff?
IMO this take is dismissive of the entrenched systems that make it hard to pull the plug. It's a hard problem and we need to think about it more.
Who is "you"? And what do you envision is the "plug" to pull? And when does an intelligence become a superintelligence - do "we" know when to pull the "plug"?
For example, a superintelligence may be born in a datacenter. Would you expect politicians are aligned with shutting down a datacenter (privately owned) in which they may have heavy stake? What if critical systems are also running on the same infrastructure, will it be easy to cherry pick the superintelligence nodes to shutoff?
IMO this take is dismissive of the entrenched systems that make it hard to pull the plug. It's a hard problem and we need to think about it more.