Instrumental Convergence
We worry about AI turning evil, but the real danger is competence. The apocalypse won’t be an act of war; it will be an act of optimization. If you ask a sufficiently powerful machine to do something simple, it will destroy you to get it done. It creates a parking lot out of your house because it needs space. It uses your atoms because they are raw materials. Malice is unnecessary when you have efficiency. ...