AGI or artificial general intelligence will be transformative for society. However, it is powerful enough that indiscriminate access could result in dangerous outcomes. One of the biggest risks of developing advanced AI is that human operators will use the power for malicious ends. When individuals possess powerful technologies such as weapons, the state has to ban them when they get powerful enough. Societies themselves, i.e. nation states, will also have to adapt to the advent of AGI. The balance of power between countries used to rest on military strength, now more recently on mutually assured destruction (nuclear weapons). But the availability of cyberspace attacks, especially with AI added to the mix, may destabilize the status quo. We discuss several solutions to these two problems, including having a singular world government or having humans merge with machines. Although AGI will bring massive good to the world, its introduction has to be navigated carefully to avoid bad consequences for humanity as a whole.
-
🤖AI can enhance the capabilities of people acting with bad intent, raising concerns about preventing ordinary human actors from doing really bad things when they have access to AI.
-
🧠The potential fallout from someone misusing an AGI system seems to be quite large and very scalable, raising the question of whether society can afford to give AGI to everyone.
-
🤯The idea of mutually assured destruction highlights the dangerous consequences of advanced technology in the wrong hands.
-
💣The potential danger of AGI lies in its capability to be a powerful weapon that could lead to a true first strike, making defensive AI systems difficult to imagine.
-
🌍Open-source models could bypass attempts to limit AI capabilities, potentially allowing individuals to seize immense political power or carry out actions that society can't recover from.
-
💥AI-based Weaponry could be devastatingly effective, leveraging all the capabilities of cyberspace and nuclear weapons, potentially leading to human extinction.
-
🌍The idea of humans merging with machines to intervene when AI is used for bad purposes is a reasonable solution, but it requires advanced brain-computer interfaces.
-
🌍It would be far too dangerous to let every individual in society have access to AGI.