Maybe the real unaligned super intelligence were the corporations we made along the way 🥺 - eviltoast
  • gerikson@awful.systems
    link
    fedilink
    English
    arrow-up
    1
    ·
    6 months ago

    Banks neatly sidesteps the “AI will inevitably kill us” scenario by making the Minds keep humans around for amusement/freshness. Part of the reasons for the Culture-Idiran war in Consider Phlebas and Look to Windward was that the Idirans did not want Minds in charge of their society.

    • smiletolerantly@awful.systems
      link
      fedilink
      English
      arrow-up
      2
      ·
      6 months ago

      Noone was trying to force that on them though, the actual reason IIRC correctly that Idirans had a religious imperative for expansion, and the Culture had a moral imperative to prevent other sentients’ suffering at the hands of the Idirans.

      IMO he mostly sidestepped the issue by clarifying that this is NOT a future version of “us”

        • smiletolerantly@awful.systems
          link
          fedilink
          English
          arrow-up
          1
          ·
          6 months ago

          Yep. They leave us alone so we’ll function as a control group. This way contact can later point at us and go “look! That’s what happens if we don’t intervene!”

      • gerikson@awful.systems
        link
        fedilink
        English
        arrow-up
        2
        ·
        6 months ago

        OK I misremembered that part. It makes sense that after suffering trillions of losses the Culture would take steps to prevent the Idirans from doing it again.

        And by “us” I meant fleshy meatbags, as opposed to Minds. Although in Excession he does raise the issue that there might be “psychotic” Minds. Gray Area’s heart(?) is in the right place but it’s easy to imagine them becoming a vigilante and pre-emptively nuking an especially annoying civilization.