• 7 Posts
  • 678 Comments
Joined 3 years ago
cake
Cake day: August 29th, 2023

help-circle

  • I think the biggest potential motivator for doomer suicide would actually be from the other direction: some AI apparently advances enough the rationalist become convinced it is going to secretly produce nanobot factories in preparation for a takeover (such as in this story or this classic piece of rationalist mythos building CW ponies) and they need to die before they can get “I Have No Mouth, and I Must Scream”'d (or uploaded into a pastel dystopia) by the malevolent AI. The current paradigm probably has a bubble bust before it can produce anything that gets them that scarred (even with how good they are at taking the hype at face value) but it still feels like a landmine… any news story that they can delude themselves into being an imminent indicator of unfriendly AI takeover might prompt such action.

    Agreed the boosters and e/accs might have some crashouts when the LLM industry crashes.



  • Yeah, they’ve been claiming credit for predictions that were only partially right and/or require a pretty skewed perspective (i.e. buying the claim that vibe-coders really can create quality stuff just letting the agent act freely).

    Of course the big thing is the claim of superintelligence by 2027 they initially slightly eased up on, then doubled down recently over the past few months. Somehow, I don’t think /r/singularity (and similar booster discussion areas) will give them credit for a bunch of intermediate predictions if they prove blatantly wrong about the really big really critical one.


  • Don’t forget that they also think an “aligned”/“friendly” AI would make magic nanotech and cure death of old age (at minimum, at maximum maybe it jumps straight to brain uploading and everyone gets totally immortality through cloud backups, just ignore any philosophical quandaries about being copied).

    They only have to be patient enough to wait for figuring out alignment properly. …Well, Eliezer has given up on this being achievable by the current generation (because if he couldn’t do it, obviously no currently living human could) and instead wants humanity to engage in ill-thought out genetic engineering plans while AI is paused for decades, but most rationalists haven’t given up on the original align AI plan.

    Also, to the extent people like Peter Thiel or Elon Musk were counted as believers in the singularity by rationalists, they should have realized they were asking aging billionaires to patiently put the good of humanity over gambling everything on achieving AI that could make them immortal within their lifetime. Framing it like that, it is obvious what the outcome would be.







  • the reason Saltman was fired was because the board didn’t like his stupid face.

    He did lie to them and tell different people different stories to try to manipulate them. But you are right among the things he lied about was AI “safety” related stuff, and (as samvines describes) he lied because he wasn’t a true believer like they were. (And thus tried to bypass their oversight in releasing ChatGPT.)

    Lukewarm take, considering that LLM models could use a lot more thought and care in their development and release (although obviously not for the reasons the doomers want), the EA cultists were actually the (relatively) good guys in that whole saga of Sam Altman’s firing and rehiring.




  • There is loads of stuff they didn’t get right. The benefit of the narrative format is that any specific details they get blatantly wrong they can spin as just being part of the fictional narrative and not a deliberate prediction. They can also shrug off discredit by claiming that only their technical predictions were accurate, not their political predictions. In fact, as we’ve speculated recently on sneerclub, one move the boosters may make is blaming the bubble pop (and subsequent failure to produce AGI) on the Democrats winning, because obviously if the LLM companies had kept up their momentum just a few months longer surely they would have achieved recursive self improvement (and all the wondrous things they’ve promised that would follow).






  • “Since you [Ms. Congresswoman] can’t manage an international total ban on all AI progress, the only alternative is to bail out my company so we can maybe have a small chance at inventing a friendly AI before the Chiense invent skynet. Obviously the Chinese are behind us and have only gotten as far as they have by distillation attacks to steal our models’ training (that we trained with honestly-and-totally-fair-use acquired scrapes of the entire internet), BUT if you just make us stop, the Chinese will catch up in 6 months and then we will have Chinese skynet, or worse yet, an AI aligned to communism and the CPP!”