OpenAI does not want you delving into o1 Strawberry’s alleged “chain of thought” - eviltoast
  • DarkCloud@lemmy.world
    link
    fedilink
    English
    arrow-up
    18
    ·
    edit-2
    2 months ago

    I don’t think it’s alive, I think it’s talking to its self. They’re making a Chinese whisper machine, and it will remain so until it has embodiment, subjective and changing goals, and a will of it’s own.

    That’s part of intelligence, but it’s still a reverse engineering take on things.

    In actuality we have intelligence because our threat detection and social protection/survival goals became abstract enough for self-awareness to occur.

    EDIT: Telephone game is what I meant.

    • swlabr@awful.systems
      link
      fedilink
      English
      arrow-up
      45
      ·
      2 months ago

      Chinese whisper

      Complete tangent but outside of the commonwealth, this game is referred to by the much less racist moniker “telephone”

      • Codex@lemmy.world
        link
        fedilink
        English
        arrow-up
        16
        ·
        2 months ago

        I figured he was talking about Searle’s Chinese room thought experiment. Searle sucks though, so that’s probably also racist (in addition to being stupid.)

        • Sailor Sega Saturn@awful.systems
          link
          fedilink
          English
          arrow-up
          19
          ·
          edit-2
          2 months ago

          In 2024 it is, at the very least, extremely uncomfortable to read Searle describe Chinese writing as “meaningless scribbles”, “formal symbols”*, “squiggle squiggle”, and “squoggle squoggle”. Basically taking Chinese, ignoring the fact that it’s a real language used by real people and is not alien nor inscrutable nor mathematical, and using it as a prop to purposefully obfuscate a thought experiment.

          But that’s like, just my opinion man.

          * The paper never seems to get around to calling English letters symbols I wonder why.

          • froztbyte@awful.systems
            link
            fedilink
            English
            arrow-up
            11
            ·
            2 months ago

            the other thing about this that’s often come to mind for me is that the “who” picked in such things tends to be telling of the speaker, and of their perception of “impenetrable”

            relatedly, a somewhat common phrase around this side of the world is/was “it’s greek to me”. I don’t know the history of why it came into public lexicon around here (whether it was imported or grew locally), but been curious.

            which leads to my sidebar and sneer: it’d be nice if it were easier to research things like this, and good god the modern internet makes it hard to do that. holy fuck what a tsunami of dogshit. and then fucking LLMs and openai come around, going “HOLD MY BEER”. le sigh.

            • Mike@awful.systems
              link
              fedilink
              English
              arrow-up
              8
              ·
              2 months ago

              relatedly, a somewhat common phrase around this side of the world is/was “it’s greek to me”. I don’t know the history of why it came into public lexicon around here (whether it was imported or grew locally), but been curious.

              Wikipedia has quite a comprehensive list of similar idioms from a lot of different languages. Chinese gets a lot of mentions, but so do Greek and Spanish. Plus Turkish and Hebrew. As far as I can tell the Chinese describe any incomprehensible language as “Martian”. But “It’s Greek to me” goes right back to the Romans.

              https://en.wikipedia.org/wiki/Greek_to_me

              • Architeuthis@awful.systems
                link
                fedilink
                English
                arrow-up
                10
                ·
                edit-2
                2 months ago

                But “It’s Greek to me” goes right back to the Romans.

                The wiki seems to say the aphorism originates with medieval scribes and Shakespeare’s Julius Caesar.

                The actual ancient Romans are unlikely to have had such qualms, since at the time Greek was much more widely understood than Latin, so much so that many important roman works like Caesar’s Memoirs and Marcus Aurelius’ Meditations were originally written in Greek, with the Latin versions being translations.

                • zogwarg@awful.systems
                  link
                  fedilink
                  English
                  arrow-up
                  10
                  ·
                  2 months ago

                  Pedantic note: Yes, Meditations (a phisosophical treatise) was written in Koine, Commentarii de Bello Gallico (veni, vedi, vici—self-aggrandizing combat-reports meant for the senate and propaganda) or other “published” works from Caesar were not.

                  Although bonus points, the ancient sources portray Caesar (a proper educated major family Patrician) as speaking his dying words—if reported saying anything at all—in Greek, not in Latin: “Καὶ σὺ τέκνον” (Even you, child) rendered in Shakespeare as “Et tu, Brute”.

        • pyre@lemmy.world
          link
          fedilink
          English
          arrow-up
          3
          arrow-down
          8
          ·
          2 months ago

          the reason to pick Chinese may be racist (possibly due to the writing system looking complicated) but the thought experiment itself doesn’t have racist connotations imo, and i don’t think it’s stupid either. doesn’t have to involve Chinese or a specific language at all.

          it’s a logical question to ask: if i can mimic speaking in a language to a point that it convinces native speakers, but don’t understand what I’m saying myself, am I considered a genuine speaker of that language? does what i say matter or have any value?

          • froztbyte@awful.systems
            link
            fedilink
            English
            arrow-up
            8
            ·
            2 months ago

            well it appears like you’re posting english, but actually you’re posting nonsense

            so the answer to your question is no

          • swlabr@awful.systems
            link
            fedilink
            English
            arrow-up
            8
            ·
            2 months ago

            Scientists: weird, we didn’t slip this piece of paper saying “mansplain the chinese room thought experiment” through the door, and yet that’s all the room seems to want to do. I guess we just have to conclude the room is an idiot?

            • pyre@lemmy.world
              link
              fedilink
              English
              arrow-up
              2
              arrow-down
              10
              ·
              edit-2
              2 months ago

              what an unnecessarily aggressive comment. mansplain? am i even responding to a woman? also i wasn’t trying to explain it; i was saying the central question doesn’t have to involve a specific language at all and it still a worthy question, especially with all this AI bullshit being pushed all over.

              • swlabr@awful.systems
                link
                fedilink
                English
                arrow-up
                7
                ·
                2 months ago

                Enter these into your data banks, you automaton:

                • The chinese room was brought up as a tangent to a tangent, why are you doubling down on mansplaining it?
                • nobody was even talking about whether or not it is a good or bad concept, we were just talking about the problematic dude that made it popular
                • you are in a space that is critical of AI. The chinese room was brought up by name. You think we don’t know what it is and that you need to explain it?
                • literally all you had to do to seem even remotely reasonable was to not try mansplain AGAIN and yet you did. I wrote what I said as a joke and you went ahead and turned it into prophecy. You buffoon!!!
      • magnetosphere@fedia.io
        link
        fedilink
        arrow-up
        13
        ·
        2 months ago

        Thank you for explaining that. I, an American, have never heard the term “Chinese whisper”, but I’ve definitely heard of the telephone game.

        • swlabr@awful.systems
          link
          fedilink
          English
          arrow-up
          9
          ·
          2 months ago

          It’s much more awkward as a subject of the crown. I tried explaining the game “telestrations” as pictionary + chinese whispers before I had this knowledge. I didn’t know!!! It’s even right there in the name!!! I swear I’m not racist!!!

          (Note: I am of chinese origin and have heard my extended family mangle messages through the telephone. So both names are real to me)

          • Mike@awful.systems
            link
            fedilink
            English
            arrow-up
            9
            ·
            2 months ago

            As a white British dude the problem is that “Telephone” is an Americanism, so I think the solution is that we find an entirely new name to describe speech-like yet utterly incomprehensible-to-the-listener noises that’s completely devoid of cultural appropriation. I suggest “This is all Trump to me”. The game could be “Trump Tweets”.

  • zbyte64@awful.systems
    link
    fedilink
    English
    arrow-up
    7
    ·
    edit-2
    2 months ago

    That’s OpenAI admitting that o1’s “chain of thought” is faked after the fact. The “chain of thought” does not show any internal processes of the LLM — o1 just returns something that looks a bit like a logical chain of reasoning.

    I think it’s fake “reasoning” but I don’t know if (all of) OpenAI thinks that. They probably think hiding this data prevents cot training data from being extracted. I just don’t know how deep the stupid runs.