Over just a few months, ChatGPT went from correctly answering a simple math problem 98% of the time to just 2%, study finds - eviltoast
  • Fisk400@lemmy.world
    link
    fedilink
    arrow-up
    2
    ·
    1 year ago

    We have people that assign numerical values to peoples ability to read and write every day. They are english teachers. They test all kinds of stuff like vocabulary, reading comprehension and grammar and in the end they assign grades to those skills. I don’t even need tiny professors in my pocket, they are just out there being teachers to children of all ages.

    One of the task I have chatGPT was to name and describe 10 dwarven characters. Their names have to be adjectives like grumpy but the description can not be based on him being grumpy. He has to be something other than grumpy.

    ChatGPT wrote 5 dwarves that followed the instructions and then defaulted to describing each dwarf based on their name. Sneezy was sickly, yawny was lazy and so on. This gives a score of 5/10 on the task I gave it.

    There is a tapestry of clever tests you can give it with language in focus to test the ability of a natural language model without giving it a bunch of numbers.

    • ThreeHalflings@sh.itjust.works
      link
      fedilink
      arrow-up
      2
      ·
      1 year ago

      OK, you go get a panel of highschool English teachers together and see how useful their opinions are. Lemme know when your post is up, I’ll be interested then.

      • Fisk400@lemmy.world
        link
        fedilink
        arrow-up
        2
        ·
        1 year ago

        Sorry, I thought we were having a discussion when we were supposed to just be smug cunts. I will correct my behaviour in the future.