Remember how ChatGPT totally aced the bar exam? Wow! yeah, turns out that was just a lie - eviltoast
  • vin@lemmynsfw.com
    link
    fedilink
    English
    arrow-up
    45
    arrow-down
    3
    ·
    6 months ago

    Though making an unreliable intern is amazing and was impossible 5 years ago…

      • QuaternionsRock@lemmy.world
        link
        fedilink
        English
        arrow-up
        9
        arrow-down
        10
        ·
        6 months ago

        I mean, it’s not shit at everything; it can be quite useful in the right context (GitHub Copilot is a prime example). Still, it doesn’t surprise me that these first-party LLM benchmarks are full of smoke and mirrors.

          • QuaternionsRock@lemmy.world
            link
            fedilink
            English
            arrow-up
            3
            arrow-down
            8
            ·
            6 months ago

            Not to be confused with Microsoft Copilot, which I have yet to find a use for. Do you not like GH Copilot either?

            • FredFig@awful.systems
              link
              fedilink
              English
              arrow-up
              16
              ·
              6 months ago

              Eclipse could generate templates for me, and I think we collectively agreed to stop using Eclipse like 20 years ago, so why are we trying to bring it back.

              • froztbyte@awful.systems
                link
                fedilink
                English
                arrow-up
                9
                ·
                edit-2
                6 months ago

                hey hey hey, don’t forget about android studio! that kept inflicting the pain of eclipse on many for years!

          • self@awful.systems
            link
            fedilink
            English
            arrow-up
            21
            ·
            6 months ago

            it’s always fucking “boilerplate” with these assholes, isn’t it? I don’t know how so many people got into this field and didn’t figure out the template, snippet, or macro engines in their editors

            • froztbyte@awful.systems
              link
              fedilink
              English
              arrow-up
              12
              ·
              6 months ago

              “hey copilot buddy please write me a http server for a guestbook application I can demo on my blog”

          • QuaternionsRock@lemmy.world
            link
            fedilink
            English
            arrow-up
            15
            arrow-down
            9
            ·
            6 months ago

            That GitHub Copilot and friends are useful? I would argue that their utility is rather subjective, but there are indications that it improves developer productivity.

            I’m unsure if you’ve used tools like GH Copilot before, but it primarily operates through “completions” (“spicy autocorrect” in its truest form) rather than a chatbot-like interface. It’s mostly good for filling out boilerplate and code that has a single obvious solution; not game-changing intelligence by any means, but useful in relieving the programmer of various menial tasks.

            May I ask, what evidence are you hoping to see in particular?

              • o7___o7@awful.systems
                link
                fedilink
                English
                arrow-up
                5
                ·
                edit-2
                6 months ago

                all in all: underwhelming. I remain promptdubious.

                I know I’m six months late to the party but how do you like “promptcritical”?

                • skillissuer@discuss.tchncs.de
                  link
                  fedilink
                  English
                  arrow-up
                  5
                  ·
                  edit-2
                  6 months ago

                  prompt critical is already a term of art in nuclear energy and it’s a state that you’d very, very much avoid (unless that was the intention of course)

                • froztbyte@awful.systems
                  link
                  fedilink
                  English
                  arrow-up
                  4
                  ·
                  6 months ago

                  yar I thought of that at the time too but with “gendercritical” having been used by ghouls I felt like the well might’ve been poisoned. still don’t really have a good one :|

                  • o7___o7@awful.systems
                    link
                    fedilink
                    English
                    arrow-up
                    4
                    ·
                    6 months ago

                    oof, yeah, you’re right I reckon!

                    It really chafes how awful dipshits can turn chunks of language into superfund sites.

            • self@awful.systems
              link
              fedilink
              English
              arrow-up
              13
              ·
              6 months ago

              May I ask, what evidence are you hoping to see in particular?

              holy fuck shut the fuck up

            • Krauerking@lemy.lol
              link
              fedilink
              English
              arrow-up
              13
              ·
              6 months ago

              I too want a taxi driver that doesn’t know how to drive a car but can adjust the little TV content in the back.
              Psh I mean all he has to do is step on the gas pedal and the car does all the work anyways right? I’m glad he doesn’t have to think to much about so he has more time to get the thermostat just right.

                • o7___o7@awful.systems
                  link
                  fedilink
                  English
                  arrow-up
                  4
                  ·
                  edit-2
                  6 months ago

                  The moral equivalent of “peril-sensitive shades” will be the killer app for augmented reality headsets.

              • LargeMarge@sh.itjust.works
                link
                fedilink
                English
                arrow-up
                5
                arrow-down
                7
                ·
                6 months ago

                I mean…yea? That’s kind of the point. It’s not driving, it’s the copilot. You’re the one driving, and it will get the thermostat right because you’re busy operating the vehicle and want to keep your attention on the road. That seems useful to me.

                If you already have an idea of the code you want to write and start typing it, Copilot can help auto complete so you can focus on actually solving whatever problem you’re working on instead of searching for the correct syntax online. I understand shitting on AI is fun and there’s plenty of valid criticisms to be made, but this is actually kind of useful.

                • self@awful.systems
                  link
                  fedilink
                  English
                  arrow-up
                  12
                  ·
                  6 months ago

                  how could we possibly be critical of the technology that at best replicates basic editor functionality (templating, syntax completion), outputs wildly incorrect code, and burns rainforests?

                  • LargeMarge@sh.itjust.works
                    link
                    fedilink
                    English
                    arrow-up
                    3
                    arrow-down
                    5
                    ·
                    edit-2
                    6 months ago

                    I’m not saying you can’t be critical of it, but templating and syntax completion is in fact useful. Suggesting incorrect code is obviously bad, but all of this stuff is still relatively new and I’m sure it’ll get better with time. Can’t we at least try to be a little optimistic about what this stuff is capable of when we give our criticisms, instead of having knee jerk reactions that make this out to be the harbinger of the apocalypse?

                    Side point to address the linked article: yes, computing systems use energy. If our energy grid is overly reliant on the burning of fossil fuels that release harmful emissions, that doesn’t mean we need to stop the advancement of our computers. It means we need to stop using so much fossil fuels in our grid.

                • froztbyte@awful.systems
                  link
                  fedilink
                  English
                  arrow-up
                  10
                  ·
                  6 months ago

                  “Ah but see, there is no agency, there is merely emergent behaviour! It is none of our choices that drive this, but merely the ideas some have had that drive this engine of our doom. Alas, we can do nothing about this outcome!”

                  • LargeMarge@sh.itjust.works
                    link
                    fedilink
                    English
                    arrow-up
                    3
                    arrow-down
                    5
                    ·
                    6 months ago

                    I have no idea what you mean by this comment. All I’m saying is that an auto complete feature when writing code is useful, which is largely what this was designed for.