Building an early warning system for LLM-aided biological threat creation - eviltoast
We’re developing a blueprint for evaluating the risk that a large language model (LLM) could aid someone in creating a biological threat. In an evaluation involving both biology experts and students, we found that GPT-4 provides at most a mild uplift in biological threat creation accuracy. While this uplift is not large enough to be conclusive, our finding is a starting point for continued research and community deliberation.
  • froztbyte@awful.systemsEnglish
    3·
    11 months ago

    one of the most recent things I’ve seen in this space is https://www.biscuitsec.org/, which is built on datalog and aims to solve a problem in a fairly interesting domain. I still mean to try it out on a few things, to see how well it maps to use in reality

    • self@awful.systemsMEnglish
      2·
      11 months ago

      that seems very cool! I’ve been frustrated in the past by rules-based auth libraries implementing half-baked but complex declarative DSLs when Datalog is right there, so I’m hoping it works well in practice because I’d love to use it too

      • froztbyte@awful.systemsEnglish
        2·
        11 months ago

        what, you don’t like the 10~15y old pattern of someone slapping together a DSL in a weekend because they read a blogpost about it last week, and then having to deal with the evolving half-restricted half-allows-eval mess in [ruby,erlang,…] with its syntax denoted in some way that isn’t equivalent between operating languages? sheesh. what kind of modern web engineer are you?!

        • froztbyte@awful.systemsEnglish
          2·
          11 months ago

          lukewarm take: the fact that “yaml engineer” exists as a joking self-deprecating referential description of what so many people do is both an indictment of their competencies (so, so many of these people would rather twiddle variables than even think of learning to write a small bit of programming), but also of the tools that claim to provide more abstractions and an “easier way” to do things

          (yes I have a whole rant about this bullshit stored up)

          • self@awful.systemsMEnglish
            2·
            11 months ago

            one day the things we do with yaml will correctly be seen as a crime, but very likely only after yaml is replaced by something significantly worse, cause our field stubbornly refuses to learn a damn thing. it’s probably not a coincidence that the only declarative languages I know that aren’t monstrosities are from academia, and they’re extremely unpopular compared to the approach where a terrible heap of unreadable yaml is made worse by shoving an awful macro language into every field