LLM Inference for Legal Document Summarization? - eviltoast

Hey everyone, I’ve been parsing through the Huggingface website and am having a bit of trouble picking out an LLM inference to help me parse through legal documents. I am not a lawyer, but I would like to understand my rights and how to search for answers to legal questions with concrete answers using an inference.

I have heard a multitude of things around Llama being a privacy nightmare and something about Gerganov ML files? GGMU is also a nebulous term to me and I understand the basics about how a model is trained and validated, but not how to pick one for personal use that isn’t GPT-4.

Any suggestions or things to add on to the discussion?

  • mo_ztt ✅@lemmy.worldEnglish
    17·
    1 year ago
    1. I would be very hesitant to put my legal reasoning in the hands of an LLM. They’re not AIs, they just come up with plausible text completions. There have actually been cases by now of lawyers who’ve gotten fucked by using AI to try to save themselves effort and then it not being good enough for what they were expecting from it.
    2. If you’re convinced you want to do this, there are basic tutorials on Youtube - I’m not 100% sure but I think that instead of “fine-tuning” in the same way you would do to fit an LLM to a problem space, you want to import the legal documents into something like Chroma, then use something like Llama as hooked up to the Chroma DB. But again, I wouldn’t. For messing around with some things it’s fine, but for legal documents you really want a sentient intelligence involved in the process.
    • hillbicks@feddit.deEnglish
      3·
      1 year ago

      Man, Ryan Reynolds the lawyer is really entertaining. Thanks for posting the video, I thoroughly enjoyed it.

    • gronjo45@lemm.eeOPEnglish
      1·
      1 year ago

      Is there another archetype of Machine Learning technologies that would be better suited to the task of locating useful information enciphered in legalese? I know Lex Machina exists, but that’s more of a specialized software for someone in case law.

      I don’t plan on using what the Agent tells me in a court of law, nor do I plan on using it to blindly form a legal opinion. I remember watching the Legal Eagle video about a lawyer who submitted a legal brief containing case law that didn’t even exist because GPT-4 hallucinated it! Sounds like a nightmarish scenario to find yourself losing your J.D. over it lol

      • mo_ztt ✅@lemmy.worldEnglish
        3·
        1 year ago

        I think an LLM+Chroma is probably as good as it gets, and who knows, it might work. Just I’d be very careful of getting screwed by the process. As I’m sure you know LLMs are right at that inflection point where they’re good enough to seem trustworthy but they can still completely malfunction (and they tend to do so in ways that are actually really difficult to spot because they seem perfectly plausible.)

        Yeah the Legal Eagle video was hilarious. The guy used GPT-4 to make his legal briefs, then when it hallucinated cases he lied to the judge and said he’d researched them and the cases existed, then when faced with the clearly obvious fact that they didn’t, he finally came clean but still sort of tried to weasel out of responsibility for the whole thing and the judge quite rightly tore him a new one. And, I have some vague memory of it being discovered that GPT had basically tried to tell him it wasn’t qualified to make his legal briefs and he insisted to it that it needed to do it anyway. It was just an absolute casserole from start to finish.