Self hosted LLM - eviltoast

Hello internet users. I have tried gpt4all and like it, but it is very slow on my laptop. I was wondering if anyone here knows of any solutions I could run on my server (debian 12, amd cpu, intel a380 gpu) through a web interface. Has anyone found any good way to do this?

  • HumanPerson@sh.itjust.worksOP
    link
    fedilink
    English
    arrow-up
    1
    ·
    9 months ago

    I don’t really want to try to get that to work. I wonder how hard it would be to create my own webui using gpt4all’s Python package.