Is there a good ai thats run fast on a amd rx 7600? - eviltoast

Ive been playing koboldai horde but the queue annoys me. I want a nsfw ai for playing on tavernai chat

  • projectmoon@lemm.ee
    link
    fedilink
    English
    arrow-up
    1
    ·
    4 months ago

    How much speed are you actually getting on Mixtral (I assume that’s the 8x7b). I have 64 GB of RAM and an AMD RX 6800 XT with 16 GB of VRAM. I get like 4 tokens per second with Q5_K_M quant.