What is this? (Its OC!) - eviltoast

List of icons/services suggested:

  • Calibre
  • Jitsi
  • Kiwix
  • Monero (Node)
  • Nextcloud
  • Pihole
  • Ollama (Should at least be able to run tiny-llama 1.1B)
  • Open Media Vault
  • Syncthing
  • VLC Media Player Media Server
  • brucethemoose@lemmy.world
    link
    fedilink
    arrow-up
    2
    ·
    edit-2
    5 months ago

    Can you afford an Arc A770 or an old RTX 3060?

    Used P100s are another good option. Even an RTX 2060 would help a ton.

    27B is just really chunky on CPU, unfortunately. There’s no way around it. But you may have better luck with MoE models like deepseek chat or Mixtral.