What is this? (Its OC!) - eviltoast

List of icons/services suggested:

  • Calibre
  • Jitsi
  • Kiwix
  • Monero (Node)
  • Nextcloud
  • Pihole
  • Ollama (Should at least be able to run tiny-llama 1.1B)
  • Open Media Vault
  • Syncthing
  • VLC Media Player Media Server
  • Smokeydope@lemmy.worldOP
    link
    fedilink
    English
    arrow-up
    30
    ·
    edit-2
    4 months ago

    tinyllama 1.1b would probably run reasonably fast. Dumb as a rock for sure. But hey its a start! My 2015 t460 thinkpad laptop with an i7 6600U 2.6GhZ duo core was able to do llama 3.1 8B at 1.2T-1.7T/s which while definitely slow at about a word per second. Still, was also just fast enough to have fun in real time with conversation.