disconnects? - eviltoast

The site seems to go down intermittently. One time I got a message about too many requests, but mostly it just gives a generic lemmy error page.

Sometimes the issue goes away real quick and it drops my login session. Other times, it lasts long enough for me to give up and try a different site.

I’m not doing anything particularly unusual. I browse lemmy as I would reddit, killing an hour or two. When eviltoast stops working for me, I’ve tried testing from other browsers and computers. Yes I’m using ublock on firefox, but I’ve gotcha whitelisted. My chrome is clean, same issue.

I have access to remote systems to test as well, and it seems the issue is not related to me or my connection. Or that’s the way it looks so far. I could do more remote testing if needed.

Are we hitting a throttle of some sort?

  • eviltoast adminMAEnglish
    2·
    10 days ago

    Hey @dbtng@eviltoast.org , thanks for the report.

    There has been a lot more traffic than normally lately, mostly due to ai bot scraping/crawling. I think the “too many requests” errors you saw are probably from that. Hitting an overall rate limit in the webserver vs an individual limit.

    Looking at the logs, there have also been more errors lately, but it’s hard to tell how many are real errors. For example, a “couldnt_find_post” error can show up as a 500 internal server error instead of a 404. I’ll dig into it some more

    I normally use one of the apps (Jerboa or Voyager) and haven’t noticed any issues, but they might be retrying requests transparently.

    • dbtngOPEnglish
      2·
      4 days ago

      Hey thanks for providing a place to hang out and attending to the maintenance.

      I don’t see any mention of how you fund this other than it shares space with something else you host. It wouldn’t kill me to kick out $20 every now and then to help out. Particularly if it helps pay for bandwidth.

      But I hear ya. The freakin AI bots are hell. I’d block all the bots.
      Seen some creative ways of doing that recently.
      Other people and sites share this AI pain, and they are coming up with answers.

      • eviltoast adminMAEnglish
        2·
        3 days ago

        Thanks for the kind words!

        I don’t see any mention of how you fund this other than it shares space with something else you host. It wouldn’t kill me to kick out $20 every now and then to help out. Particularly if it helps pay for bandwidth.

        Everything’s funded out of pocket by me currently, but it’s not in risk of going away. I am planning on migrating to newer hardware in the near future though, and considering accepting donations since it will be more expensive to run.

        ai bots

        I’ve blocked the “abusive” ones for the most part, but do probably need to come up with something better. I saw some creative methods recently too that could be fun. Some are harder to block the normal ways because they are coming from residential IPs and using fake user agents.

    • eviltoast adminMAEnglish
      2·
      10 days ago

      I did find one consistent error related to postgres not having enough memory. That should be fixed now, but it looked like it was triggered by some internal queries from lemmy.