Image uploads, federation delays, and performance - eviltoast

Happy new year!

Hi folks! I hope everybody had a good holiday period and I wish you all the best for 2024. I have some quick updates to share about lemm.ee:

Image uploads

Image uploads are now enabled for all lemm.ee users 4 weeks after account creation. The upload size limit is currently set to 500kb.

The 4 week account age requirement is in place to discourage spam and abuse. It is of course not a fool-proof solution, but let’s give it a go and see what the results are.

Please note that lemm.ee is not intended to be a image hosting service! Feel free to upload avatars and banners for your profile and communities, but please be aware that we reserve the right to modify the upload limits going forward, as well as delete old images if storage costs become too high.

For image posts and comments, it would still be preferable for you to use an external image hosting service.

Federation delays

Over the holidays, our outgoing federation workers began experiencing some significant delays. I have been working on this problem for the past few days, and after updating to 0.19.1, applying some additional patches to the code, and changing our infrastructure a bit, I believe the issue has been resolved.

The good news is that now that we are on 0.19, problems such as this do not cause Lemmy to completely drop federated activities, as we now retain a persistent queue of federation activities for all linked instances. This means that after the issue was resolved, our federation workers started going through the backlog of likes, comments, and posts which you had made over the past several days, and sending these out to other instances. Essentially, all of your activities did end up reaching their target servers, just with some additional delay.

One quick side-note here, while we are now federating your activities in real-time again to most big instances, there is still a bit of a backlog left on the lemm.ee -> lemmy.world federation (it is a few days behind). I expect this to also catch up by tomorrow.

Performance

The new persistent federation queue is still quite a new feature in Lemmy, so it’s a bit rough around the edges - after resolving the federation issues, our federation workers started going through the queue at extreme speed, which caused intense additional load on our database. This was one of the reasons for some performance degradation many of you noticed over the past few days.

Additionally, since updating to 0.19, there have been regular performance issues for many users. I have managed to solve a few of these by making some changes in our infrastructure, but I am also aware of a few more issues which I will continue to monitor and hopefully improve in the near future. Sorry for the inconvenience, I hope that the changes I have made so far will help make it a bit smoother already!

That’s all from me for now, as always, feel free to comment if you have any thoughts, and have a nice day!

  • TWeaK@lemm.ee
    link
    fedilink
    English
    arrow-up
    3
    ·
    10 months ago

    You’re being modest. “Here and there” does nothing to credit the significance of what you do. Penicillin is used “here and there”.

    I’m sure there are other professionals who could do more, just as I’m sure that you aren’t 100% focused on lemm.ee (not that you should have to be), but history so far has shown that you’re far ahead of the curve in providing a stable platform - one that other instances should aspire to.