• 2 Posts
  • 20 Comments
Joined 6 months ago
cake
Cake day: August 5th, 2025

help-circle


  • Yes, the votes table is absolutely massive (10s of GB) so even with perfect indexes, queries & joins on it are slow enough to become a problem if you are doing many things. For example when listing a page of posts in a community when there are 70 posts per page that’d mean lots of interaction with the votes table, that I’d rather avoid.

    Instead, we get the last 3000 votes the viewer cast (a single DB query, or less if it’s in redis already, which is another layer of cache) and hold that in memory. While creating the list of posts a “viewer has voted for this” flag is set if the post id is in the list of votes from earlier. The list of votes is sorted so an efficient binary search can be done.

    That’s just one part of it, there are about 3 or 4 other similar optimizations where front-loading some data drastically reduces the work involved in getting a lot of posts at once. I’ve put days into speeding up the ‘get a list of posts’ API endpoint and it’s become acceptable but still not great.

    The next version of the API is going to be a lot slimmer both in the responses it generates and the CPU work involved in making them. At present we are trying to produce API responses that are very close to what Lemmy provides and that brings a lot of baggage.