I recently setup a new Lemmy instance and was surprised when my feed was mostly empty. I’ve since learned that a key part of Lemmy’s federation is based on a user from your instance subscribing to communities on other instances. Only then, will your instance pull in posts from the subscribed community to your “All” feed.

This means that subscribing to new communities is especially important if you’re on a young Lemmy instance since it helps to build out everyone’s feed on that instance.

I’ve found discovering new communities to subscribe to on other instances can be difficult. To help me search for new communities I may be interested in, I tried aggregating as much of the Lemmy fediverse together into a single feed by subscribing to the widest range of Lemmy communities possible. This offers a Lemmy feed that’s kind of like reddit.com/r/all. If it’s interesting to anyone else, you can find the instance here: https://lemmy.directory.

Hopefully this offers another way to find new communities to subscribe to on other instances.

Here’s a better description of my understanding on how Lemmy federates communities and why you might be interested in checking out lemmy.directory: https://lemmy.directory/post/34207.

Hope this helps ease the orientation to how Lemmy federation and communities work.

  • PriorProject@lemmy.worldEnglish
    0·
    1 year ago

    This offers a Lemmy feed that’s kind of like reddit.com/r/all. If it’s interesting to anyone else, you can find the instance here: https://lemmy.directory.

    You might want to update your instance link to be https://lemmy.directory/home/data_type/Post/listing_type/All/sort/Active/page/1. Ironically, it defaults to the local feed which is… empty. You could probably make nginx rewrite the homepage to be the all feed as well so the simple/nice url does what you want.

    You might also want to add a section to your post writeup about federation load and why Lemmy doesn’t do this by default. In a world where Lemmy is very successful and there are lots of instances (many thousands) that subscribe to all communities like you’re doing…

    • An instance hosting lots of niche/unpopular communities is going to be sending thousands of copies of every post/comment off to instances where no one is going to read them. The federated network is small right now, and federation replication load is not a big issue… but the way it remains a non-issue as the network grows is for communities to replicate only when users on an instance are interested in them.
    • Each instance would have to handle the replication and storage of the entire lemmyverse. I don’t know if that’s a meaningful workload yet… but again… part of what keeps single-user instances viable in the future as the network grows is ensuring that they only have to fetch and store content that their user is interested in.

    Very cool project though. Having an “all” instance sounds like a great service for discoverability. I’d also be interested to see a writeup from you about the hosting requirements. Does it take a lot of CPU/ram/disk to receive the full lemmyverse right now, or is it trivial? Your performance profile could be an interesting leading indicator of replication load as the network grows.

    • kiwi@lemmy.directoryOPEnglish
      0·
      1 year ago

      You might want to update your instance link to be https://lemmy.directory/home/data_type/Post/listing_type/All/sort/Active/page/1

      Thanks for the tip, I updated the link! And I’ll work on drafting some more text about the implications of subscribing to everything.

      An instance hosting lots of niche/unpopular communities is going to be sending thousands of copies of every post/comment off to instances where no one is going to read them.

      The rate a small instance would need to send off copies would be tied to the frequency of posts to the community, right? I’m interested in aggregating the lemmyverse, but wouldn’t want to overburden other instances in doing so. Although, honestly, I’m more worried about overburdening my own trying. Were you suggesting that it might be rude to subscribe to a community just for discoverability?

      So far the hosting requirements seem moderate, although I’d image if more people browse the feed the CPU and database reads will spike. I’ll keep an eye on them. I’m not sure if this project is feasible/needed yet, but I thought it might be interesting to try. And if ever help, then I’d imagine during the start of a bunch of new instances.