I was told that I should post this here.
cross-posted from: https://lemmy.world/post/932750
Say you decide to self-host a Lemmy instance. When you create that instance, do you immediately need to download and store all the data that has ever been posted to all federated Lemmy instances? Or perhaps you only need to download and store everything that is posted to the federated Lemmy instances from that point forward? Or better yet, do you only store what the users on that instance do (i.e. their posts, and posts to the communities hosted on that instance)?
I’m not really sure how this is supposed to work long-term, then. I can’t imagine anyone wants to be on an instance with only a fraction of the content available. It makes perfect sense when subscribing, but surfing All loses its appeal. I understand the challenges, but I hope there’s a creative solution at some point. It seems like folks will gravitate to the instances with the most stability and users.
I think you’re right. People will gravitate to the most stable large instances because their “All” will be as close to 100% as possible without doing anything special. I wrote a script to seed instances and update subscriptions, but it uses a single account that is subscribed to everything so that other users can see everything. That’s not something that would normally happen. Maybe that needs to be part of the base software?
I don’t suppose your script is published anywhere? My comment adjacent to yours mentions how something like Mastodon’s Relay system would really help solve this issue, and it sounds like what you’ve made is probably the closest thing we’d have to a relay system for a while (given the core devs being super super busy with the existing issues).
On a side note, I wish there was also a way to set the homepage of an instance to “All” as well (which can be done user-side, but not globally), my instance only has a meta-community for announcements, so I can imagine that it just looks like an absolute ghost town to anyone who stops by.
https://github.com/jheidecker/lemmony
That’s cool, thanks i’ll check it out. I also found https://github.com/Fmstrat/lcs
That‘s so helpful, thank you for sharing it with us! One question: If I want to update my known communities in a month or so, can I just rerun the script or will that cause issues?
You should be able to rerun it anytime. It only gets stuff that doesn’t exist on your instance. That’s how it was designed. It is dependent on browse.feddit.de however. :(
Have you got a link to that script? I want to seed my local private instance!
https://github.com/jheidecker/lemmony
Knowing that instances only pull posts/comments that occur after the first subscription, it will become less and less viable to choose a small instance if Lemmy doesn’t add the option of adjustable pull settings.
It’d be really cool if we had something similar to Mastodon’s “Relays” where you basically subscribe to a firehose of posts from everyone whose on a server connected to that relay (they show up in the “Federated Timeline”).
I don’t know exactly how this would work for Lemmy, but it seems like if we had a system like this it could really help tackle this issue.
I think there is already some software that fetches content but it’s early in development.