- cross-posted to:
- selfhost@lemmy.ml
- cross-posted to:
- selfhost@lemmy.ml
In the past few days, I’ve seen a number of people having trouble getting Lemmy set up on their own servers. That motivated me to create Lemmy-Easy-Deploy, a dead-simple solution to deploying Lemmy using Docker Compose under the hood.
To accommodate people new to Docker or self hosting, I’ve made it as simple as I possibly could. Edit the config file to specify your domain, then run the script. That’s it! No manual configuration is needed. Your self hosted Lemmy instance will be up and running in about a minute or less. Everything is taken care of for you. Random passwords are created for Lemmy’s microservices, and HTTPS is handled automatically by Caddy.
Updates are automatic too! Run the script again to detect and deploy updates to Lemmy automatically.
If you are an advanced user, plenty of config options are available. You can set this to compile Lemmy from source if you want, which is useful for trying out Release Candidate versions. You can also specify a Cloudflare API token, and if you do, HTTPS certificates will use the DNS challenge instead. This is helpful for Cloudflare proxy users, who can have issues with HTTPS certificates sometimes.
Try it out and let me know what you think!
You must log in or register to comment.
Literally been thinking about this so thank you beautiful brained individual. Would you mind if I shouted this in the YSK group?
Thank you very much for the kind words!
Please be my guest! It would make me happy to know this was helping people join Lemmy!
Been pounding my head against the desk for the last TWO DAYS trying to get everything to work. Then you came along and solved all of my problems and it only took me 10 minutes to set up (mostly due to waiting on DNS to flush!)
THANK YOU SO MUCH for creating this, and PLEASE continue to maintain! I will gib coffees if need be along the way!!
Really awesome work. We need more Lemmy servers!
seriously, distributing the load helps a LOT. Though if you can’t spin up your own instance one thing you can do is try and host pictures externally, in !youshouldknow!youshouldknow@lemmy.world a post mentioned how to do it for images in comments since by default it has you upload if you don’t manually put in
The check
$LEMMY_HOSTNAME == http*will give a false positive if (for whatever reason) the domain name starts with httpThanks! Fix pushed.
I will definitely try this out. I already have my domain and SSL certificate. This will work on linode right?
It will work on pretty much anything that has a public IP and a domain pointing to that IP. The only thing that won’t work “out of the box” for most users is email, as most VPS providers block port 25. If you’ve requested access to port 25 and have been approved to use it, you can edit
config.envto turn on the email service.As for your SSL certificate, unfortunately this does not support importing your own certificate. It’s made for beginners, after all :p
But there should be no problems with Caddy simply requesting a new one for you!
I think I have successfully got the server running on my linode.
I’m glad to hear that! Thanks for letting me know, it’s nice to hear people were able to use my script to get up and running :)
how well do those email servers work to begin with? i just flat out disabled the postfix relay on my instance and simply configured sendgrid, which works perfectly, no delays or spam folder issues (although i did have to disable a bunch of tracking bullshit). doing so with similar services has been my go-to card in freelance webdev, because getting other mail servers to trust you can be hard, so i’m interested in the experience people are having with those. (i’d much prefer to self-host email too, but providing a good experience is the primary goal)
I haven’t actually used the embedded postfix server at all, I keep mine disabled. I only include it because it’s “included” in the official Docker deployment files, and I try to keep this deployment as close to that as possible.
I’m considering adding support for an external email service, as you mentioned, but I have nearly zero experience in using managed email services, and I’m not sure if non-technical users would be able to navigate the configuration of things I can’t do for them (i.e. on a web dashboard somewhere). And if I can’t do it for them, it means more issues for me, so I hesitate to add support for it at all.
I’d love to hear your experience in setting up sendgrid and how easy that was. And the tracking stuff you mentioned as well.
On the surface, setting up an external mail server is actually super easy. Pretty much every service supports SMTP, so all you need is to find your credentials (an endpoint, a username, and a password) and input that in your lemmy.hjson file, along with a sender address. There is a verification process with every email provider, you will usually have to set some DNS records to prove you own the domain you’re trying to send from, but once you do these two things, emails will just start flying out.
The actual process tends to be a little more tedious though. Pretty much every mail provider wants to sell you on their specific super awesome API and only mentions SMTP as a footnote, for reasons that I can only parse as an attempted vendor lock-in. They also sometimes make the verification process difficult. SendGrid, for example, immediately redirects you to their “sender identities” which are more useful for their marketing automation tooling than they are for automated emails from a website, and you also have to dig a fair bit to find the SMTP credentials. (The endpoint is
smtp.sendgrid.net,apikeyon the username, and whatever your API key is for the password, you’ll need to set one of those up.) They also have a number of different tracking things that you’ll have to find and turn off in the UI (luckily all of them can be toggled in the same place), which would normally actually help you by providing you statistics about how many emails were opened and how many people clicked on links, but for the emails Lemmy sends, it’s useless and I didn’t want SendGrid to collect that data on my instance users, so I shut that off.Initially, I wanted to use Amazon SES, but they have an even more tedious setup: you have to talk to an actual human to have them turn it on for you, otherwise you’re stuck in sandbox mode and can only send email to verified receiver addresses. Their service works well though, I used it a lot in my freelancer days, and they do make it fairly straightforward to find the SMTP credentials, but the verification process can be slow.
I also remember using Mailgun for some project ages ago, it was pretty much the same deal as all the others: you have to pry the SMTP details out of their docs, but once you do and you verify your stuff, it works well.
Ultimately, I think the best you could do to help out users here is to provide them with documentation for a few of these services: exactly where to go, which buttons to click and whatnot. Or just stick to the embedded service for a bit, but the real issue there is the bounce rates from spammers. They made my instance send out like 60-70 verification emails for bogus addresses on gmail, yahoo, hotmail, and live.com on the first day I got federation working, and if you don’t already have a good reputation – which is what these third parties help you with – then you can easily get yourself blacklisted, either internally or externally.
I see. Thanks a lot for this!
I really don’t have the capacity to support a bunch of different email services, so it sounds like the best I can do right now is make the SMTP settings accessible without also running the postfix server. So if someone wants to run their own email somewhere else, they can configure it. But otherwise, I’ll leave it to the user to figure out what happens after an email request leaves Lemmy.
Does that sound fair, and like something you would have used? Essentially just an interface in
config.envthat puts the right SMTP address/credentials inlemmy.hjson.yup, that sounds perfect
i can’t track down a non vendor-specific guide right now but if you just expose the same settings in
config.envthat you’d have in thelemmy.hjsonfile that should be more than sufficient to get an external service goingThanks!
I’ve got that included in a staging version I’m preparing to release, if you wanted to take a look at the changes:
Will try this tomorrow. Tried them all. Nothing seems to work! I have been at it the whole week trying.
I’m relatively competent installing server software, but the Lemmy instructions completely flummoxed me. Their docker instructions just don’t work.
I ended up using the ansible docker scripts and filling out the blanks because I’m unfamiliar with ansible.
If this is as good as it sounds, you’re doing everyone a massive favour.
Works great on Linode!!
Thanks for the helpful tool! Posting this from my new single-user Lemmy instance. I ended up tweaking the compose template a bit to remove Caddy since I already have it running on this VPS for other services. Wasn’t too bad to just take the Caddyfile information and add it to my own existing framework.
This was absolutely amazing. I was having some trouble with the build process using the docker compose from Lemmy itself, but this just instantly worked. Thank you!
Nice! Looks like it even has update checker as well. Is there any reason why
pictrsis not included in the update checker and hardcoded to version 0.3.1?The Lemmy maintainers themselves seem to lock it at 0.3.1, and I wanted to maintain parity with their deployment. I know
pictrsis up to at least0.3.3, and has a release candidate for0.4, but upstream Lemmy uses0.3.1for whatever reason, so that’s why I lock it there.It’s excluded from the update checker because I don’t have a stable way to check what version upstream is using. The Lemmy update checker just checks to see what the latest tag on
LemmyNet/lemmyis. I could try and pull the latest Gitea tag forpictrs, but since upstream Lemmy isn’t using the latest version, that’s not really an option as something might break.I considered trying to parse their docker-compose.yml file to see what version they use, but they seem to be restructuring their
dockerfolder right now. The folder inmainis completely different from the one tagged0.17.4. If I assume a certain directory path for that file for every version after this, but they move it, my script will break. Sadly, until their Docker deployment files seem like they’re going unchanged for a good few versions, I’ll have to do it manually for now.I see, looks like it’s a correct decision to me. Let the Lemmy developers worry about which version of pictrs to use themselves.
Looks really good. I did it pretty much the same way, myself - but if I were looking to start again, I would definitely use this.
Edit: Ran it on a fresh AWS Ubuntu instance and it worked perfectly fine.
Were you able to get AWS to open port 25 for smtp support?
I didn’t bother, as I was just testing. But you are right, port 25 outbound is blocked by default. They have a defined process for you to ask for it to be unblocked and you have to tell them what you are using it for and how you are preventing spam from being sent. In this case it might be enough to say that you aren’t allowing port 25 inbound, so it can’t be used as an open relay.
Yeah, I tried and failed. They denied it and told me to use SES lol.
Whatever haha
I am not 100% surprised they refuse to do it for new accounts. If you have an account that has been with them for a while, they most likely would open it.
Problem with SES is that you start sandboxed and can only deliver to specific email addresses - which obviously won’t work here.
Yeah maybe, i’ve had my account for a few years but who knows.
If i decide do use email i’ll just make an smtp mail server on my homelab
I just made a post about having issues with getting a fully functional instance so I think I’m gonna give this a try.
I hope it works out for you!
Quick note: For email, pretty much every VPS provider out there blocks port 25, which is needed for emails to send. They do this to prevent spam emails from being sent en-masse from their servers. This is likely why your Ansible installation is not sending emails.
Since it’s uncommon for servers to support email, this script disables it by default. If your provider supports port 25 (or you get approved to use it, some VPS providers allow you to request access), check
config.envand setUSE_EMAILtotrue. This will set up everything you need for email.I haven’t been able to test email, so let me know how it works if you do! This doesn’t do any of the DNS verification some email provders require, so your emails might be sent to spam. Lemmy doesn’t really have documentation about how to set this up properly. If someone makes guidelines for this, I can update my project to do that automatically as well.
Looks great my dude.
If you expanded out the environment variables a ton, making it more customizable, (with default values in place of couse) this could appeal to a huge range of people.Can you explain? I provide an interface for everything available in
lemmy.hjson, so I am not sure what else I would add.I will note though, this is primarily intended for beginners. More advanced users would probably prefer to manage this on their own with Docker Compose, and those people will be very well versed in messing with the environment variables and all that.
If there are variables you want to pass in, you can simply edit
docker-compose.yml.templateto import anenv_file, that way you can pass anything you want into the container.
So stupid question - how do you shut them down? Docker compose down or stop don’t seem to work.
Also, as an intermediate (spoiled Unraid) user, if I’m behind nginx proxy manager with custom ports do I just change 80 and 443 in the lemmy docker-compose? I’m doing this in a Debian VM.
