Setting up a mail server with OpenSMTPD, Dovecot and Rspamd

No description provided.

Awesome article, thanks for all your work!

neat, been waiting for this article for some days now since I read your "mail's not hard" article just when I was trying to set up an openbsd smtpd server myself.
Since I'm still flailing a bit (it is working but I managed to create local mail loops for instance), I'll try and recreate and tweak your setup next :) Hints on how to set up virtual users are most welcome (and how I can relay local user mail to those virtual users)

Anyway: Found a small typo it should say options.inc (not options.conf) in this line: "This is done by adding the dkim_signing filter to the filters key configuration in /etc/rspamd/options.conf:"

When I find more I'll edit this post. :) Thanks again Gilles!

@mchack23

@mchack23 typo fixed

the source markdown for the articles are in this repository, feel free to submit pull requests for typos and such :-)

I have been using Vultr 1GB VPS for about a year with a LEMP server spitting out transactional mail without issue. A few months ago I moved my MiaB over to the same service and haven't had any real issues, never submitted a ticket for any sort of filtering. Are you using a European DC?

Have you found dnswl.org to be useful? I see a lot of the big guys are there.

I look forward to your article on Microsoft because I'm not sure it can be done, as well as the DKIM key size as this is the first I've been aware of this.

While I originally taught myself how to configure my own mail server, I moved to MiaB in order on a simple VPS because as a personal mail server, it's fantastic and trouble-free, as much as a mail server can be.

However, I am wondering if you ever cover capacity limits? For some of the organizations I think you are targeting, they may quickly max out a VPS so capacity planning seems like a very relevant topic. What is considered for CPU limits in a mail server? What is using up memory and how painful is swap? How much storage should I plan on using, especially when forecasting growth rates? Are there some rules of thumb for any of these? Can I share block storage with more than one server? Reverse proxy? Etc...

I have also wondered about IPv6. What happens when spammers can just use a new IP address for every email and blast you a million times a second into eternity? Can I get by with just IPv4?

If reading about how you can build such setups is of any interest, I could also write about it to help people build MORE hosting alternatives which is ultimately what I hope for.

Yes, I'm interested.

The hard part is 5 years later when your current mailserver's OS goes end of life and you have to port everything to a new setup using different versions of things.

I'd like to suggest incorporating this in your howto:

OpenSMTPD could use LMTP (or execute dovecot-lda, not sure which works in OpenSTMPD) to deliver mail via dovecot, the benefit is that then dovecot updates it's indexes immediately, and you can benefit from sieve filtering for incoming mail. Then you can make dovecot filter e.g. spam messages to Spam folder directly.

@benwaffle

Thanks for the great writeup. As @benwaffle mentioned, I'm also interested in large-scale email server setup especially about virtual users and multiple domain managements.

I will write instructions assuming the upcoming 6.6.0 release which is due in a few weeks now. It brings a ton of improvements and I don’t want the instructions to be obsoleted and need a rewrite in October.

Out of curiosity, which is the OpenSMTPD v6.6.0 specific settings? Will there be similar breaking changes on OpenSMTPD in the future?

PR #18, I think the flag does not match the text.

@myfirstnameispaul

@myfirstnameispaul

I have been using Vultr 1GB VPS for about a year with a LEMP server spitting out transactional mail without issue. A few months ago I moved my MiaB over to the same service and haven't had any real issues, never submitted a ticket for any sort of filtering. Are you using a European DC?

For my own needs, I'm using multiple DC (Amsterdam, Frankfurt, London, Paris) all in Europe yes.

Have you found dnswl.org to be useful? I see a lot of the big guys are there.

In all honesty no, I know dnswl.org but I can't even recall if I ever subscribed mine, so they may be useful but you can certainly do without.

I'm 100% positive that I didn't subscribe most of the domains I plugged.

However, I am wondering if you ever cover capacity limits? For some of the organizations I think you are targeting, they may quickly max out a VPS so capacity planning seems like a very relevant topic.
What is considered for CPU limits in a mail server?
What is using up memory and how painful is swap?
How much storage should I plan on using, especially when forecasting growth rates?

I don't because it really depends on what you are going after.

I documented a very simple setup which would work fine for a lot of people on the cheapest VPS, because that's how I test these setups and even load them a little, but how people need to handle capacity planning depends on many factors.

On my own setup, Rspamd instances run on machines of their own so I can adjust them depending on CPU and RAM usage, whereas Dovecot runs on a separate machine with decent disks, whereas my OpenSMTPD instances are basically CPU-bound, delivering the messages to a remote machine and not the local disk. My planning here is per-service and since services are per-machine, this is something I'd expect a company to do but not a lambda user, my recipes won't work for them.

Then, even on a simpler setup, some users will want MariaDB to handle their accounts, others will want an elk stack to monitor, others will want a webmail and maybe some fancy log analysers, and depending on where you run these, it changes radically how you do your planning. A machine could run very fine with your mail infra and load like hell just with elastic installed, so you might need to go for a real dedicated server or much higher-frequency CPUs just for one of the components.

I think capacity planning is not very different with mail than with any other service and, as far as I'm concerned, it goes by graphing and monitoring how things evolve in time to adjust, I don't have a secret works-for-all recipe. Some more competent SREs surely have better solutions though.

A very cheap VPS will do as they are more powerful than most mail servers from the last decade and the software have not grown more hungry in resources (only speaking about opensource solutions, not the commercial ones which will plain not work on a VPS).

For Rspamd, I have no idea what the minimal resources are, but I had never had a problem running it on the cheapest VPS I could get my hands on. It mostly does CPU, that's about all.

For Dovecot, this is VERY dependant on how many users you have, how they consume their mail, if they search mailboxes and such, so again I had never had a problem running on the cheapest VPS BUT your mileage may vary and graphing resources consumption to adjust is key. It mostly does CPU and DISK but is also mostly idle if not on a busy server.

For OpenSMTPD, not only the cheapest VPS will do but SMTP servers are mostly idle, so testing the limits is quite difficult. It can run (not just build but actually run) on very cheap machines, people use Raspberry boards, as well as on some ancient or not-so-powerful architectures supported by OpenBSD (https://www.openbsd.org/plat.html). It does CPU and DISK but it can also deliver remotely (what I do) in which case it mostly does CPU when it's not idle, even on a relatively busy server.

A lot of people thing that a lot of CPU is needed for all the TLS handling but this is not correct, I could share stories about that ;-p

I don't observe variations in terms of memory usage across time and on a server that has been running for a while, the memory consumption of just mapping the Go runtime is an order of magnitude higher than the memory consumption of any process from Dovecot or OpenSMTPD for instance.

When looking into how good I'm with resources, my main concern is if I should add disk or not.

Are there some rules of thumb for any of these? Can I share block storage with more than one server? Reverse proxy? Etc...

The sharing of block storage is a tricky part and depends on what system you're using.
If using Maildir backend, you're supposedly safe when sharing block storage because Dovecot keeps its state internal to a Maildir, and OpenSMTPD delivers atomically and lockless inside Maildir.
Then questions arise if you're not only sharing block storage but also clustering / cloning the SMTP instances or the IMAP instances. Both are technically doable, not necessarily in a very tricky way, but I would not say it's easy.

Reverse proxying is simple in both cases, both Dovecot and OpenSMTPD can support proxy-v2 and as a matter of fact I'm using an haproxy myself.

I have also wondered about IPv6. What happens when spammers can just use a new IP address for every email and blast you a million times a second into eternity? Can I get by with just IPv4?

That's a problem.

But since it's not globally solved by Big Mailer Corps, IPv6 remains insignificant when it comes to mail as of today and you can certainly just get by with IPv4.

@cmouse

@cmouse

I'd like to suggest incorporating this in your howto:

OpenSMTPD could use LMTP (or execute dovecot-lda, not sure which works in OpenSTMPD) to deliver mail via dovecot, the benefit is that then dovecot updates it's indexes immediately, and you can benefit from sieve filtering for incoming mail. Then you can make dovecot filter e.g. spam messages to Spam folder directly.

I'd rather see people write such howtos ;-)

@ynakao

@ynakao

Thanks for the great writeup. As @benwaffle mentioned, I'm also interested in large-scale email server setup especially about virtual users and multiple domain managements.

I will write instructions assuming the upcoming 6.6.0 release which is due in a few weeks now. It brings a ton of improvements and I don’t want the instructions to be obsoleted and need a rewrite in October.

Out of curiosity, which is the OpenSMTPD v6.6.0 specific settings?

All the filter stuff comes with v6.6.0 so while you could have rspamd, dkim and such, it was done through a slighly trickier method of "queue reinjection" where you tell OpenSMTPD to relay to an Rspamd proxy or a DKIM proxy and you tell the proxy to re-enqueue in the daemon.

It was not necessarily much more complex in terms of configuration but it was a bit more complex in terms of understanding how it works and far less elegant than just adding a filter keyword to a listen line :-)

Will there be similar breaking changes on OpenSMTPD in the future?

In this case the change is not breaking, the configurations that worked before would still work, but they would be far less elegant and efficient at solving the issue, or would even be missing some of the improvements.

That being said, OpenSMTPD is a very active project with a release cycle of six months and we'll have breaking changes every now and then because we'd rather have users change a keyword than let technical debt accumulate.

We have had a very big breaking change a year ago where the configuration file was completely revamped, this was the first time since the project was first started ten years ago, so a huge breaking change should not happen before another decade :-) but it's likely that within the next year, we'll change a keyword here or there to help improve the project.

I'd like to suggest incorporating this in your howto:

OpenSMTPD could use LMTP (or execute dovecot-lda, not sure which works in OpenSTMPD) to deliver mail via dovecot, the benefit is that then dovecot updates it's indexes immediately, and you can benefit from sieve filtering for incoming mail. Then you can make dovecot filter e.g. spam messages to Spam folder directly.

I'd rather see people write such howtos ;-)

Perhaps. But I think it would make sense to involve dovecot on the delivery phase always, as it would enable much more functionality in dovecot and would allow dovecot to update it's indexes during delivery.

My only concern for HA mail setup are mails waiting in a queue to be delivered, eg. a mail server in DMZ with paused delivery aka "deffered relaying" (cf 9.2.1 Postponing Mail Delivery in Postfix: Definitive Guide; would that be doable with smtpd -P mta?) and which waits for a manual trigger to try to deliver via SMTP to a mail server in intranet. If the mail server with this setup would fail, the mails will be lost. IIRC 'extras' used to allow Python plugins which could save mails into redis etc. to have a queue "distributed"...

Thank you!

Hi Gilles,
thank you for the great article! It finally got me to host my own email.
I do have a problem however: It seems like rspamd is not DKIM signing my outgoing mail despite setting up /etc/rspamd/local.d/dkim_signing.conf and using filter-rspamd in /etc/mail/smtpd.conf.
Is there something else that needs to be enabled in Rspamd or OpenSMTPD to make this work?

Edit: It ended up being a simple typo, my /etc/rspamd/local.d/dkim_signing.conf erroneously set allow_user_name_mismatch=true; instead of the CORRECT allow_username_mismatch=true;, note the extra underscore in the incorrect version.

Another thing that might be interesting to mention is to include the mask-src option for the submission listening clause of /etc/mail/smtpd.conf. That way the E-Mail doesn't contain the information from which IP address the mail was submitted to the mail relay.

Hi, CAVEAT(s): I am: - using postfix+rspamd for public MTA (MX routing) - using multi-node email setup This is my /etc/rspamd/local.d/dkim_signing.conf allow_username_mismatch = true; domain { my_some_domain.xxx { path = "/var/lib/rspamd/dkim/dkim-2021q1.key"; selector = "2021q1"; } } sign_networks = [ "aa.bb.cc.dd", ] I had to add the `sign_networks' setting for *my* setup since locally originating email comes from a separate VM instance. I also have /etc/rspamd/local.d/options.inc local_addrs = [ "127.0.0.0/8", "aa.bb.cc.dd", ] This is to let rspamd know which IP(s)/networks it should consider "local" - originators for email from our own domain. The "sign_networks" setting in dkim_signing.conf is really redundant because rspamd should sign outgoing mails from "local" sources (sign_local = true setting). You can get a dump of all rspamd settings by issuing: rspamadm configdump > rspam-settings.txt Hope it is somewhat helpful. didar

…

-- The Wright Bothers weren't the first to fly. They were just the first not to crash.

Hello,

When I test the mail with this service "www.mail-tester.com", I got a warning about the lack of SPF and DMARC records.

"You do not have a SPF record, please add the following one to your domain mx1.domain.net"
"You do not have a DMARC record, please add a TXT record to your domain _dmarc.mx1.domain.net"

In my understanding of the SPF and DMARC records of the guide, it covers the MX servers or the domain.
Correct ?

Here is the zone of my domain :
`$ORIGIN mydomain.net.
$TTL 3600 ; 1 hour

; Start Of Authority
;===================
@ IN SOA ns1 admin.mydomain.net. (blabla serial, refresh, etc)

@ IN NS ns1
@ IN NS ns2
@ IN MX 10 mx1

; NS Records
;===========
ns1 IN A ip_address_ns1
ns2 IN A ip_address_ns2

; MX Records
;===========
mx1 IN A ip_address_mx1

; SPF Records
;============
@ IN TXT "v=spf1 mx -all"

; DMARC Records
;==============
_dmarc.mydomain.net. IN TXT "v=DMARC1;p=none;pct=100"

; DKIM Records
;=============
20210328._domainkey.mydomain.net. IN TXT "v=DKIM1;k=rsa;p=MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCpUznIPZz8gc4aZ94a/87/Tvaezhe3hgR2nLbUs3q23gUsZ5tPyLaPqRdVMxXpC00Yk6VGMwMaGHpFx1w4GuoOAAIwX1PFT6WyNCR+JQzOaCocI4xO+G97gV9p94zubpNd6nvxSsjEbheJ1ILYwUNOnTPrDNCToRfN9O/F67jzDwIDAQAB"
`

The mails are not also signed with DKIM and I have no clue of the reason :-(

Any idea please ?

I solved the SPF and DMARC complain by sending e-mail with the command :
"mail -r foo@mydomain.net -s test xxx@srv1.mail-tester.com"

For information, before, there was no "-r foo@mydomain.net".

But the missing DKIM signature is still there :-(

Is your DKIM record correct? If you copy it from the console (cat /etc/mail/dkim/domain.com.pub for example) it will have line breaks, so I'd take a look at that.

I finally understood that the messages are signed only when sent from a an external client that authenticate itself first.

By curiosity, what is the configuration to get DKIM signed messages when sent locally (directly from the server) ?

I tried to ad to the configuration the part "learn to Dovecot to train rspamd" and it does not work.

It seems the pigeonhole is not compatible : on OpenBSD 6.8, once I installed the official package "dovecot-pigeonhole-0.5.11v1" I can not restart dovecot.
I get this error message :

root@mx1 [11:27:17]:~ # dovecot -F
doveconf: Error: Couldn't load plugin /usr/local/lib/dovecot/settings/libmanagesieve_login_settings.so: Module is for different ABI version 2.3.ABIv11(2.3.11.3) (we have 2.3.ABIv13(2.3.13))
doveconf: Error: Couldn't load plugin /usr/local/lib/dovecot/settings/libmanagesieve_settings.so: Module is for different ABI version 2.3.ABIv11(2.3.11.3) (we have 2.3.ABIv13(2.3.13))
doveconf: Error: Couldn't load plugin /usr/local/lib/dovecot/settings/libpigeonhole_settings.so: Module is for different ABI version 2.3.ABIv11(2.3.11.3) (we have 2.3.ABIv13(2.3.13))
doveconf: Warning: NOTE: You can get a new clean config file with: doveconf -Pn > dovecot-new.conf
doveconf: Warning: Obsolete setting in /etc/dovecot/dovecot.conf:25: 'imaps' protocol can no longer be specified (use protocols=imap). to disable non-ssl imap, use service imap-login { inet_listener imap { port=0 } }
doveconf: Fatal: Error in configuration file /etc/dovecot/dovecot.conf: protocols: Unknown protocol: sieve

I can restart Dovecot normally after deleting the package.

Edit on 06th May :

After reinstalled the server with OpenBSD v6.9, the issue disappeared.
I'am waiting incoming spam to check the learning feature does work.

Just wanted to share that I have followed this guide both on 6.8-stable and 6.9-stable and it worked really well.

FWIW, I used this guide to setup my email server on OpenBSD 6.7 I believe and upgraded the system twice since then. Neither of those upgrades broke the email server.

…

On 21/05/22 07:44AM, whateverthisisafasf wrote: @hagnert, yeah same for me (mutt login). Been getting following error in my logs: `SSL failed: error:1408F10B:SSL routines:ssl3_get_record:wrong version number` I have to admit that I didn't search for to long, but would be very thankful for any helpful advice. -- You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: #17 (comment)

@poolpOrg

@poolpOrg

I read some of the previous posts on this thread, and though this might come across as a support issue, I think the following is worthwhile mentioning since its an issue with the midpoint of the blog post itself, so it kind of teeters towards something that the blog author could clarify.

To begin, I wanted to add the following points:

Thanks for the post itself, seems pretty decent, and was straight forward up to this point.
I followed the points up to a T, so I am fairly certain I did not miss anything.

Within the "Installing and configuring Rspamd" section, you listed the following command:
mail$ doas pkg_add redis rspamd opensmtpd-filter-rspamd

However, when I have done this, you are forced to utilize the 'doas' configuration file before you're allowed to do this. I would ask if you could update your blog post to reflect this because this wasn't mentioned. This might seem straight forward to someone experienced with this, but keep in mind, most people like myself use Linux for this kind of thing, so even using OpenBSD for this whole process is absolutely new to me.

So, anyway, I basically copied the standard 'doas' configuration file into /etc/ with the following command:
cp doas.conf /etc/doas.conf

I then proceeded onward with the remaining sections of that post.

The part that I'm currently stuck on is "Also make sure the dkim key file is readable by members of the _rspamd group." instruction.

The problem with this directive is that when I use the 'groups' command, I get the following output:
wheel kmem sys tty operator staff guest

As you can see, I don't see an '_rspamd' group at all.

I'm assuming that your 'doas' command was supposed to perhaps install rspamd as its own user or group (not sure, totally new to all this).

I did a bit of research, and if you want to make a file readable by a group, then the related permissions are '040' (https://kb.iu.edu/d/abdb)

However, I'm not sure what exactly to do in this scenario for the '/etc/mail/dkim/(mydomainname).key' file itself's permissions since the '_rspamd' group doesn't clearly exist on the OpenBSD VM I'm running on Vultr.

This is more apparent since when I try to start the 'redis' and 'rspamd' processes with the 'rcctl' commands, I get the following output:

# rcctl start redis
redis(ok)
# rcctl start rspamd
rspamd(failed)

Any help on this?

Thanks!

Navigation Menu

Repository navigation

Setting up a mail server with OpenSMTPD, Dovecot and Rspamd #17

95 remaining items

Edit on 06th May :

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Setting up a mail server with OpenSMTPD, Dovecot and Rspamd #17

Description

Activity

leo-unglaub commented on Sep 14, 2019

mchack23 commented on Sep 14, 2019

poolpOrg commented on Sep 14, 2019

myfirstnameispaul commented on Sep 14, 2019

benwaffle commented on Sep 14, 2019

superkuh commented on Sep 14, 2019

cmouse commented on Sep 15, 2019

ynakao commented on Sep 15, 2019

omoerbeek commented on Sep 15, 2019

poolpOrg commented on Sep 15, 2019

poolpOrg commented on Sep 15, 2019

poolpOrg commented on Sep 15, 2019

cmouse commented on Sep 15, 2019

jirib commented on Sep 16, 2019

95 remaining items

redeltaglio commented on Mar 2, 2021

knightshrub commented on Mar 9, 2021

didarhossain commented on Mar 10, 2021

yf4Yeh4ZLt89lT commented on Mar 28, 2021

yf4Yeh4ZLt89lT commented on Mar 29, 2021

netstx commented on Apr 3, 2021

yf4Yeh4ZLt89lT commented on Apr 10, 2021

yf4Yeh4ZLt89lT commented on Apr 29, 2021

Edit on 06th May :

netstx commented on May 6, 2021

hakova commented on May 22, 2021

RennisDitchie commented on Jun 6, 2021

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions