Stop going to Spam: DMARC, SPF and DKIM Explained

Welcome to Syntax on this Monday Sanity treat. We're gonna be talking about email.

All about stop going to spam, DMARC, SPF, DKIM.

These are all things that you typically just accept as existing whenever you connect to your email clients and largely might ignore. But we're going to be talking all about what they are and how it works within your transactional email. My name is Scott Tolinski. I'm a developer from Denver.

With me JS always is Wes Bos. What's up, Wes?

Hey.

Excited to talk about DMARC Wes and DKIM.

I there's some news sender requirements both from Google and Yahoo in February, which is now.

And if you do not, if you send any type of email, whether it's just marketing email, transactional email or just from your own domain name, you're going to want you have to get this implemented. Otherwise, you're gonna be sent to spam.

Yeah. You get better bring that SPF or else you're gonna get burned. And you also might get burned if you don't have the correct air and exception handling tracking service for your software.

And I'm talking about Sentry. This podcast is presented by Sentry, but, honestly, it's such a great service. If you're working with any of this stuff, you got a server, you're sending an email. It's good to know what's going on. Maybe your email's not even sending because you have a bug. And, Sentry JS gonna find that bug for you and then tell it tell you how to fix it really quick.

So let's get going here. Let's get going on this custom email stuff.

Wes, like you, I have written a lot of applications that send email. Yep. I've done so through many different email clients, many different email delivery Vercel, and the part in which you get to add things to your DNS, connect your service, and send email has always been a big mystery to me.

I go to the docs.

I go step by step. I copy and paste the things they tell me to do it, and then it usually just works.

Yeah. So what what is all this stuff? I know you've done a little bit of a dive here. I'm interested in hearing about, like, what all these things actually are in this whole process here. Yes. So we're gonna be talking about SPF, DMARC, and

DKIM.

Being DMARC aligned is a way to get your DNS set up so that when you send an email as your domain name via any method, that it will be properly approved, properly signed and be allowed. And anything else, like if anyone else is trying to send email as you, then you can immediately tell all the other email clients to send that directly to spam or to quarantine. Then we'll talk about what the different options are. So let's start by just talking about, like, different ways that you might want to send email. You have a domain name, syntax, data from and you're going to want to send email in several different ways. First of all, you're probably going to want to send email via Google workspaces because we have a custom Google workspace set up or Outlook or whatever you're using for email, for your from your domain name. Right. So Google needs to be able to send email on your behalf as your domain name. Right. So that's 1. And then you also have like transactional email. So if we are sending a receipt or like for my own courses, when somebody buys a course, they reset their password.

There's an update to something that's a single email that is sent to a single user that is called transactional email. And you often will use services like Amazon, Postmark, SendGrid Mandrill. There's lots of them out there.

And those Vercel, like you're likely not running your own mail servers.

You're likely using a service that will send them on your behalf because they have all the infrastructure

so that's what's referred to as transactional email. And then then you also have email marketing. Hold on. Yeah. Sorry. Before you move on from that, let me even put another one in front of you JS resend JS one that I've been using lately that I recently resend.com.

It was really easy to get started.

Email for developers, it it it felt very nice as an onboarding process for this thing, and it it worked effortlessly. So shout out to resend.

So what is the resend? Like, is it it's. I don't think it's not it's not just another, like, transactional utility. Right? They also have some services on top of it?

Yeah. There are some some services on top of it. There's, like, React stuff. I think there's even some, like, template stuff inside of Oh, yeah. They'll compile your emails as well. Yeah.

But for the most part, I found it to be pretty cheap. For $0 a month, you get 3,000 emails a month.

A 100 emails a day. And if you're running a full side project

Yeah. It's not that bad. That's awesome. And then the the the next one we have is email marketing. So if you're sending out a newsletter ESLint Syntax, you should sign up for the Syntax newsletter. By the way, go to Syntax Scott FM for SNAPBACK.

And when you want to when you want to send a newsletter out via I use drip syntax, we use ConvertKit Mailchimp is a big one as well. There's lots of them out there.

They need to they have their own email service, so they need to do it. So like you generally have like at least 3 different ways to send email from 3 different Vercel, but you all want them to be sent as your domain at Wes or syntax.

Right. Other things like Shopify, like if Shopify needs to send a you buy something from the Shopify store and you need to send a receipt and use your own domain name.

Shopify needs to be able to send email on your behalf. Now, the tricky part is, is that, yes, you want other people to send email on your behalf. But no, you don't want other people to send email on your behalf because there's spammers can, because of the way email works, spammers can send email and and literally just spoof the fact that it's coming from Wes or Syntax Data Firm. And whether or not that shows up in your spam or not is up to Google or up to Outlook or whatever email you're using. But there are several different ways that you can ensure and sort of whitelist possible servers to allow them to send email on your behalf. And that's Wes gonna be talking about right now. So you're ready? I'm so ready,

because this is something that I think a lot of people do hit. They either get put in the spam zone or they, you know, I I feel like the the difference between transactional and all these different types of email in your application, it can get hairy if you've never done it or haven't done it before.

Honestly, like like me, if if I encounter a problem, I have to reach for customer support because I straight up Yeah. Am uneducated

in this stuff. So So the first one is SPF. There's 2 different ways that you can sort of, like, validate and and guarantee that your email is being sent, SPF and DKIM, and those 2 together make up what's called alignment or compliance. So allows you to specify which IP addresses or host names are permitted to send email on behalf of your domain name. So what I have to do is I go into my DNS records for my domain name and I say, okay, here is my record and these are the domain names that are allowed to send email on my behalf. So I have to include Google on there because Google is allowed to send email. But for me, I have to include Postmark on there because I use Postmark for transactional email and then I have to use whatever the drip one is. So drip itself doesn't actually send the email. They they use another service, so you have to add that 1 on there. And what will happen with those SPF records is they will go out to each of those domain names and find all the IP addresses that are associated with those domain names and basically create a whole list of IP addresses and say these servers around the world are allowed to send email, as Wes or syntax.fm.

And so to be clear, if you have multiple of these services in your SPF, do they all go in 1 record, 1 text record, or you do multiple text records? May only have 1

SPF record.

It's different for DKIM, but for SPF, you may only have 1 single record, and you can add multiple, hostnames in that that text record.

Nice.

You may also see a pnpm, and what those you generally won't have to dip into those. Those will also allow your subdomains and your MX records to send on your behalf. You can Scott of just say include those. But unless you're sending from subdomains like merch, Scott, dotfm, you don't have to add those to yours. Now, at the end of that SPF record, you're going to see an all property And it's either plus all tilde, all or minus all. And those will advise the receiver what to do with the emails if they don't meet SPF.

So I send an email to somebody with an Outlook inbox.

The Outlook will look up my domain name and check, hey, this person sent an email from this server. Was that allowed? Is that on the list of possible servers that are allowed to send email for Wes? And if it is not, you can. Plus, all will accept it anyway.

Tilda all will accept it, but it probably will go into spam or minus all. They're just going to reject it. It's not even going to go into spam. It's going to not even land in the user's inbox. It's just going to reject it all out.

So that's the first Node. Then we also have DKIM and Kim is stands for domain keys identified mail. And what it does is it adds a public key to your DNS records to allow email receivers to verify your outgoing sign email. So what does that mean? Well, public key, private key, we've talked about it in the past. I'm not going to go super deep into it, but your email sender. So in my case, postmark that sending a piece of transactional email, what they'll do is they will add a signature to the header of your email that is signed with your private key.

Then the receiver, Gmail, Outlook, whoever is actually receiving the email, will be able to look up your public key. So you sign it with your private key. Nobody can see that. You don't have to do any of the signing. Your email sender takes care of that, And then the receiver will get the public key and it will say, okay, well, I have the public key. Let me verify that the hash that was generated in the headers is is actually something that could have been generated, given that we now have the public key.

And if it's not, that means 1 of 2 things. 1, the the contents of the email were changed at some point along the Node. So it was intercepted and changed. And you can say, all right, well, same with like SSL certificate. Right. If somebody intercepts your Wes site between your server and the browser, the public key is going to allow you to say, oh, it's not someone someone goofed with it. Or the other thing is that someone just sent it from a place that was not allowed. And obviously the private key, public key are not going to align, and it's going to say, hey. It doesn't work.

Yeah. Which is it's interesting that the more we learn about a lot of these, like, security techniques, everything ends up kind of always being just, like, public and private keys. You know? Yes.

It's the Node tech that we've been using forever and ever.

As a self certificates, passkeys. We had the 1 password folks on to talk about, pass keys. And Node this and it's all over the place. The one thing about this is it's it's not encrypting your email.

It's encrypting. It's adding an encrypted header to the email based on some pieces of the pieces of the email, some of the content, some of the sender details.

And if again, if any of that data changes, then you're sort of out of luck. These public keys get added to your domain domain name via 1 of 2 ways a text record or adding a CNAME record. And again, you don't have to really worry about it. Your whatever service you're using will tell you, Hey, copy paste this thing, add it to your domain Node, and you're good to go.

So D Kim and SPF, 2 ways to verify that an email was sent by someone who is allowing that server to send email on their behalf.

Node, and, and, and you need both of them to be clear.

Well, no, you don't.

You should have both of them. There are 2 ways to verify that an email is legit.

And now to take a step further, someone can still send email from a domain name, and it's up to the ESLint to to catch that spam. Right now, this is where Demark alignment comes in.

And when your email passes and or DKIM, that is referred to as being demark aligned.

And the demark policy is a third thing that you add to your domain Node. And it's telling the email clients or the email receivers what to do when they receive email that is sent on your behalf.

So you are Outlook.

You receive an email from Bos and Outlook will say, Okay, what do you want me to do with this? This this email from Wes.

Wes, do you? I'll check if it's SPF compliant.

I'll check if it's, DKIM compliant.

And if it doesn't meet either of those, ideally that meets both of them. But if it meets 1 of them, then it will say, all right, it's valid. We'll throw it in the inbox. But if it's not, you can tell Outlook what to do with that email because you say, if anybody is sending email on my behalf and it doesn't meet these strict records that I've set out, then I need you to do 1 of 3 things. 1st, none do nothing to quarantine it. So maybe put it in the spam folder.

And then 3, reject all out. Very similar to the SPF ones. And that's really important because I'll give an example of when we started the ConvertKit newsletter at Syntax.

Right.

So we fired up the syntax domain name and we fired up a new convert kit and we started sending email as syntax Scott of them. And the next morning we had an email from I. T. And said, Hey, you send an email from Syntax Data Firm.

And they immediately were notified. That said because Century has a very good I. T. Security.

They said somebody is sending email on behalf of syntax Scott of them, and it is not in the list of allowed ways to send email. At the time, we only had allowed Google to send email from Syntax Scott of them, and now we are sending email from ConvertKit.

So we said, oh, yeah, that's us. So he said, okay, no problem. Let's Wes added ConvertKit to the list of allowable senders. And then that stuff was was Kim and DMARC aligned.

That's all very interesting.

Yeah.

Very interesting. And I thought, like, as soon as that happened, like like, immediately, we got a message from them. And because, like, companies that send email are extremely protective over bad parties sending email on their behalf. So you have to be very strict and pretty much just whitelist in who is allowed. Right.

And the way that they knew that is via reporting.

So another part of the demark entry in your domain name is you specify an email address that email clients can send back to you when something goes wrong. So it's called an R. U. A. And I don't know what Sanity for. Probably return something.

And the Outlook, Yahoo, Gmail, anyone that accepts email will send a report back to that email address telling you what had happened, telling you Wes it compliant? Was it DKIM compliant? And, and was it DMARC aligned? And then those emails can you can you can send them right to your inbox. But there are services out there that will compile them all and tell you how you're doing, because when you want to like, for example, for me, when I wanted to move to demark strict or demark reject, you have to say, okay, well, like who is who am I sending emails on my behalf? Right. You might have a Shopify.

You might have a Snipkart.

You might have, but I probably have 5 or 6 different things that were sending email on my behalf. And I was like, oh, man, like, I don't even know if I know all the different services that are going on there. So the way that it works is you set it up, you set up demark reporting and you set the the P, which is what happens when something is not demark compliant. You set it to none. So basically you say, give me all the reports and then you let email send for a couple of days and you can look at your report and say, all right, this JS a list of everyone that's sending email on my behalf. And you can look at it and say, yeah, I recognize that. I recognize that. I recognize that. And then also on that list, there will be a list of, in my case, there's 6 or 7 different like you're going to see lots of spammers because spammers will use every single domain name on there. You're going to see people that are trying to Sanity email as you. And one funny thing that popped up was my courses.

So in my courses, we send email.

And in my courses, we use Mailtrap.

And that's just sending it doesn't send real email. It just sends it to a service where you can spoof the inbox.

And that's really nice. But some people were taking my course and then hooking it up to a real email SMTP service.

But they were not taking the Wes Bos Scott domain name out because we're coding along. I was like, Well, the from address is Wes at West Boston.com.

And people were like, Okay, Wes, the from address is Wes Bos.

So people were there's a couple of people that had apps that were just sending email as me.

You would not believe how many when I'm doing courses, how many times I have to be like, listen, do not use this database path. I I'm not blurring out the keys or the database URL. Yeah. So that way you can see exactly what it looks like, but I will be deleting this the moment that this course is over. And then Yeah. People will still try to use that. And they're like, why isn't this working?

Oh, it it was hilarious. So, luckily, it was only a couple, but I recognize that. And then in there, you can also get a couple of spammers that are trying to send email as you as well. So you Sanity it all. You fix any issues. And the whole idea of monitoring it is if you have something that you forgot about, like Stripe is sending email on your behalf for receipts or something weird like that, you can go, oh, I forgot about that one. Let me adjust my SPF and DKIM records.

And then after monitoring for a couple of days, you can set it up to quarantine, which is Scott of one level stricter Sanity for a couple of days and see like, I don't know. Do you get it? You get, hey, your email went to spam or, hey, I never got an email from you. Check your spam. Oh, there it is. You know, and luckily now that happened to me. I've been running quarantine for about a week now. And then finally you go full go, which is reject. Right. And then you say, Okay, now I have put in place these are the 3 areas that can send email my behalf. Anything else? Anyone else that's using my email as a course thing or whatever, immediately it will reject and you have Node, strong, email sending full demark compliance.

Wow.

So Crazy.

Yeah. And so I I guess the 1 question I have is I know that you said that, you know, email providers are very, very concerned about who's sending email on whose behalf and spam and things like that. Yep. Is that just because if too many emails get marked as spam from an address, does that does that hurt the IP address that it's coming from, or does that hurt the the sender?

Yeah. So, like like, for example, ConvertKit, Drip, Postmark, they will be constantly monitoring spam complaints against your emails.

And if you go above a certain percentage, they're gonna say, like, I I sign into my postmark probably at least once a week and, just take a look at it. And this morning it was 0, which is good. And every now and then you get people that are like, oh, well, who the hell is this guy? You know, they market as spam and sometimes those are false positives.

But yeah, you get too many. Then that IP address starts to get blacklisted and other people that are on that service, their emails will start to affect the quality as well. So they are very, very aggressive with that type of stuff.

That all tracks. Cool. Yeah.

Other things, my Google Workspace, I use Bos TypeScript, which is like my company, right? I just signed up as the the main domain as Bos. I've Scott. But I don't use that to send email. I use Wes to send email and my wife uses Kate Boss.com to send email. So we have aliases set up in Google Workspace. And I you cannot get DKIM alignment set up with aliases in Google Workspace, only SPF, which at first I was like, oh, crap. But I had talked to a couple of people online and they said that's that's standard stuff. So, again, you only have to be DKIM or SPF. Ideally, both. But in that case, I could only do SPF.

So an aliases is not is that, like, a straight up Google feature? I know that I've used aliases on Google before, but is is there any sort of spec there for doing aliases outside of that? Yeah. Yeah. Like, the the whole email,

spec is like if you look at the headers of a specific email and let me actually pull 1 up real quick. Node sec.

So when you send an email, there are headers and it's just like a like a Web request, like information about. And there's 2 things. There's a from address.

Who do you want it to appear as? But then you also have like a return address, and that's who's actually sending the email. So if you take a look at any of my emails that are sent from West Bos, you'll see the front address is West Bos.

But the reply to address is Bos.

And that's a feature that you need because sometimes you need to send it from you. You're technically sending it from 1 domain name, but you want it to visually appear as a different domain name.

Same thing with with Century. Right. Like we can send email as syntax out of them, but it's technically being sent from Century IO. Right. Those are aliases between the 2. So that's why you need this pnpm DKIM alignment to be able to say, hey, I know I'm sending it from this server, but in Sanity, I warp it to show up as this different, actual URL.

Very interesting.

Yeah. That's that's good stuff. So this isn't a 100% of staying out of the spam folder. There's whole other stuff pruning your list, frequency of sending, contents that are in there. There's all kinds of stuff that you can sort of play ESLint, But because Gmail and Yahoo have gone so aggressive to say Wes will not even accept an email if you do not have your demark compliance in order is you should certainly check it in the one place this is going to be a problem for a lot of people that they're probably not thinking about is WordPress plug ins. You do a password reset on WordPress.

It's just using the PHP mail function. Right. It's not using unless you're setting up some plug in to use WordPress.

Some external transactional email JS just sending an email via the Pnpm mail function. It's sending it straight from your server. And if you do not have your server's address of your PHP server in your, SPF and and DKIM setup, then those emails are not gonna get to you, and you're not gonna be able to reset your password when you need to. So that's that's probably Node place that people aren't necessarily thinking about. So I'd certainly check for that. Awesome. Wow.

Thank you. Thank you for all this to us. Yeah. That's that's all we got. Hopefully, you enjoy that, and, we'll catch you later.

Well, thank you. Peace.