How to Test Email: A Personal Journey
Denys Kontorskyy
I’ve been with Mailtrap since 2021, and I have tested a gazillion emails (using Mailtrap) and talked to god-knows how many SMEs to wrap my mind around all the intricacies.
This piece is an honest account of my ‘how to test email’ journey. It covers every aspect you can test, their importance, and the examples of best practices.
Sure, I used Mailtrap almost exclusively, so the following sections feature a bunch of references and pointers to our platform. But, you’ll also encounter other tools and tips on how to incorporate them into your workflow.
What aspects of an email can you test?
You can test ALL aspects of your email and here’s a quick list:
Email deliverability
SPAM score
Headers
Technical tests (DNS authentication, inbox placement, etc)
Email components and interaction elements
Also when testing, a structured approach is crucial, or you’ll get lost in all the data. Worse yet, you’ll waste a lot of time testing the wrong things.
Deliverability testing
Deliverability testing is the first thing to do because it’s about making sure your emails hit the inbox, not the spam folder. And my dear colleague Piotr wrote a great guide on “Email deliverability: Everything you need to know”.
HTML check
The HTML check is crucial in email content testing. Since this is not only about different content elements working but also about rendering. If the rendering is bad, the customer will think you are a spammer – spam complaint = sending reputation drop.
Note that I’ll be covering sandbox tests with Mailtrap to check emails before sending them. And that’s without spamming real recipients.
Later, I’ll talk about production email tests (with Mailtrap and otherwise). You can safely run these with a preselected cohort of your subscribers. For example, production email tests are A/B tests and specific aspects of trigger point tests like email heat maps.
Finally, it’s time to move to WHATS and HOWS.
The HTML check takes a deep dive into various elements (HTML and CSS) of your email template and assesses overall market support across different email clients, and different devices; covering web, mobile, and desktop.
There’s a neat “Only” button, which pops up when you hover left of each provider. Hitting it allows you to see the support for only the client you select. And if you’re dreading deliverability to Apple Mail, for example, it’s a cool trick to see how to fix your template so that Apple loves it.
On that note, I’ll move on to show the other email template elements you can test under HTML Check.
Email body
The <body> element section shows you the support for the body of your email template. Note that this isn’t body copy of an email. It’s the HTML body of your email and how deliverable it is across different clients.
In this example, you can see that there’s partial support for the <body> element and the problematic area is on line 6. When I click 6, next to “Found on lines:”, I’m taken to the HTML Source tab, and I can immediately see the problematic line, so it’s straightforward to fix it.
Yeah, it seems the <style> and the <body> elements are in some conflict. So, I’m moving on to check that.
Styling and fonts
The <style> element section checks the support for template’s styling. I’m still using the same example, and as you can see below, some clients partially support the element, it appears buggy, and it’s not supported across all the email client versions.
Again, clicking on the number next to “Found on lines” takes me to the template and clearly shows what’s wrong. The best thing is that if a line gets declared as “Buggy”, there’s a quick explanation why it’s buggy and how to fix it, which may cut template debugging in half.
As for the fonts, one of the key parameters Mailtrap checks is the font-weight.Of course, this is assuming you’re using a standard font family.
Like before, the notes show you that the font-weight is partially supported, plus there’s a quick explanation of why it’s like that. But the check doesn’t stop there, you can also inspect the class selector, max-width, background-color, width property, max-width, and any other CSS or HTML element that might appear in the email.
I know, it may seem that there’s something seriously wrong with the template in the example, but it’s not. As you can see from the first screenshot, it has 93.8% support across different email clients. So it’s a good template, but there are options to make it even better.
When you’re done checking the template, it’s time to see whether it has a chance to trigger spam bots.
SPAM score
The spam testing results are under the Spam Analysis and they give you the overall score for an email. In the background, we’re running an up-to-date SpamAssassin filter to determine the score. And you can see which elements of your template drive the score up.
At the same time, under Blacklists Report, you’ll see if your domain and IP address appear in any of the most common blacklists.
The template I’m using scored 3.3 which is okay. Generally, the recommendation is to keep the spam score under 5. But if you wanna get the score even higher, feel free to fix the rules as described in the table.
If you need more info on how to steer clear of the spam folder, check our blog post on “How to Avoid Emails Going to Spam?”.
That out of the way, you can proceed to check the headers.
Email headers
The Tech Info tab is the header information checker. You should know that it displays the original header values, which might be altered by an MTA or email service provider when sending the emails. However, the data we display are more than enough for debugging purposes.
Also, pointers to header information or buggy headers may also appear under the HTML Check tab, and the Spam Analysis tabs. For example, the Spam Analysis flags ‘Content-Type’ without the proper MIME header (check the screenshot below).
Automation in Email Testing (via Mailtrap API)
Yes, you can automate email testing and if you’re sending a lot of emails, you should consider doing it. Mailtrap allows you to automate via our RESTful API (available on the free pricing plan).
Sure, it requires some coding but you can let the API run the dull tasks and notify you if there’s a problem.
Since I mentioned Mailtrap HTML Check a few times, here’s an exemplary request script for the HTML analysis using PHP/cURL.
<?php
$curl = curl_init();
curl_setopt_array($curl, [
CURLOPT_URL => "https://mailtrap.io/api/accounts/account_id/inboxes/inbox_id/messages/message_id/analyze",
CURLOPT_RETURNTRANSFER => true,
CURLOPT_ENCODING => "",
CURLOPT_MAXREDIRS => 10,
CURLOPT_TIMEOUT => 30,
CURLOPT_HTTP_VERSION => CURL_HTTP_VERSION_1_1,
CURLOPT_CUSTOMREQUEST => "GET",
CURLOPT_HTTPHEADER => [
"Accept: application/json",
"Api-Token: 123"
],
]);
$response = curl_exec($curl);
$err = curl_error($curl);
curl_close($curl);
if ($err) {
echo "cURL Error #:" . $err;
} else {
echo $response;
}
And here’s an exemplary response to the request.
{
"report": {
"status": "success",
"errors": [
{
"error_line": 15,
"rule_name": "style",
"email_clients": {
"desktop": [
"Notes 6 / 7"
],
"mobile": [
"Gmail"
]
}
},
{
"error_line": 7,
"rule_name": "display",
"email_clients": {
"desktop": [
"AOL Desktop",
"IBM Notes 9",
"Outlook 2000-03",
"Outlook 2007-16",
"Outlook Express",
"Postbox",
"Windows 10 Mail",
"Windows Live Mail"
],
"mobile": [
"Android 4.2.2 Mail",
"Android 4.4.4 Mail",
"BlackBerry",
"Gmail Android app IMAP",
"Google Inbox Android app",
"Windows Phone 8 Mail",
"Yahoo! Mail Android app",
"Yahoo! Mail iOS app"
],
"web": [
"Yahoo! Mail"
]
}
}
]
}
}
The response you’re looking for is 200. If there are any issues, the system will return 401, 404, or 404 error messages.
Thank you for exploring this snapshot of Mailtrap's guide on effective email testing. To dive into the complete article, offering a wealth of strategies, tools, and best practices, continue your journey by using this link.
Happy testing!
Subscribe to my newsletter
Read articles from Denys Kontorskyy directly inside your inbox. Subscribe to the newsletter, and don't miss out.
Written by