Search

Understanding Email Spam Fighting Technologies




Email SPAM has become a significant issue for consumers and businesses. At times SPAM can be funny, annoying, and, unfortunately, very dangerous. Over half of the email traffic is SPAM. For April 2020 alone, an analysis from Cisco Talos Intelligence put the average daily spam volume at 320 billion. Google's email service blocks more than 100 million phishing emails every day, and during the pandemic, they said out of those, 18 million were related to COVID-19 in some way. 94% of malware is delivered over email, and one in every 3,000 email messages contain malware payloads. 76% of companies have fallen prey to phishing scams. Every year SPAM costs businesses over $20.5 billion. In a few years, analysts predict it will reach $257 billion annually. Email SPAM is a big problem.


Companies are investing a significant amount of money in fighting email SPAM. The global enterprise SPAM market is growing at such a rapid pace. In 2018 the market was valued at $849 million and will increase in the next several years to $2,675 million at a CAGR of around 17.5%. Significant money is being invested into fighting SPAM, making it very hard for companies to send cold outreach emails. Google's machine learning algorithm that powers the SPAM filter for 1.5 billion Google email users is believed to be 99.5% accurate. As more advanced SPAM technologies are being developed, businesses need to understand these technologies and learn how to play within the rules to ensure their emails get delivered.

SPAM Technologies Protect The Inbox

To solve the massive SPAM problem, mailbox providers ended up inventing SPAM filters, Firewalls, and Blacklist directories. SPAM technologies is software that uses artificial intelligence to decide what emails should exist in your inbox and which ones should go to your spam folder or blocked altogether. Inbox providers are highly incentivized to provide better SPAM technologies. Since 70% of email is SPAM, a good SPAM filtering technology will reduce the number of emails being processed by your inbox and increase the businesses' profit margin. Also, SPAM technologies protect their customers from malware and phishing attacks, which helps drive customer loyalty. In a highly competitive market, like the inbox market, it is essential to maintain a reputable brand to keep your customers from opening another email box with your competitors.



How Spam Technologies Work


To understand SPAM technologies, we must first understand the building block technologies ensuring your emails get delivered. Here are some of the tools you need to know:


Email Service Provider (ESP)- ESP or email marketing software (like Sales Innovator) helps you send emails at scale.


Internet Service Providers (ISP) - ISP stands for Internet Service Provider. When we're talking email marketing, ISP refers to the major email providers: AOL, Hotmail, Outlook, Yahoo, Gmail, Comcast, and so on. Their customers are commonly your email recipients. It's the job of the ISP to protect their customers from receiving unwanted or unsolicited emails.


Spam Filters - Decides whether the email should be present in the inbox, promotions tab, spam folder, or whether it should be blocked altogether.


Internet Protocol Address (IP Address) - This is a unique public address of the server over which your emails go. Your ESP typically maintains your IP Address.


Blacklists - Databases that store IP/Domain spammers. There are a lot of them, and their criteria are specific. Just keep in mind that they mainly refer to your email sending activity: sending frequency, sending volume, bounce rate, spam reports.


Custom Blocking - It is common for ISP's and corporate networks to create their SPAM blocking criteria. ISP uses information from blacklists and content filters in a "weighted" system that gives "spam points" for each offensive piece of the message.


Domain Name System (DNS) - The domain name system connects URLs with their IP address. With DNS, it's possible to type words instead of a string of numbers into a browser, allowing people to search for websites and send emails using familiar names. An email address is matched to a domain name, and the domain is matched to an IP address. The DNS matches the address on the email message to its destination and delivers the email. The DNS can access the public key to decode your DKIM and publish the SPF record to authenticate your email.


Now that we are clear with our definitions, you can better understand how these technologies work together. Business senders use email service providers to send emails via ISP's. ISP's use SPF,DKIM, DMARC, and MX Records to verify the sender and recipient's identity. Once a connection is made SPAM technologies help determine if emails should be blocked, added to the SPAM folder, put in the inbox, or filtered to another destination. SPAM technologies use customized technologies such as SPAM

filters and blacklists to decide where the emails should go.



For the rest of this article, we will further describe how SPAM technologies work and how you can learn how to play safe and maintain a high email deliverability level.



Types of SPAM Filters

There are broadly three types of SPAM filters that you will encounter:

Gateway SPAM Filters: A gateway spam filter is an on-premise software solution for filtering spam. This software is installed as a virtual appliance behind the network firewall. Gateway SPAM filters are the first line of defense against spam. Companies providing this solution include Cisco's IronPort and Barracuda.


Hosted SPAM Filters - These types of SPAM filters are cloud-based solutions. Instead of supporting one company, these solutions support various companies and access a lot more data, giving them a more sophisticated SPAM filter. Hosted SPAM filters deliver messages to an inbox, move messages to spam, quarantine messages, and block messages. Two of the leading solutions in this space include Cloudmark & MessageLabs.


Desktop SPAM Filters - These are third-party SPAM filters that exist in a user's system. Desktop spam filters are highly customizable and are more personalized to a specific user. Outlook uses Microsoft's SmartScreen anti-spam filter to filter email.



Major Factors Influencing Spam Filters



Whatever type of SPAM filter your inbox provider users three major factors will impact their decision to filter your message:


  1. Email Reputation - Email, sender, or domain reputation is the measurement of your email's sending practices and how closely you abide by the standards set by your Internet Service provider (ISP). ISP's use different criteria to filter spam and unwanted email. Your email reputation is primarily measured for a sending IP address. Your email reputation is comparable to your credit score. It takes time to build up a good credit score, but it is easy to destroy it if you are not careful.

  2. Email Authentication - Email authentication is a technical solution that certifies your email as a valid email. It affirms that your email comes from who registered your email. Email authentication is used to block harmful or fraudulent uses of email, such as phishing and spam. The most common types of email certifications include SPF, DKIM, and DMARC.

  3. Email Content - Email content is the analysis of your email content and how it follows the guidelines set by your ISP. Email filers tool for emails containing "spammy" language, too many uses of images, broken HTML, attachments, proper use of hyperlinks, and many other variables that signal SPAM. Most importantly, if email content is low-quality, your users will not engage, and your reputation will be affected.



How GMAIL's SPAM Filter Works

Google's SPAM filter is most likely one of the most advanced SPAM filters. Each day over 1.5 billion people use Gmail, and over 5 million businesses pay Google to use Gmail as part of G Suite. Gmail is the second-largest email client (38%), next to Apple Mail (40%). G Suite is the leading office suite provider with a 58% market share vs. Office 365's 40.93% market share. With a commanding market share, Google's SPAM filter claims to have a 99% accuracy.


Reputation


Gmail looks at both the domain and IP address of senders when determining where to place an email. Setting up proper authentication for your outgoing email is very important for your email to get accepted and not filtered or blocked by Gmail. Ensure you properly authenticate yourself with your ISP and setup your SPF, DKIM, and DMARC.


User Engagement


User engagement is vital to Gmail's algorithm for determining what should be filtered or blocked. Gmail looks at each person's action in their inbox when deciding what should be filtered. What could be SPAM for one person could be a valuable email for another person, so personalizing filtering is important to Gmail. Here are some examples of the user actions Gmail algorithms take into account when filtering:


  • Messages deleted before being read.

  • Messages marked as SPAM

  • Emails voted as not SPAM

  • Messages moved to the promotions folder

  • Starred messages

  • Messages read

  • Messages replied to

  • SPAM reports or complaints


Because Google highly values engagement, companies need to clean up their subscriber lists and remove subscribers with minimal engagement. Emailing those users will bring down your overall reputation and will send more of your emails to SPAM. Keep the same email cadence with your active users and choose an alternative strategy for your passive subscribers. By segmenting your emails by engagement, you will maintain a good reputation with Google for your engaged subscribers while using an alternative email address and domain for your riskier subscribers.


Content


Content is another factor Gmail analyzes when deciding what content should be blocked or sent to SPAM. Gmail will analyze your email's header, body, images, and links to determine where they should send your email. Avoid "spammy" words is important, but it is not as crucial as a sender's reputation. Gmail highly ranks a sender's reputation as a critical factor in ensuring your emails reach the inbox.


Sending History


If you are using a dedicated IP and have a new IP address, you should warm up your email. Instead of sending a lot of emails right away, you should slowly scale your sending. Gmail blocks new IP addresses for the first 2-24 hours, and after that, it starts delivering a few emails into the inbox and sends some to spam to test recipients' reactions.


If their initial test results in high complaints, then your emails will most likely all land in SPAM. On the other hand, if your emails receive a high engagement rate, you will most likely end up in the inbox. Your first set of emails is critical to determining your future relationship with Gmail.


If you were not successful passing Gmail's test and need to be rescued, your recipients will need to go to their spam folder and mark your emails as "This is not spam". After you receive several of these, your email will be marked safe for the inbox.


Monitoring Your GMAIL Reputation


Gmail provides a tool, "Postmaster Tool" to give you visibility into your current domain reputation, IP reputation, spam complaint rate, and other deliverability statistics. Use this tool to monitor your reputation with Gmail continuously to ensure deliverability.



https://www.gmail.com/postmaster/

How Outlooks SPAM Filter Works

Microsoft is the oldest webmail provider, having launched Hotmail.com in 1996 and then rebranding Hotmail.com to Outlook.com. Currently, Outlook has over 400,000 active users. Microsoft also supports businesses with its Microsoft Exchange and Office 365 email and office products, which control 87.5% of the email and authoring market.


Microsoft SPAM filters look at engagement, complaints, spam traps, and other factors similar to other service providers. Although, unlike their competition, they use the Microsoft Sender Reputation Data Network to decide what emails are spam.


Microsoft Sender Reputation Data Network (SRD)


The SRP or SPAM fighters program uses a panel of voters selected randomly from active outlook users to help train their filters. These voters are asked to vote whether the original email was "Junk" or "Not Junk". The votes decide your fate on having your emails sent to SPAM or not. The spam fighter program has been more reliable at finding spam instead of just monitoring complaint rates. Complain rates are a lot easier to manipulate since senders can increase their email volume to reduce the total percentage of complaints.


Monitoring Your Outlook Reputation


Microsoft provides several tools to help you manage your reputation.


Microsoft's Smart Network Data Service (SNDS) - For those using a dedicated IP address, senders can use SNDS to see data about subscriber complaints, email volume, spam trap hits, etc.