Freemail Detection

Detection of Freemail accounts is now available for our Online API users. Email validation will continue as before with the same result codes, but a new result field named “freemail” has been added. This field will take the value “true” if an address is from a Freemail provider, or false if not.
Existing implementations using our Online API will continue to work without changes, but to benefit from the new feature, the code has to be updated to parse the “freemail” field”.

Here is the example output for a valid hotmail.com Email address:

{
"status":200,
"info":"OK - Valid Address",
"details":"The mail address is valid.",
"freemail":true
}

See also:

Freemail Revisited

We have struggled a long time now with Freemail accounts, and only recently decided to disallow their use for free trial runs of our validation services. Like Disposable Addresses, they are far to easy to obtain, and frequently used for abuse and scam schemes.

The capability to detect Freemail accounts is now available to our API customers in an open beta-test. Customers can participate by adding “Version=beta” to the argument list of the API requests. A valid Freemail address will then return

211 OK - Freemail Address

instead of the usual 200 OK - Valid Address validation result for good addresses. All other results remain unchanged, in particular bad Freemail addresses will return 410 Bad Address like any other bad address.

Don’t be surprised that the big three, Google (gmail.com), Microsoft (hotmail), and Yahoo will be reported as Freemail, along with many other well-known names. They are Freemail services, and they did not get this big by putting up roadblocks to prevent users from registering.

With Freemail and Disposable Address detection, you can protect your service against abusers registering as many new Email addresses as they want. Freemail detection also helps with the Google + and . Email address obfuscation trick. This type of address is almost disposable (the original account name is still plainly visible), but they can still create as many different Email addresses as they want.

Let us know what you think of this new feature, and drop a comment or mail us at support@byteplant.com.

Sender Score Reputation and Small Business

Getting newsletters to your customers gets increasingly difficult. Email providers rate mail servers using opaque and sometimes bizarre measures, and the odds are increasingly stacked against you, especially if you are a small business.

Some providers are relying on ReturnPath’s Sender Score to evaluate mail servers, and depending on the score they will impose rate limits, or even bounce messages, even when the address is good. Fortunately there is some insight into the way Sender Score works, and it may serve as an example how ratings are usually created. A good Sender Score can be achieved if you meet the following requirements:

  • correct server setup (reverse DNS etc.)
  • no spam complaints
  • no spam traps
  • consistent email volume
  • less than 2% unknown users

It is usually a simple thing to meet the first three requirements, but not so easy for the other two:

  • A typical small business is unlikely to generate a consistent mail volume, and your Sender Score will be all over the place, especially if you send a newsletter occasionally, like once per month or even less frequent.
  • If you do not contact your customers frequently (once every two weeks), it is likely that a sizable percentage (certainly more than 2%) will be invalid, just for the reason that the majority of all email addresses ever created are either disposable or temporary only, used for the sole purpose of bypassing web service registration requirements.

What can you do?

It helps to validate email addresses before trying to send a newsletter, but the volume problem still remains.

If you have deep pockets, you can talk to ReturnPath about their certification program. Chances are they won’t even bother to talk to you, but if they do, you may end up buying a “Get Into The Inbox Free” card for real money. This might help or not, there are still far too many providers ignoring ReturnPath certification. Either way, it is time to quote Tony Soprano here:

I won’t pay. I know too much about extortion.

There is another obvious solution: Send mail constantly to your customers, like every two weeks, with just enough content to make sure no one complains. This way you can make sure they still exist, and it keeps your email volume constant. My guess is that Spammers have also already thought about that.

Talk about unintended consequences?

Statistics

Here are some of last month’s statistics of all email addresses checked on our servers, and how they break down to the different large providers:

Google
33%
Microsoft
22%
Yahoo
17%
Europe
11%
US other
5%
World
12%

The Google number includes all email addresses under the gmail.com domain, and the accounts hosted by Google, but running under a different domain name (about 1%).

The most common Microsoft domains still are all variations of hotmail, followed by live, msn, outlook, and many other hosted domains.

Yahoo email addresses (yahoo, ymail.com, and the venerable rocketmail.com) make overall third place. The number also includes other domain names with email hosting by Yahoo.

The European number includes the large European email portals, dominated by services in France, Germany and the United Kingdom.

The “US other” number represents other large US cable and communications services other than the big three, but with big names: AT&T, AOL, Comcast, UOL, Verizon.

Finally “World” represents all the other email services and domains that did not come to attention by volume. This includes names such as Telstra, qq.com, or Yandex.

Presented without comment.

International Email Delivery

First, a definition of the term “International Email Delivery” for all non-US residents: International email delivery is about getting an email delivered to an US email address, while being located outside of the US (within the US=national, outside of the US=international, get it?).

Now that you know what I am talking about, you have probably already guessed what the problem is: international email delivery is shaky. Lots of US-based people and businesses implement some kind of IP-geolocation based mail filtering. If some IP database says your IP address is somewhere outside of the US, your mail is not delivered.

Many US residents are often unaware of this, as this policy is implemented by their Email service providers without their knowledge. While it makes sense to assume that a plumber based in Rolling Meadows, IL, is unlikely to receive Email messages from Poland, others might be using an Internet service hosted in Poland, and this filtering policy renders their email address useless for this service.

Some mail servers are straight-forward about location-based filtering, and state the reason for non-delivery openly:

550 country IP access denied

Other servers are more enigmatic, leaving room for interpretation:

550 blocked

How to interpret this answer? Maybe the server does not like the country of origin, or maybe the requested email address simpy does not exist? At least, it is an answer, other servers might not even bother to accept a connection if they do not like the originating IP address.

In all these cases, the email address might actually work, if the right conditions are met, but because our Email validation service is intended for customers worldwide, we try to be on the safe side and flag addresses with potential international delivery problems with the “317 Server Reject” reply.

In the past weeks we have spent a lot of effort in refining our detection algorithms and databases, so we can now detect mail servers with international delivery problems more safely and faster than before.

Why IPv6 is Completely Useless

IPv6 has been around for more than fifteen years now. IP4 address space has run out. So why is IPv6 adoption still so pitiful?

When you look at the latest Google IPv6 statistics, at the time of this writing Google’s web servers see about 6% of their traffic in native IPv6. That’s not very much, is it? Ok, growth is exponential, so with the percentage doubling every year, we will see 6000% of IPv6 traffic in 10 years — but hey, wait a minute, there must be something wrong with this prediction.

When you look at the top 50 websites, you will find that some rely on Akamai or similar services (thus supporting IPv6). There are also a few that run a webserver supporting IPv6, but the rest just do not bother.

That’s only webservers. When it comes to other services, like messaging or Email – at the time of this writing, there are still only two of the major Email providers supporting IPv6. They are, and have been for years now, in alphabetical order (drumroll):

  • Comcast
  • Google

If Google and Comcast can do it, why can’t all the others?

Wrong question. They could, if they wanted to. There are simply no compelling reasons to support IPv6 for a mail server, on the contrary, it is more like there are good reasons to avoid IPv6:

  • IPv6 is not compatible to IPv4, so every mail server listening for IPv6 connections must continue to listen for IPv4 connections as well. Why bother with IPv6 then?
  • IPv4 blacklists are completely useless to filter Spam for an IPv6 connection. Enabling IPv6 for your mail server invites Spammers.
  • IPv6 is useless to send outgoing mail. Practically no other server (with the two notable exceptions listed above) will be talking to you.

All IPv6 transition scenarios and projections forget the important thing: There is still no such thing as a server only connected to the Internet by IPv6 (if there was, no one would want to have any business with it). In the end you can still reach everybody and everything by IPv4. If there is zero benefit in adopting IPv6, then why use it in the first place?

SMTP/Email will be solidly IPv4, for many years to come. Probably even SMTP will be replaced with something else, before the world even begins to transition to IPv6 in earnest.

420 Domain Name Misspelled

Sometimes we get to see email addresses which are obviously misspelled. Here are a few examples:

someone@yahooo.com
someone@gnail.com
someone@hotmial.com

Two things are worth to mention:

  • Usually, the correct email address is obvious, and correcting the email address is often very easy. However, we do never automatically correct spelling errors, as US and European Anti-Spam legislations for good reasons do not allow you to send emails to any “fixed” or “corrected” email addresses without the explicit consent of the owner of the email address.
  • There are quite a lot of mail servers actually accepting mails sent to some of these misspelled domains. This poses a severe security risk, as confidential or sensitive information might be misdirected to a possibly dishonest recipient.
    There are even tools that help miscreants find likely domains and misspellings, like the domain typo generator you can find here.

With countless domains and possible misspellings, any algorithm to identify them will always be incomplete, so some will always slip through. The Email Validator result will be 207 Catch-All in this case, because a malicious mail grabber will accept any mail address for the misspelled domain.

Another reason to be wary when using mail addresses from a domain with Catch-All.

317 Server Reject

Sometimes email addresses just can’t be validated:

  • a mail server accepts every email address (resulting in 207 Catch-All)
  • a mail server rejects every email address (resulting in 317 Server Reject)

You may ask yourself: What is the point of running a mail server, when all email is rejected? There is no point, of course, but typically these servers only look as if they do not accept any mail. They employ IP geolocation filters, where email simply does not work unless your server is in the same country as the recipient server.

This is the main reason for a “317 Server Reject” message to appear, but there are others as well:

  • The mail server is known to employ Greylisting with a prohibitively long delay time (up to twelve hours).
  • The mail server uses outdated or discontinued blacklists.
  • It is not possible to discern with reasonable confidence whether a mail server employs any of the above methods.
  • The mail server just can’t be bothered.

In summary, a 317 Server Reject email address is not safe to use internationally, but may still work when used in the same country as the recipient. Luckily, “317 Server Reject” replies occur only very rarely.

Top 3 Myths About Email Address Validation

In a recent post on the Spamhaus news page, a member of the Spamhaus team shares his thoughts on email address verification services.

In the Spamhaus news article, there are a number of common misconceptions and we would like to take this opportunity to share our view on these myths:

  • “Listwashing and spamtrap washing services help spammers”
    No, spammers do not use email address verification – why should they? It costs money and they do not have to care about invalid email addresses at all. Period.
  • “They do not help list owners who have done proper opt-in acquisition and list maintenance”
    Everybody in the industry knows that email addresses are volatile and even addresses acquired through confirmed opt-in and carefully maintained will sometimes be invalid within only a few weeks without any notice.
    Verification helps websites to identify mistyped email addresses, and visitors can correct their input before they have left the site.
  • “[…] nor do they help mail receivers (including both Postmasters and mailbox owners) who rely on spamtrap data to keep spam off their servers and out of their mailboxes.”
    Simple list verification does not identify spam traps, a spam trap address is any email address like any other, and it can be either valid or invalid.
    The whole concept of a “spamtrap” seems a silly idea anyway – how would a confirmed opt-in (COI) protect you as a legitimate mail sender from sending an email to a spamtrap address if somebody (e.g. a competitor) entered this address into one of your website forms?

Email verification seems to be an issue DNSBL providers now have to deal with, causing more headaches than the dreaded impact of wide-spread IPv6 adoption by mail servers, if it ever comes, but that’s a topic for another post.

Minority Report – MailChimp Edition

From time to time MailChimp users [MailChimp is an email marketing service provider] are requesting our help because MailChimp threatens to suspend their accounts – allegedly because of ‘abusive behaviour’.

MailChimp is trying to predict email delivery issues and abusive behaviour with some sort of heuristic pre-send evaluation algorithm.
Everybody else knows that prediction is very difficult, especially if it’s about the future, unless you are a Precog, but Precogs do not exist, or do they?

What we do know is that MailChimp does not have Precogs. Their approach does not really work well and leads to lots of false positives – effectively keeping many legitimate emails from being sent through MailChimp.

Most important, MailChimp is likely to refuse your existing email list even after it has been validated and cleaned – they simply continue to flag the entire list as not being safe to send, but without providing specific reasons or a detailed explanation.

If you fell victim to MailChimp’s “Minority Report”-like abusive behaviour prediction algorithm, or if you are experiencing similar issues with your email marketing provider:

  • We strongly recommend that – especially with MailChimp – you remove all Catch-All addresses from your email list and use only email addresses with a “200 OK – Valid Address” validation result for your email campaigns.
  • If all else fails, you can still try this (from the MailChimp knowledgebase):
    “Keep in mind, you may still send to the list without making changes. If you choose to do this, MailChimp will slowly send to the list over a period of hours to monitor the bounce and complaint rates. If the rates remain low, we’ll complete sending to the remainder of the recipients. All existing addresses previously sent to through the account, and any addresses collected through our double opt-in process will be sent first.”