This post will give a bit more depth on SmartScreen® and our approach to spam filtering, and on a particularly evil kind of spam called a phishing attack. I'll' also explain why individual e-mail accounts experience different levels of spam, and what you can do to reduce spam in your own account.
In my last post, I gave two key measurements:
Let's take another look at those numbers. The first is the amount of spam that our filters catch relative to the total amount of spam that comes into the system. The second is amount of spam in the inbox relative to good messages in the inbox.
Here's the math:
There are different approaches to measuring spam. Our approach is to use real user data to measure how much spam gets through our system. We select a cross-section of customers who reflect the broad population of Hotmail customers in several dimensions‑such as age of account, country or region‑and invite those customers to participate in our Feedback Loop program. The participation rate in the program is high, with more than 50% of participants classifying messages every day.
Another approach, which you see reported by some e-mail services, is simply to measure the rates at which users report spam using the "Junk" button or an equivalent. This approach suffers from a few flaws as a reliable metric, including false positives, biases in the feed, susceptibility to gaming, and dependence on customer enthusiasm to report spam. For many spam attacks, the rate of spam that gets reported is less than 2% of the total messages delivered. That's because most of the messages are never even opened or reported as spam. So, using this method of reporting can result in a deceptively low rate of spam ("under 2%!") even with no filter whatsoever.
We think our approach of measuring true spam by using a feedback loop is better. Our approach yields a statistical view of how Hotmail customers perceive a random selection of e-mail that we send to them for classification. It's the difference between a controlled experiment and observational studies. (But for those of you interested in the rate of "Junk" reports at Hotmail, it's consistently under 0.5%, which is comparable to what you'll see from other services.)
It can be difficult to make an apples to apples comparison of spam-fighting effectiveness across e-mail services. You're generally comparing your experience on different accounts, and there are several factors that affect the spam volume in an individual account, including:
That last one is interesting, and perhaps a bit counterintuitive. We know that Hotmail is a big target for spammers because of the large number of customers. But our spam-fighting technology has become effective precisely because we see all of the spammer attacks that occur on the Internet. We use all of the data from those attacks to get smarter about battling spam. We may be a big target, but we've built up some strong armor!
Finally, it's worth talking a bit about perception and our approach to dealing with spam.
Each major e-mail service has a philosophy and criteria for dealing with spam. Some services put a large volume of messages in your junk folder. This might seem impressive, but it can make finding those false positives particularly hard, because you have to dig through so many spam messages. Hotmail, on the other hand, has a policy of aggressively deleting spam (although we never delete a message unless we're sure it's spam), and we also delete messages that have been in the Junk folder for more than 10 days. A customer who uses Hotmail as their primary account will see very few spam messages either in the Inbox or the Junk folder, and will occasionally find false positives in the Junk folder. Of course, classifying those false positives helps us make the system even better.
We think our approach of aggressively deleting spam and giving you the tools to control spam in your account makes a lot of sense.
One thing that makes fighting spam challenging is that spammers are constantly working to find new ways to exploit Hotmail and other e-mail services. Spam is big business and it only works if the spam messages (even just a tiny percentage of them) get delivered. There's a strong profit motive for spammers to find holes to exploit, so we see the techniques evolve every day.
What worked to prevent spam yesterday might not work today. That's why we take a balanced approach between long-term investments that will yield lasting improvements and short-term efforts to react to the latest attack. So, while we're working to keep the latest "cheap electronics store" message from showing up again, we're also focused on stopping any and all spam, period. It's a long battle.
Here's one of the graphs from my last post. The green triangles show when Hotmail released new spam-fighting technology. Those are typically the more strategic, long-term bets that yield major improvements, and you can see that they usually result in a substantial drop in spam. The blue circles highlight the day-to-day challenges of constantly evolving spam attacks‑and the measures we took to tackle those attacks, as well as sustain the gains from our strategic investments. Those blue circles also highlight another reality of spam: there can be significant day-to-day variance in the amount that gets through the system. Some days you'll experience more, some days less.
One of the most egregious forms of spam is phishing–a type of spam attack that attempts to acquire some sensitive information (like your password or credit card number) through fraudulent, misleading e-mail.
You've probably seen some of these phishing scams. Common scams include:
Phishing scams, like all other spam, can be very effective even with extremely low success rates, because the spammer simply needs to crank up the volume of e-mail in order to profit.
Our SmartScreen technology fights phishing scams by aggressively deleting or filtering these kinds of messages. SmartScreen uses several techniques, including:
When a message is deemed dangerous or suspicious, Hotmail displays the red safety bar at the top of the message.
SmartScreen is also built right into Internet Explorer, so you get the benefit of being alerted to phishing sites and other suspicious sites whenever you're browsing the web.
Here are some tips to avoid getting scammed:
You may recall from my last post that graymail refers to those messages in your inbox that are unwanted, but that aren't unsolicited. Common types of graymail include newsletters, social networking notifications, and alerts.
The "right" way to handle graymail is not so black and white; different recipients will disagree on whether or not a given message is spam. In fact, it's neither the content of the message nor the sender of the message that best determines whether or not the message is wanted; rather, it's your own relationship to the content or to the sender that determines whether or not you want to see the message in your inbox. What is perhaps most interesting is that your opinion on whether a particular e-mail is spam can actually change over time. (Advertisements for TVs are annoying until you're looking for a new TV.)
The good news is that Hotmail puts you in control. We provide several tools that help you decide what messages you do and don't want in your inbox, including:
These tools give you a lot of control over what shows up in your inbox, but our research has shown that we can do even more. This is one area where we will continue to make big investments. You can expect to hear more once we're ready to release new technology.
That's all for now. Next time around you'll hear from John Scarrow, my counterpart on the Windows Live Safety Platform team.
Until then, I hope you'll keep using Hotmail and keep the comments and feedback coming.
Dick Craddock, Group Program Manager, Windows Live Hotmail
@jordanmills - One of the things we do to help prevent spammers from using Hotmail to send spam is to limit how much mail a new account can send. New accounts that are used by real people (as opposed to spammers) quickly establish a good reputation just by using the account in a normal way, and the limits get out of the way. We routinely tweak the settings in order to make sure the experience for our real customers is good, while still making it tough on spammers. Recently, one of our tweaks resulted in some older (good) accounts hitting these limits when they shouldn't have. That was unintended, and we have since found and corrected the issue, so you shouldn’t still be seeing this problem.
Does this have anything to do with the hundreds of live mail accounts that suddenly got limited to twenty messages a day? How can we get someone to look at our accounts? Posts to the sending limits forum seem to be ignored.
@rfcjat - We sometimes see issues with user preferences that can cause this to happen, e.g., sometimes the sender's address is on the Blocked Senders List (usually accidentally). As Chris said, we'd love to take a look at the emails.
@nutterguy - That may be a content issue with the URLs. If you can provide the URLs, we can take a look to see why they might be getting filtered.
@nutterguy @rfcjat Sorry to hear about this. It would be great to look at emails that have bounced so we can see what's going on. I'll contact you directly.
I have the same problem as rfcjat actually, kind of.
I made a nice signature for myself with a few links in it and I can't send it to any of my hotmail contacts because it never reaches them.
It reaches all of my other non-hotmail friends though.
As soon as I took out the links it sent again just fine.
Links where to some of my different site accounts on reputable sites on the net.
How do you deal with real mail that accidentally gets categorized as junk and doesn't even make it to the Junk email folder... in other words I will never see it again. For example, I sent myself a test message the other day and I never got it. I sent it to a different account and I received it at the other account.