From the day Windows Phone 7 made its debut, we’ve gotten lots of kudos for its on-screen keyboard and intelligent suggestions and corrections. Engadget called it “one of the best and most accurate virtual keyboards we’ve used on any platform.” Gizmodo declared: “The keyboard is boss.” Even the New York Times loved the “smart auto-suggestions.”
In Windows Phone 8, we set out to make our critically-acclaimed keyboard even more intuitive, accurate, and personal. As one of the program managers for the feature, I wanted to tell you about improvements in the new release, and also explain some of the science behind our prediction technology, something we get a lot of questions about.
Introducing Word Flow
On average people type 20 to 30 messages a day on their phones; that’s over 10,000 a year. They also make a lot of mistakes: about one in every three words, we’ve found. Windows Phone 8 introduces Word Flow—an improved and renamed version of the Quick Correct feature from Windows Phone 7.5—to help prevent time-wasting typos and focus on what matters: communicating with the people you care about. (See also: Tips for typing faster and smarter)
Word Flow is designed to reflect how real people talk to each other—and so it knows everything from pop culture to slang. When building Word Flow we reviewed over 2.5 billionEnglish words from sources ranging from the dictionary to the Internet. Because phones lack infinite storage space, we picked some 600,000 of the most common words and phrases that people actually type on their phones to power Word Flow’s corrections and suggestions.
As a result, Windows Phone’s auto-correction skills are 94% accurate, on average, out of the box. But we all have our own communication style, so Word Flow is also designed to be a keen student of your writing habits and what you care about. It’s always learning, so it can help personalize the suggestions and corrections to suit each individual person.
The Word Flow keyboard in Windows Phone 8 is designed to learn your style over time to help improve the accuracy of its suggestions.
How we do it
Building a great keyboard is challenging; we’ve all seen smartphones make frustratingly goofy fixes. If you need proof, just check out the thousands of hilarious submissions on sites like Damn You Autocorrect.
To prevent Windows Phone from being the star of those sites, it needs to know a lot of words so it can figure out what you’re trying to type. So we have a built-in virtual dictionary akin to the dusty old copy of Webster’s on your shelf. But there’s one crucial difference: our dictionary contains not only words but data about how frequently real people, on average, use them on their mobile phones.
Imagine, for example, that someone starts tapping out “h-a-p-p”. What word should the phone suggest next? “Happen” is one possibility from the dictionary—unless the person was actually about to say “happy.” The Office team has been researching commonly-used words for more than 20 years to power similar features in Word and Outlook, and they partnered with us to build our first dictionaries in Windows Phone 7. Office uses a variety of sources—from linguistic research to frequency analysis of documents, books, and web pages—to build their dictionaries. Since “happy” is used more often by people than the word “happen,” we offer “happy” as the first suggestion.
Even a “smarter” dictionary, however, doesn’t prevent strange suggestions and unwanted auto-corrections. This happens for two reasons.
First, it’s very important that the dictionary be relevant to what you would type. A dictionary based only on words found in scientific papers and textbooks wouldn’t offer suggestions relevant to everyday conversation (although science teachers across the world would rejoice). For example, if you typed “h” you might see “hydrogen” as the first suggestion.
Second, the likelihood of a word depends on what comes before it. If you type “H-a-p-p-y-b,” “birthday” is much more likely than other common b-words like “be.” Some words are very easy to guess based on the words before it, for example “New York C” is almost always “New York City”
What word should we suggest next? To answer that question, Windows Phone 8 has to know not just a lot of words, but how frequently real people use them.
Designing for real people
Windows Phone 8 is designed to solve both of these problems by taking into greater account how real people use language on their phones. In the real world, people use slang, nonstandard spelling, and other casual shortcuts. Thus we needed different dictionaries than what you might find in Microsoft Word.
How did we build them? You helped us! Remember the little checkbox during phone set up (and in Settings) that talks about helping us improve text suggestions and build a better product? When someone gives us permission, we collect anonymous typing data—free of passwords, names, numbers, and other personal info—to help create and test Word Flow.
To make the phone smarter about pop culture and slang, we also turned to Twitter and Wikipedia to help inform our dictionaries. As fun as it would’ve been to read Twitter and Wikipedia all day, we instead built programs to “crawl” these sites for new and commonly-used words. (Although we still have to manually review the results, tossing out irrelevant content like twitter hashtags and web addresses.)
Finding the things, places, people, and topics people care about helps us make more relevant suggestions and corrections, so you can get relevant text suggestions ranging from Harry Potter (examples: Ravenclaw and Slytherin) to famous people (examples: Obama and Clooney). Of course, what’s popular in the U.S. may not mean anything in Russia, so we built Word Flow uniquely for each language. What’s also great about using Twitter and Wikipedia is they help us build dictionaries that are appropriate to specific countries or dialects.
The second problem I mentioned earlier is understanding context: what words are the most likely to come before and after the one you just typed. Specifically, we look at three words of context (the fancy word for this is “trigram”) to figure out what words are best to suggest. Imagine if a user typed “I miss” and then tapped the letter “y.” Can you guess what most people are trying to type? The answer is “you,” of course, because we mostly use our phones to stay in touch with people we care about. (If you said “yodeling,” you’re in a very small minority.) Trigrams are great for common SMS phrases like “I love you”. But it also means Windows Phone knows that when you type “Harry,” you’re probably going to type “Potter” afterwards. Here are the top 12 phrases from our dictionary—how many have you used recently in a text message?
I love you I love u How are you I know I miss you What’s up Where are you Thank you I don’t know Oh ok Love you too Me too
Another engineering challenge involving accuracy is the “fat finger” problem—which has less to do with finger size than screen size on smartphones. If a user’s finger covers more than one letter on the on-screen keyboard— a fairly common occurrence—which one did he actually intend to type?
The solution we use in Windows Phone is to change each key’s invisible “hit target”—the touch-sensitive area around each letter—as you type. When your finger touches a hit target, that letter is inserted. Hit targets are constantly changing size, depending on what word the keyboard thinks you’re trying to type. Take a look at the video see this in action.
How do we know which keys’ hit targets to make bigger? We use that same dictionary to help us decide which letters you’re likely to tap next. Going back to the happy birthday example, after typing “h-a-p-p-y-b,“ the hit target of the key “i” gets bigger and the hit targets of the surrounding keys—“j,” “k,” “o,” and “u”—get smaller because it’s likely you’re going to tap on “i” rather than any of those other letters. Many of our users continue to tell us that typing on Windows Phone is just easier than any other smartphone out there; hit target resizing is a big reason why. Getting everything we did to work smoothly, by the way, required some serious scientific horsepower—which is why we partnered closely with Microsoft Research.
Your custom dictionary
Finally, the most important part of the phone’s dictionary starts out blank when you buy the phone: the custom dictionary. That’s where we store on your phone the words and phrases that youuse. As you type on your phone, it learns how commonly you use language. We then use the custom dictionary in all of the places we describe above—suggestions, corrections, keyboard hit-target adjustment—for improved typing accuracy.
The result is a keyboard designed to be the most intelligent on any smartphone—and our team is just getting started. We’re excited about continuing to make typing on your phone more intuitive, accurate, and personal. Let us know what you think.