The secrets of the Windows Phone 8 keyboard

The secrets of the Windows Phone 8 keyboard

  • Comments 33
  • Likes

From the day Windows Phone 7 made its debut, we’ve gotten lots of kudos for its on-screen keyboard and intelligent suggestions and corrections. Engadget called it “one of the best and most accurate virtual keyboards we've used on any platform.” Gizmodo declared: “The keyboard is boss.” Even the New York Times loved the “smart auto-suggestions.”

In Windows Phone 8, we set out to make our critically-acclaimed keyboard even more intuitive, accurate, and personal. As one of the program managers for the feature, I wanted to tell you about improvements in the new release, and also explain some of the science behind our prediction technology, something we get a lot of questions about.

Introducing Word Flow

On average people type 20 to 30 messages a day on their phones; that’s over 10,000 a year. They also make a lot of mistakes: about one in every three words, we’ve found. Windows Phone 8 introduces Word Flow—an improved and renamed version of the Quick Correct feature from Windows Phone 7.5—to help prevent time-wasting typos and focus on what matters: communicating with the people you care about. (See also: Tips for typing faster and smarter)

Word Flow is designed to reflect how real people talk to each other—and so it knows everything from pop culture to slang. When building Word Flow we reviewed over 2.5 billionEnglish words from sources ranging from the dictionary to the Internet. Because phones lack infinite storage space, we picked some 600,000 of the most common words and phrases that people actually type on their phones to power Word Flow’s corrections and suggestions.

As a result, Windows Phone’s auto-correction skills are 94% accurate, on average, out of the box. But we all have our own communication style, so Word Flow is also designed to be a keen student of your writing habits and what you care about. It’s always learning, so it can help personalize the suggestions and corrections to suit each individual person.

imageimage

The Word Flow keyboard in Windows Phone 8 is designed to learn your style over time to help improve the accuracy of its suggestions.

How we do it

Building a great keyboard is challenging; we’ve all seen smartphones make frustratingly goofy fixes. If you need proof, just check out the thousands of hilarious submissions on sites like Damn You Autocorrect.

To prevent Windows Phone from being the star of those sites, it needs to know a lot of words so it can figure out what you’re trying to type. So we have a built-in virtual dictionary akin to the dusty old copy of Webster’s on your shelf. But there’s one crucial difference: our dictionary contains not only words but data about how frequently real people, on average, use them on their mobile phones.

Imagine, for example, that someone starts tapping out “h-a-p-p”. What word should the phone suggest next? “Happen” is one possibility from the dictionary—unless the person was actually about to say “happy.” The Office team has been researching commonly-used words for more than 20 years to power similar features in Word and Outlook, and they partnered with us to build our first dictionaries in Windows Phone 7. Office uses a variety of sources—from linguistic research to frequency analysis of documents, books, and web pages—to build their dictionaries. Since “happy” is used more often by people than the word “happen,” we offer “happy” as the first suggestion.

Even a “smarter” dictionary, however, doesn’t prevent strange suggestions and unwanted auto-corrections. This happens for two reasons.

First, it’s very important that the dictionary be relevant to what you would type. A dictionary based only on words found in scientific papers and textbooks wouldn’t offer suggestions relevant to everyday conversation (although science teachers across the world would rejoice). For example, if you typed “h” you might see “hydrogen” as the first suggestion.

Second, the likelihood of a word depends on what comes before it. If you type “H-a-p-p-y-b,” “birthday” is much more likely than other common b-words like “be.” Some words are very easy to guess based on the words before it, for example “New York C” is almost always “New York City”

imageimage 

What word should we suggest next? To answer that question, Windows Phone 8 has to know not just a lot of words, but how frequently real people use them.

Designing for real people

Windows Phone 8 is designed to solve both of these problems by taking into greater account how real people use language on their phones. In the real world, people use slang, nonstandard spelling, and other casual shortcuts. Thus we needed different dictionaries than what you might find in Microsoft Word.

How did we build them? You helped us! Remember the little checkbox during phone set up (and in Settings) that talks about helping us improve text suggestions and build a better product? When someone gives us permission, we collect anonymous typing data—free of passwords, names, numbers, and other personal info—to help create and test Word Flow.

To make the phone smarter about pop culture and slang, we also turned to Twitter and Wikipedia to help inform our dictionaries. As fun as it would’ve been to read Twitter and Wikipedia all day, we instead built programs to “crawl” these sites for new and commonly-used words. (Although we still have to manually review the results, tossing out irrelevant content like twitter hashtags and web addresses.)

Finding the things, places, people, and topics people care about helps us make more relevant suggestions and corrections, so you can get relevant text suggestions ranging from Harry Potter (examples: Ravenclaw and Slytherin) to famous people (examples: Obama and Clooney). Of course, what’s popular in the U.S. may not mean anything in Russia, so we built Word Flow uniquely for each language. What’s also great about using Twitter and Wikipedia is they help us build dictionaries that are appropriate to specific countries or dialects.

The second problem I mentioned earlier is understanding context: what words are the most likely to come before and after the one you just typed. Specifically, we look at three words of context (the fancy word for this is “trigram”) to figure out what words are best to suggest. Imagine if a user typed “I miss” and then tapped the letter “y.” Can you guess what most people are trying to type? The answer is “you,” of course, because we mostly use our phones to stay in touch with people we care about. (If you said “yodeling,” you’re in a very small minority.) Trigrams are great for common SMS phrases like “I love you”. But it also means Windows Phone knows that when you type “Harry,” you’re probably going to type “Potter” afterwards. Here are the top 12 phrases from our dictionary—how many have you used recently in a text message?

I love you I love u
How are you I know
I miss you What's up
Where are you Thank you
I don't know Oh ok
Love you too Me too

Another engineering challenge involving accuracy is the “fat finger” problem—which has less to do with finger size than screen size on smartphones. If a user’s finger covers more than one letter on the on-screen keyboard— a fairly common occurrence—which one did he actually intend to type?

The solution we use in Windows Phone is to change each key’s invisible “hit target”—the touch-sensitive area around each letter—as you type. When your finger touches a hit target, that letter is inserted. Hit targets are constantly changing size, depending on what word the keyboard thinks you’re trying to type. Take a look at the video see this in action.

How do we know which keys’ hit targets to make bigger? We use that same dictionary to help us decide which letters you’re likely to tap next. Going back to the happy birthday example, after typing “h-a-p-p-y-b,“ the hit target of the key “i” gets bigger and the hit targets of the surrounding keys—“j,” “k,” “o,” and “u”—get smaller because it’s likely you’re going to tap on “i” rather than any of those other letters. Many of our users continue to tell us that typing on Windows Phone is just easier than any other smartphone out there; hit target resizing is a big reason why. Getting everything we did to work smoothly, by the way, required some serious scientific horsepower—which is why we partnered closely with Microsoft Research.

Your custom dictionary

Finally, the most important part of the phone’s dictionary starts out blank when you buy the phone: the custom dictionary. That’s where we store on your phone the words and phrases that youuse. As you type on your phone, it learns how commonly you use language. We then use the custom dictionary in all of the places we describe above—suggestions, corrections, keyboard hit-target adjustment—for improved typing accuracy.

The result is a keyboard designed to be the most intelligent on any smartphone—and our team is just getting started. We’re excited about continuing to make typing on your phone more intuitive, accurate, and personal. Let us know what you think.

Related: How to use the Windows Phone 8 keyboard

33 Comments
You must be logged in to comment. Sign in or Join Now
  • I have a Nokia 822 with the vanishing keyboard problem. Occurs on internet forums. Tried disabling background services, soft re-set, hard re-set, phone has the latest update, and have had zero luck with clearing out this issue.  Are there any new workarounds for this vanishing keyboard? It pops up, then drops back down instantly. Very frustrating. Thanks.

  • This is neat tech, but no on-screen keyboard can replace the passive tactile feedback of a physical keyboard.  Why aren't there any windows phone 8 models with physical keyboards?  Is that an OEM decision or is MS pressuring them?  Why is there no HID profile in the WP8 Bluetooth stack?  It was present in WM6.5, but dropped in WP7 and has been missing since.

  • Is there a possibility that Word Flow comes to WP7 users?

  • the context-based suggestions are great! Awesome work! :)

    However, my 2 suggestions:

    -Auto save words people type to the personal dictionary/ automatically detect a new word and give an option for saving ( currently we have to select every word to add to the dictionary)

    - A word saved in the dictionary retains its capitalized letters from when it was saved. When you use it again, it has the same capitalization irrespective of the context.( Eg: If you save STOP to your dictionary, the next time you select it from suggestions, it'll remain STOP and there is no way you can change it to stop or sToP if you will). It is very annoying in case of larger words.

  • It would be really useful to have a facility to enter commonly used phrases. The best example is an email address. When there is free wifi available it often asks me to log in with my email address and maybe a simple password. Word Flow does not help at all with this because every element of the email address counts as a different word so it cannot be stored a suggestion. Typing in my whole email address one character at a time is a real pain. I actually just wrote a simple app in TouchDevelop to do this but it would be an obvious thing to implement.

    I must say that overall I am very impressed with the keyboard in WP8 and for the first time ever I've been persuaded to give up using  a physical keyboard.

  • I think the keyboard is very good. I really love the Word Flow feature where suggested words come up.

    My main bugbear is that i am ALWAYS hitting,comma,instead,of,the,spacebar.

    The other issue is that as a UK keyboard user i too suffer from lack of capitalised 'i' etc. Please make ENG-GB a first class citizen!

    Finally, while this,may not be a keyboard issue, editing text in a browser text input/area is often difficult: tapping a word just places the cursor most of the time (or more annoyingly blurs focus from the control), rather than,selecting the word,as is the case in non-browser controls. This makes it very difficult to use the dictionary corrections features. Also placing the cursor where you want it in a browser text area is sometimes impossible, which is very frustrating. I hope the UX around browser text editing gets a lot of attention soon.

  • I would like to have a swype style interface.

  • So Jason it is now understood that the UK keyboard has an issue with Capitalizing I's which is like the most common letter that is said in the UK "I think", "I will be", "Yes I can do", etc etc so hopefully in another update (Not the latest one that I received last night) you can correct this :)

  • arrow22
    50 Posts

    Thanks for the response Jason, but I should have clarified. The keyboard in WP7.5 capitalizes "true" and "false" under the "English UK" keyboard (which I use since there is no Canadian keyboard and Americans don't know how to spell :P).

    It seems people are experiencing lots of issues with this keyboard. Perhaps your testing methodology needs some work.

  • So having used both the UK and US keyboards in WP8 extensively, I have to say the US Word Flow is far better at getting the right next word (it knew Windows Phone 8 which the UK keyboard didn't) and I get the same issue of i not being auto capitalised, which works perfectly on the US keyboard. I am also finding I keep hitting the comma instead of the space bar all the time; if I was doing that on the US keyboard it was getting auto-corrected for me. Please, can the non US keyboards get the same great performance as the US one? This isn't nearly as bad as the difference between US and UK keyboards in NoDo and Mango but it's still annoying!

  • @henbo - press and hold to get an insertion cursor you can drag into place in any word

  • I notice the Microphone icon at the bottom of those snapshots.  Never seen that, is this part of the next WP8 update?

  • henbo
    0 Posts

    Hmm. Why not show screenshots in Landscape on standard resolution? Maybe with Excel?

    Or with a text message where I cannot read what I typed three lines above?

    Why not explain how to reliably place the cursor inside a word? Or how to easily switch from one input field in a web form to another? Or how to enter a degree character ° (U+00B0) on DEU.

    How to use the keyboard on a website without input fields but that supports keystrokes (Google Reader Mobile)?

    I could go on. Nothing can easily beat the good old five-row HTC TouchPro physical keyboard.

    The completion feature and the quick language switch are pretty slick thou (easy: it actually works). It could make me happier if it preferred known names (phone book) over frequent names.

  • faruq
    0 Posts

    great awesome work!!!

    "The Office team has been researching commonly-used words for more than 20 years to power similar features in Word and Outlook, and they partnered with us to build our first dictionaries in Windows Phone 7"

  • We love the enthusiasm everyone has around the keyboard---thanks for all the great suggestions and feedback!  We’ll take them into consideration for future releases. Meanwhile, here are a few responses to specific questions:

    @arrow22: On Windows Phone 8 when you type phrases like, “This is false,” we leave it as “false.”  Further, if you make a mistake and type, “This is falsw,” we auto-correct to “false” in lowercase.  Also, to avoid auto-correction from happening, tap on the white space directly in front of the word. If the auto-correction does occur, tap on the word that auto-corrected and you’ll notice the first word is what you previously typed.  Tap on it and we’ll replace the word with what you wanted (and in Windows Phone 8, begin to learn that’s what you want in the future)

    @Ootpekl: As@QZee pointed out, you can remove the language switching key by having only a single keyboard enabled.  On Windows Phone 8, go into Settings>Keyboard, then press and hold on the keyboard languages you don't want, then tap Remove.  When only one language remains, the language toggle key will be removed from the keyboard.

    @Arostino: There isn't currently a way to back up the custom dictionary and take it with you to a new phone. I understand this can be frustrating, and the team will take this feedback into consideration for future planning.  

  • Corno85
    0 Posts

    I completely agree with the phone (HTC 8X) not correcting i... and I do have the settings for correcting mispelled words, and nothing seems to change it.

    Another problem I have come across is when I want to type " I'll " when using the keyboard you have to add the apostrophe yourself, on my blackberry I could set up a macro to change il into I'll, made typing a lot easier!

    I made a mistake the other day, and added a word to my dictionary that is misspelled, does anybody know how you can delete a word from the dictionary without destroying you 'typing history' because then the keyboard will have to learn my language again : s

  • SimbaP
    0 Posts

    Shame on you and that keyboard for not allowing users to save swear words. I get not including it in the box, but prohibiting a willing adult from adding a word he chooses to the dictionary? And you have the audacity to claim the keyboard is "personal"?? Get outta here.

  • Prayaas
    1 Posts

    Is that Windows Phone x86? :O

  • Ootpek
    0 Posts

    Also...if you are running English United Kingdom as the language it will NOT capitalize the I.  If you run English United States as the language it will.

  • Make WP8 keyboard better in one simple step:  Swype.  I recently migrated back to Windows Phone from Android...but the one thing I really miss is Swype.

  • Ootpek
    0 Posts

    For the capital I issue. Do you have "correct misspelled words" turned off in the keyboard settings?

    If I turn that off it no longer capitalizes i

  • Ootpek
    0 Posts

    THANKS QZee!  That's a great tip!

    As far as the capital "i" thing goes...mine actually does capitalize it when I am set to English... but does NOT when I am set to French.  

  • QZee
    0 Posts

    Ootpek, this language selection button on the keyboard drove me mad too, leads to endless typos.

    To get rid of it, go to settings, then into keyboard and press and hold the second language you want to remove and select delete.

    This will restore the good old WP7 keyboard layout.

  • QZee
    0 Posts

    djSupport, I also am frustrated by my Lumia 820 noit capitalising the letter i.

    I have tried reseting dictonary to default but isn't helping.

    smallmountain, thanks for your input, we know its not manufacturer specific as now we know there is a Lumia and 8X user who are suffering the problem and yourself an 8X user who doesn't have the problem. Just to clarify, when you type the letter I mid sentance, does it capitalise it for you? My Lumia will capitalise the letter I if it is first word in sentance only.

  • Ootpek
    0 Posts

    Ok that points out one important thing to me regarding my "keyboard"...it has an extra button!

    Beside the keyboard type selector button I have a language selector that says ENG.  Simply hitting it switches me to FRE!

    Unfortunately I have hit this button by accident already which produces very poor auto-correct results when I am always typing in English.  Is there a way I can disable that button so my keyboard looks like the one in your examples?  

  • On the issue others have mentioned about WP8 not capitalizing "i" when its by itself:  WP8 *does* capitalize them for me, in text messaging and mail.  Maybe some more info is needed about scenarios.

    Slightly OT, but the keyboard's audio feedback lags too much to be helpful.  It's actually distracting. I had to turn it off.  I think I'd like it if it didn't lag.  I have HTC 8X.

  • mog0
    8 Posts

    Too early to make a definitive judgement but after upgrading this week from an LG Optimus 7 running WP7.5 to a Nokia Lumia 920, I've actually found a significant DROP in accuracy, despite the bigger screen / keys. I'm hoping that it will learn and surpass my old LG though.

    There are some real annoyances though, for example on 7.5 if I type "i" on its own with a space either side, it would get auto corrected to "I", but WP8 doesn't do this so I have to go back and correct, which is tricky as it's a single narrow character and tricking to tap on. When I do tap on it, the only suggestion it comes up with is "I", so why wasn't it just done automatically?

    Overall, love the OS though, just some minor irritations.

  • This is good work and shows how much effort is required to make a on-screen touch keyboard great.  I would love to see the keyboard improved in the following ways:-

    • Editable custom dictionary.

    • Option for haptic feedback on the keyboard.

    • Advanced text formatting (bold / italic / underline etc).

    • Option to remove the automatic space after selecting a suggested word.  This is most annoying when you want to punctuate after a word using a comma, semi-colon or period.

    • More Autocorrect options like change a word to Title case or upper case – that would save so much time and effort, fiddling around trying to place the cursor.

    • Speaking of the cursor, ever since copy&paste was introduced, the keyboard is way too biased in favour of highlighting a whole word when you just want to place the cursor somewhere specific.  Very annoying.

    • When a word is misspelled, a red underline appears under the word.  When I select it, very often there are no suggestions to correct it at all.

  • What drives me absolutely Nuts is for some reason my WP8 HTC 8x no longer capitalizes the I so when typing fast I get "Hi i think i should be going now" rather than "Hi I think I Should be going now" I have reset the dictionary and all the settings are on to correct but yet I still get the i, WP7 always Capitalized my I!!!

  • Will my custom dictionary after two years of WP7 be backed up and stored in a new WP8 phone as soon as I will buy a new WP8 phone?

    And later, from WP8 to a new WP 8 phone, also?

  • abm
    268 Posts

    Very well. Now lets talk about voice recognition feature of Windows Phone.

    Take a look what IRON MAN / Tony Stark has to say about Windows Phone speech recognition while comparing with others www.youtube.com/watch 8=)

    I have tried to contact any TellMe team member, but unable to find any blog, feedback form, contact info etc. If you know how to contact them, this comparison is a good start.

    [Message from community]

    In your on-going effort, "We need a next-generation speech-to-text system, which detects the names like iOS does and does the real-time/offline speech recognition like Android does and continuously evolving / updating this offline pack. We need an option to enable/disable auto punctuation and improve the auto-punctuation algorithm. We NEED all it because Microsoft has much more experience in this domain than others, and we expect their product to be better than others".

  • arrow22
    50 Posts

    Hope some of the WP7 issues have been fixed (like changing "false" to "FALSE". Very annoying.) I too have some issues with WP7 resizing touch targets for less common words, which forces me to type the wrong letter and then to go back and correct it.

    Perhaps at the touches to the visible limits could be recorded to offer up the word typed on visible targets as a suggestion?

    Last thing, which may have been fixed, sometimes I notice that the correction is bold, and it's going to correct my word, but I don't want it to. What do I do?

    All told though, it's an awesome keyboard and you and your team have done an amazing job!

  • Very good work done on the keyboard in WP8. I've really been giving Word Flow a workout, lol. I've become much more efficient when typing with it. :)

    However, my issue is with the hit areas. A lot of times with acronyms, your hit areas shrink to the point of screwing me up more than leaving them alone. Also, I almost always have issues when using double-letters, or repeated keystrokes like repeated-backspace for fixing errors.

    Don't get me wrong here, I'm not really complaining just pointing out an area that I hope you'll keep improving, because I'm infinitely more productive with WP8 keyboard than any other keyboard option I've used with the exception of Swype. (Side note: I swear I'm the Swype F-ing Master, lol... you should work with that team for a WP version! :))