February 8, 2016 10:00 am

Optical Character Recognition (OCR) for Windows 10

Optical Character Recognition (OCR) is part of the Universal Windows Platform (UWP), which means that it can be used in all apps targeting Windows 10. With OCR you can extract text and text layout information from images. It’s designed to handle various types of images, from scanned documents to photos. At the same time, it is highly optimized and runs entirely on the device without requiring an Internet connection. The Windows 10 November update enables OCR for four new languages, bringing the total number of supported languages to 25.

This technology has been heavily tested inside Microsoft for years. It’s used in major products like Word, OneNote, OneDrive, Bing, Office Lens, and Translator for various scenarios, including image indexing, document reconstruction, and augmented reality.

Using the OCR API is very simple, as demonstrated by this code snippet:



SoftwareBitmap bitmap;

// ...
// Acquire bitmap from file, camera, ...
// ...

OcrEngine ocrEngine = OcrEngine.TryCreateFromUserProfileLanguages();
OcrResult ocrResult = await ocrEngine.RecognizeAsync(bitmap);

string extractedText = ocrResult.Text;


Running the code on the image below extracts the following text:

“TRAVEL THE WORLD SEE THE SIGHTS HAVE GREAT DAYS AND BETTER NIGHTS”

1_traveltheworld

OcrResult can be also used to retrieve text line by line or find position of each individual word. For more information, please check out the code sample at GitHub and MSDN documentation.

This work is the evolution of Microsoft OCR library for Windows Runtime, released on NuGet in 2014. If you used this library in an app for Windows/Windows Phone 8.1, moving to the new OCR API in Windows 10 will be straightforward.

In case you need to use OCR as a service, we’ve got you covered as well. The same technology is released as part of Project Oxford (a set of services for natural data understanding—we recommend you check it out as it brings a number of possibilities for your apps). Compared to the Windows.Media.Ocr namespace, the service has additional features such as language detection and text orientation detection. Try experimenting with the OCR service demo and get detailed info in the API documentation.

We’re looking forward to your new, amazing OCR-powered apps in the Windows Store. If you’d like to share feedback, ask a question, or start a discussion, please post on MSDN forum. We’ll be eagerly waiting to hear back from you!

Written by Pavle Josipovic, a Software Engineer on the Analog team

Updated February 8, 2016 10:22 am

Join the conversation

  1. Why are these features still not available even on Microsoft office apps such as One Note Windows 10 mobile platform in spite of the claims that these are universal apps?

    • As a matter of fact the OCR feature is already there in OneNote. Copy and paste the image you want to do OCR on, then right click on the image & click on “Copy text from picture”. Finally paste it from clipboard & voila! This is an example: http://i.imgur.com/QurXA5x.png

      • Sorry. I understand that this works in a surface tablet/laptops. My question was about usability of this feature through One Note app (assuming that it is currently universal) in Windows 10 mobile phones. For productive use, we would want these features in mobile phones (Windows mobile phones) where we capture a picture when using One Note during a demo/presentation and extract the words from the picture into One Note or MS word. But I am not able to do this using windows phones even with the latest build of windows 10 mobile. I know that this can be done through Office Lens. But for that one has to open office lens separately and do this. Office lens currently cannot even be accessed through lenses option in windows camera app in windows phone triggered through camera button (That was possible in WP 8). Also Office lens tends to save this as a new separate one note page in a different location in windows 10 mobile which needs to be again copied and pasted into the one note file where I am taking quick notes. In other words, most of these apps are disjointed as a complete eco system in their current state. If One Note as a universal app is supposed to provide same user experience regardless of the device being used, why is this not possible on Windows’ own mobile platform? Are there any plans to address these differences in the near future? Or am I missing something here? I just hope that Microsoft is focused on illustrating/demonstrating the full potential of universal apps in its own platforms (both mobile and desktop) using its own apps. Only such strong demonstrations will get the attention of developers and motivate them to develop innovative and useful universal windows apps. It is okay to get the attention of users to try/use Microsoft apps in other rival platforms. But for the users to choose using these universal apps in windows platform (otherwise, why would anyone prefer UWP compared to apps in rival platforms or the age old x86 programs?) , Microsoft teams currently need to get the attention (through proper demos with their own apps) of creative developers towards the full potential of universal apps.

      • I’m using One Note on Windows 10, right clicking on an image doesn’t offer the option to copy text! Can you help?

  2. Is there any channel to discuss troubles with recognizing text? I have some image with I think quite clear text, but this library has trouble to recognize that

  3. I’m using One Note on Windows 10, right clicking on an image doesn’t offer the option to copy text! Can you help?

    • if u r using one note in win10. right click on the picture > picture > copy text > then u can paste it where ever

  4. When I see a how to article start off with “EASY” I know “WE” are in trouble. What a tech that totally understands what he/she is talking about does not have the ability to dumb down their techy talk for the clueless lay person. It’s impossible.
    Can someone tell me in everyday words how to use my win 10 scanner as an OCR without using one technical term? “the thingamjimmy on the top left button works for me”. Get down to the Kindergarten level – NO words like – Microsoft OCR library for Windows Runtime; MSDN forum; API documentation…I’m going fishing, I’ll check back in a couple weeks and see if I got a bite.
    Thanks, Bob