April 6, 2016 2:00 pm

Create, ideate, and collaborate: Build apps powered by Windows Ink

Handheld tools have always been a key component of human communications and productivity. From the earliest stone carvings to the modern pen, handwriting is not only familiar to us, it is second nature. With Windows, we’ve consistently seen that people who use the pen are happier, more engaged, and more productive. From our research, 72% of the people we surveyed still depend on pen and paper for one or more hours daily, and 32% of them use pen and paper for three or more hours daily.

Since the release of the first tablet PC, digital pen and inking have been a target for investment at Microsoft, and the Surface Products are one of the results. For Windows 10, we built an inking experience that complements hardware from Microsoft and our partners. We also introduced DirectInk, that was re-imagined from the ground up for modern touch and pen scenarios.

Windows Ink

In Windows 10 Anniversary Update, we renew our commitment to inking with the introduction of Windows Ink—a family of end-to-end user experiences, platform advances, and hardware capabilities that will propel Windows to the next generation of computing and help bring about the introduction of the next generation of devices. Windows Ink starts with an experience that greets you at the click of the pen button. This experience, the Windows Ink Workspace, is also always accessible through the taskbar, no matter where you are.

The Windows Ink Workspace puts all your pen experiences right at your pen tip, fully integrated with Windows for quick access and zero setup. We hear from our users that the pen is very powerful for capturing quick thoughts, sketching out ideas, and sharing quick annotations with friends. We also hear from our users that they want more readily available access to their pen apps and to be able to find content more easily. Check out the Windows Ink Workspace—coming to you this summer.

Sticky Notes in the Windows Ink Workspace helps users leverage the power of their pen to not only quickly capture a thought, but also convert that thought swiftly into action. Write a reminder to “Call Mom tomorrow” and that thought will be directly integrated with Cortana across all your devices. She’ll be able to remind you wherever you are. Need to pick up a friend from the airport? Write flight number “AA 316” down and Bing will keep you up to date with the latest flight status. Write down a grocery list to buy items from a store and watch Sticky Notes convert that to a smart list that you can easily manage. Just like that, your handwriting just turned into a powerful digital tool without you having to think twice about it.

Have an idea you need to sketch out? A picture paints a thousand words, and sometimes it’s simply easier to draw your ideas out and let the creative juices flow. With sketchpad in the Windows Ink Workspace, you can do exactly that. This is your space to create, doodle, and quickly get your ideas down. There’s even a built-in digital ruler to help draw straight lines and charts.

Found an awesome picture you want to share on Twitter or Facebook, but first you want to add a personal touch with funny handwritten notes? Screen sketch on Windows Ink Workspace or a simple double click of your pen button lets you do exactly that without breaking the flow of what you are doing—even if it’s a funny scene from a movie you are watching, or an epic move you just pulled off in a game.

Looking for more apps to use with your pen? We now have a curated list of suggested apps in the Windows Ink Workspace to bring you the latest and greatest pen apps from the Windows Store, plus a dedicated link to all pen apps right at your pen tip. You’re a pen power user? The Windows Ink Workspace makes it super easy to see the apps you last used with your pen, so you can get to your favorites fast and be more productive.

Love the Windows Ink Workspace and want all these experiences without having to unlock your PC? You can open the Windows Ink Workspace while your PC is locked with just one click of your pen button. Plus, customize what each pen click shortcut does in the pen Settings page.

A more natural interaction model

Starting with the Windows 10 Anniversary Update, users can begin to create more naturally using both of their hands—they can draw or write with one hand while their non-dominant hand manipulates tools or the canvas. We call this “simultaneous pen and touch.” If you’d like to try out this feature yourself, check out the Windows Ink Workspace with the ruler tool in sketchpad or screen sketch.

Simultaneous pen and touch is turned on by default in the OS. Your app can now receive both touch and pen input streams simultaneously via the existing Windows Pointer APIs (WM_POINTER and PointerPoint for Win32 and UWP apps respectively). You can also leverage the power of Windows Ink to handle both input types by using InkPresenter, CoreWetStrokeUpdateSource to manipulate ink, as well as GestureRecognizer to detect and process touch gestures. The combination of these APIs allows you to build your own stencils and process input in custom ways. Simultaneous pen and touch is a powerful gateway to more natural interactions. While tools like the ruler are likely just the first (and most obvious) way to prove this, the platform now offers you many other ways to do it as well. For more examples, take a look at our Build 2016 talk Pen and Ink: Inking at the Speed of Thought starting at 43:21.

We have been working with various 3rd party app developers to integrate simultaneous pen and touch. Below is a video of how Adobe Illustrator is partnering with us on simultaneous pen and touch with Windows Ink.

Adoption and differentiation made easier

Opening up developers to the world of simultaneous pen and touch is just one benefit of the Windows Ink platform. Windows Ink also provides a unique opportunity for developers to differentiate their apps.

We have tried to make it as easy as possible to jump into Windows Ink. In Windows 10, we introduced the DirectInk platform. DirectInk powers all the Windows Ink experiences. We created a simple way to add inking to your apps via one line of code–the InkCanvas.



<InkCanvas x:Name="myInkCanvas"></InkCanvas>


This single line adds a control that allows you to quickly enable inking for users. In its simplest form, this is a canvas that users can write on, but you can also leverage it as an overlay for an existing app. Imagine that you have a Sudoku app where users can type numbers in to solve. With InkCanvas, users can now use their own handwriting to write their answers.

In the Windows 10 Anniversary Update, we are introducing even more building blocks in response to your feedback. These building blocks help to deliver a standardized drawing experience in your app and help you jump into simultaneous pen and touch more easily.

Next there is the InkToolbar—with an additional line of code, you can add this XAML control to get a consistent inking user experience featuring three brush types (pen, pencil, and highlighter), a common color and width picker, an eraser, and the ruler.



<InkToolbar TargetInkCanvas="{x:Bind myInkCanvas}"></InkToolbar>


1_inkbrushtypes

With these two building blocks—InkCanvas and InkToolbar—you now have the basic foundation for enabling ink in your apps. Adding both of these lines of code also gives you your first simultaneous pen and touch experience with the ruler. You can find out more about the InkCanvas in this MSDN link and information on the InkToolbar will be available soon.

Not only are the InkToolbar and InkCanvas easy to use, they are both highly customizable to meet the needs of your app. Let’s take a look at how easy it is to add a new toggle button to the toolbar.



<InkCanvas x:Name="myInkCanvas" />
<InkToolbar TargetInkCanvas="{x:Bind myInkCanvas}" />
   <InkToolbarCustomToggleButton Content="ITB" />
</InkToolbar>


2_inktoolbar

In addition to adding new buttons, you can also use custom color schemes for button visuals or states, override built-in defaults (ink color, ink size, ruler color, ruler size, ruler orientation), change default controls or buttons, add custom buttons, or choose to not bind the InkToolBar to the InkCanvas.

If you need even more customization for your app, you can take advantage of the flexibility provided by InkPresenter. This MSDN article provides additional details and we will also have a post that will go deeper into this at a later date.

Whether you build a simple game or app today, or compete with the best creative tools out there, ink has a home in your app. Ink adds depth and emotion to any experience and it is the most efficient way to convey thoughts and ideas between humans. It is a powerful medium that captures the personality and essence of the individual, and last but not least, it is a great teaching tool. Ink adds a new dimension to existing scenarios, like writing down what you did on a vacation on the back of a photo, that you could never easily do before digitally.

The platform gets you closer to how humans want to interact and that is a powerful opportunity.

Windows Ink enables an entire app ecosystem

In addition to the Windows Ink Workspace there are a number of inbox and first party apps that showcase Windows Ink. Microsoft Office and OneNote continue to innovate in the pen productivity space. Office plans to release many features that center around ink including ink to shape, ruler, new pen types, and revision of documents using the pen. Building off of work in 2015 which introduced ink to shape, OneNote plans to expand their pen features to include ink to math recognition, ink replay, ruler, and new pen types in the upcoming year. And Maps is harnessing the power of ink as a planning tool. You can easily draw a route or landmark on top of a map for your own reference or to share.

Additionally, we are engaging with third party app developers to provide a rich ecosystem of ink optimized apps. Adobe Illustrator’s blob brush is built on the Windows Ink platform which allows them to make the brush super-responsive to the user. They will be introducing simultaneous pen and touch with their stencil tools—creating is now a more natural experience on your PC. We have also been working with FluidMath who will be building a Universal Windows Platform (UWP) app using the Windows Ink platform. Over the next year, you’ll see even more first party and third party experiences light up with Windows Ink.

Windows Ink is built to support a wide variety of hardware

Not only does Windows Ink include updated experiences and a powerful platform, it also includes an investment in the pen hardware ecosystem. With Windows Ink, we will license the Microsoft Pen Protocol to hardware manufacturers, which enables turnkey solutions for high quality, affordable, and interoperable devices at a variety of price points. You will start seeing these devices reach market in holiday of this year.

In addition, we are also partnering with the world leader in digital pens, Wacom, on Windows Ink. Wacom has licensed the Microsoft Pen Protocol and will build pens that are compatible with a broad variety of PCs that are already in the market as well as the lineup of devices coming this holiday. By combining what we have learnt from Surface and Wacom DNA into a high-quality writing instrument built for Windows Ink, users will have an easy choice when they need accessory pens this holiday. Through these efforts, there will be more pens in the ecosystem at more price points, and more users demanding pen applications.

Join the Windows Ink revolution

With Windows Ink, we are renewing our commitment to inking in Windows; this is just the beginning. With the Windows 10 Anniversary Update, we are unleashing the Windows Ink Workspace, new in-box apps that harness the power of inking, as well as new third-party apps that will reinforce inking as a powerful medium.

We hope you will think about how your app can join us in this revolution. Let’s make every PC as simple to use as pen and paper but as powerful as a PC.

Written by Li-Chen Miller (Group Program Manager) and Jerry Koh (Group Program Manager) for Windows Ink

Updated October 20, 2016 9:04 am

Join the conversation

  1. Do you need to use a specific API to be eligible for Windows Ink Workspace? E.g. we are working on a UWP app that use PointerPoint for when the user is using a styles/pen and processing this data our self. What else is needed to be eligible for Windows Ink Workspace?

  2. Ink strokes captured by the new InkCanvas force the WPF ink classes to crash.
    As soon as even a single stroke from the new InkCanvas is transferred into ISF (file or clipboard) and loaded by WPF System.Windows.Ink, the WPF implementation crashes.

    See Feedback Hub report for details : feedback-hub:?contextid=386&feedbackid=ae34a583-1294-47e7-bf33-1a687f6a4960&form=1&src=2

    • Hi Philippe, this is Francis from the Windows Ink engineering team. Thank you for your feedback, we are trying to repro the crash internally and investigate the root cause. Please stay tuned.

      • Let me know if you need more details, I have sample code for TH2 UWP, RS1 UWP, WPF and Win32+msinkaut as well as ISF files to repro.

        The WPF part can be as simple as the following :
        using System.Windows.Ink; // WPF/Avalon ink support
        static void Main(string[] args)
        {
        System.IO.FileStream fs = new System.IO.FileStream(@”.\SampleInk.isf”, System.IO.FileMode.Open, System.IO.FileAccess.Read);
        StrokeCollection strokes = new StrokeCollection(fs); // Crashes if ISF contains any TH2 UWP stroke
        }

        Everything is interoperable except RS1 UWP with WPF.
        TH2 UWP and Win32+msinkaut have no problem with strokes coming from RS1 UWP InkCanvas, but WPF will crash when encountering ISF including a stroke capture in RS1 UWP.
        This leads me to believe the bug actually is in WPF and related to the ISF structure of a individual stroke. Creating an ISF file with msinkaut to be loaded by WPF will make WPF crash as soon as a single stroke has been imported into msinkaut IInkDip through the clipboard from a RS1 UWP InkCanvas.

        Also, it would be nice to allow UWP to save ink as ISF alone (without embedding it in a GIF). This is important for people interested in very small data transfers, for embedding ISF into our own file formats, and for the sake of interoperability between the 3 Ink implementations as the WPF one actually supports only ISF alone, making it harder than necessary to load ink coming from UWP GIF+ISF format.
        I suspect this could be the reason the bug in question hasn’t been noticed earlier, as there is no direct path between the UWP and WPF implementations.

      • 3 weeks later, build 14352 available, still the same bug and no news…

        Redstone1 is supposed to put the focus on Windows Ink, yet it breaks ink compatibility between UWP and WPF apps.

      • Hi Francis, the issue is now fixed in build 14361. Hopefully it means you found the cause and got it fixed, I really hope there won’t be regression on this one.

  3. It should be noted that almost all of this existed going back to Windows.Forms, albeit, the addition the InkTool bar is a nice shortcut for having to write your own. In fact, on a lark, I spent a bit of time last weekend replicating Windows Ink Workspace using 10586 version of Win10 – although the code would work on Win8 an Win 7 – then added some extra features like shape and text recognition.

    Stroke rendering is already there – and it’s not hard to write your own renderer.. but again, more ‘out of the box’ renderers isn’t a bad thing.

    I spent two years writing whiteboard apps for a major smartboard company and got to know Microsoft Ink really well. Not much of what I’m seeing here is new – but it is nice to see it get more face time.

    Side note, the UI design on the Ink tool bar is actually not great. The mashing together of pen selection and drop down for colour is really finicky to use (at least at is it in the Anniversary Edition prereleases), especially if you switch between pen and touch. In my version, after duplicating your design, I finally decided to add a ‘palette’ button to bring up a richer colour/style/width toolbar that can be stretched out or glued to the side of the screen and made very narrow.

    I’m actually having similar discussions with the Autodesk Sketchbook team to improve their UI/UX on Surfaces. Their UI gets in the way and their response to touch and pen interactions can get very tedious to use.

    • Most of this existed as COM components using GDI in Windows XP Tablet PC Edition (end of 2002), but the UWP implementation was very limited previously, with no renderer at all on Windows 8 WinRT, then just basic features in Windows 10 UWP. It’s good to see they are adding features to make ink easier to include in every app.

  4. Fortunately, the original bug that crashed Avalon/WPF has been fixed before release.
    Unfortunately I believe I found another serious issue in the release version.

    Pencil strokes seems to record invalid pressure data.
    Their points descriptions define X, Y, and NormalPressure ({7307502d-f9f4-4e18-b3f2-2ce1b1a3610c}), and that NormalPressure has Minimum=0, Maximum=1023 and Resolution=1.
    However, looking at the points data, they contain NormalPressure with values well above 1023, I observed up to 16383 on pencil strokes with high pressure.

    Ignoring the points descriptions and assuming a Maximum=16383 for pencil strokes seems to provide correct points pressures.
    This leads me to believe pencil strokes pressure do not respect the digitizer’s capabilities and do not set the points descriptions properly.

    This breaks pressure handling in every other implementation, best practice would be to use the range of pressures supported by the digitizer, and bad practice would be to use arbitrary pressure range but at least set that range properly in the points descriptions. Current implementation does neither, forcing other implementations to inclure quirks handling for pencil.

    • Same problem with standard pressure-sensitive Pen strokes generated by the new UWP implementation. When rendered by Avalon/WPF, they appear much larger than they should.

  5. When using the ink function on Windows 10, it looks great in onenote and “sticky note”. The image quality is much lower quality when using the ink function to annotate a word document, though. This is shown in the screenshots I posted here:
    https://answers.microsoft.com/en-us/windows/forum/windows_10-touch/why-is-ink-quality-much-worse-on-word-than-other/95f58a11-4b4e-4da6-95df-12a98e543918?tm=1474264587722

    I am interested to know:
    (1) Is this behavior normal or do I have a problem with my system that can be fixed?
    (2) If this behavior is currently normal, do MS plan to make the ink for annotating word documents the same as other applications (such as onenote and Sticky Note)?

    Thanks in advance to anyone who can shed some light on this.

    Regards,
    David.

  6. Hey everybody!

    I’am using a digital pen comming from pulse/livescribe. Herewith i can collect stroke data like those be collected by an ink canvas.
    What i’ve already done is to interact the “external” collected stroke data with the handwriting recognition of microsoft ink – a really exciting couple!

    But i’ve reached a border of usage – perhaps anyone of you can help me to cross it:

    Base idea: I want to share the pen with a collegue. The collegue has got an individual handwriting – like anybody has

    As Microsoft INK supports indiviudal learning – i’am looking for the possibility to deliver “external” collected strokes into the “learning Environment” of Microsoft INK instead of capturing ink data via the canvas within the learinng app.
    Is there any possibility to do it?

    Further more, is it possible to Switch beteween multiple “individual handwriting recognition sources” by i.e. just selecting a person within a drop box – load “external captured ink data” to be recongized with the person’s related hwr source?

    Thanks for your ideas and answers in advance

    Oliver

  7. Hey everybody!

    I’am using a digital pen comming from pulse/livescribe. Herewith i can collect stroke data like those be collected by an ink canvas.
    What i’ve already done is to interact the “external” collected stroke data with the handwriting recognition of microsoft ink – a really exciting couple!

    But i’ve reached a border of usage – perhaps anyone of you can help me to cross it:

    Base idea: I want to share the pen with a collegue. The collegue has got an individual handwriting – like anybody has

    As Microsoft INK supports indiviudal learning – i’am looking for the possibility to deliver “external” collected strokes into the “learning Environment” of Microsoft INK instead of capturing ink data via the canvas within the learinng app.
    Is there any possibility to do it?

    Further more, is it possible to Switch beteween multiple “individual handwriting recognition sources” by i.e. just selecting a person within a drop box – load “external captured ink data” to be recongized with the person’s related hwr source?

    Thanks for your ideas and answers in advance

    Oliver