August 11, 2015 3:52 pm

Contextual Sensing in Windows 10

This blog was written by Rinku Sreedhar, Senior Program Manager

In Windows 10, we are very excited to introduce several APIs for contextual sensing that can help you build apps to improve your customers’ daily lives. These include apps that detect presence when customers approach their devices, apps that can detect whether your customer is walking or driving, apps that can help customers track their fitness routines, and more. Using these APIs you can anticipate customers’ needs and proactively deliver targeted, personalized, and relevant content or services to enhance and simplify their lives. It is very powerful both in Consumer and Enterprise scenarios.

sensorCloud

Detecting activities

We are very happy to introduce the new Activity Detection APIs, which will help detect the user’s motion context. These APIs attempt to detect activity states based on current motion of your customer, such as Walking, Running, In Vehicle, Biking, Stationary, and Idle. The Stationary state is returned when a user has the device with them, while the Idle state is returned when the user has the device laying idle on a table or still surface. You also have the ability to create background triggers and get history details for up to 30 days.

A few usage scenarios are:

  • Surface information based on the user’s motion context (such as an activity-based playlist)
  • Change the app behavior based on the user’s motion context (such as auto-adjusting the camera focus when you detect that the user is capturing an image while walking or running)
  • Health and fitness tracking
  • Navigation and maps
  • Power saving (for example, avoiding constantly polling location or wifi when the device is idle or stationary)

The API pattern is shown below:

// Getting the current activity
var reading = await activitySensor.GetCurrentReadingAsync();

// Subscribing to activity changes
activitySensor.ReadingChanged += new TypedEventHandler<ActivitySensor,
    ActivitySensorReadingChangedEventArgs>(ReadingChanged);

// Getting history of up to 30 days
DateTimeOffset yesterday = ...
var history = await ActivitySensor.GetSystemHistoryAsync(yesterday);
foreach (var entry in history) { ... }

// Using a background task
var trigger = new
    Windows.ApplicationModel.Background.ActivitySensorTrigger(reportIntervalMs);
trigger.SubscribedActivities.Add(ActivityType.InVehicle);

// .. register the trigger, etc..

More detailed API reference for activity detection can be found here and UWP SDK Samples can be found here.

Counting steps

Another new addition is the pedometer which counts the user’s steps both when walking and running. As with activity detection, you can access history details for up to 30 days.

A typical application would be health and fitness tracking (without requiring an additional gadget/ wearable device). The pedometer can also combine with the wearable and Windows device sensor data.

The API pattern is shown below:

//Getting the pedometer reading changes
pedometer.ReadingChanged += new TypedEventHandler<Pedometer,
    PedometerReadingChangedEventArgs>(ReadingChanged);

void ReadingChanged(Pedometer sender, PedometerReadingChangedEventArgs args)
   {
    PedometerReading reading = args.Reading;
    if (reading.StepKind == PedometerStepKind.Walking)
        walkingSteps = reading.CumulativeSteps;
}

//Getting pedometer step count history
var history = await Pedometer.GetSystemHistoryAsync(yesterday);

More detailed API reference for pedometer can be found here and UWP SDK Samples can be found here.

Barometer and altitude sensing

We are very happy to introduce the Barometer API that reports barometric station pressure, and the Altimeter API that reports relative altitude i.e. changes in elevation.

Typical applications include:

  • Health and fitness tracking: knowing the relative altitude, you can determine if the user is going up or down and adjust calorie expenditure calculations accordingly
  • Detect what floor the user is on for indoor navigation
  • Weather forecasting

The API pattern is shown below:

Barometer barometer = Barometer.GetDefault();
BarometerReading reading = barometer.GetCurrentReading();

double pressure = reading.StationPressureInHectopascals;
barometer.ReadingChanged += ...

Altimeter altimeter = Altimeter.GetDefault();
AltimeterReading altimeterReading = altimeter.GetCurrentReading();

double altitudeChange = altimeterReading.AltitudeChangeInMeters;
altimeter.ReadingChanged += ...

//Selecting a report interval
mySensor.ReportInterval = 500;

Detailed API reference for Barometer can be found here and UWP SDK Samples can be found here. Detailed API reference for Altimeter can be found here and UWP SDK Samples can be found here.

Detecting presence

We now support APIs for both short range and long range presence detection. Short range sensors can detect for up to 2-3cm, while some of the longer presence sensors can detect presence up to a 12 meter range.

A few usage scenarios are:

  • Enabling a device to wake on user approach or log off when the user has departed. The Surface Hub uses these APIs to wake on approach when a user walks into a conference room and log off when they depart.
  • Automatic turn off of the display during a phone call
  • Understand and ignore accidental clicks when it is detected that the device is in the user’s pocket
  • Detection of gestures

The API pattern using Proximity is as shown below:

using Windows.Devices.Sensors;

//Getting the current reading
ProximitySensorReading reading = sensor.GetCurrentReading();
bool isDetected = reading.IsDetected;

//Subscribing to reading changes
sensor.ReadingChanged += ReadingChanged;
void ReadingChanged(ProximitySensor s, ProximitySensorReadingChangedEventArgs e)
{
    ProximitySensorReading reading = e.Reading;
    bool isDetected = reading.isDetected
}

//Selecting a report interval
mySensor.ReportInterval = 500;

More detailed API reference for Proximity can be found here and UWP SDK Samples can be found here.

Stay tuned; we will be sharing more details of new Windows 10 features upcoming blog posts. You can also find more information in our Build 2015 session on building contextually-aware UWP apps using sensors.

Updated August 12, 2015 10:53 am

Join the conversation

  1. Does this mean that we’ll finally see Motion Data and SensorCore come to non-Lumia devices, and if not why can’t Microsoft just release the APIs to other OEMs, just listen at this, please bundle the Motion Data setting with Windows 10 for capable devices as it would otherwise be a multi-billion dollar waste to buy Nokia’s telephone division(s) and never use the awesome software they’ve developed to enrich the core Windows 10 Mobile, please release the Lumia Imaging SDK next, no more exclusives, and please bundle the likes of Lumia Beamer and Lumia Car App with Windows 10 Mobile’s settings (the OS, not as Lumia firmware) so other OEMs could make competitive devices and Microsoft will become less dependent on the sales of Lumia hardware, Lumia software = awesome and should be telephone agnostic, Lumia hardware = should be the example other OEMs follow like how the Surface introduced a concept and Lenovo perfected it with their Yoga.

    • Thanks Nga for your feedback. Starting Windows 10, Activity Sensors and Steps (Pedometer) that were originally part of SensorCore SDK will be available as a UWP API – which means it will be supported on all Windows device families on Mobile, Desktop, IoT etc. that has the required hardware and drivers. Hope this helps.

  2. And please just call it ”SensorCore” on mobile devices, perhaps rename the Nokia Lumia SensorCore to ”Microsoft SensorCore” (or keep the first name, it’s awesome too), the name is already known in the dev. world and Microsoft could simply rename the publisher called ”Lumia SDK” (which was previously called ”Nokia Developer”) to ”Windows Mobile SDK” and release other API testing “applications” with that account to demonstrate the real world usage of those APIs.

  3. How is the altimeter calibrated if at all?

    Barometric altimeters have to be calibrated via QNH, QFE &al prior to each departure from an airfield. These values change on a day to day basis for a particular geographic region so how trustworthy is the altimeter reading from the API?

    • Looking at the API the altimeter is relative – it is intended to report the change in altitude, not an absolute altitude.

    • The altimeter API does not return the absolute altitude, but relative altitude which is a delta in the altitude changes derived from the station pressure, hence the calibration issues should not impact the readings. We did initially look into surfacing Absolute altitude and looked into an auto-calibration process, but later decided to surface relative altitude.

      • Thanks for clarifying that.

        I take it the altimeter’s just using ISA model atmosphere and not trying to correct for local weather conditions? I hope so because Microsoft’s weather data for my home town in Scotland is never correct.

        Just to note, the API documentation on the altimeter makes no mention of it measuring an altitude delta, even digging into the AltimeterReading class incorrectly describes the .AltitudeChangeInMeters property as “Gets the current altitude determined by the altimeter sensor”.

        • Yes, you are right, we are using the ISA model and assumes constant temperature, molar mass and gravitational acceleration at the troposphere level. When the local area has different sea level than the mean sea level, the altitude data won’t be accurate without calibrating it, as you mentioned, which is why we decided to surface relative altitude.
          Thanks a lot for your feedback on the API documentation, we have updated it to make this more clear. May take a few days before it shows up on the public site.

  4. But how can I simulate these new sensors? a Pedometer for example.
    I couldn’t find info about this anywhere.

  5. ActivitySensor.GetDefaultAsync() is returning null on devices with build 10512. Those Lumia devices previously worked fine with SensorCore (and still do, when I try from SensorCore demo apps that were published to the Store, such as “Activities – Lumia SensorCore SDK sample”), so the activity sensor is clearly there. Is this a known issue? The samples you have on GitHub are also failing. Any workarounds? Thanks.

    • Hi Igor, Thanks for your feedback. It is returning NULL, because the Sensor is not present due to hardware constraints. Activity and Pedometer Sensors and the UWP APIs are supported on new hardware with Windows 10. We do understand that this is not ideal, so we are looking at ways to make it easier and will give you an update soon.

      In the meanwhile, as a workaround, we will provide a sample that uses both the UWP APIs and the Lumia APIs (as a fallback option) so it works on both Windows 10 hardware as well as the older hardware. I’ll give you an update as soon as we have it ready.

      • Thanks Rinku. Yeah, it’s a bit disappointing that the old hardware won’t work with the new APIs. It would be even cooler if the ActivitySensor was wrapping Lumia APIs (as a fallback option) for non-Windows 10 hardware 🙂 but handling it in our code shouldn’t be that difficult either.

        The reason why I was asking about this is because I wrote some state triggers for contextual sensing to help us devs use them more easily in XAML (http://igrali.com/2015/08/13/contextual-activity-and-proximity-state-triggers-in-windows-10-uwp/), and I just couldn’t get the ActivitySensor to show.

        Anyway, looking forward to more updates on this!

  6. Thank for the blog post. I still cannot access the barometer with my Lumia 1020, the API call returns null, will this ever work? Thanks!

    • Sorry Pedro for the very late response. You will need a Windows 10 device with Barometer, the APIs will not work with the Lumia 1020 device.

  7. Is there a resource that describes how the activity detection API infers your activity?

    For example, how does it distinguish walking, cycling, and in-vehicle? What determines a high vs low confidence?

    If not, I’d love to hear more about that.