Aligning sensors with your app’s orientation

The sensors supported by Windows Store apps offer a powerful way to spice up your app’s functionality. Sensors let your app know the relationship between the device and the physical world around it; the direction, orientation, and movement in those realms. These sensors can help make your game, augmented reality app, or utility app more useful and interactive by providing a unique form of input such as using the motion of the tablet to arrange the characters on the screen or to simulate the user being in a cockpit with the device as the steering wheel.

In this blog post I’ll review the supported sensors and offer some guidelines and suggestions for their best use. Devices are now much more varied in size, shape, and native orientation, and making a bad assumption about how sensors behave in various situations could cause negative reviews for your otherwise killer app. Check out the Sensors Overview video to learn more.


Download this video to view it in your favorite media player:
High quality MP4 | Lower quality MP4

For a sample that uses the logic contained in this blog post, see the Display Orientation sample. For more info on choosing the best sensor for your app, see the Choosing the right sensor blog post.

Display orientation vs. device orientation

Think of display orientation as the direction the pictures on the screen face whereas device orientation describes the physical positioning of the device. Display orientation doesn’t necessarily conform to device orientation. Take, for example, this device, where the arrow represents the display orientation:

The device orientation is in landscape as well as the display orientation

In this picture the device orientation is in landscape as well as the display orientation. However, take these two:

Both tablets have the same device orientation (LandscapeFlipped) but different display orientations

Both tablets have the same device orientation (LandscapeFlipped) but different display orientations. The tablet on the left has display orientation in LandscapeFlipped while the tablet on the right remains in landscape.

Sensor-related apps that match their display orientation to the device orientation like the left-hand example above must validate their sensor data before using it. For instance, if the display orientation LandscapeFlipped, then the accelerometer data will also be flipped, and you’ll need to negate the data to get appropriate values. Understanding these conversions are simple after you understand how reference axes work.

Manipulating reference axes based on display orientation

When manufacturers integrate components into PCs, they must do so in a unified and consistent way so that everyone operates within the same reference frame.

We use reference axes to define how sensors report their data. The device’s landscape orientation, not current orientation, sets these axes as seen below with natively landscape and natively portrait devices:

Tablet with landscape orientationTablet with portrait orientation

Current orientation relatively defines the horizontal and vertical axes. Thus, in the above picture the device has +X projecting from the right while in landscape orientation, but have –X projecting from the right while in LandscapeFlipped (pictured below).

You can always query these values through DisplayInformation by using the method GetForCurrentView with the properties CurrentOrientation, then create logic by comparing against the DisplayOrientations enumeration. Remember that for every orientation you support, you have to support a conversion of the reference axes to that orientation.

You can use this chart to understand how the reference axes translate for each display orientation. Notice that the Z axis stays constant while the X and Y axes often switch and require negation in order to establish new relative reference axes based on a transform of the absolute reference axes.

Table showing tablets in various orientations

Display orientation and accelerometer and gyrometer

Now to start applying your understanding of display orientation and reference axes to get to correct sensor data: let’s start with the accelerometer and gyrometer. Because both the accelerometer and gyrometer map directly to the X, Y, and Z axes, you can transform these in a straightforward manner. Simply use the chart above and draw +X coming out of the right of the device, +Y coming out of the top of the device, and +Z coming out of the front of the device and compare against the absolute reference axes.

Device orientation table

Accelerometer and gyroscope thus follow the same basic logic in mapping. If your app needed to apply these conversions to the gyrometer, for instance, use the below code:

C#:

private void ReadingChanged(object sender, GyrometerReadingChangedEventArgs e)
{
GyrometerReading reading = e.Reading;

// Calculate the gyrometer axes based on
// the current display orientation.
DisplayInformation displayInfo = DisplayInformation.GetForCurrentView();
switch (displayInfo.CurrentOrientation)
{
case DisplayOrientations.Landscape:
x_Axis = reading.AngularVelocityX;
y_Axis = reading.AngularVelocityY;
z_Axis = reading.AngularVelocityZ;
break;
case DisplayOrientations.Portrait:
x_Axis = reading.AngularVelocityY;
y_Axis = -1 * reading.AngularVelocityX;
z_Axis = reading.AngularVelocityZ;
break;
case DisplayOrientations.LandscapeFlipped:
x_Axis = -1 * reading.AngularVelocityX;
y_Axis = -1 * reading.AngularVelocityY;
z_Axis = reading.AngularVelocityZ;
break;
case DisplayOrientations.PortraitFlipped:
x_Axis = -1 * reading.AngularVelocityY;
y_Axis = reading.AngularVelocityX;
z_Axis = reading.AngularVelocityZ;
break;
}


// Update the UI...
}
JavaScript:
function readingChanged(e) {
var reading = e.reading;
var displayOffset;

// Calculate the gyrometer axes based on
// the current display orientation.
var displayInfo = Windows.Graphics.Display.DisplayInformation.getForCurrentView();
switch (displayInfo.currentOrientation) {
case Windows.Graphics.Display.DisplayOrientations.landscape:
x_Axis = reading.angularVelocityX;
y_Axis = reading.angularVelocityY;
z_Axis = reading.angularVelocityZ;
break;
case Windows.Graphics.Display.DisplayOrientations.portrait:
x_Axis = reading.angularVelocityY;
y_Axis = -1 * reading.angularVelocityX;
z_Axis = reading.angularVelocityZ;
break;
case Windows.Graphics.Display.DisplayOrientations.landscapeFlipped:
x_Axis = -1 * reading.angularVelocityX;
y_Axis = -1 * reading.angularVelocityY;
z_Axis = reading.angularVelocityZ;
break;
case Windows.Graphics.Display.DisplayOrientations.portraitFlipped:
x_Axis = -1 * reading.angularVelocityY;
y_Axis = reading.angularVelocityX;
z_Axis = reading.angularVelocityZ;
break;
}

// Update the UI...
}

Display orientation and compass heading

Compass heading returns 2D data by projecting 3D magnetometer data onto a 2D plane, making the transform slightly trickier than for the sensors mentioned earlier. You need to compensate the returned heading with your display orientation or your heading will face the wrong direction:

landscape display table

The API compass heading must be modified as shown in this table to correctly display the heading:

C#:

private void ReadingChanged(object sender, CompassReadingChangedEventArgs e)
{
double heading = e.Reading.HeadingMagneticNorth;
double displayOffset;

// Calculate the compass heading offset based on
// the current display orientation.
DisplayInformation displayInfo = DisplayInformation.GetForCurrentView();

switch (displayInfo.CurrentOrientation)
{
case DisplayOrientations.Landscape:
displayOffset = 0;
break;
case DisplayOrientations.Portrait:
displayOffset = 270;
break;
case DisplayOrientations.LandscapeFlipped:
displayOffset = 180;
break;
case DisplayOrientations.PortraitFlipped:
displayOffset = 90;
break;
}


double displayCompensatedHeading = (heading + displayOffset) % 360;

// Update the UI...
}
JavaScript:
function readingChanged(e) {
var heading = e.reading.headingMagneticNorth;
var displayOffset;

// Calculate the compass heading offset based on
// the current display orientation.
var displayInfo = Windows.Graphics.Display.DisplayInformation.getForCurrentView();

switch (displayInfo.currentOrientation) {
case Windows.Graphics.Display.DisplayOrientations.landscape:
displayOffset = 0;
break;
case Windows.Graphics.Display.DisplayOrientations.portrait:
displayOffset = 270;
break;
case Windows.Graphics.Display.DisplayOrientations.landscapeFlipped:
displayOffset = 180;
break;
case Windows.Graphics.Display.DisplayOrientations.portraitFlipped:
displayOffset = 90;
break;
}

var displayCompensatedHeading = (heading + displayOffset) % 360;

// Update the UI...
}

Display orientation and inclinometer

While the inclinometer sensor provides robust data in an easy to understand format, the nature of its bounds and discontinuities make it considerably harder to mathematically manipulate than other sensor data. To make your life easier, I recommend you support only landscape display orientation when using the inclinometer in your app. You can do this by setting only setting landscape in the InitialRotationPreference in the app’s manifest.

Display orientation and orientation sensor

The orientation sensor uses advanced mathematical constructs (quaternion and rotation matrix) to represent the absolute physical orientation of the device in space. The math for its transformation, while somewhat straightforward, isn’t easy to visualize or conceptualize. Keep in mind that these different orientations can be thought of as rotations counterclockwise to the Z axis, so we need to do the reverse of the rotation to get back the user’s orientation. For quaternion data, we can use Euler’s formula to define a rotation with a reference quaternion and use a reference rotation matrix as well:

Euler’s formula to define a rotation with a reference quaternion

A reference rotation matrix

To obtain the desired relative orientation, you must multiply the reference object against the absolute object (Note: this math is not commutative):

Multiply the reference object against the absolute object

where the absolute object is returned by the sensor data.

display orientation table

Wrapping up

The wide array of sensors supported by Windows 8.1 Store apps offers you lots of options for making your apps more interactive. As more and more devices of various sizes become available, keep in mind that readings they get from sensors are rationalized against the display orientation and device orientation. By following the guidelines outlined in this blog post, your app will be able to run elegantly on any device and in any orientation.

–Brian Rockwell, Program Manager for Sensors on Windows