The commercial media industry is undergoing a major transition as content providers move away from proprietary web plug-in based delivery mechanisms (such as Flash or Silverlight), and replace them with unified plug-in free video players that are based on HTML5 specifications and commercial media encoding capabilities. Browsers are moving away from plug-ins as well, as Chrome is with NPAPI and Microsoft Edge is with ActiveX, and toward more secure extension models.
The transition to plug-in free media has been enabled through the recent development of new specifications:
- From W3C: Media Source Extensions for adaptive streaming and Encrypted Media Extensions for content protection.
- From the Moving Picture Experts Group (MPEG): DASH and Common Encryption (CENC).
These specs were designed and developed to enable interoperable streaming to a variety of media platforms and devices. By focusing on interoperable solutions, content providers are able to reduce costs and at the same time users are able to access the content they want on the device they prefer using the app or web browser of their choice. Microsoft believes that this is a huge benefit to both content producers and consumers, and is committed to supporting companies that make this transition.
This is a long blog, and we don’t want you to miss a topic that interests you. Here’s a glimpse of what we’ll cover:
- Some information on Microsoft Edge and Silverlight
- An overview of interoperable web media.
- Some challenges and options to address them:
- The simplest form of DASH streaming.
- A website demo that uses a library to play Smooth content.
- Services from Azure Media Services that can help.
- A simple method for creating a Universal Windows Platform (UWP) app based on website code.
- A demo UWP that integrates video playback with Cortana voice commands.
Microsoft Edge and Silverlight
Support for ActiveX has been discontinued in Microsoft Edge, and that includes removing support for Silverlight. The reasons for this have been discussed in previous blogs and include the emergence of viable and secure media solutions based on HTML5 extensions. Microsoft continues to support Silverlight, and Silverlight out-of-browser apps can continue to use it. Silverlight will also continue to be supported in Internet Explorer 11, so sites continue to have Silverlight options in Windows 10. At the same time, we encourage companies that are using Silverlight for media to begin the transition to DASH/MSE/CENC/EME based designs and to follow a single, DRM-interoperable encoding work flow enabled by CENC. This represents the most broadly interoperable solution across browsers, platforms, content and devices going forward.
Interoperable Media across Browsers
Plug-ins like Silverlight were intended to support interoperable media by providing versions for every browser they supported. This became more difficult as devices and platforms that support browsers multiplied. Now, as the old plug-in models are being removed, replacements for them are needed. For media, a great forward looking replacement can be based on DASH, MSE, EME and CENC.
Windows 10 and Microsoft Edge support DASH, MSE, EME and CENC natively, and other major browsers ship implementations of MSE and CENC compliant EME. This support allows developers to build plug-in free web video apps that runs across a huge range of platforms and devices, with each MSE/EME implementation built on top of a different media pipeline and DRM provider.
In the days when DRM systems used proprietary file formats and encryption methods, this variation in DRM providers by browser would have presented a significant issue. With the development and use of Common Encryption (CENC), the problem is substantially reduced because the files are compressed in standard formats and encrypted using global industry standards. The service provider issues the keys and licenses necessary to consume the content in a given browser, but the website code, content and encryption keys are common across all of them, regardless of which DRM is in use. An example of such an implementation is DASH.js, the open source industry reference player used to prove out these technologies and which serves as the basis for many players across the web today.
As shown above, Microsoft’s PlayReady DRM supports two modes of DRM protection: “Software DRM”, which uses our traditional software protected media path, and “Hardware DRM”, which moves the media path into secured hardware when available on the device. Hardware DRM was designed to meet the highest requirements for commercial media content protection, and will allow streaming of the highest quality content available. Not all devices will be Hardware DRM capable, but sites built on MSE/EME can accommodate the difference and stream the best possible content quality supported by the browser or device.
Support from Microsoft
Like any new technology, the transition to DASH/MSE/EME/CENC can present some challenges. These include:
- Large existing media libraries have been encoded in formats which are not directly supported by MSE/EME. These libraries must be either supported in some way on the new APIs or re-encoded. Silverlight Smooth Streaming is an example format which was designed and built for sites using Silverlight plug-ins. A solution that could play this content directly would be useful for any Silverlight replacement technology.
- MSE/EME are maturing, but still undergoing change that may present interop challenges working with different media formats and across browsers.
As part of helping the industry move to interoperable media delivery solutions, we are investing in technologies to address these challenges.
DASH Type 1: MSE Made Easy
DASH content usually consists of media files encoded at various quality levels and a manifest that provides information on the files to the media application. An MSE based player is then responsible for parsing these files, downloading the most appropriate content and feeding it into the media element’s sourceBuffer(s). This is very flexible, but requires either investment in authoring site MSE implementations or use of a reference MSE implementation such as the aforementioned DASH.js.
There is another much easier option: native DASH streaming, where the site code simply sets the manifest as the media element source, and playback is automatically managed by the browser’s built-in streaming engine. This approach allows web developers to leverage the learnings and investments made by browser makers and easily provide a premium media experience on their site. We have added support for native DASH streaming to Windows 10 and Microsoft Edge, and more details on this are available in our previous blog: Simplified Adaptive Video Streaming: Announcing support for HLS and DASH in Windows 10.
A number of websites have large media libraries encoded in the Smooth Streaming format and are looking to move to an interoperable HTML5-based solution. One possible solution is to use a library that supports their current content through MSE/EME without re-encoding. Libraries are available now that are capable of playing back Smooth Streaming content using MSE and EME. For example, a version of the “hasplayer.js” library can do just that and is available for download at https://github.com/eiximenis/hasplayer.js.
That makes it very simple to support streaming Smooth content on a website. We’ve provided a sample in the Contoso Video Sample GitHub Repository that uses this library to play video. You can try the webpage yourself at the Contoso Video Demo Website.
It is possible to provide a client-side translation library for Smooth Streaming content because the Protected Interoperable File Format (PIFF) underlying the Smooth Streaming protocol was a primary input into the ISO Base Media File Format (ISOBMFF) spec used with DASH and because PIFF introduced the multi-DRM protocol which was standardized as ISO MPEG Common Encryption (CENC). Two PIFF formats are in broad use today – PIFF 1.1 and PIFF 1.3 – and the hasplayer.js Smooth Streaming MSE/EME library supports both formats. These formats are supported by on-the-fly conversion from the PIFF format to the Common Media Format (CMF) used with DASH. This ensures that all browser content played back by the library is conformant with DASH CMF and capable of playback in all MSE-capable browsers.
Some content owners would prefer to focus on producing great content, not the technical details required to deliver their content to consumers. These companies may wish to leverage hosted media services that prepare content for web delivery, handle the streaming logic and player UI, and manage the DRM license servers. Azure Media Services offers this capability today, with support for both PlayReady and Widevine DRM systems. This service provides support for both Video on Demand (VoD) and live streaming. A single high quality file/stream is provided to Azure, where it undergoes dynamic compression and encryption into CENC protected content that can be streamed to end devices, and a complete player solution is available for developers to simply add to their website. Some details of this service are newly announced in Azure Media Services delivers Widevine encrypted stream by partnering with castLabs.
Hosted Web Apps
One powerful advantage of moving to DASH/MSE/EME/CENC streaming is that the same code running on your website can be packaged as a Universal Windows Platform (UWP) app. UWP applications run on all devices with Windows 10. A website developer can create both an interoperable website based player and a Windows app that uses the same code, but offers a richer experience using Windows platform capabilities. Their common code will thus be able to handle UI and media streaming details, AND take advantage of capabilities only available to apps through WinRT APIs.
These hosted web apps:
- Are offered through the Windows Store
- Are able to interact with Cortana (“Contoso Video play elephants”)
- Are able to offer notifications (“NBA Finals streaming starts in 15 minutes”)
- Have access to advanced adaptive streaming support
- Have access to enhanced content protection for Full HD and Ultra-HD video playback
- Are able to light up live tiles
- And more!
Hosted App Demo
Voice Command Definition (VCD) file
The Voice Command Definition (VCD) File specifies the actions you want to support with voice commands. The code below informs Cortana of the app name (Contoso Video), that the app supports a “play” command, and how the “playing” state should be represented in the app UI.
The HTML File
The HTML file must add a meta element pointing to a VCD file on your server.
The full source for the Contoso Video website and demo app is available in the Contoso Video Sample GitHub Repository.
DASH/MSE/EME/CENC offer compelling advantages over plug-in based solutions, and we are quickly moving towards an era of broadly interoperable streaming media. Both content providers and consumers will benefit from this transition. While the adoption of these technologies may present short-term challenges, the features and options discussed in this blog are provided to assist companies as they make this change.
We’re eager for your feedback so we can further improve our streaming media offerings, and are looking forward to seeing what you do with the tools and approaches we’ve discussed here!
– David Mebane, Program Manager, Windows Media
– Jerry Smith, Senior Program Manager, Microsoft Edge
– Kevin Hill, Senior Program Manager, Microsoft Edge
Updated June 28, 2018 7:30 am