June 5, 2015 12:00 pm

Using FFmpeg in Windows Applications

It is important to Microsoft that developers can leverage open-source software when building apps for Windows. As part of this effort we’re very pleased to announce FFmpeg support for Window 8.1, Windows Phone 8.1, and Windows 10 applications.

What is FFmpeg?
FFmpeg is a free, open-source multimedia framework that includes a set of tools which can be used by end users for transcoding, streaming, and playing, as well as a set of libraries for developers to use in applications. These libraries are valuable to developers as they add support for virtually all media formats including .mkv, .flv, and .mov. Projects that use FFmpeg libraries include Google Chrome, VLC, and many more.

What’s new?
We have released a set of Compilation Instructions for a WinRT compatible build of FFmpeg. Typically, after including FFmpeg libraries, developers have to go through the costly effort of rendering audio and video frames themselves. To solve this issue, we have released an open-source project, FFmpegInterop library for Windows. This project includes the FFmpegInterop WinRT component which processes media with FFmpeg and then leverages the Windows Media Framework for playback. This approach works well because the Windows Media Framework automatically renders and synchronizes media samples while reaping the benefits of hardware acceleration.

To demonstrate this we have also included three sample Media Player applications in C#, C++, and JavaScript.

Compilation Instructions
In order to leverage the powerful and diverse multimedia features of FFmpeg, we have contributed a set of compilation instructions for Windows 8.1, Windows Phone 8.1, and Windows 10 applications. After following the compilation instructions, developers can include FFmpeg libraries in their applications, enabling apps to play or stream almost any video format.

FFmpegInterop Object
FFmpegInterop is a WindowsRuntime Component that produces a MediaStreamSource which can be assigned to a XAML MediaElement or HTML5 video component for playback of any media file format supported by FFmpeg. This FFmpegInterop component can be used as a template to utilize further functionality of the FFmpeg libraries. Moreover the FFmpegInterop object can be accessed by multiple programming languages including C#, CX/C++, and JavaScript.

Media Player Application Samples
Also included with the FFmpegInterop library for Windows are sample Media Player applications in C#, C++, and JavaScript for Windows 8.1, Windows Phone 8.1, and Windows 10. They demonstrate how to achieve the following:

  1. Create a FFmpegInterop object that plugs easily into a XAML MediaElement or WinJS HTML5 Video component
  2. Open a local file using FileOpenPicker and pass it as a stream to FFmpegInterop
  3. Project the API to the different Windows Runtime languages (CX/C++, C#, and JavaScript)

Example: Using the FFmpeg MediaStreamSource Object in a XAML Media Element.

C++

// Instantiate FFmpeg object and pass the stream from opened file
IRandomAccessStream^ readStream = stream.get();
FFmpegMSS = FFmpegInteropMSS::CreateFFmpegInteropMSSFromStream(readStream, false, false);

// Pass MediaStreamSource to Media Element
mediaElement->SetMediaStreamSource(FFmpegMSS->GetMediaStreamSource());

C#

// Instantiate FFmpeg object and pass the stream from opened file
IRandomAccessStream readStream = await file.OpenAsync(FileAccessMode.Read);
FFmpegMSS = FFmpegInteropMSS.CreateFFmpegInteropMSSFromStream(readStream, false, false);

// Pass MediaStreamSource to Media Element
mediaElement.SetMediaStreamSource(FFmpegMSS.GetMediaStreamSource());

To submit feedback, pull-requests, or issues, please check out the GitHub repo: https://github.com/Microsoft/FFmpegInterop

If you have a suggestion for other open source projects that would help you make great applications, please let us know in the comments.

For more information about Microsoft’s involvement with open source check out these sites:

Updated June 11, 2015 8:25 am

Join the conversation

  1. Hi,

    You need to follow the instructions to build FFmpeg. While FFmpegInterop leverages it, you still need to build it yourself.

  2. I’ve followed the instructions from the above link to build FFmpeg, but I’m getting a lot of syntax errors when running make. Compilation fails with error 2 on ffmpeg/libavcodec/dxva2_h264.c with the following errors, to name just a handful:

    C:/ffmpeg/libavcodec/dxva2_h264.c(29): error C2061: syntax error: identifier ‘DXVA_PicParams_H264’
    C:/ffmpeg/libavcodec/dxva2_h264.c(30): error C2061: syntax error: identifier ‘qm’
    C:/ffmpeg/libavcodec/dxva2_h264.c(30): error C2059: syntax error: ‘;’
    C:/ffmpeg/libavcodec/dxva2_h264.c(32): error C2061: syntax error: identifier ‘slice_short’

    I’ve cloned the latest copy of the FFmpeg source from the official GitHub and have not made any changes. I get the same set of errors at the same point in compilation no matter which architecture I choose to compile for. I’m using Windows 10 Build 10130 with Visual Studio Community 2015 RC installed and up-to-date. Compiling libraries for Windows is a little out of my element (I have experience developing WinRT apps with JS and have only minimal C#/C++ knowledge, and am familiar with Java and Android development), so it’s altogether quite possible I’m missing something simple.

    Any guesses off the top of your head about what I’m doing wrong? I’d be happy to provide more details if that would help (all that said and done, I’m really excited about FFmpeg on the Universal Windows platform and am looking forward to using it in a Store app I’m developing for Windows 10).

    • A recent change in FFmpeg has enabled DXVA when DX11 is being used. We are working to update the build instructions to account for this issue. As a workaround, you can go back to an older build of FFmpeg.

  3. Hi
    I was able compile both in Windows 10 and Windows 8.1 Works. But the video is pixelated and audio is garbled. Any idea ? I have tried .wmv and .ts files.
    thanks

    • Is it possible to use the FFmpegInterop library in a windows 7 app? I don’t know if there is only Windows RT stuff in de examples that they give or also in the library itself.

  4. I have a problem. Video starts but after few seconds picture stops, however CurrentState is Playing. Any ideas how to fix it?

  5. I cannot pass certification because of this:
    API getenv in api-ms-win-crt-environment-l1-1-0.dll is not supported for this application type. avutil-54.dll calls this API.
    ◦API CryptAcquireContextA in advapi32.dll is not supported for this application type. avutil-54.dll calls this API.
    ◦API CryptGenRandom in advapi32.dll is not supported for this application type. avutil-54.dll calls this API.
    ◦API CoCreateInstance in ole32.dll is not supported for this application type. avdevice-56.dll calls this API.
    ◦API CoGetMalloc in ole32.dll is not supported for this application type. avdevice-56.dll calls this API.
    ◦API CoInitialize in ole32.dll is not supported for this application type. avdevice-56.dll calls this API.
    ◦API OleLoadFromStream in ole32.dll is not supported for this application type. avdevice-56.dll calls this API.
    ◦API OleSaveToStream in ole32.dll is not supported for this application type. avdevice-56.dll calls this API.
    ◦API BitBlt in gdi32.dll is not supported for this application type. avdevice-56.dll calls this API.
    ◦API CombineRgn in gdi32.dll is not supported for this application type. avdevice-56.dll calls this API.
    ◦API CreateCompatibleDC in gdi32.dll is not supported for this application type. avdevice-56.dll calls this API.
    ◦API CreateDIBSection in gdi32.dll is not supported for this application type. avdevice-56.dll calls this API.
    ◦API CreateRectRgn in gdi32.dll is not supported for this application type. avdevice-56.dll calls this API.
    ◦API DeleteDC in gdi32.dll is not supported for this application type. avdevice-56.dll calls this API.
    ◦API DeleteObject in gdi32.dll is not supported for this application type. avdevice-56.dll calls this API.
    ◦API GetDIBColorTable in gdi32.dll is not supported for this application type. avdevice-56.dll calls this API.
    ◦API GetDeviceCaps in gdi32.dll is not supported for this application type. avdevice-56.dll calls this API.
    ◦API GetObjectA in gdi32.dll is not supported for this application type. avdevice-56.dll calls this API.
    ◦API GetStockObject in gdi32.dll is not supported for this application type. avdevice-56.dll calls this API.
    ◦API SelectObject in gdi32.dll is not supported for this application type. avdevice-56.dll calls this API.
    ◦API capCreateCaptureWindowA in avicap32.dll is not supported for this application type. avdevice-56.dll calls this API.
    ◦API capGetDriverDescriptionA in avicap32.dll is not supported for this application type. avdevice-56.dll calls this API.
    ◦API AdjustWindowRectEx in user32.dll is not supported for this application type. avdevice-56.dll calls this API.
    ◦API BeginPaint in user32.dll is not supported for this application type. avdevice-56.dll calls this API.
    ◦API CopyIcon in user32.dll is not supported for this application type. avdevice-56.dll calls this API.
    ◦API CreateWindowExA in user32.dll is not supported for this application type. avdevice-56.dll calls this API.
    ◦API DefWindowProcA in user32.dll is not supported for this application type. avdevice-56.dll calls this API.
    ◦API DestroyCursor in user32.dll is not supported for this application type. avdevice-56.dll calls this API.
    ◦API DestroyWindow in user32.dll is not supported for this application type. avdevice-56.dll calls this API.
    ◦API DispatchMessageA in user32.dll is not supported for this application type. avdevice-56.dll calls this API.
    ◦API DrawIcon in user32.dll is not supported for this application type. avdevice-56.dll calls this API.
    ◦API EndPaint in user32.dll is not supported for this application type. avdevice-56.dll calls this API.
    ◦API FindWindowA in user32.dll is not supported for this application type. avdevice-56.dll calls this API.
    ◦API FrameRect in user32.dll is not supported for this application type. avdevice-56.dll calls this API.
    ◦API GetClientRect in user32.dll is not supported for this application type. avdevice-56.dll calls this API.
    ◦API GetCursorInfo in user32.dll is not supported for this application type. avdevice-56.dll calls this API.
    ◦API GetDC in user32.dll is not supported for this application type. avdevice-56.dll calls this API.
    ◦API GetIconInfo in user32.dll is not supported for this application type. avdevice-56.dll calls this API.
    ◦API GetSystemMetrics in user32.dll is not supported for this application type. avdevice-56.dll calls this API.
    ◦API GetWindowLongPtrA in user32.dll is not supported for this application type. avdevice-56.dll calls this API.
    ◦API GetWindowRect in user32.dll is not supported for this application type. avdevice-56.dll calls this API.
    ◦API LoadCursorA in user32.dll is not supported for this application type. avdevice-56.dll calls this API.
    ◦API PeekMessageA in user32.dll is not supported for this application type. avdevice-56.dll calls this API.
    ◦API ReleaseDC in user32.dll is not supported for this application type. avdevice-56.dll calls this API.
    ◦API SendMessageA in user32.dll is not supported for this application type. avdevice-56.dll calls this API.
    ◦API SetWindowLongPtrA in user32.dll is not supported for this application type. avdevice-56.dll calls this API.
    ◦API SetWindowRgn in user32.dll is not supported for this application type. avdevice-56.dll calls this API.
    ◦API ShowWindow in user32.dll is not supported for this application type. avdevice-56.dll calls this API.
    ◦API OleCreatePropertyFrame in oleaut32.dll is not supported for this application type. avdevice-56.dll calls this API.
    ◦API SHCreateStreamOnFileA in shlwapi.dll is not supported for this application type. avdevice-56.dll calls this API.
    ◦API getenv in api-ms-win-crt-environment-l1-1-0.dll is not supported for this application type. avformat-56.dll calls this API.

  6. Hi, I successfully build ffmpeg for ffmpeginterop and I added reference nodes to the dll in the .csproj file, however when I run my project as soon as FFmpegInteropMSS.CreateFFmpegInteropMSSFromStream(avStream, false, false) is executed I’m hit with a System.IO.FileNotFound exception. I did paste the ffmpeg dlls with the ffmpeginterop and also tried in the project dir with copy to output, no change. Any ideas would be greatly appreciated.

    Thaanks

  7. Nice job, can the MediaPlayerJS run on a webserver like IIS or AZURE so that the website with the FFmpeg-player can be run from anywhere and on any client-browser (Chrome, IE , Safari) and OS?

    Thanks, Adri

  8. can I use ffmpeginterop to record from an ip camera, and save to video file the streaming, do you know how?, thanks in advance.