Building Bing Smart Search for Windows 8.1

This blog post is part of a series of guest posts we’re publishing this week from different people in groups across Microsoft who helped us build Windows 8.1. – Brandon

My name is Kieran Snyder, and I’m a Group Program Manager in the Bing User Experience team. My team was responsible for working hand-in-hand with Windows engineering to build the new Bing Smart Search in Windows 8.1. I’m excited to tell you a little bit about our work with the Windows team to build what is one of the coolest and most visible improvements to Windows 8, and highlight a few of my favorite things, now that it will be broadly available to everyone tomorrow.

Kieran Smart Search

Integration from the Start

The Bing/Windows collaboration on Smart Search for the Windows 8.1 update started with our work on Windows 8 when our teams worked together to build the Bing search app. The focus for that app was a cross-team effort to create a modern, fast and fluid search experience that was great for touch, and took advantage of what the revolutionary Windows 8 platform offered. Both teams brought different perspectives to the table in those first days of collaboration. Bing brought a deep knowledge of core search, including experience with experimentation and agile development, expertise in machine learning and relevance, and specialized knowledge in multimedia. Windows was embracing modern design, and brought the kind of unrelenting attention to detail and craftsmanship that is essential when you’re shipping software to a billion people around the world. And as we worked through that initial project across teams – and I was actually a Group Program Manager in the Windows team at the time – we realized how much more opportunity there was to rethink what search should be and to build something really great for people that brought all the pieces together.

From the start of Windows 8.1 development both the Windows and Bing engineering teams knew that we wanted to go all in on search – together. For me personally, the Windows 8 collaboration brought me back to my old roots in natural language processing and got me hooked on the search space, so when work began on Windows 8.1 I moved to the Bing team to learn everything I could about it. But, to make a cross-company collaborative project work at this scale, you need a truly diverse set of individuals involved. On the Bing side we started building a new team to work on Smart Search that drew from a wide variety of expertise and backgrounds. Our team now includes people with advanced graduate degrees and some straight out of college; folks with varied perspective as designers, developers, researchers and marketers; people with experience in Windows, Office, Internet Explorer, Visual Studio, as well as companies outside of Microsoft and the startup scene. And that’s just our small corner of Bing. All told, nearly 1,000 people in Bing have contributed to Smart Search in some way, including program managers, developers, testers, designers, researchers and our colleagues in business development and marketing.

Rethinking what search should and could be

The promise of creating something together – making a modern search experience that includes the best of Windows search and the best of web search all in one place was an exciting challenge. We knew we wanted to help people find what they needed, no matter where it lived or what it was. We asked ourselves, “Why should someone have to remember where they saved something? What if there is a forgotten note or picture tucked away that can be helpful; how would you know to go look for it? Shouldn’t Windows help you find all of that?” That was the underlying goal with Smart Search – let you simply swipe or type to find everything, all in one place. Whether it is a document on your PC, an app you downloaded months ago, a picture stored in SkyDrive or a website, Bing Smart Search will bring it to the surface. As you’ll see, Smart Search helps you find stuff fast wherever it lives.

A challenge with designing search that works everywhere was reconciling how people have historically searched on Windows versus how they have historically searched on the Web. Consider what we call “query formulation”, which is when you enter your search term in the search box. On the web, when you enter a query in the Bing search box, hitting enter launches a search query for exactly the string you’ve typed – like this:

skydrive

But in the world of Windows when typing “sky” and pressing enter, you expect to launch SkyDrive directly, rather than searching for “sky” and getting a web search results page. It took us many rounds of iteration to make sure the Smart Search experience exactly preserved web search expectations, but didn’t compromise the efficiency of quick commanding in Windows. Notice how typing “sky” in Smart Search surfaces SkyDrive, as well as related Control Panel settings, and more:

Win 8 sky

That is just one example of the challenges we had ahead of us to rethink the search experience for a new and improved Windows experience. We quickly realized that even with a great team and a strong partnership, this wasn’t going to be easy!

Leveraging Satori for the Windows canvas

Within the varied backgrounds of our two teams were also many strengths that have given us the opportunity to look beyond the constraints of a traditional search box and create transformative experiences that are unlike anything you can find in other products. My favorite example of this is a feature we call “search heroes.”

To provide background on the search heroes I want to outline a little context on some pretty cool technology that most people don’t know about. For many years, Bing has invested in Satori, which is essentially a repository of the many people, places, events, and objects in the world – what we call entities – and more importantly the relationships between them. At last count, and this changes due to the constant changing nature of the web, Satori includes hundreds of millions of entities and understand billions of points of information linking them together. Satori provides the foundation for our Snapshot experiences on Bing.com, so that when you search for your favorite celebrity or famous landmark, you get an arrangement of facts, biographical information, images and other key information in addition to the most relevant web results.

Windows 8.1 provides a fantastic, full-bleed canvas where apps can shine, and we realized pretty quickly how the investment made in Satori could be leveraged across this canvas for a Smart Search experience that is beautiful as well as functional. Search heroes take this vast set of structured data that Bing has about the world and turns it into something that feels beautiful, curated, and relevant – something that looks and feels like the best modern apps. Thanks to Satori and Bing, you can think of these experiences as mini-apps that Windows creates on the fly. As you would expect of an experience built for Windows, they include deep links into your Windows applications to help you get stuff done.

Take music, as an example, which has been one of the most popular categories in the Release Preview. With Bing, Windows, and the Xbox team working together, we can present a big, beautiful answer that puts the most relevant information and actions front and center. We are able to include immediate music playback of all the most popular songs in the Xbox Music catalog, serve up music videos, launch apps that can help you go deeper and get more done, and surface other information that is easily accessible with a swipe of your finger. Across Windows, Xbox, and Bing, we built something that none of these teams could have made in isolation.

DAFT Punk

Dynamic and agile

There isn’t a single part of the Smart Search experience that hasn’t been built in collaboration across our teams, including our state of the art image search and our results page that combines relevant results from Windows, apps, the web, and your personal content from SkyDrive. But perhaps the most critical investment that we’ve made together is our update pipeline and flighting service, which allows Bing to deploy new search features and refinements to Windows every single day. Now you can get critical search updates every time you use Windows – versus waiting for periodic updates. With Smart Search we dynamically reflect the ever-changing web, we can learn what’s popular and what’s not, we can try out new experiences to see how well people like them, and we can improve your experience daily. This is the largest-scale investment in server-side development for Windows that Microsoft has ever made, and it took incredible commitment on the part of the Windows team to build this. More than any other part of our collaboration, this is what sets us up to make new, dynamic experiences together for years to come, delivering what people need when they need it.

One Microsoft

Building something together in such a close way is often hard at a big company. From the very beginning of this project, Katie Frigon, my counterpart in Windows, and I have had full commitment to a shared design process. For fifteen months now, every User Interface PM and designer on the project has gotten together for two hours every Friday morning to share work in progress and to critique and discuss together. These design forums are meetings we continue to have as we look to our next set of features and investments. There is just no substitute for having everyone on a project in the same room together week in and week out to make the project work. We hold shared architecture reviews and code reviews for developers. The lead designers on the project recently gave a presentation to user experience leads around the company about this design process so they can share their learnings. Windows and Bing user researchers test each other’s features in the lab and run shared studies. At all levels of the company and across all disciplines, collaboration has been open, effective, and frequent, and I’m thrilled about what this has allowed us to build together and excited about what comes next.

Smart Search is just the beginning of what’s new in Windows 8.1. We can’t wait for you to check it out! If you have Windows 8, (tomorrow) just go to the Windows Store on the Start screen and get the update. If you don’t yet have Windows 8 or are waiting for all the great devices coming this holiday season, you can head to www.bing.com/smartsearch to see it in action and get a feel for what you’re missing.

Kieran Snyder
Group Program Manager
Bing User Experience Team