Libraries Under the Hood

Libraries Under the Hood

  • Comments 9
  • Likes

By now, you should have a better understanding of what Windows 7 libraries represent and how they can benefit users as well as developers. This is the third post in a series of Windows 7 Libraries posts. In the previous post, Understanding Windows 7 Libraries, we explained what libraries are and the important role they play in users’ day-to-day work. In this post, we explain the underlying architecture that supports libraries.

As usual, let’s start with a quick historic review of Libraries' ancestors. Windows Vista introduced a new storage scenario and a new user profile namespace called the “Known Folders.” This represented the next evolutionary step from “standard folders” that were based on a constant special item ID list (CSIDL). The CSIDL values provide a system-independent way to identify special folders used frequently by applications. In Windows Vista, we replaced these values with the Known Folders system where a KNOWNFOLDERID constant identifies each special folder.

Both CSIDL and Known Folders included support for users’ specific content types such as Document, Music, Pictures and Videos – these folders are part of the user storage profile. In Windows 7, Libraries extend some of the common user Known Folders, like My Documents, into the Documents Library. If you review the KNOWNFOLDERID in Windows 7 Beta SDK, you will find that the default Libraries are Known Folders themselves, which make sense since we want a unified programmatic way to access user’s content. By default, each Library contains two physical file locations:

  • The user's personal folder by type (Documents, Pictures, Video…), which is represented by the default Known Folder that is included in a Library (that is, My Documents in the Documents Library), and it is also the default save location.
  • The public folder by type

For custom Libraries, the default save location is the first folder added.

In some ways, a Library is similar to a folder. From a user's point of view, a library looks and behaves just like a regular folder. However, unlike a folder, a Library gathers files that are stored in several locations into a single view. This is a subtle, but important difference. Libraries don't actually store users’ items, they only monitor folders that contain items, and let users access and arrange these items in different ways. They also let users and developers access the list of folders and their contents. For example, if you try to save a text document (from Notepad) directly to the Documents Library, the file will be saved. However, we just said that Libraries are NOT regular folders that contain files, so what’s going on? Well, the text file is saved to the Documents Library default save location, which is, by default, the user’s Documents folder – also known as “My Documents” and is part of the user profile. The user has full control over the default save location as well as the other locations that are part of a library. As shown in the next figure, the Picture Library management dialog (which is accessible directly from the Windows Explorer UI) contains several locations on my local hard drive with one of the folders designated as the default save location.

Library Management Dialog

It is important to note that the Pictures Library didn’t replace the My Pictures folder from Windows Vista; it simply includes that folder in the Pictures library as one of many locations. Also, please note that the CSIDL system and its APIs system are still supported for application compatibility. However, we do not recommend that you use them in new application development.We recommend that you use the known folder system instead

A Library in Windows 7 is stored as an XML definition file that has a file extension of “.library-ms.” The file name is the actual library’s name. For example, the Documents Library is represented by an XML file called Documents.library-ms. Library descriptions are saved on disk in the %appdata% Roaming\Microsoft\Windows\Libraries folder (also known as FOLDERID_Libraries). You can use the Library definition files to monitor changes to the library as we will explain in future posts

Let’s dig into the Documents Library definition file schema. The XML structure is pretty self-explanatory, but let’s explain a few of its elements. At the top of the file, we can find the Library “header” information:

<libraryDescription xmlns="http://schemas.microsoft.com/windows/2009/library">
<name>@shell32.dll,-34575</name>
<ownerSID>S-1-5-21-2127521184-1604012920-1887927527-4897363</ownerSID>
<version>4</version>
<isLibraryPinned>true</isLibraryPinned>
<iconReference>imageres.dll,-1002</iconReference>
<templateInfo>
<folderType>{7D49D726-3C21-4F05-99AA-FDC2C9474656}</folderType>
</templateInfo>

The root XML element is libraryDescription, which contains child elements that define the library:

  • <ownerSID> is an element that defines the Security ID of the user who created this library to isolate Libraries and protect user data from other users
  • <isLibraryPinned> is a Boolean element that defines if the library is pinned to the left navigation pane in Windows Explorer and NOT to the Taskbar
  • <version> defines the content version of this library, which reflects the number of times the library definition file has been changed
  • <templateInfo> is an optional container element that lets the author specify the folder type (Documents, Pictures, Video) for controlling the arrangements of views in Windows Explorer
  • <iconReference> defines an icon resource using standard Windows Shell resource style; for example: <iconReference> C:\Windows\system32\imageres.dll,-65 </iconReference>. This icon presents the library in the Windows Explorer

Users can’t change the default libraries icon via Windows Explorer, nor can they assign the icon to a new customized user-defined library. However, this can be done programmatically using the API, which we will cover in future posts.

A key part of the XML is the list of locations that the Library represents:

<searchConnectorDescriptionList>
<searchConnectorDescription publisher="Microsoft" product="Windows">
<description>@shell32.dll,-34577</description>
<isDefaultSaveLocation>true</isDefaultSaveLocation>
<isSupported>true</isSupported>
<simpleLocation>
<url>knownfolder:{FDD39AD0-238F-46AF-ADB4-6C85480369C7}</url>
<serialized>MBAAAE…. </serialized>
</simpleLocation>
</searchConnectorDescription>
  • <searchConnectorDescriptionList> contains one or more search connectors that map to physical locations included in this library
  • <searchConnectorDescription> contains a <simpleLocation> element that describes one location included in the library
  • <url> defines a URL for this location and it is there for human-readability only, this cannot be used by developers since it is likely to be out of date.
  • <serialized> this is the actual representation (Base64) of a location in a library which is as a serialized ShellLink object

One final note - Applications should never attempt to access or edit Library description files. Instead, applications should always use the Shell programming model or the IShellLibrary API to consume and manipulate Library contents.

In the next post we will dive into the Shell programming model.

9 Comments
You must be logged in to comment. Sign in or Join Now
  • Great stuff. I really like your writing style.

  • You wrote very interesting post i love your blog it contains very informative article i will share it with my friends.

  • Hello Yochay,

    How can I know if an item (let's say a file) inside a library was added, removed or changed?

    Thank you,

    Javier Andrés Cáceres Alvis

    Blog Personal: speechflow.spaces.live.com

    Blog Intel: software.intel.com/.../javierandrescaceres

  • Duane Duane
    52463 Posts

    with images and code you provided when you were trying to do something but could not manage the information we receive from you and then we will do thanks

  • dwashin
    2 Posts

    @MaryBranscombe

    One option is that you can press Win+2 to launch Libraries since it is the second option in the taskbar.

    You make a good point about supported locations. Even though users can not directly add unindexable locations to libraries developers can use the IShellLibrary API to add these add these folders to library. This has the consequence of putting the library in "safe mode" with reduced functionality. Look for more detail on this in Yochay's later posts.

    I want to reiterate Yochay's comment that users and developers should never edit the library description XML directly.

  • conversely, I'm a huge library fan and what I cannot yet find is how to redirect the Explorer Winkey-E shortcut back to libraries instead of my computer; can this be done in the registry or through GPO?

    Also, the XML suggests libraries can be deployed programmatically/through scripts; what happens if an XML file points at a library that the user cannot add manually (ie one that gives the Cannot add library because it cannot be indexed error?)

  • 7flavor
    352 Posts

    "Users can’t change the default libraries icon via Windows Explorer, nor can they assign the icon to a new customized user-defined library." Why can't this minor behavior change by RTM? Can you allow the icon of any library to be changed/customized by the user from Windows Explorer instead of editing the XML file programmatically?

  • Tommyinoz
    46 Posts

    I'm afraid I do not fully understand the point of this feature.  What can a Windows library do that a symbolic link can't?

    As an example, can I not just create a symbolic link pointing to a public folder from the users My Documents folder?  Would this not resolve issues like requiring new software that is 'Windows library' aware etc?  

  • PRab
    3 Posts

    As much as I have liked the Windows 7 beta, this one feature that I do not support. It seems to blur the filesystem in unnecessary ways. See my comment from "Understanding Windows 7 Libraries". As I expected, Libraries did add another layer of complexity not found in any previous version of windows. If I want to make a simple program such as "ls" or "dir" I now have to read an XML(indirectly) to make it fully match the functionality of windows explorer.

    I can't wait to get a call from my mom when some of her pictures are missing because she unplugged an external harddrive that was part of her Pictures library. (Side note: What happens if the drive letter changes?)