In this post I’ll discuss how you can optimize performance, cost, flexibility, and resiliency when building a Windows 8 storage solution. Windows 8 provides two new storage capabilities that enable optimized storage solutions based on these considerations: Storage Spaces and File History. I won’t get into the details of File History in this post, but you can read my complete write-up here. In this post I’ll use my recent video editing PC overhaul as a case study for how to build a cost-efficient high performance storage solution on Windows 8.
The 8 drives used in “VIDEO2” a PC that I recently rebuilt
As a creative professional, I depend on high-performance hardware and apps to get my work done. When working with apps like Adobe After Effects and Adobe Premiere Pro, the storage demands are staggering. A typical 1080p uncompressed After Effects animation render can easily occupy 100GB of disk space or more. Working with video is also very demanding for storage. For some projects, I’ll store 10 or more video clips at several Gigabytes each, and then output several renders in different formats. These apps also use “Scratch Disks” to store project data that can’t be stored in RAM when previewing, rendering, and for other similar tasks. Scratch disks need to have super-fast I/O to keep up with the demands of these dynamic tasks. Of course, all of my project, media, documents and other files need to be backed up. I want to minimize the risk of losing data and I want my system to have excellent performance fundamentals.
Before I discuss how I implemented my Windows 8 storage solution, let’s first take a look at the factors that influenced this design.
Here are the considerations that I have for a complete storage solution on my PC:
- Performance
- Extensibility
- Resiliency
- Data history
- Configurability
- Cost
To frame the design discussion, I’ll use an app that represents an “extreme scenario” for storage as a use-case. Let’s consider how Adobe Premiere Pro CS6 uses storage when launching, editing, and rendering files:
- Premiere Pro loads application files from disk into memory (app launch)
- Data is read from and written to a “scratch disk” during previews and renders
- Source media files are read from
- Output files are written to
Adobe documentation provides helpful guidelines for system hardware configuration when running programs like Premiere Pro CS6 and After Effects CS6. From these guidelines and with knowledge about Windows storage technology, I created the following storage plan for my high performance PC:
Video editing file interactions shown at a high level using Premiere Pro CS6 as an example
After landing on this high-level storage layout plan, I did some thinking considering the following:
- Existing storage hardware I had on hand
- Requirements and operating parameters for each logical storage device in my plan
- Budget and options for additional storage hardware
Here are the simplified high-level requirements and parameters for each logical storage device:
Logical storage devices (rows) and corresponding storage requirements (columns)
From this high-level planning exercise, I felt prepared for hardware selection and storage configuration in Windows 8. I’ll summarize here how I implemented each logical storage device.
System Storage
As I evaluated the Windows/apps logical storage device, the primary concern was performance. This storage device needed to handle both random reads/writes and sequential reads/writes really fast. Resiliency was not a primary concern since I knew I could easily reinstall Windows and my apps in the event of a hardware issue. The solution was relatively simple: this logical storage device would be a straight-up SSD drive.
A 240GB SSD was selected for OS and apps
Scratch
The app scratch drive would need to offer high performance for both random and sequential reads/writes. This scratch storage would not need any form of resiliency since the files stored would all be temp files. An SSD would be a great option for this logical storage device, but I didn’t have an extra SSD on hand. What I ended up using was the two Western Digital 10,000 RPM drives that I had been running in RAID configuration in the Windows 7 incarnation of the PC known as “VIDEO2”. This time I would use these drives differently. Windows 8 Storage Spaces would offer RAID-like performance, but would be much easier to setup. I created the scratch Storage Space, and did not use resiliency so as to optimize write performance.
With just a few clicks the scratch Storage Space was configured and ready for use
The creation of this scratch Storage Space would be a test of sorts, and some simple tests with Adobe Premiere Pro CS6 would tell me quickly whether this scratch Space was going to offer adequate performance when previewing and rendering. If there were any issues, I knew it would be super-easy to delete this Storage Space, and I could then repurpose these drives and obtain an SSD for scratch storage. You have to read the rest of the article to find out what happens. Naming this Storage Space “Scratch Space” made it easy to find from other apps when configuring storage options.
Project Files (Bulk Storage)
For project files there were a few considerations. First, I needed a large extensible space to store a lot of really big files. Second, I absolutely needed a form of resiliency to ensure these files would never be vulnerable to hardware failure. Third, this space would need to keep up with the sequential read demands from various apps (Premiere Pro CS6 for the use case presented in this post). The solution was to create a Storage Space with two-way mirrored resiliency. This would offer RAID-like performance, resiliency, and extensibility. With Storage Spaces you can easily add storage at any time, so it’s a great way to implement a single storage repository for large quantities of data. I was excited about the idea that I wouldn’t have to split up my files between drives, all of my important data could live on this single extensible Storage Space.
Creating the Storage Space for bulk file storage
With this bulk Storage Space, I would always have redundant copies of my data. This would change my strategy for backups. Bulk backups would now be a mitigation for a fire, flood, and other natural disasters. Using File History to protect my primary documents would give me a history for my data and offer another form of protection. Finally, SkyDrive would add another form of redundancy for my files. I was feeling good knowing that my data would be safer than ever.
I chose to use a total of four inexpensive Seagate Barracuda 7200 RPM drives for larger logical storage devices
Just like with my scratch Storage Space, I would need to perform some simple tests to confirm that sequential reads wouldn’t be a bottleneck for video editing performance with this bulk storage Space. With RAID-like read performance for this multi-drive setup, I wasn’t too worried.
Utility and Renders
The final logical storage device was targeted at utility and storage of output renders. Since these files would be stored temporarily, this logical storage device would not need any kind of resiliency. The main concern for this space would be sequential write performance. I chose to go with the simple solution of dedicating a single Seagate Barracuda 2TB drive for this space.
Here’s an updated diagram that shows the entire storage solution as implemented on VIDEO2:
This storage implementation was quick and easy to setup, but I was really wondering how it would perform!
Here’s how I defined my “minimum bar” when using Adobe Premiere Pro CS6:
- Smooth 30fps previews of 1080p video files taken directly from my cameras (AVCHD and H.264) in the program monitor at full resolution, full quality
- Acceptable smooth playback of 4K REDCODE RAW files taken directly from RED cameras at lower quality and lower resolution.
Output render performance was also a consideration, but this was second to preview performance since it has no impact on the editing experience.
What I found was that the entire system (hardware and software) met these criteria with no issues whatsoever. I used FRAPS to check my preview performance both with 1080p HD files, and with 4K REDCODE RAW files. In both cases I had some effects applied that I use frequently: brightness/contrast adjustment, sharpening, and the “fast color corrector”.
Premiere Pro CS6 previewing a 30fps AVCHD file at 100% magnification and full quality
Premiere Pro CS6 previewing a 24fps 4K REDCODE RAW file at 25% magnification and 1/4 quality
These quick tests confirmed that I would have more than acceptable editing workflow for both 1080p files and 4K files on VIDEO2 wen running Adobe Premiere Pro CS6.
This was great, but I was curious to know if this setup was going to perform better from a storage perspective than an all-on-one-SSD storage setup that would represent a simple “default” storage scenario.
As a storage test, I rendered a 4-minute uncompressed AVI clip with two storage configurations:
- Everything on the SSD (OS/apps, scratch, source and output files)
- Optimized storage configuration (as outlined in this write-up)
What I found was that my optimized storage solution performed better than an all-on-SSD configuration. Here are the results of my quick test:
- Optimized storage render: 41 seconds
- All-on-SSD storage render: 57 seconds
SUCCESS! My storage overhaul for VIDEO2 is a huge improvement over the storage setup I had been running before the overhaul. Now I don’t need to worry about losing data at any time- I have mirrored resiliency. When I need more storage (it’s not if, but when) I can easily add more drives to my bulk Storage Space. With File History I can selectively store a history for files that are important to me. This system also boots super-fast, and launches apps super-fast. Finally, this storage setup didn’t break the bank.
I’m happy.
Want to know more about Storage Spaces in Windows 8? Here’s a great Storage Spaces FAQ that provides a lot of great insights and information.
Stay up to date, follow me on Twitter!