WRITTEN BY JEFF SHIFFMAN, CO-OWNER OF BOOM BOX POST
One challenge any modern business owner will inevitably face is data sharing and storage. It's not sexy. In fact, it's super unsexy. However big data is a fact of life and in post-production we collect A LOT of data. So I'm going to throw caution to the wind and dive right in. At Boom Box Post we've gone through a few iterations. Here's how (through trial and error) we solved our big data issues. As a post sound facility, our needs fall into three categories:
Library
Our ever growing library of original sound effects needs to be accessible to all editors while working in house.
Drop Share
Often times we need a way to quickly share individual sounds, sound effects builds, picture and even software among two or more workstations in the studio.
Archives
In addition to simply needing to keep all of our work safely archived, we need the ability to share entire sessions of completed past work so editors can reference existing materials and pull any reoccurring elements.
Attempt #1: Clunky and Inefficient
Our first method of storing and sharing data was simply stacks of external hard drives. Each large project received a 2TB hard drive and would hold all necessary data for the show, essentially serving as our archives. My go-to drives are the Mercury Elite Pro Mini from Other World Computing. We would pass these drives around as needed for referencing past work. In terms of a library solution, every edit room had an identical Library drive permanently attached to each workstation. Drop share was the most disorganized. Everyone just grabbed the nearest drive for quick transfers. I should clarify. This was less of a method for storage and sharing and more of a default m.o. as the company got under way. It's this mess that drove us toward finding smarter solutions.
The Good: Simple and cheap. Requires no brain power to set up.
The Bad: Passing hard drives around is a stopgap solution. Besides the evident danger of a single drive failure, you're asking for an organizational mess when running your projects through that many hands. These same drives inevitably became a mess with all the random drop share data. And what happens when there are multiple needs for the drive at once? As for our library, a big part of our culture here at Boom Box Post is the constant creation and sharing of new sound effects so the library drives were constantly in need of revising to keep each workstation up to date.
Conclusion: This approach is one (small) step above data chaos (I may have just coined a new term!). I would only recommend this method if you need to get up and running immediately and have a very small team. Regardless, it won't be long until you're looking for a better solution.
Attempt #2: Getting the Network Involved
For our second try, we decided to centralize our library and drop share. With its high storage capacity and easy setup, the Apple Aiport Extreme Time Capsule seemed like a good solution. We were able to fit our entire library onto the 3TB version, running us roughly $400. Using Soundminer HD+, we simply scanned the entire library folder into a database and crossed our fingers. As long as the drive was mounted on a workstation, it worked like a charm. The databases are easily updated and shared amongst all user's workstations, making universal library updates a snap.
A digression about Soundminer
For years I worked in the Pro Tools workspace window, ignorant to the wonders of Soundminer. I tell any editor that asks to be careful; Soundminer is dangerous. It's the type of program so essential that if you try working with it even once, you'll have to purchase it. That's how essential it is for agile sound editorial. In our offices, all of our databases are built in Soundminer. With a few versions on the market, I had my fingers crossed that Soundminer HD Plus (the most affordable version that includes the spot to DAW feature) would work by referencing a network drive. Bonus: Once you scan your database, you can share that same database file, making updates a breeze. Just navagate to /Library/SMData/Databases and drop the database file on each workstation.
For the drop share, we added into the mix was a spare hard drive, connected via USB. The Airport treats the external drive as an extension of itself, easy to navigate to. It should be noted, this method did not have a solution for our archiving needs, so we ended up sharing larger sessions piece meal via this same external hard drive.
There were days when Soundminer searches were rock solid and super fast, but more often than not we got the beach ball for at least a second or two on each search. When you're searching hundreds of times a day, this small lag time can really add up. Ultimately we weren't able to prove the problem, but suspect it had something to do with the Airport Extreme's internal drive (which was not solid state) needing to spin up every time it was pinged by one of our users. In addition, the USB external drive kept dropping offline and the only fix was to physically power it off and on again. We even had a drive die on us entirely after only a few weeks of use.
The Good: Reasonable price point with included 3TB hard drive. Easy to set up, easily discoverable by our macs.
The Bad: Linking our library back to the Airport Extreme via the Soundminer software was hit or miss. Unreliable read speeds. External USB hard drive prone to dropping offline.
Conclusion: For the budget conscious on a small scale, this isn't a terrible solution. We had no idea if our Soundminer software would function referring back to a networked drive like this. We basically faked a server and with pretty good success. If your traffic needs are low, this could be a low cost work around. In the end however, inconsistency and instability ultimately sunk this system for us. Clearly the Airport Extreme's internal drive and USB pipeline wasn't built to handle this kind of consistent traffic. After the Airport itself started needing constant reboots, we decided it was time to search for another solution.
Attempt #3: Investing in the Future
After you go through two failed configurations, you start to scratch your head and google a lot. In my case, I searched for 'top NAS devices 2015.' I had some personal experience with Network Attached Storage in my home (A Drobo that I'm hot or cold on depending on what day you ask) so I knew a little bit about these devices. I decided to look into one as a server solution for our post production needs because I knew you could fill them with hard drives that were built to run 24/7; namely the Western Digital RED drives. This, I figured would be a solution to all our dropped network issues. My google searched turned up two models by Synology, a company I hadn't heard of. Upon further research I found nothing but glowing reviews and youtube videos extolling the many virtues of their products.
This is the part of the post where I get down into the specifics of our Synology DS1515+. However, before I get too deep, you should know that the CNET Best Network Attached Storage of 2015 article was already out of date when I found it. You couldn't even find it on the Synology website. These devices seem to be updating faster than people can write about them and it's safe to say the basics of each product line will probably carry over from model to model (the DS1515+ look identical to the older model, it simply has better/faster hardware and more updates available via it's operating system). In other words, take this as a review of the Synology small/medium business class devices and see what's on the market as of your reading of this article.
And so we have landed on the Synology DS1515+. This little guy is a beast. I've only been working with it for a few weeks but I can safely report that thus far I am very pleased. The days of buying or building a large server (or hiring someone to do it for you) seem to be behind us with the advent of these NAS devices.
I had read that you need some experience in networking to get it up and running, but I found the process to be extremely straight forward. This particular model doesn't require any tools for installing hard drives. You simply slide the drives into a sled and snap on two side pieces to hold it in place. The easy installation is nice, but the operating system is the big sell here. Within a web browser I can log on to the device and see all kinds of options laid out in front of me. Most important for our particular needs is the ability to manage sharing down to the smallest detail.
Library
As with The Airport Extreme, our original sound library lives on the NAS referenced via a database in Soundminer. Users log in and to gain read only access. The big difference here is the speed. No more beach ball!
Drop Share
Each user has a folder with permission to read/write, making tracking activity and organization simple.
Archives
Each project has been given a folder that is read only by the user associated with each project. The Synology is easily fast enough on our basic cable network to directly run Pro Tools Sessions. This means to reference older materials, users don't need a local copy. As a bonus, I can attach external 4 USB hard drives with all of our archives for the initial transfer to the NAS, avoiding the network all together.
The Good: Very little setup time. Ability to micro-manage sharing.
The Bad: Priced reasonably for it's functionality, however probably too expensive once you factor in hard drive cost for businesses with less than 4 users.
Conclusion: Our problems have been solved. It should be noted that I've barely scratched the surface here in terms of the usefulness of this device for a small post production business. The device has many more features including remote login, hosting your own cloud service and syncing with existing cloud services (google drive, dropbox, etc). For the time being, we just needed a simple place to host all of our files and a straight forward way to manage access. Found!
QUESTION: HAVE YOU FOUND ANY CREATIVE SOLUTIONS FOR BIG DATA IN YOUR WORKFLOW? LET US KNOW IN THE COMMENTS.
*Please note: Boom Box Post does not officially endorse any of the aforementioned products, nor do we consider ourselves experts in the field of networking.