Hi. I posted this over at the Apple Aperture Support Community (I am one of the top contributors there), but haven’t received much in the way of replies, so I’m posted here in the hope of getting some pointers. Thanks.
Best storage set-ups for large Libraries?
Let’s say “large Library” means 500 GB and/or 500,000 Images, and above.
What set-ups for storage, on a Mac maximized with RAM, will provide the best Aperture performance? The best bang-for-the-buck? Assume Thunderbolt is available. I don’t know much about hardware: RAID, hybrid drives, etc.
What sensible considerations should one make for backup and data-redundancy?
How can I determine if the storage media is a bottleneck in the performance of Aperture on my system?
I run most libraries off of external USB-3 drives mounted (sometimes directly, often via powered USB-3 hubs) to my
MacBook Pro
- Retina, 15-inch, Early 2013
- Processor 2.7 GHz Intel Core i7
- Memory 16 GB 1600 MHz DDR3
- Graphics NVIDIA GeForce GT 650M 1024 MB
- System Drive APPLE SSD SD512E
This works well for small and medium-size Libraries, but for large Libraries, I’m spending costly time waiting for the system to catch up.
Some delays, demonstrably, come from running a three-monitor set-up (which this provides some welcome advantages to me), but many more seem to be I/O issues.
Additionally, some back-ups and database repairs now take 12 hr. or longer, so any significantly faster throughput will be useful.
—Kirby.
Hi Kirby, you have helped me a number of times over on the apple site. My lib are reference on an older droboS and not nearly as large as yours. I have split my main lib into four lib which help with speed of updating coping and trouble shooting, etc. I do miss not being able to search all at one time. My guess for you and me at this time is one large SSD per Lib attached with Thunderbolt, daisy chained. For backup superduper or Copycloner to a normal Larger HD, one Lib copy per HD, I use a toast type caddy with bare drives. I suppose three copies are better than two. Not an inexpensive solution but raids are not cheap either. Networked raids as you know are problematic and usb hubs can be a bad connection too. Perhaps the new “photos” with all images everywhere will be the best answer in the Fall. Cheers
davidbmoore@mac.com
Twitter= @davidbmoore
Scottsdale AZ
Hi David,
Thanks for the reply. I agree that a dedicated T-Bolted SSD for each Library is almost certainly going to give the fastest throughput short of an SSD system drive (and I don’t have the knowledge to compare those two). I have long used SuperDuper to clone all Libraries and Referenced Originals after each use, leap-frogging duplicate drives weekly and never having all three copies (one working copy, two back-ups) in the same physical location.
I’d like to know if the bus is a bottleneck. I’m getting a BlackMagic rating of c. 370 (iirc, MB/s) for my installed SSD system drive, but only seeing c. 50 - 70 for USB-3 external drives. I think USB-3 is theoretically capable of 500 MB/s. It seems possible that for other reasons I will never have a drive mounted to this machine that is speed-limited by the USB-3 bus. A USB-3 SSD is considerably less expensive than a T-Bolt SSD.
Security and throughput concerns leave me uninterested in any cloudy solutions at this point.
—Kirby.
—Kirby.
www.kirbykrieger.com
Have you had Activity Monitor running so you can see what is happening during the slow periods? CPU hog? Disk I/O? Swap space climbing? David had some good points about SSD and Thunderbolt, if the disk is indeed your bottleneck. Also wondering if you have Faces turned on or off, which can be a hog.
Bill Jurasz
Austin Texas
Faces has been off since I got over the 200,000 Images mark. Thanks for the suggestion. Faces _is_ the most sluggish part of Aperture, IME.
I have been watching Activity Monitor, but with Mavericks I don’t know what to watch for, especially in memory use. My large Libraries regularly take 5 - 8 GB of RAM. With Aperture running, my physical RAM is almost always maxed out (e.g.: 15.64 of 16 GB).
Can you point me to some pointers on how to use Mavericks’ Activity Monitor to locate the bottleneck in my Aperture performance? Thanks.
—Kirby.
—Kirby.
www.kirbykrieger.com
I too could use a new refresher course on the new Mavericks’ Activiey monitor and Aperture. By they way Activity Monitor has crashed a few times and I have a report in to AP. We’ll see what happens.
davidbmoore@mac.com
Twitter= @davidbmoore
Scottsdale AZ
I have been using LaCie Thunderbolt (1) 5 big raid unit. Using blackmagic speed test it runs at reads/writes at 794/763 MB/s. Now there is a thunderbolt -2 that will read/write over 1000 MB/s. However I don’t think that for most people that will make any difference. I have have found by using different configurations that once once the drive speed has gotten above approx. 300 MB/s there is no detectable difference in speed. Probably because the time spent reading/writing the file has become insignificant to the overall process. At that point a more powerful video card with more video memory will be beneficial. For the price these thunderbolt drives running in raid 0 will provide much faster read/write speeds than you will see with an SSD drive.
LaCie - LaCie 5big
The unit was purchased as a dedicated drive for my aperture library, so its a bit of overkill, 10 TB, and I have it configured to raid 0 for speed. That means there is no redundancy. If one of the drives fail all data is lost. That necessitates a good back up plan. For that I have two LaCie 2 big thunderbolt units that I have configured to raid 1 (mirrored), Using Chronosync to back of the library every 4 hours to each unit. I also have purchased a spare drive, just in case I should need it.
LaCie - LaCie 2big Thunderbolt Series
For my offsite backup up I use Backblaze. The first backup takes awhile, after that its just updates.
Now as a disclaimer my library is running around 360 GB, with some 50,000 plus images. No where near al large as yours. I’m using an iMac, late 2012. 3.4 Ghz i7 processor with 32 GB is ram, GTX 680MX video card with 2048 MB. I do notice that as you scroll through the images in aperture the available memory keep decreasing. I suspect that Aperture is keeping as much in memory as possible. So no mater how much memory you have it will eventually use it all. My main drive is a fusion drive with only 35 GB used so my system should be running from the flash memory. If I had a do-over I would get the SSD for the main drive.
The performance of the library is excellent. Most things are instantaneous. Making adjustments, scrolling around the library etc.. I do not use the vault system for backup. I just use Chronosync to maintain a duplicate copy’s of my library.
Mike
This is a timely discussion. I’ve been using a MacBook Pro 2011 w external 2T USB drive. I have been importing photos from a windows box over time. Ive noticed a performance degradation as I’ve added more photos to our aperture libraries. In the end ill have approx 500k+ photos jpeg/RAW.
Been looking at upgrading to a new Mac Pro and a RAID 5 array. But would like to stick with the MBPro for a bit longer if possible. Been looking at Promise Pegasus 2 (dont know if the older Pegasus TBolt 1s are still avail) for RAID.
Any thoughts on LaCie vs Promise RAIDS?
I can’t really comment on Promise RAIDS since I don’t have one. I do see that it can be configured in the different arrays, i.e. 0, 1, 5, 6, 10. I would think that it would have to have its own CPU, Memory, and OS to manage the different Raids. The LaCie is a software raid configured using Apple’s disk utility. So that limits it to raid 0 or 1. I didn’t look up prices, but since the promise has to be a more sophisticated device I’m sure its a bit more expensive. While both systems would require a backup system (raid is not a backup) , you really need to keep your backup’s current when running raid 0. I have mine scheduled to backup to two mirrored raid units every other hour from 8am to 12 pm.
USB 2 or USB 3 hard disk? Also I’ve heard rumors that disk fragmentation, while not normally an issue for Macs, actually is an issue for Aperture, and wondering if anyone knows about that.
Bill Jurasz
Austin Texas
I’ve always been partial to and have had great performance from OWC external drive options.
Since not long before Thunderbolt was introduce I have been using OWC Mercury Elite Pro QX2 … I have one four-drive unit with the images and video files from the past three years in RAID 5 with four 2TB drives and another with legacy files from earlier in RAID 0 with four 1TB drives. The currently offered housings support hardware RAID 0, 1, 5 ,6 and 10 and JBOD. They also offer eSATA, FW800 and USB 3 (mine are older and have USB2) interfaces and include all cables.
I initially used these with FW800 as that was the fasted connection my then iMac offered. When upgraded the iMac to a newer model that offered TB, I added the LaCie Thunderbolt eSATA hub … which is much faster.
I plan on retaining these housings as backup units when I purchase the TB versions OWC is now offering. Though, currently the Thunderbay IV only offers software RAID.