Search:
Submit Search


SSD drive... worth it? here are some results

Buying or building computers for Hauptwerk, recommendations, troubleshooting computer hardware issues.

SSD drive... worth it? here are some results

Postby micdev » Wed Mar 28, 2012 7:36 am

Dear all,

Once in a while someone ask me "Should I buy an SSD drive for Hauptwerk?", "Will my sample sets within seconds?"

Of course I knew that samples wouldn't load in a few seconds, all I knew was that it would later but by how much. Reading the tech specs of SSD drives, they seems incredibly fast, but between theorical specs and real world one there is probably a large difference.

The type of files used/accessed, software used, computer running will influence the end result. Lately, I took the plunge and bought a 250Gb OCZ Agility sata III SSD drive to install in my Hauptwerk computer (i7-2600k). This drive is used exclusively for the cached files. Note that once format the 250Gb gave me 223Gb free!!! Talk about a "premium" just to format it.

So you will say, "Is it worth it?". As I said earlier results from theory and real life can be different. The load time did as expected improved... but by how much? Tell me François, can I load the Caen surround in 20 seconds? How about the PAB Gravissimo... 30 seconds max right? Wrong :!:

I could give you comparative time between Sata III WD Black Caviar drive and OCZ drive, but your results will probably be different than mine, since the cached file size will be different (a 16bit cached file will be smaller than 20bit one), so for the same organ but loaded with different parameters we will have different load time. Also the complexity of the ODF will influence the end result also.

So what can you expect. Well, on my system I can tell you that Hauptwerk loads (because Hauptwerk is the program doing the job) about 11Gb a minute. In practice this means for me that the Caen surround that used to load in 1:47 is now loading in about 1:06 a 38% improvement. A smaller set like the Trost saved 35% loading in about 30s. The Velesovo surround used to load in 1:26 and is now loading in 0:52, about 39% improvement in loading speed.

On my system I can expect load time cut by 35-40%. Is it worth it to spend a few hundred box to save 15-20-30 seconds when loading a set? I don't know. What I can tell you thou is that I now personnally change sample sets more often when practicing, selecting the right organ for the repertoire I'm playing.... maybe 30 seconds less loading time did made a difference for me.

All the best
François
Last edited by micdev on Wed Aug 15, 2012 1:11 pm, edited 1 time in total.
Virtually sharing my enthusiasm and experience with you
Worldwide technical assistance, consultation and ready to play system.

http://www.HauptwerkConsultant.com
User avatar
micdev
Member
 
Posts: 1424
Joined: Sat Feb 09, 2008 10:24 am
Location: Canada, Quebec

Re: SSD driver... worth it? here are some results

Postby polikimre » Wed Mar 28, 2012 8:51 am

Thanks for the numbers, I'm in the process of putting together a Hauptwerk pc with an ssd, so these are useful.

However, 11Gb (which I interpret here is 11 gigabytes) per minute is about 183 megabytes per second. I'd assume that loading a cache file falls in the realm of sequential data read, which is up to 300 megabytes per sec for sata II drives and up to 500 megabytes epr sec for sata III ones. Compared to those, 183 seems a bit low. It looks like the bottleneck in your system is somewhere else and you could gain some more speed.

Is loading the cache file still processed by a single cpu core? That was the case a couple of years ago.

(Stupid question: do you have a sata III interface on your notherboard, or only a sata II? If you have some sparse time, would you mind performing some data read and write tests on your ssd?)
User avatar
polikimre
Member
 
Posts: 675
Joined: Tue Sep 12, 2006 8:39 pm
Location: USA, NC, Cary

Re: SSD driver... worth it? here are some results

Postby mdyde » Wed Mar 28, 2012 9:34 am

Hello Imre,

Is loading the cache file still processed by a single cpu core? That was the case a couple of years ago.


No - since Hauptwerk v3.10 Hauptwerk will utilise up to four CPU cores when loading from the sample set data cache, i.e. when loading a sample set normally. (Only when regenerating a sample set data cache is only a single CPU core currently used.)

I'd assume that loading a cache file falls in the realm of sequential data read, which is up to 300 megabytes per sec for sata II drives and up to 500 megabytes epr sec for sata III ones. Compared to those, 183 seems a bit low. It looks like the bottleneck in your system is somewhere else and you could gain some more speed.


Hauptwerk's sample set loading mechanism was designed to be able to take full advantage of up to four CPU cores and the sustained read rates that a pair of 7200 RPM hard-drives in a RAID 0 array could achieve. With storage speeds much beyond that the CPUs will currently (Hauptwerk v4.0) be the bottleneck.

(Not to be confused with Hauptwerk's audio engine, which can take advantage of more than four CPU cores.)

In the longer-term we're planning to look at updating Hauptwerk's loading mechanism to be able to take better advantage of more than four CPU cores and the sustained read rates that current high-performance SSDs can achieve.
Best regards,
Martin.

[Please use email or the Contact page if you need to contact us privately, rather than private forum messages.]

Image
User avatar
mdyde
Moderator
 
Posts: 10644
Joined: Fri Mar 14, 2003 2:19 pm
Location: UK

Re: SSD driver... worth it? here are some results

Postby polikimre » Wed Mar 28, 2012 10:58 am

Thanks Martin, this also settles another question I had: SATA III vs SATA II. It seems like that the maximum transfer rate of SATA II is not the bottleneck in loading a sample set. The all-in-one I'm considering has only a SATA II interface.
User avatar
polikimre
Member
 
Posts: 675
Joined: Tue Sep 12, 2006 8:39 pm
Location: USA, NC, Cary

Re: SSD driver... worth it? here are some results

Postby micdev » Wed Mar 28, 2012 12:35 pm

Imre,

Yes I do have the drives (WD Black Caviar and OCZ SSD) connected to the sata III interface. Also the bios was set properly to handle the additional throughput.

I don't know how the theorical thoughput is calculed/done, but I guess that they are done by using highly optimised "machine coded" programs. In real life the result are quite different. When I got my drive I tested it using one of the benchmark program used by some websites testing SSD and my results were quite similar, so I'm pretty confident that my system is set correctly.

Thanks Martin for the additional information and I'm looking forward to the new updated loading mechanism.

Still like the improved loading speed I got thou :D

Regards
François
Virtually sharing my enthusiasm and experience with you
Worldwide technical assistance, consultation and ready to play system.

http://www.HauptwerkConsultant.com
User avatar
micdev
Member
 
Posts: 1424
Joined: Sat Feb 09, 2008 10:24 am
Location: Canada, Quebec

Re: SSD driver... worth it? here are some results

Postby schantzplayer » Wed Mar 28, 2012 2:47 pm

I did a comparison of load times after installing SSD's in my MAC Pro.The largest sample set I own is the Silver Octopus 80 and it takes about 75 seconds to load. The St Annes loads in about 6 seconds. I can't test anything bigger as this set almost maxes out my 32 GB of RAM.
User avatar
schantzplayer
Member
 
Posts: 485
Joined: Tue Jun 22, 2010 5:50 pm
Location: Mifflinville, PA USA

Re: SSD driver... worth it? here are some results

Postby ernst » Wed Mar 28, 2012 9:57 pm

I have carried out such an experiment as well.
Sadly, in Peru, where I live, after 2 weeks of searching I could just buy one make and model of SSD - it is a G-Skill Phoenix Pro FM-25S25 of 125GB, with a SATA II specification. Moreover I can't buy here a SATA III interface (of course could import it from the US but I have reached my limit of allowed imports this year).
The results were very disappointing.

I have also been swapping hard disks then. Originally I had a Seagate Barracuda ST31000 (SATA II, 7200 rpm 1TB) with 512b sectors. I wanted to buy a second one, but could only get the newer model ST1000 (also 7200 rpm 1TB) but with 4k sectors and SATA III, which appeared to be MUCH faster. Then I tried a RAID0 with 2 new ST1000 hard disks.

I got the following results: (note that the loading times I mention are the 1st time of loading, because if you repeat it the loading times are shorter, probably because of the cache effect of the HDs and/or the CPU). Loading times in seconds.
I have a i7-930 CPU.

Code: Select all
------------------- RAM ---- ST31000 ---- SSD -- ST1000 --- RAID0 of 2*ST1000
Dom Bedos ----- 21.8 GB -----241.0 s ------ ? --- 140.0 -------- 112.5 (-53%)
St. Michel ---- 11.7 GB ----- 93.5 s --- 84.3 ---- 72.3 --------- 59.5 (-36%)
Rabstejn ------- 3.5 GB ----- 60.0 s --- 57.0 ---- 42.0 --------- 19.0 (-88%)
St.Anne -------- 2.9 GB ----- 23.0 s --- 20.3 ---- 18.8 --------- 16.5 (-28%)


You may notice that savings differ a lot for the various sample sets.
I have made a graph of 18 sample sets, RAM size versus loading time, and the relation is roughly linear with the remarkable exception of Pusztaszabolcs, which takes a relatively longer time.

In conclusion, in my specific situation, the SSD has no benefit to mention, and I use it now as my system disk. The RAID0 consists of 2 Seagate Barracuda ST1000 hard drives (4k sectors,7200 rpm, 1 TB) and my old Seagate Barracuda ST31000 is the built-in back-up drive, given the unreliability of a RAID0.

I used tu use Norton and later Acronis for my backups, but I find that Windows (Image) backup is a LOT more reliable. With my experiments with RAIDs and swapping all disks around I have at least 4 times installed Windows and tried image restores with Acronis and Windows. Windows Backup is fast and flawless. Because I have tweaked Windows seriously in order to solve many Hauptwerk problems, I have saved two small Image backups, just in case: a personalized Windows version with internet capabilities, a personalized version optimised for Hauptwerk without internet capabillities (both around 20GB), and the full image (still expanding) with all organ data on the RAID0. Because Windows puts its backup files in the root of your backup drive, you have to separate the above 3 versions physically, which I have done by making 3 partitions. I check every once in a while those images (with the repair function).

A note. When you remove Windows or Image Backups, sometimes large invisible files remain (about 5GB!). That annoyed me and I have formatted the drives repeatedly to get rid of those. But be warned that Windows activation (which checks your version of Windows and the detailed composition of your PC, which are linked) puts an invisible indentification file on your system disk. If you format that disk (or when you change major PC components) you have to activate Windows again. With internet profoundly unabled, that's a problem.

Ernst
User avatar
ernst
Member
 
Posts: 406
Joined: Sat Dec 18, 2010 10:27 am
Location: Lima, Peru

Re: SSD driver... worth it? here are some results

Postby mdyde » Thu Mar 29, 2012 6:59 am

(Ernst - I've edited the formatting of your results table so that the columns line up.)
Best regards,
Martin.

[Please use email or the Contact page if you need to contact us privately, rather than private forum messages.]

Image
User avatar
mdyde
Moderator
 
Posts: 10644
Joined: Fri Mar 14, 2003 2:19 pm
Location: UK

Re: SSD driver... worth it? here are some results

Postby mdyde » Thu Mar 29, 2012 7:13 am

In case it's of any interest, my brand new MacBook Pro (early-2012 model, 2.5 GHz quad-core i7 CPU with 8 MB cache and HyperThreading, 16 GB RAM and 512 GB Apple SSD) takes about 10.4 seconds to load St. Anne's (after a reboot, to ensure that it wasn't already cached in RAM) in full with all default rank options, with an average sustained read rate of about 243 MB/s from the SSD during Hauptwerk's disk-reader activity.

Overall Hauptwerk manages to load the sample data at an average rate of about 136 MB/s. That's a bit better than I was expecting (based on other Hauptwerk user reports of SSD performance), and is probably contributed to significantly by the higher processing power available in the recent i7 models.

(Hauptwerk reports the loading statistics, including average sustained read rate from the disk/SSD, sample data loading time, and total size of sample data read, in its log straight after an organ has finished loading.)
Best regards,
Martin.

[Please use email or the Contact page if you need to contact us privately, rather than private forum messages.]

Image
User avatar
mdyde
Moderator
 
Posts: 10644
Joined: Fri Mar 14, 2003 2:19 pm
Location: UK

Re: SSD driver... worth it? here are some results

Postby micdev » Thu Mar 29, 2012 7:30 am

Hello Ernst,

Interesting results; your idea of using 4K blocks is something we don't often think to do and looking at your results seems to provide interesting improvement at no cost. Your raid0 configuration provides about the same speed as my sata III SSD.

A bit like standard hard-drive, SSD drives are not all the same; cache size, interface, controller and memory chips used will greatly influence the final result.

Raid0 or sata III SSD seems to hit the ceiling of loading performances for a given processor. Future HW version will probably provide even faster performances (see Martin' previous post). Inspired Acoustics' organ are behaving a bit differently. The ODF must be built differently; extended graphical elements may also cause some delays building the screen etc.

Martin' comment about rebooting the computer is important to make sure the cache is cleared.

Thanks for sharing
François
Virtually sharing my enthusiasm and experience with you
Worldwide technical assistance, consultation and ready to play system.

http://www.HauptwerkConsultant.com
User avatar
micdev
Member
 
Posts: 1424
Joined: Sat Feb 09, 2008 10:24 am
Location: Canada, Quebec

Re: SSD driver... worth it? here are some results

Postby mdyde » Thu Mar 29, 2012 7:32 am

P.S. For comparison, my old 2006-model quad-core Mac Pro (2x dual-core 2.66 GHz Xeon 5150s, 8 MB CPU cache in total) takes about 30 seconds (with 45.3 MB/s average sustained read rate) to load St. Anne's in full with all default rank options from a 500 GB 7200 RPM hard-drive.

Hence the combination of a recent high-performance SSD and a recent recent high-performance CPU in the current Mac Book Pro have increased overall loading speed by about 3 times and sustained read rate by about 5.4 times, based on St. Anne's, compared to the 2006-model Mac Pro (which was the higher-performance model when it was made), even with the same number of physical CPU cores and approximately the same CPU clock speed and amount of CPU cache.
Best regards,
Martin.

[Please use email or the Contact page if you need to contact us privately, rather than private forum messages.]

Image
User avatar
mdyde
Moderator
 
Posts: 10644
Joined: Fri Mar 14, 2003 2:19 pm
Location: UK

Re: SSD driver... worth it? here are some results

Postby polikimre » Thu Mar 29, 2012 9:07 am

Ernst,

in researching SSD drives I came accross the following excellent tutorial:

http://www.bit-tech.net/hardware/storag ... rs-guide/5

In particular it discusses the different controllers used in SSD drives. Your G-skill Phoenix uses the new sandforce controller, which compresses the data before writing it to the disk. For most applications this will result in an improved throughput, since the data read/written is actually less, but for Hauptwerk it actually hurts performance, because the cache files are already compressed. Most probably the disk's on-the-fly compression increases the size of the data thus decreases the read rate.
User avatar
polikimre
Member
 
Posts: 675
Joined: Tue Sep 12, 2006 8:39 pm
Location: USA, NC, Cary

Re: SSD driver... worth it? here are some results

Postby ernst » Fri Mar 30, 2012 7:43 pm

Hello Imre,

Thanks for posting that article. But, mind you, my Phoenix SSD uses the first generation of SandForce controller. The current generation of SATA III SSD's which use the 2nd generation of SandForce are much faster (and even more when used with an internally RAID0 arrangement).


Hello Martin,

Thanks for straitening my table; still don't know how to do that myself.
I had not thought of using the log file, and therefore loaded those 4 VPO's again and looked at the log file now. The loading times are all about 1 s shorter than I had measured with a stopwatch (looks I have a19th century mind), and the difference will be the initiation processes I assume. The thing that really struck me were the so-called "average data read rate during disk reader activity", which I don't understand looking at the data you mention for your new MacBook Pro of about 243 MB/s. (You call it "average sustained read rate from the disk" but I have seen just one mention of a read rate in the log file). I have found:
- Dom Bedos 614.66 MB/s
- St. Michel 687.54 MB/s
- Rabstejn 610.63 MB/s
- St.Anne's 1360.94 MB/s (!!!)
In tests with CrystalDiskMark for sequential read of 1000MB data blocks I got 339 MB/s for the RAID0.

Am I doing something wrong in this comparison? Sure can't believe those speeds.

Ernst
User avatar
ernst
Member
 
Posts: 406
Joined: Sat Dec 18, 2010 10:27 am
Location: Lima, Peru

Re: SSD driver... worth it? here are some results

Postby pat17 » Sat Mar 31, 2012 2:52 am

Very interesting subject, which gives us the pleasure to read from François! :D

Sorry I am not very much on the technical side, yet I would like to make sure I understand well -

- as of today, Hauptwerk's loading performances are limited to i7 4 cores machines using an Hard Disks' Raid 0 configuration.
- a SSD can be equivalent to a Raid 0 configuration.

If my understanding is correct, that means the following configuration I'm having in mind can take all what Hauptwerk needs to be at its best - regardless of future improvements indeed -

- Mac Mini 2011 Server (2.0GHz quad-core Intel Core i7)
- RAM expanded to 16 MB (OWC)
- one of the two internal 7200 RPM hard drives replaced by a SSD - from what I can read on the net, configuring the two HD in RAID 0 is not really recommended on a Mac mini unless you know how to tweak it correctly which is beyond my skills.

Is that correct? Indeed, I could do better by going to a Mac Pro, but the budget is not the same...

Besides, when Martin is mentioning he has selected the Apple SSD, was there any specific reason behind this choice? I've read a bit everywhere the SSD supplied by Apple were not the best value for money - their only advantage is they are supported by Lion's TRIM feature unlike third party models.
User avatar
pat17
Member
 
Posts: 1094
Joined: Wed Jan 06, 2010 8:53 am
Location: France, La Rochelle

Re: SSD driver... worth it? here are some results

Postby mdyde » Sat Mar 31, 2012 7:47 am

Hello Enrst,

Thanks for straitening my table; still don't know how to do that myself.


You're welcome. I just used the 'code' tags, which make the forum software display it with a fixed-width font (same width for each character).

I had not thought of using the log file, and therefore loaded those 4 VPO's again and looked at the log file now. The loading times are all about 1 s shorter than I had measured with a stopwatch (looks I have a19th century mind), and the difference will be the initiation processes I assume.


Yes - the difference is probably due to the time that the audio and MIDI drivers take to start, which isn't included in the log file's loading statistics.

The thing that really struck me were the so-called "average data read rate during disk reader activity", which I don't understand looking at the data you mention for your new MacBook Pro of about 243 MB/s. (You call it "average sustained read rate from the disk" but I have seen just one mention of a read rate in the log file).


This is an example from the log after loading St. Anne's:

2012-03-29-12-03-54: INF:2157 The organ StAnnesMoseley.Organ_Hauptwerk_xml has been loaded. Metrics:

Organ ID: 000010.
Organ ver.: 4.10.
Organ min. Hauptwerk ver.: 4.0.0.000.
Mem.: very approx. est. physical mem. free/usable: 12999.20 MB.
Mem.: very approx. est. sample/object mem. in use by Hauptwerk (excl. other data): 1427.55 MB.
Mem.: total raw audio/trem. sample data mem. (excl. pre-proc. audio, images, or other data): 1292.90 MB.
Mem.: overall compression ratio for compressed ranks: 41.78 pct.
Total time: overall: 10.436 sec.
Total time: sample data: 9.485 sec.
Total time: all except sample data: 0.951 sec.
Time: prep.: 0.156 sec.
Time: organ def. loading/parsing: 0.218 sec.
Time: organ def. object/key validation: 0.032 sec.
Time: organ def. prep.: 0.015 sec.
Time: prep. for loading settings: 0.031 sec.
Time: settings loading/parsing: 0.016 sec.
Time: settings object/key validation: 0.031 sec.
Time: settings prep.: 0.000 sec.
Time: data cache validation/prep.: 0.000 sec.
Time: audio data: 9.469 sec.
Time: trem. data: 0.016 sec.
Time: images: 0.125 sec.
Time: comb. and temper. files: 0.327 sec.
Sample loader: loaded from data cache: Y.
Sample loader: multi-threaded load: Y.
Sample loader: concurrent threads: 4.
Sample loader: data cache total disk size: 1330.91 MB.
Sample loader: buffers: 9.
Sample loader: approx. loader peak mem. usage during audio loading: 102.95 MB.
Sample loader: loader def. mem. usage during audio loading: 97.27 MB.
Sample loader: approx. loader mem. usage during trem. loading: 22.57 MB.
Sample loader: approx. avg. data read rate during disk reader activity: 243.41 MB/s.
Sample loader: pct. data loading time each stage busy: disk I/O: 57.65.
Sample loader: pct. sample loading time stage busy: 1st proc.: 1.27.
Sample loader: pct. sample loading time stage busy: 2nd proc.: 76.81.
Sample loader: pct. sample loading time stage busy: 3rd proc.: 93.25.
Sample loader: pct. thread activity due to stage: disk I/O: 25.18.
Sample loader: pct. thread activity due to stage: 1st proc.: 0.55.
Sample loader: pct. thread activity due to stage: 2nd proc.: 33.55.
Sample loader: pct. thread activity due to stage: 3rd proc.: 40.72.


To work out the overall average data loading rate, divide 'Mem.: total raw audio/trem. sample data mem. (excl. pre-proc. audio, images, or other data)' by 'Total time: sample data', i.e. in this example:

Average data loading rate = 1292.90 MB / 9.485 sec =approx= 136 MB/s

The statistic that shows the average raw read from the disk is 'Sample loader: approx. avg. data read rate during disk reader activity', which is about 243 MB/s in this case. However, that will usually be higher than the overall average data loading rate because Hauptwerk needs the CPUs to perform some processing on each chunk of sample data after it's been read from disk (for example, decrypting it). That happens simultaneously (by using other CPU cores) but with extremely fast storage such as an SSD the CPUs will still currently be the bottleneck overall.

I have found:
- Dom Bedos 614.66 MB/s
- St. Michel 687.54 MB/s
- Rabstejn 610.63 MB/s
- St.Anne's 1360.94 MB/s (!!!)
In tests with CrystalDiskMark for sequential read of 1000MB data blocks I got 339 MB/s for the RAID0.

Am I doing something wrong in this comparison? Sure can't believe those speeds.


Did you reboot the computer prior to each test? My guess is that Windows still had the raw file data cached in memory (its file system cache), so it wasn't actually reading the data from the SSD at all, at least for the highest of those figures.

If you did reboot, perhaps Windows SuperFetch had pre-loaded some of the sample data anyway, which again would result in the data not actually being read from the SSD:

http://blog.tune-up.com/myth-buster/myth-busted-why-disabling-superfetch-on-vista-and-windows-7-is-a-bad-idea/
Best regards,
Martin.

[Please use email or the Contact page if you need to contact us privately, rather than private forum messages.]

Image
User avatar
mdyde
Moderator
 
Posts: 10644
Joined: Fri Mar 14, 2003 2:19 pm
Location: UK

Next

Return to Computer hardware / specs

Who is online

Users browsing this forum: No registered users and 2 guests