It is currently Fri Mar 29, 2024 1:27 am


How To Decrease Sample Loading Times

Buying or building computers for Hauptwerk, recommendations, troubleshooting computer hardware issues.
  • Author
  • Message
Offline

johannusfan

Member

  • Posts: 31
  • Joined: Wed Dec 15, 2010 12:49 pm
  • Location: Concord, California

How To Decrease Sample Loading Times

PostFri Mar 08, 2013 12:43 pm

I have been steadily beefing up my PC builds to optimize how Haup. runs and loads. I have it pretty well dialed-in, but there is one question: where is the typical bottleneck for loading samples when I'm ready to play?

I'm chiefly interested in loading the very largest organs routinely (like the Sonus Paradisi Rotterdam organ). Is it the bus rate for loading the RAM or the data leaving the hard drive (or both) that should be eyed carefully as a bottleneck source. I can go make and have a coffee break whilst this thing pokes into readiness! I don't want to consider the S.S. HD's yet.

PC specs:
Win 7 64
Intel DQ77MK mobo
32G dual-channel DDR3 1600 RAM
Intel SRCSAS144E SATA/SAS PCI-E RAID Controller w/512M memory
Seagate ST3300656SS 15K rpm HD
ASUS Xonar Phoebus sound card (a must for the 24-bit folks)
Nitrous injectors? :D

For those considering the Rotterdam samples, ya need a mainframe computer (64G+ RAM) to load the whole thing!
32G just makes it run.

Thanks folks.
Offline

amun

Member

  • Posts: 242
  • Joined: Sun Mar 13, 2011 3:20 am

Re: How To Decrease Sample Loading Times

PostFri Mar 08, 2013 1:23 pm

I can go make and have a coffee break whilst this thing pokes into readiness!
:twisted:

So do I: with an ASUS Laptop, i5, Win 7/64, 8 GB RAM, Paramount 332 ( ca 6 GB) setup, programms preferred, WLAN,Virusscan etc disabled.

@johannusfan:
Is Your PC an ASUS breed?
Did You try to wait until Your HD is running idle. My observation is that laoding the 6 GB samples then requires only about a felt minute or so?

My guess is that although the PC-performance is set to programs preference there are so many other ( for our purpose useless ) SW-modules with higher priority to be processed before the sample set gets a greater chance than a minimum of timesharing.

ASUS provides a table were the start of programs and services can be delayed, but his does not help in the case of samples.

Amun
Offline

Organorak

Member

  • Posts: 685
  • Joined: Sun Nov 22, 2009 6:55 am

Re: How To Decrease Sample Loading Times

PostFri Mar 08, 2013 2:28 pm

I don't know how much RAM I'm using but out of the 24GB on my system Hauptwerk says I have about 0.9GB free after loading Zwolle (though I shall need to change something there as I get occasional audiopops, maybe I should go for 20 bit rather than 24bit samples etc) Originally my Samsung 7200rpm spinpoint disks were RAID0 and loaded the sample in just over 1 minute flat, then I went to RAID1 and they load in about 3 minutes. I haven't got the luxury of SSDs.

But in saying that I've just described three things you might try, loading lower bit samples where possible (maybe for the rear sample quieter stops), having RAID0 and having SSD. If you combine all three you might not need the nitrous injector you mentioned earlier - let us know how you get on!
Offline

johannusfan

Member

  • Posts: 31
  • Joined: Wed Dec 15, 2010 12:49 pm
  • Location: Concord, California

Re: How To Decrease Sample Loading Times

PostFri Mar 08, 2013 5:06 pm

"@johannusfan:
Is Your PC an ASUS breed?
Did You try to wait until Your HD is running idle. My observation is that loading the 6 GB samples then requires only about a felt minute or so?" "Reduce sample rate"

No, it is not ASUS - I custom build and tune my own desktops. The system is not network-attached, so it has no other programs running in the background - very simple and clean. It is special to running Haup. The drive is quiet before the samples load. XP would have been a nightmare in that it has so much [stuff] happening in the background all the time. All you folks should work hard to implement 7 when you can. Much cleaner, in my opinion and experience.

Sample rates: The day I have to reduce the sample rate below 24 bits is when I have to totally rethink my computer platform. The sound difference is stunning. I so enjoy the sweetness and smoothness this high rate affords. I am an audiophile and the quality of these fine organs is nothing short of mission critical! Good suggestions, though. Keep them up.
Offline

1961TC4ME

Member

  • Posts: 3144
  • Joined: Mon Feb 07, 2011 1:45 pm
  • Location: Lake Minnetonka, Minnesota

Re: How To Decrease Sample Loading Times

PostFri Mar 08, 2013 7:30 pm

I think I'd try the 150 shot of nitrous! :D Don't stand on it too hard though, you might throw a hard drive out the side of the computer case! :lol:

Marc
Offline
User avatar

polikimre

Member

  • Posts: 676
  • Joined: Tue Sep 12, 2006 7:39 pm
  • Location: USA, NC, Cary

Re: How To Decrease Sample Loading Times

PostFri Mar 08, 2013 8:30 pm

While the sample set is loading start task manager and go into the Resource monitor. See the load on your CPUs, the read rate from your hard drives, and compare these to the figures listed for your equipment. That will give you the bottleneck.

In most cases the bottleneck is the hard drive, followed by the CPU, as the caching of sets is not fully optimized for multiple threads (maybe only up to 4?, I don't remember exactly).
Offline

johannusfan

Member

  • Posts: 31
  • Joined: Wed Dec 15, 2010 12:49 pm
  • Location: Concord, California

Re: How To Decrease Sample Loading Times

PostFri Mar 08, 2013 9:03 pm

CPU is poking along at well below 10%. Hard drive on any system is definitely a very real consideration. I don't know how to see the HD throughput, but at maximum "warp", sustained data rate is 164Mb/s. Therefore, if my math is accurate, a 32Gb sample should load in less than four minutes, yes? Something sounds constipated... :?

Is there another user out there who's PC blasts these in at a reasonable speed? Using a special blessing program??
Offline
User avatar

NickNelson

Member

  • Posts: 880
  • Joined: Tue Dec 20, 2005 10:31 am
  • Location: Yorkshire, UK

Re: How To Decrease Sample Loading Times

PostSat Mar 09, 2013 2:03 am

I think this may depend on what the reported figures actually mean. If the connection is serial it is possible that 164 Mb/s means 164 million bits per second rather than 164 million bytes per second (MB/s). Since there are 8 bits to a byte, and RAM capacities are given in bytes, you might have to multiply the calculated time by 8 to get a real estimate of loading time.

Or maybe I'm barking up the wrong tree (though SATA data rates are definitely in bits per second).

Nick
Offline

amun

Member

  • Posts: 242
  • Joined: Sun Mar 13, 2011 3:20 am

Re: How To Decrease Sample Loading Times

PostSat Mar 09, 2013 4:52 am

polikimre wrote:While the sample set is loading start task manager and go into the Resource monitor. See the load on your CPUs, the read rate from your hard drives, and compare these to the figures listed for your equipment. That will give you the bottleneck.
In most cases the bottleneck is the hard drive, followed by the CPU, as the caching of sets is not fully optimized for multiple threads (maybe only up to 4?, I don't remember exactly).


IMHO a good point to follow... but for solid decision where to locate and tackle the bottleneck the workload of the HD should be evaluated in relation to the number of tasks processed.
Roughly I found two different statesof the system over the timeline:
- a warm-up period of about 5-10 min. A huge number of processes are started. Loading samples ( not the HW prorgam) of whatever size becomes a test of ones patience.
- a waiting-for-action period. A process is started now and then.The loading of samples is fast.

An evaluation of this result opens roughly three options for a solution:
- a faster HD...but what will be its job when the loading is done, which is a very short, almost neglictable period in comparison to the time when the organ is played and there is no need for a HD at all,
- reduce the "warm-up" processes to a minimum...which, if done in a moderate way can help within limits , but may end in difficulties, if done the severe way
- live with the "warm-up" period and make use of it: dry agility exercises ( as in former times prof-pianists did with their "silent piano" ), concentrate on the music...

The decision: for me a matter of personal preferences and budget...

Amun :wink:
Offline

johannusfan

Member

  • Posts: 31
  • Joined: Wed Dec 15, 2010 12:49 pm
  • Location: Concord, California

Re: How To Decrease Sample Loading Times

PostSat Mar 09, 2013 3:11 pm

Nick - you are absolutely correct - d'oh! I feel like a dufus. Of course RAM is BYTES and disk xfer rate is BITS. That changes the 32G to 256M bits. A "slight" increase in bit volume - by 8x. If one is in a hurry, stick to the small positiv organs.

So that increases the factor significantly. So apparently - as I tell my geek kids - the HD is the slowest thing in the box (after the CD drive). Oh well, I like coffee anyhoo. If I'm in a hurry, I can always turn on the organ speakers and play them until those 7,000 pipes wake up! :D

I tried another experiment and put the system in sleep mode with the large organ loaded and running. The next time I brought the system up out of sleep mode it only took three minutes 'till it was up and ready to play again. Go figure.
Offline

johannusfan

Member

  • Posts: 31
  • Joined: Wed Dec 15, 2010 12:49 pm
  • Location: Concord, California

Re: How To Decrease Sample Loading Times

PostThu Mar 21, 2013 9:53 am

Update. I have upgraded the system to make organ loading very quick. Motherboard has RAID 0 and two SATAIII (6Gb/sec) slots. Purchased two WD VelociRaptor hard drives. With the combined parallel ultra-high speed data paths from the drives to the RAM, loading times have been slashed. Highly recommended!
Offline

Organorak

Member

  • Posts: 685
  • Joined: Sun Nov 22, 2009 6:55 am

Re: How To Decrease Sample Loading Times

PostTue Apr 09, 2013 10:48 am

The question of RAID does pop up from time to time, so here's something I'm wondering. I started off with two 7200 hard drives in RAID0 and loading times were seriously FAST. Then I got concerned about the possibility of disk failure and curious about trying RAID1. Loading times are now very much slower.

Before I go the whole hog of making a disk image and reinstalling everything into RAID0 again, I was wondering if there is another option. Can I create two partitions across my two hard drives (e.g. C: and D:) and retain C: in RAID1, which will store operating system, all other programs plus all my data, and put the D: partition of the disks into RAID0 configuration. I would only use that for storing a copy of whatever sound sample files it is that Hauptwerk loads when it loads an organ - assuming that HW allows me to copy (or move) those files to a different location on the hard drives.

Is that even possible, and if so what should I look out for? I have installed Intel rapid storage technology, if that's any help (though does anyone know how it reacts if a disk image created from RAID1 is installed onto a RAID0 pair of disks?). Even in RAID1 my disk size is way more than I presently need so I won't be running out of disk space any time soon.
Offline

Sander

Member

  • Posts: 418
  • Joined: Sun Oct 07, 2007 5:08 pm
  • Location: Assen, NL

Re: How To Decrease Sample Loading Times

PostMon Apr 15, 2013 6:55 pm

In theory, reading from RAID 1 is just as fast as from RAID 0, but this depends on the quality of your RAID controller. It may help a lot if this is a special device, instead of the onboard controller you are probably using now and which usually are of lesser quality.
Offline

johannusfan

Member

  • Posts: 31
  • Joined: Wed Dec 15, 2010 12:49 pm
  • Location: Concord, California

Re: How To Decrease Sample Loading Times

PostTue Apr 30, 2013 4:30 pm

Excellent points. I have tried both the on-board RAID (Intel DQ77MK mobo) and the Intel SRCSAS144E RAID controller board. Both performed visually at the same speed. There could have been a stop-watch difference, but I'll go for simplicity and lower power/heat. Also, visually, the 0 seemed a bit faster than the 1 configuration. With good drives, the HD failure scare is minimal, but I went ahead and installed a backup hard drive where I stored a total system image.

I've read a lot of debates about RAID 0 and other scenarios, but my experience indicates RAID 0 is optimal. It just took a bit of time, education, and experimentation to get it working properly and optimally. I have not, however, experimented with the new SSD hardware. $$$ and other concerns.
Offline

Organorak

Member

  • Posts: 685
  • Joined: Sun Nov 22, 2009 6:55 am

Re: How To Decrease Sample Loading Times

PostWed May 01, 2013 3:54 am

On a related thread I recently posted results from the three possible configurations that my two identical 1TB Samsung F3 Spinpoint disks and Intel RAID controller permit:

viewtopic.php?f=16&t=11350&p=87285#p87281

Results:
_______._____RAID0____.____RAID1____.____RAID0 drive for cached files plus RAID1 C:\ drive
Caen_________.__86_____._____158____.____121
Paramount 310____8_____._______6____.______6
Salisbury________67_______.___122____.____100
Zwolle_____._____91______.____188____.____123

Thus loading times where my two disks are partitioned into a RAID1 C-drive (with operating system, documents and Hauptwerk sample files) and a RAID0 D-drive (Hauptwerk cache) are about 40% slower than then configured as a single RAID0 C-drive, which in turn is around 40% slower than if configured as a single RAID1 C-drive. I should probably make a mirror of the C drive in case of disk failure, though with RAID1 that is hopefully unlikely to happen (and the only time I ever made a disk mirror using Windows 7 mirroring program, it wasn't recognised when I came to reinstall and I have to do a clean reinstall of everything, so maybe that's the better option in any case...) I have copied all the D-drive Hauptwerk cache folders and files onto the C drive and in the event of the RAID0 breaking it should be a simple matter to delete, reformat and rebuild it as a RAID0 partition, rename it D drive and transfer the cache folders back again.

I could halve the time spent waiting for samples to load if I did away with all this and just got an SSD, but if I do that I have less money available for my next sample set. Choices, choices?
Next

Return to Computer hardware / specs

Who is online

Users browsing this forum: No registered users and 5 guests