It is currently Thu Jul 09, 2020 11:40 pm


Slow Load Times - OSX64

Hauptwerk software technical support only. Please make sure you have read the manual, tutorials and FAQ pages before requesting support.
  • Author
  • Message
Offline

deWaverley

Member

  • Posts: 321
  • Joined: Fri Feb 29, 2008 2:33 pm
  • Location: UK, Bath

Slow Load Times - OSX64

PostSun Jun 28, 2009 4:01 pm

Does anyone have any ideas what might be causing my Organ load times to have suddenly gone really slow?

Vollenhove now takes 2 mins 15 seconds on a macpro! Unless I've gone mad, I remember it being around 20 seconds previously.

You would think it must have started rebuilding the cache each time, but the messagebox is normal (i.e. doesn't give the spiel about 'this first time will be longer...') so I'm at a loss.

The only thing I can think is that I experimented with multiple audio outs a few weeks ago, but I've put it back how it was now.

Any ideas would be very gratefully received.

deW
Offline
User avatar

mdyde

Moderator

  • Posts: 12149
  • Joined: Fri Mar 14, 2003 2:19 pm
  • Location: UK

Re: Slow Load Times - OSX64

PostSun Jun 28, 2009 4:11 pm

Is the hard-disk busy doing something else at the same time (e.g. something like an automatic background defragment, or downloading an OS X update)?

Could the disk be nearly full?

Your Hauptwerk log files (Help menu) show detailed diagnostic load timing information, so you can verify there whether the load time was previously 20 seconds (which sounds about right to me for the Vollenhove).

Send us a diagnostic file (File | Create diagnostic file) if you like, and we'll see if it gives any clues, although unless the sample set cache is regenerating (which you say isn't the case), or memory is very low (which presumably isn't the case) then I'm not sure it will help much.

Probably a silly question, but I assume you haven't disabled Hauptwerk's high-speed loading mechanism (on the 'General settings | General options' screen)?
Best regards, Martin.
Hauptwerk software designer/developer, Milan Digital Audio.
Offline

deWaverley

Member

  • Posts: 321
  • Joined: Fri Feb 29, 2008 2:33 pm
  • Location: UK, Bath

Re: Slow Load Times - OSX64

PostSun Jun 28, 2009 8:21 pm

As ever - absolutely staggered by how generous you are with your time...an instant detailed reply at 9pm on a Sunday night...almost makes me want to cry!

- nothing happening in the background
- 100 GB of free disk space
- high-speed loading not switched off

But...the Log file does warn from time to time (while loading) that free memory is dangerously low - which is odd, as I have 6GB installed and Vollenhove takes just under 4GB (& I have no other programs running).

I did a fresh "Load Organ adjusting ranks..." and it took 15 minutes to load - is that normal do you think?

Sauer happily loads from the cache in 50-60 secs.

I'll have another investigate tomorrow morning.

Thanks again for your reply.
Offline
User avatar

mdyde

Moderator

  • Posts: 12149
  • Joined: Fri Mar 14, 2003 2:19 pm
  • Location: UK

Re: Slow Load Times - OSX64

PostMon Jun 29, 2009 6:09 am

Thanks - no problem.

But...the Log file does warn from time to time (while loading) that free memory is dangerously low - which is odd, as I have 6GB installed and Vollenhove takes just under 4GB (& I have no other programs running).


The maximum amount of memory that Mac OS X 10.5.x allows to be active before it starts paging data out to disk is usually around 63-67 percent of the installed physical memory. Hauptwerk will give the low-memory warning approximately when you reach that threshold, so the problem is almost certainly just that you're loading more than OS X will reliably allow to remain in active (non-paged out) memory.

That will make loading extremely slow because some of the data have to be read to/from disk at least three times (once for Hauptwerk to load initially, once for the OS to write out to the page file, and once for the OS to read back in again when Hauptwerk re-accesses it towards the end of the loading process), and to/from different parts of the disk (disk 'thrashing'). You'll probably also see a warning message in the log file to the effect that data appear to have been paged out, and warning of poor performance as a result.

Hence just changing the rank options to use a bit less memory will almost certainly solve the issue. With 6 GB of installed memory, for maximum performance/reliability it's probably best to aim to keep Hauptwerk's total memory consumption (shown in the log) to about 3.7 GB or less.

Given that you noticed the message in the log file, it sounds as though perhaps you've disabled Hauptwerk's low-memory warning prompt (on the 'General settings | General options' screen). I'd recommend leaving that prompt enabled because the low-memory warning gives a strong hint that performance issues are highly likely (in terms of slow loading and/or audio glitches).

Of course, installing some extra memory is probably the ideal option. I have third-party Mac Pro-compatible made by Kingston in my Mac Pro, which has always worked fine for me, and is very reasonably priced:

http://www.kingston.com/ukroot/
Best regards, Martin.
Hauptwerk software designer/developer, Milan Digital Audio.
Offline

deWaverley

Member

  • Posts: 321
  • Joined: Fri Feb 29, 2008 2:33 pm
  • Location: UK, Bath

Re: Slow Load Times - OSX64

PostMon Jun 29, 2009 1:46 pm

Thank you so much again Martin for that long, detailed and very helpful reply...I do think you should allow yourself Sundays off though!

That was all very interesting, and I will certainly re-tick the low memory warning box. I had let Hauptwerk load Vollenhove whilst watching the memory page of "Activity Monitor", so I was starting to realise something quite laborious was going on.

It's funny because, when I first read the Log file, and before I had even replied to your first comment, I went on to Crucial's website and ordered another 4GB of memory! So that should help (briefly!).

Thanks so much.

deW
Offline

deWaverley

Member

  • Posts: 321
  • Joined: Fri Feb 29, 2008 2:33 pm
  • Location: UK, Bath

Re: Slow Load Times - OSX64

PostWed Jul 01, 2009 11:24 am

Sorry - I'm back again!

So, I load Hauptwerk and then load Vollenhove (4GB sampleset in my 10GB mac pro). That takes exactly 120 secs.

If I then reload Vollenhove it takes 20 seconds.

(There were no error messages in the Log file for either load, and no other activity on the computer).

I can see why the first load might take longer, but should it really be 2 minutes versus 20 seconds?

Many thanks

deW
Offline
User avatar

mdyde

Moderator

  • Posts: 12149
  • Joined: Fri Mar 14, 2003 2:19 pm
  • Location: UK

Re: Slow Load Times - OSX64

PostWed Jul 01, 2009 11:40 am

That sounds perfectly normal.

The first time (only) you load a sample set (with a given set of rank options) Hauptwerk has to regenerate its sample cache, which is a very processor-intensive (slow) and involves lots of disk reads and writes (also slow).

Subsequently it loads from cache, which is specifically designed for maximum loading speed, and is optimised for multiple CPUs/cores, as well as using asynchronous I/O to get the maximum possible sustained read-rate from your hard-drive(s).
Best regards, Martin.
Hauptwerk software designer/developer, Milan Digital Audio.
Offline

deWaverley

Member

  • Posts: 321
  • Joined: Fri Feb 29, 2008 2:33 pm
  • Location: UK, Bath

Re: Slow Load Times - OSX64

PostWed Jul 01, 2009 3:16 pm

Thanks Martin - but the point is, the first of those loadings was not an "Adjusting rank...", it was a normal load from cache...and yet it takes 2 minutes. An identical subsequent load takes 20 seconds.

[EDIT]...the long load-time happens every time I restart Hauptwerk and load the organ.
Offline
User avatar

mdyde

Moderator

  • Posts: 12149
  • Joined: Fri Mar 14, 2003 2:19 pm
  • Location: UK

Re: Slow Load Times - OSX64

PostWed Jul 01, 2009 4:10 pm

Have you installed the extra memory you bought yet? If not, have you disabled some ranks to get the sample set to load in 3.7 GB or so or less, as I previously suggested?

If you've done one or both of those and you still have the problem with a slow initial load then please send us a diagnostic file so that we can see your timings etc.

If you haven't done either of those things, then it's almost certainly just OS X paging data (hard-disk thrashing), since you're trying to load more data than OS X will comfortably allow to be in memory. It might page more after a fresh reboot than subsequently - that's up to the operating system, and not something Hauptwerk has any control over.
Best regards, Martin.
Hauptwerk software designer/developer, Milan Digital Audio.
Offline

deWaverley

Member

  • Posts: 321
  • Joined: Fri Feb 29, 2008 2:33 pm
  • Location: UK, Bath

Re: Slow Load Times - OSX64

PostWed Jul 01, 2009 6:42 pm

Yes, as I said, this is a 4.1GB sample set loaded into a 10GB mac pro, and there are no memory warnings in the Log file.
Also, it's not just after a reboot that it happens, but every time I restart Hauptwerk.

I get this when Hauptwerk has just been loaded:

****
2009-07-01-20-23-59: INF:2106 Starting to load the organ file OrganART 1720 Bosch-SchnitgerV3.0-extd.Organ.Hauptwerk.hbx. Very approximate estimate of total physical memory currently free/usable: 6081 MB. Very approximate estimate of sample and object memory currently in use by Hauptwerk (not including other data): 18 MB. Multi-threaded loading disabled: N. Organ definition auto-compacting disabled: N. [ACT:2001 Loading an organ.]

2009-07-01-20-26-03: INF:2157 The organ OrganART 1720 Bosch-SchnitgerV3.0-extd.Organ.Hauptwerk.hbx has been loaded successfully. Metrics:

Organ ID: 000227.
Internal diagnostic code: 01000000.
Memory usage: very approximate estimate of total physical memory currently free/usable: 1744.60 MB.
Memory usage: very approximate estimate of sample and object memory currently in use by Hauptwerk (not including other data): 4129.61 MB.
Memory usage: overall compression ratio achieved for all ranks for which memory compression enabled: 42.38 percent.
Total loading time: overall: 123.931 seconds.
Total loading time: sample data: 122.562 seconds.
********

and this if I keep Hauptwerk running and just reload the organ:

***
2009-07-01-20-20-29: INF:2106 Starting to load the organ file OrganART 1720 Bosch-SchnitgerV3.0-extd.Organ.Hauptwerk.hbx. Very approximate estimate of total physical memory currently free/usable: 5870 MB. Very approximate estimate of sample and object memory currently in use by Hauptwerk (not including other data): 18 MB. Multi-threaded loading disabled: N. Organ definition auto-compacting disabled: N. [ACT:2001 Loading an organ.]

2009-07-01-20-20-49: INF:2157 The organ OrganART 1720 Bosch-SchnitgerV3.0-extd.Organ.Hauptwerk.hbx has been loaded successfully. Metrics:

Organ ID: 000227.
Internal diagnostic code: 01000000.
Memory usage: very approximate estimate of total physical memory currently free/usable: 1554.90 MB.
Memory usage: very approximate estimate of sample and object memory currently in use by Hauptwerk (not including other data): 4129.61 MB.
Memory usage: overall compression ratio achieved for all ranks for which memory compression enabled: 42.38 percent.
Total loading time: overall: 20.264 seconds.
Total loading time: sample data: 19.245 seconds.
********

If you need more I will happily send a diagnostic file.

Thanks

deW
Offline
User avatar

B. Milan

Site Admin

  • Posts: 4376
  • Joined: Sat Aug 30, 2003 11:15 am
  • Location: Sarasota, FL. USA

Re: Slow Load Times - OSX64

PostWed Jul 01, 2009 8:11 pm

Are you reloading the Vollenhove without loading other organs in between when you are able to get it loading in 20 seconds? My guess would be that the data is still in the RAM when you simply reload the organ and that it therefore loads much faster. Very often the RAM cache will not be cleared completely and some data remains in memory if nothing else has been loaded to override that data in memory, so it would make sense that from a fresh start it would take over 2 minutes (which actually seems more correct than 20 seconds for a set loading in 4GB or so.) This has more to do with the operating system and how it handles memory allocation rather than Hauptwerk since it deals with the RAM usage on a more global level.

If you can test this same situation with other organs let us know if you get a similar result in the ratio of loading times.
Brett Milan
Owner
MILAN DIGITAL AUDIO

Image
Offline
User avatar

Grant_Youngman

Member

  • Posts: 1156
  • Joined: Sat Jan 20, 2007 7:50 pm
  • Location: Savannah, Ga

Re: Slow Load Times - OSX64

PostWed Jul 01, 2009 9:33 pm

My Mac Pro is a 2007 4-core, with a 1TB WD SE3 drive.

Vollenhove loads fresh in about 42 seconds. If I load it again it loads in about half that time. That's at 24 bits with all features.

By the way, the 64 bit Mac HW version won't let me load another organ without UNLOADING anything currently loaded. If I don't unload the current instrument I get one of those "HW quit unexpectedly" messages. Sometimes HW will relaunch, more often than not it won't. The crash logs show a Qt problem. I figured there are still a myriad of Qt bugs, so haven't worried about it too much.

I had pulled out some Frescobaldi and was loading up Brescia St. Carlo a while ago. I just sat there for a minute thinking about the great fortune of having all of these wonderful instruments to play (badly), and how I almost find myself taking it for granted. When I think that I was playing "odeur de Rodgers, 1981" as my only instrument just 2 years or so ago. It's just unbelievable how life has changed for the better :-)
Grant
Offline

deWaverley

Member

  • Posts: 321
  • Joined: Fri Feb 29, 2008 2:33 pm
  • Location: UK, Bath

Re: Slow Load Times - OSX64

PostThu Jul 02, 2009 4:58 am

Hello Brett - thanks so much for your comment. Yes, the 20 sec load was without loading another organ in between. I can see that it makes sense that it would load faster in those circumstances. But I was just wondering if 2 minutes was longer than it should be - and Grant Youngman's comment bears this out, when his mac pro (older than mine) loads the same set fresh in 40 seconds. It makes me wonder whether the hard disk itself is starting to fail or something.

I will certainly try testing other sets, as you suggest.

Grant, thank you for your experiences. That is very interesting that you get 40 secs compared to my 2 minutes. I'm convinced that mine used to load in around 40 secs - which is why I'm so mystified. It's also strange that I don't get your reload-crash problem at all - everything is absolutely stable here.

I completely agree with you though that what we have to play at the end of the 2 minute wait is absolutely breathtaking organ heaven...I have access to real organs to practise on, but in many ways I'd rather stay in and play my Schnitger, Silbermann, Sauer or CC!

deW
Last edited by deWaverley on Thu Jul 02, 2009 8:10 am, edited 1 time in total.
Offline
User avatar

gingercat

Member

  • Posts: 971
  • Joined: Sun Apr 11, 2004 9:20 am
  • Location: Lancashire, UK

Re: Slow Load Times - OSX64

PostThu Jul 02, 2009 7:19 am

Do Macs have the equivalent of disk defragmentor? Might be worth a try to speed up your disk access?
Regards,
Chris Blaylock
i5 4670k, 32GB RAM, Win7, 2xELO 1280x1024 Touchscreens, Focusrite Saffire, MIDISport 4x4
4 Manual Console, 32 R&C Pedalboard, 3xExpression, Solenoid coupler tabs
Offline
User avatar

mdyde

Moderator

  • Posts: 12149
  • Joined: Fri Mar 14, 2003 2:19 pm
  • Location: UK

Re: Slow Load Times - OSX64

PostThu Jul 02, 2009 8:19 am

Yes, as I said, this is a 4.1GB sample set loaded into a 10GB mac pro


Sorry - I should have read your post more thoroughly.

Perhaps try emptying your OS X Trash Can and then just forcing the sample cache to be regenerated by Hauptwerk for the Vollenove (load the organ with 'Organ | Load organ, adjusting rank audio output routing' and just click OK). Maybe the cache file for that sample set is fragmented on disk for some reason, making it unusually slow to read from disk. Given that you mentioned that the Sauer sample set loads in 50 seconds or so, I would doubt that there's anything wrong with the disk hardware itself.

I've just set up OS X 10.5.7 with all the latest updates on my 2006 quad-core 2.66 GHz 8 GB Mac Pro, single 500 GB SATA hard-drive (no RAID), with the Vollenhove and Hauptwerk v3.22.

Using all default rank settings, except with all ranks set to load in 24-bit (instead of the default 16-bit), load statistics when loading from sample set cache after a clean reboot in each case are as follows:

Non-extended Vollenhove:

2009-07-02-12-16-29: INF:2157 The organ OrganART 1720 Bosch-Schnitger.Organ.Hauptwerk.hbx has been loaded successfully. Metrics:

Organ ID: 000225.
Internal diagnostic code: 01000000.
Memory usage: very approximate estimate of total physical memory currently free/usable: 1568.24 MB.
Memory usage: very approximate estimate of sample and object memory currently in use by Hauptwerk (not including other data): 3128.26 MB.
Memory usage: overall compression ratio achieved for all ranks for which memory compression enabled: 41.29 percent.
Total loading time: overall: 53.613 seconds.
Total loading time: sample data: 52.461 seconds.


Extended Vollenhove:

2009-07-02-12-43-05: INF:2157 The organ OrganART 1720 Bosch-Schnitger-extd.Organ.Hauptwerk.hbx has been loaded successfully. Metrics:

Organ ID: 000226.
Internal diagnostic code: 01000000.
Memory usage: very approximate estimate of total physical memory currently free/usable: 1420.55 MB.
Memory usage: very approximate estimate of sample and object memory currently in use by Hauptwerk (not including other data): 3275.61 MB.
Memory usage: overall compression ratio achieved for all ranks for which memory compression enabled: 41.20 percent.
Total loading time: overall: 56.696 seconds.
Total loading time: sample data: 55.538 seconds.


(Re-loading the sample set straight away without closing Hauptwerk then takes around 17-20 seconds, but that's just because OS X still has the disk file in its file system cache in memory, i.e. doesn't need to read it from the physical disk, as Brett described.)

Since your Mac Pro is more recent than mine, I would expect you should be able to load the sample set (with a given set of rank settings) at least as fast as my Mac Pro can.

If you need more I will happily send a diagnostic file.


If regenerating the cache/defragmenting doesn't help, then yes please - I'm not sure exactly what rank and other settings you're using, and whether they might be relevant, but if you can send us a diagnostic file then we can see those things, and also try using your settings exactly.

Do you have all current OS X updates applied? If not, please try applying them for good measure (unlikely to make a difference, but at least then your configuration should more closely match ours).

Fragmentation within files is supposed not to be a significant issue on OS X (unlike Windows NTFS, where file fragmention can be extremely high, although Hauptwerk specifically tries to avoid that by the way it handles its files), and drive fragmentation is also supposed not to be a problem unless your drive is fairly full (say at least 70 percent), e.g.:

http://support.apple.com/kb/HT1375?viewlocale=en_US

http://www.macfixitforums.com/ubbthread ... d/1#import

http://www.macupdate.com/info.php/id/4307

The last link mentions that the Apple Care Protection Plan utilities DVD includes TechTool Deluxe, which probably has defragmenting capability, so that might be worth trying, although I haven't had a need to try it myself.
Best regards, Martin.
Hauptwerk software designer/developer, Milan Digital Audio.

Return to Technical support

Who is online

Users browsing this forum: Bing [Bot] and 2 guests