Audio drop out with SP Caen set

[imcg110]

Hi Guys

Just got the Caen CC surround up and running. Did some voicing and started playing, but I am loosing a lot of notes! Never had this before, but it is my biggest sample set so far. When I play on any more than about 60% of the organ (playing fast notes) I get lost notes - most noticeable when the reeds drop out. I can make it glitch and crack in the traditional manner if I push polyphony up to >6500, The buffer is set at 1024 (general settings). I should get 4800 ish polyphony on my system and other other sets are OK at this. The processor never goes above 54% load. The only new recent feature is an additional audio channel to help with the surround. Tweaking the General Settings buffer doesn't make much difference.

Mac Pro Quad 2.66
OSX 10.5.6 HW3.21 64 bit
13GB RAM
Set loads in 9.6GB
Saffire Focusrite 4xstereo channels used
HW Set to maximise for polyphony

Given the plethora of knobs to tweak I thought I would ask advice first. This is new to me - I have always spent my time playing, not tweaking up till now!!

[B. Milan]

Hello Iain,

When you say you are losing notes, do you mean they are just not playing or that the polyphony management system is fading out notes?

Note that adding more audio outputs will increase the processor overhead, so this is most likely why. Also make sure you are not exceeding the RAM limits on OS X (approximtely 2/3 of overall RAM can safely be used before disk swapping occurs). Since you have exceeded that amount using over 9GB out of 13 GB this is probably a reason why as well.

Also if you are running the wet and rear versions together that will double the polyphony. So, if it is a case of the polyphony management working as it should, then you are just exceeding the polyphony limits of your computer.

[micdev]

Ian,

Don't forget that the surround asked twice the polyphony since you have the same number of ranks/pipe playing in the front as in the back (so it is like play a 110 ranks organ!), so it can indeed ask a lot to your computer.

The problem occurs when playing tutti (or close to it), try to setup HW for "more polyphony and less realism", this might help a bit.

A polyphony of 3000 reaches 6000 when using the surround!

I should receive my set in a few days and will be able to test and report if I have the same problem.

For fun, try using the surround ranks to the front and use an aux mix channel for the rear, my poor man's surround solution, and this time you will use the polyphony of only one set of ranks.

Regards
François

[gingercat]

I also get this with my Caen with the surround set - it suddenly sounds like most of the stops have been pushed in with only a few still sounding as you play more notes. The virtual keyboards do still show the keys being pressed.

Not sure if it is processor or memory related, as reducing the memory footprint (going mono, single loops etc.) does improve, but not remove the problem. I have 8GB RAM, and with the surround set loaded, I pretty much fill it.

Increasing the polyphony setting in Hauptwerk doesn't remove the problem as I then hit clipping instead. I normally run at about 80% on each core on my Q6600.

The wet set, whilst occupying the same amount of memory (everything turned "on") doesn't have the same problem, so I can only conclude it must be my processor running out of oomph with the surround set!

To conclude, I don't think this is a bug, but does give me an excuse to think about upgrading my PC...

[gingercat]

A polyphony of 3000 reaches 6000 when using the surround!

I would imagine that the Hauptwerk setting would remain the same, but the effective polyphony is halved?

[imcg110]

Mmmmm?

2 points

- if polyphony limits are mainly based on processing power, why a problem at 50% capacity? I know the surround version stresses the machine more - but the Metz in full flight gets up to 34% max in comparison. If I had the memory I could load the same organ with compression disabled and see what happens. I have already pruned it down to lots of mono and 16bit.

- the Mac memory issue is a bit belt and braces - yes it pages if the 2/3 limit is passed on caching a set, but as the set is "activated" on the final pass, there is a pause midway where the paged files are passed to the inactive memory. You can watch it happen on the activity monitor. I load the Sauer organ in >9GB 24bit with no issues.

I must admit I don't truely understand buffer issues on HW! Perhaps this has a bearing on the problem?

The bottom line is that I think I am working well within the limits of my system and am not sure where to go from here.

[gingercat]

Did you try increasing the polyphony setting? I can't because I run out of power.

[imcg110]

Yup

I am usually set at 4500
Reducing it to <1000 makes the problem worse
I can increase to up 6500 when I get clipping, pops, cracks etc but this doesn't seem to have a great effect on the problem

Reducing the buffer to lowest level (ie top line in menu) does help a little

Could there be a OSX 64 bit paging issue with buffering if the samples occupy >2/3 RAM??? The mac extended memory issue is new to us all! Not an issue with Sauer/Metz however. I somehow doubt this if Chris has a similar problem on a PC.

I suppose the 2 possible approaches are to max out the memory even more and go for uncompressed sounds to ease the processors load or prune down the memory requirements even further and see what happens. Given that this is slooooow++ I thought I would give it an airing here first.

[gingercat]

I wonder if it is possible some of the release samples are incorrectly sized, thus reducing the polyphony further (to the point new notes can't start). I say this because when hitting the wall of silence, if I stop playing, it takes a while for CPU useage to go down again.

[imcg110]

I think you might have something there Chris - that fits with my feeling too. Once the reverb tails have died out you can start again.

[B. Milan]

Hello Iain,

I think it's just a simple case of your computer not being able to cope with the instrument which seems to require more polyphony than you can give it. Hauptwerk's polyphony management system fades the tails out of previously played notes to make way for new attacks of the new notes you play.

Also by exceeding the 2/3 RAM limit in OS X, as I mentioned above, the samples *will* be paged to the hard drive. You can watch this in the activity monitor while the organ loads. This is nothing that we can control as it is a part of OS X and its built-in methods for ensuring that there is enough memory for the OS to use. Windows is more forgiving with allowing you to load nearer the maximum RAM limits, but with OS X you need to try and stay within the 2/3 mark. Feel free to contact Apple for us to see if they can change it! ;)

I would first suggest just trying to load in less memory, then see if that helps since without proper resources for the OS the software would not function properly anyhow. That coupled with setting your polyphony correctly should help further.

If all else fails, then you would either need to play with less full registrations that your computer can handle, or upgrade your computer to handle the Caen set.

[gingercat]

Hauptwerk's polyphony management system fades the tails out of previously played notes to make way for new attacks of the new notes you play.

That doesn't seem to be happening in these cases though, the new notes simply don't sound, even though the processor isn't maxed out. Even playing a single note during the "silence" only sounds a few pipes, even on Tutti, but from what you've said above, that suggests it should indeed be able to sound, unless the CPU was thrashing, in which case I would expect audio breakup, not silence.

[imcg110]

Hmmm........

Curious that we are now suggesting the need to upgrade from a macpro to run an already heavily truncated sample set:-) I hope what you are saying is that this is probably a memory issue - I cannot see that I have an issue with processing power.

Brett, your description of memory handling seems to be at odds with Martin's. He described in detail stress testing and suggested that if the "keep sample set in active memory" box was ticked that HW and OSX would run in active RAM beyond 2/3. At least that is how I read it :-)

Could someone give a brief idiots guide as to how HW handles buffering to inform my choices.

I will prune memory requirements further and report back.

I hope you will see this as feedback on a beta software rather than just whinging. If we don't push the software a bit we would have little to report back!! I think we should also push Apple - With 30GB of RAM installed how could they possibly need 10GB headroom to run the system! Thats £400 of my money for their comfort zone.

[gingercat]

I hope you will see this as feedback on a beta software

Happens with Windows too, so it isn't down to the Mac 2/3 memory issue. I don't have any paging when I see the problem.

[mdyde]

Hello Iain,

Brett, your description of memory handling seems to be at odds with Martin's. He described in detail stress testing and suggested that if the "keep sample set in active memory" box was ticked that HW and OSX would run in active RAM beyond 2/3. At least that is how I read it :-)

Not really - I previously tried exceeding the 2/3 memory barrier (using the Metz at about 7.5 GB of data on quad-core Mac Pro with 8 GB of memory) and I found that, in that instance at least, I didn't have any problems (except for extremely slow loading, etc.). However, I did say that sticking within the 2/3 limit is the safe option. Filling the memory isn't necessarily safe, although in my particular test I didn't have any problems.

My general advice would be not to exceed the 2/3 limit.

Could someone give a brief idiots guide as to how HW handles buffering to inform my choices.

I'm not sure specifically what you're referring to by 'buffering', but for the audio buffer size, broadly speaking, the smaller the buffer size the less latency but the higher CPU demands. The 'performance tuning' section in the user guide covers it in more depth:

http://www.hauptwerk.com/clientuploads/documentation/CurrentUserGuide/UserGuideRedirects/PerformanceTuning.pdf

Setting the audio buffer sizes to 1024 for all audio outputs (General settings | Audio outputs) is conservative, and should give good performance (although lower settings will give lower latency at the expense of polyphony).

I hope you will see this as feedback on a beta software rather than just whinging. If we don't push the software a bit we would have little to report back!!

The Hauptwerk audio engine code is indentical for all platforms, so it shouldn't really make any difference which platform you're using (unless paging is occurring, which is something outside of Hauptwerk's control).

I think we should also push Apple - With 30GB of RAM installed how could they possibly need 10GB headroom to run the system! Thats £400 of my money for their comfort zone.

It isn't that the operating system itself uses 1/3 of the memory - it's just that OS X tries to be proactive in handling low memory situations as unobtrusively as possible before they become catastrophic. Windows doesn't do that so much, but on Windows there is a still a risk that the OS will page data if the memory is getting full. Also Windows sometimes pages *all* of an application's data out in low memory situations (which is of course catastrophic for performance, both of the application and of the system as a whole), whereas OS X tries to avoid that by starting to take preventative action when free memory is running below about 1/3 of the physical memory. Generally speaking, that makes OS X's memory performance more robust, provided that you stick within the 2/3 limit, although from Hauptwerk's point of view it would be better if that OS safety margin could be adjusted (e.g. allowing 80 percent of physical memory to be used before the OS started taking proactive preventative measures).