Agreed. Considering that 1 GB of sample loops is about 1 hour of uncompressed audio at 24 bits, 48k samples per second, and compression ratios are fairly consistent, there's a direct relationship between memory required and how much audio time is recorded and shipped in the sample loops and releases.
So more mic perspectives, longer reverb, longer loops, and sampled trems all require more memory. There are a lot of options for reducing RAM needs on a big sampleset, including running Janke at 20 bits in less than 20 GB of RAM. But there's no way to get more detail if it's not there in the first place.
A small memory sampleset isn't automatically of lower sound quality than a memory hog, but I'd rather have more and longer recordings and leave some out if necessary.