How should a sampled synthesiser work?

burkey

Just as a bit of a coding challenge, I'm going to have a crack at building a piano synthesizer.

I realize that I will need to cache into RAM the initial attack waveform of each note - in order to give sufficient time to load the rest of the samples from file.

However I am wondering if caching the sympathetic resonances separately will also be of some benefit?

I.e. if two C strings are open - then cache the resonances of those notes in preparation that they may be struck by hammers. If the sustain pedal is down, then this would mean caching the resonances for all potential note combinations.

burkey

burkey However I am wondering if caching the sympathetic resonances separately will also be of some benefit?

On second thoughts the resonances probably don't need to be played until after the initial attack samples of the struck strings have been played. So these resonance samples probably don't need to be cached into RAM.

magicpiano

burkey If the sustain pedal is down, then this would mean caching the resonances for all potential note combinations.

It's impossible to sample all the potential combinations of sympathetic resonances that could happen on an acoustic piano. If I'm not wrong, for 88 keys they would be ((2⁸⁸) - 1).
I think some piano VST use different samples for the single notes played with sustain pedal down, so that the resonance from the other strings should be included in the sample itself... But IMHO this is just a very rough approximation of what happens on the real thing...

xooorx

Sympathetic resonances are where those open C strings start vibrating because a hammer hit an F string, and then carry on vibrating even after the F string has been damped again. Or they start vibrating -- but with a different timbre each time -- because a hammer hit a different F string or an E string or another C string or whatever other string or combination of strings. It can't be done accurately with samples there has to be some kind of modelling or synthesis or (possibly sample based) fakery going on to get these effects.

Del Vento

Take a look at https://www.fluidsynth.org/

burkey

magicpiano It's impossible to sample all the potential combinations of sympathetic resonances that could happen on an acoustic piano

I was only planning on recording the 3rds, 5ths, and octaves.

burkey

magicpiano some piano VST use different samples for the single notes played with sustain pedal down, so that the resonance from the other strings should be included in the sample itself

That actually seems like a better idea - thanks!

Although suppose I play a chord: 1-3-5, there should be resonance included in the sound of just those 3 notes. That resonance would sound different to when I have the sustain opening all strings.

MacMacMac

Yes, but you don't need to mimic every combination of sounds that a piano can produce.
You need only create those sounds that a person can hear.

Perception is not reality. But perception is all that matters.

vagfilm

MacMacMac Perception is not reality. But perception is all that matters.

magicpiano

MacMacMac Yes, but you don't need to mimic every combination of sounds that a piano can produce.
You need only create those sounds that a person can hear.

Yeah, but a person with good ears can hear much much more details and dynamics from a real grand piano compared to a sample-based VST, and I believe sympathetic resonance is one of those things where there is still much work to do.

Drew-r

MacMacMac You need only create those sounds that a person can hear.

Perception is not reality. But perception is all that matters.

But who decides to create only what are the sounds a person can hear …. the engineers ….. the engineers in cooperation with marketers …. other experts ….?

Admittedly I’m not the sharpest piano enthusiast here but sometimes i think that many others far sharper than i forget or have a reason to overlook that it is the difference in perceptions we all have that is really the basis of everything piano that gets discussed/hashed/debated and, especially, disagreed upon.

magicpiano

burkey Although suppose I play a chord: 1-3-5, there should be resonance included in the sound of just those 3 notes. That resonance would sound different to when I have the sustain opening all strings.

Yes, that's why I said sampling single notes pedal-up + single notes pedal-down is just a rough approximation. Anyway, better than nothing... The alternative would be to use some smart algorithm to "generate" the sympathetic resonance dynamically, based on the currently open strings...

burkey

magicpiano well what I can try is taking the pedal down sample and applying simple frequency filtering to isolate the 3rds, 5ths, and octaves.

MacMacMac

Drew-r But who decides to create only what are the sounds a person can hear …. the engineers ….. the
engineers in cooperation with marketers …. other experts ….?

Same as with any product. You study the market, you develop a product, and you hope it sells.

dore_m

So let me throw in a bunch of random musings based on recent experience. Hopefully some of my descriptions will be helpful.

I'm currently making my new sample library in both Kontakt and in Sfz format. I primarily use Kontakt because the tools are easier to use, such as the ability to have a filters change based on velocity, or simply the ability to control how much a velocity layer changes in volume from low to high - which makes the transitions between velocity layers less noticeable. SFZ format allows for all of this too, but you have script it rather than use a graphical tool. So I make it first in Kontakt, and then mimic the curves when I make the SFZ version.

Now, you can go crazy and sample all 88 keys at 127 velocity layers using pedal up and pedal down, release noises, una corda pedal, etc. But I've found that just recording minor thirds up the piano is enough too get good realism (always pitch down, never pitch up the samples), I did fourths at first, but when you play major scales, hearing 3 notes in succession that were pitch shifted sounds fake. Minor thirds is about the level at which you save time recording/processing/mapping without sacrificing too much realism. I record pedal up and pedal down at 4 velocity layers, which allows me to get the full breadth in brightness of the piano. Using the 4 velocity layers, you can also make "virtual velocity layers" which I can describe in more detail, but basically is just taking a louder layer, cutting the gain, and applying a strategic cutoff filter to mimic the brightness at that velocity range. Its amazing at how that sort of processing can be used to take a minimal amount of samples and create something pretty realistic.

So with those tools, a new sample based virtual instrument engine needs to have a basic sample playback engine in the very least, and then you have to decide how much pre-processing you want to do to the samples, or have the engine do the filtering/processing in realtime (like Kontakt). Feel free to use any of my sample libraries to experiment.

In recent weeks, I have be scripting half pedaling mimicking Garritan. I've found that SFZ is much easier because it allows me to do realtime amplifier envelope changes - that is, I can get CC64 values to change the release time and volume in realtime. Kontakt, however, does not allow you to do this unless you record ANOTHER sample to control the entire ADSR of that sample. So what I'm trying to say is that when you make a piano playback engine, you might want to consider how you will control ADSR envelopes.

Hope some of that makes sense.

burkey

dore_m go crazy and sample all 88 keys at 127 velocity layers using pedal up and pedal down

Thanks, I was thinking one way to halve the number of samples would be to only record the notes with sustain pedal down. Then simply filter out the extra resonance when we need to play it without sustain - it seems filtering out frequencies would be less complex to implement than adding in resonances?

magicpiano

burkey it seems filtering out frequencies would be less complex to implement than adding in resonances?

Filtering out frequencies is much harder than you think, because the "details" in a piano sound (like the sympathetic resonances, duplex-scale, undamped strings, etc.) are not just high frequencies, but they are spanned on a much bigger frequency range. So they are fused with the basic waveform in such a way that would be overkill (from a CPU standpoint) trying to extract them without interfering with the rest of the waveform.
In sound processing there is a rule I learned: adding details is much easier than removing details from a complex waveform.

dore_m

Certainly filtering to remove frequencies is going to be easier than boosting what may or may not be there. I've never tried to simulate pedal up samples by filtering pedal down samples however. Not sure it will work, exactly. In my experience, the low pass filtering I use results in a duller sound, more like a lower velocity layer than a pedal up (dampers down) sample. But I cant say I'm an expert at filtering. You can try it by just using my sample library "the Experience" on pianobook. find one sample with PU (pedal up), find the same note and velocity with pedal down (PD), and try to filter one to sound like the other.

dore_m

@magicpiano is right. I think probably sustain pedal affects the entire waveform rather than just a frequency range, whereas a higher velocity excites more partials, which can be filtered more or less.

I think by adding details you mean layering different samples? Or increasing gain to accentuate frequencies? Because with pianos I don't think you can increase anything on a pedal up sample to make it sound like a pedal down sample, you'd have to layer another sample on top.