Yet another Pianoteq critique

If I may reuse Frédéric L example. The FM-synthesis allowed for making the (in)famous DX-piano sound that basically destroyed Rhodes pianos. It didn’t sound exactly like Rhodes but nobody cared 🙂 It was kind of Rhodish and you could say it modeled a Rhodes. But it didn’t 😀 It just happened to synthesize sound that resembled Rhodes piano sound. I think to a certain degree that’s how Pianoteq works, although it’s a modern product that is specifically created to emulate/model acoustic pianos and is thus much closer to a real piano than a DX7 is to a Rhodes.

We get into semantic territory easily here. Technically, generating sound through samples is also modelling, where the model is highly complex and contains the samples for the individual notes (except some of the notes may be found via interpolation, which is truly and honestly an example of modelling).

CyberGene I think the slight stagnation in Pianoteq progress so far is largely determined by a saturation point in CPU speeds.

Yes, when PTQ came out in 2006, this problem had just started: https://cs.stackexchange.com/questions/27875/moores-law-and-clock-speed and many people (including PTQ developers, I reckon), thought it was a temporary blip, if they even were aware of it. On the other hand, storage was not on a path to catch up with computing. We know how it went: CPU clock speed not only stagnated, but regressed. The fastest CPU speeds were indeed reached around or shortly after PTQ came into existance (e.g. https://en.wikipedia.org/wiki/POWER6 and https://en.wikipedia.org/wiki/Clock_rate#Historical_milestones_and_current_records ) and then at best stayed constant, but in many case they became slower. Yes, multicore came, and a few others tricks, which mitigated the clock speed stagnation/decrease, but those are effective only for certain workloads. I am not sure what is the PTQ (computing) limiting factor, I can speculate it's the FFT, in which case multicore definitely helps and will continue to with increasing core count (which does not appear to be slowing down, even though very large core counts are not common for home computers, because they are not useful for more common home computer tasks). So PTQ in a sense painted themselves in a corner.

On the other hand, storage did catch up with computing, so arguably from this point of view samples have the advantage now (from a pure technological perspective). The only reason why they "feel" clunky is that from a commercial point of view the home computers are usually not "pushed" from a storage point of view, since most of the storage (i.e. pictures and videos of your cat) are preferred to be stored in the cloud for convenience (i.e. so you are locked with your vendor, rather than in charge of your data 🤣) -- but the technology is more readily available for the home user: you can easily buy and use a multi TB storage; whereas you can buy a 128-core CPU, but it'd be very hard if not impossible to use it on your home computer. Assuming that PTQ limiting factor is the FFT at a size which would benefit from such a CPU (like samples benefit from larger disks) and not something else yet (which will certainly become important at some point, for the Amdahl's law, and that point might be even already here, and so PTQ would not benefit from such a 128-core CPU)

CyberGene We’re probably hundreds of years from being able to model a piano in real time with finite elements. If that’s possible at all.

I think with this chip you could do it today. But I don't think you can afford to use that chip in your laptop anytime soon, and it'd probably be a long shot for using it even in your datacenter.

But the question is: would a more physical modeling sound more realistic to you and @MacMacMac ?

QuasiUnaFantasia Technically, generating sound through samples is also modelling, where the model is highly complex and contains the sample for the individual notes (except some of the notes may be found via interpolation, which is truly and honestly an example of modelling).

Agreed, besides the fact that I would call the "sampling with interpolation" a quite simple model (i.e. most of the information is in the input data and the model does very little), whereas the PTQ's way is quite complex (i.e. most of the information is in the model equation themselves, rather than in the data).

Pete14 Samples, as delvento points out, are now modelling resonances, and this is an admission that mics can’t capture the elusive and mysterious resonance(s) the way modelling does.

There are two things : the resonance of the sustain pedal : any microphone can capture them and the resonance between two strings without the pedal. Sampling a piano key per key can’t capture them because it is done key per key. Then many virtual pianos add a string resonance with either a modelled signal (some sine wave ?) or resonance samples (even there, there is a part of modelling : the law which tells a C4 and a G4 together trigger a G5 resonance at -8dB…).

Then on modern sampled virtual piano, it is not 100% sampled, but what the point : what matters is the result.

Your question ...

RIP Would a more physical modeling sound more realistic to you and @MacMacMac ?

... was answered by Frederic ...

Frédéric L what matters is the result.

Models schmodels. Samples shmamples.
I don't care if the sound is produced by monkeys banging on banyan trees. All that matters is the result.
If it sounds like a good piano then it is a good piano.

CyberGene I think the slight stagnation in Pianoteq progress so far is largely determined by a saturation point in CPU speeds.

Nowadays CPU are far more powerful than the ones 5 years ago. I would bet more money on the lack of ideas about how to enhance the product. On a sample base piano, following the progress is easy : multiply the number of layers, add resonance samples, etc. With a modelled sample with complex formulae one would have to add a second order parameter which enhance the whole : not an easy task.

CyberGene Honestly, don’t really think Pianoteq qualifies as “modeled” in the way many people imagine it.

I think we should use « algorithmic » synthesis instead of modelled. The idea of Pianoteq is to produce a sound with a bunch of mathematical formulae based on basic functions (sine, multiply…). But well anything which is not « sampled » based is named « modelled » : we have to adapt.

RIP Assuming that PTQ limiting factor is the FFT at a size

I am not sure Pianoteq is based on FFT… it would be to simple : any MP3 decoder includes a real-time FFT decoder.

The Modartt patent describes an additive synthesis with partial whose the frequency are not multiple of the FFT buffer frequency : you can’t apply (inverse) FFT in this case.

In agreement with Frédéric L, I'm fairly convinced Modartt uses physical based modelling, implying modelling the vibrations of strings and soundboard using differential equations (applied mathematics stuff). FFT does not seem at all relevant here, unless they use them for convenience to operate in the Fourier transformed space, which may or may not apply, but I suspect it doesn't.

Frédéric L I am not sure Pianoteq is based on FFT…

QuasiUnaFantasia FFT does not seem at all relevant here,

Well, we can argue whether or not FFT is "essential" or "convenience" for PTQ and we can look at their patent(s) and paper(s) for that, but in any case it's just speculation, since the software is closed source. What is definitely true is that they do use FFT transforms (and inverse), as it is clear from https://www.modartt.com/user_manual?product=pianoteq&lang=en#intellectual-property

But you guys are nitpicking on a detail. My (admittedly poorly explained) point was "the CPU improvements these days are mostly coming from increased number of cores and only a few applications can take fully advantage of that", hence PTQ can not "ride" the CPU speed increase as many applications did before 2007. Yes, PTQ can "ride" marginal increases in overall CPU performance, which are still happening, but those are marginal. On the other hand, sample-based virtual pianos can "ride" the storage speed and capacity increases which are happening at faster pace compared to CPU performance increase. And I say that with discomfort, because from a software point of view I like PTQ much better, but the hardware situation is what it is. In the end, anyway, as @MacMacMac says

MacMacMac All that matters is the result.

Didn't someone from Modartt say that Pianoteq is partly pre-rendered and that it's not actually doing all the calculations in real time. Whether that's just the software taking shortcuts to get the same result as running entirely in real time on a supercomputer, or if it's something approaching finished samples, it doesn't really matter.

I think it's very unlikely that the differences between say a Blüthner and a Steinway model come from physical measurements e.g. soundboard wood properties of actual instruments. The Bechstein model didn't even use a real piano, but a sampled one, so unlesss they use a different type of model for that, I assume they fit whatever parameters they can to match the sound.

CyberGene We have this saying: I just managed to teach the horse not to eat, but then it died due to bad luck.

Which reminds me of Frank Lloyd Wright's saying that “A doctor can bury his mistakes, but an architect can only advise his clients to plant vines.” 

I know someone who had cancer many years ago. The doctor told him that the cancer wouldn't kill him -provided he survived the chemo. Which he did and he's still going strong.

I think people can sometimes come across as ideologically opposed to a particular methodology in creating an instrument. I'm sure that when the patient eventually survives the modelling treatment Pianoteq will be great. Some sampled pianos already are. I thought TruePianos, which is a modelling/sampling hybrid, had great promise but it seems to have barely developed in recent years.

Johnstaf Didn't someone from Modartt say that Pianoteq is partly pre-rendered and that it's not actually doing all the calculations in real time.

The Modart patent describes a synthesis module (quite simple additive synthesis) and a « presynthesis » module.

Then one module pre rendered wouldn’t surprise me.
See https://patents.google.com/patent/US7915515B2/en

The idea of the patent is to pre-calculate within the presynthesis some coefficients used afterward in the synthesis module for some velocities, then with interpolation of the actual velocity we can get the coefficients to be used.

RIP But you guys are nitpicking on a detail.

MacMacMac I can't speak for today's pianos ... but Pianoteq is better than the native piano sound in my 2005-era Clav. That's not really saying much because I despise the latter and I never use it at all.

This. Miles better than my top-end 2002 Clav and its 4 layers of AWM samples, playability is dreadful (and that's not the keyboard, the GH3 might be heavy but it's actually very responsive with Pianoteq).

Never mind Laurel and Yanny, maybe the real correlation is between what people think of Pianoteq and what they think the Modartt patent is saying.

xooorx Never mind Laurel and Yanny, maybe the real correlation is between what people think of Pianoteq and what they think the Modartt patent is saying.

I've expressed that idea many times, of course I'm on the other side of the fence but I basically agree it works both ways. For instance, I think some people who love Pianoteq are mostly preconditions by the assumption of how better the modeling approach should be compared to sampling. And an opposite explanation can also be found I guess. For instance I started following Pianoteq since version 1 and at the beginning I really wanted to believe it is the superior approach but was repeatedly disappointed by those initial versions which were pure cr*p, let's face it. I guess the long history of subpar piano sound with all the Pianoteq versions up to 4 might have influenced me to the point I'm now very disappointed by modeling at all, in principle, and thus I tend to expect the worst before even hearing what the actual progress is. I think I may have exaggerated how bad Pianoteq currently is. It's not bad. But I still wouldn't use it for either personal playing, nor audio exports for my videos since I sincerely and honestly find sample-based pianos to work better for me in both cases.

Isn't this a contradiction?

CyberGene I think I may have exaggerated how bad Pianoteq currently is. It's not bad.

But I still wouldn't use it for either personal playing, nor audio exports for my videos since I sincerely and honestly find sample-based pianos to work better for me in both cases.

It's not bad … but you won't use it. To me that seems to say "it's bad".

To be perfectly frank, I find it amazing that a synthesis method (or modeling, or whatever it is called and is just a few megabytes) can produce a very realistic piano sound 🙂 Well, I wouldn't use it because I personally don't like it but I think it's more than good for what it does with the limited capabilities of their current algorithms and processing power. Kudos for that and I'm not being ironic.

CyberGene To be perfectly frank, I find it amazing that a synthesis method (or modeling, or whatever it is called and is just a few megabytes) can produce a very realistic piano sound

Finally, an admission! 😀