OK I tried various filters and some of them sort of worked a bit, but nothing worked as well as just doing the differentiation between non-adjacent samples e.g. instead of V = P[0] - P[-1], using V = (P[0] - P[-4])/4.
Cleans up the 8-bit version to this:
Cleans up the 12-bit version to this:
(Also note the plots in the previous post used a fancy 4-point differentiator:
V = (P[-2] - 8P[-1] + 8P[1] - P[2])/12
which is worse than:
V = P[0] - P[-1])
Will write more when I’m at a computer instead of the phone, but for now, I wanted to point out the (p(0)-p(-4))/4 differentiation is really just the p(0)-p(-1) followed by a length-4 unit-area boxcar. The differentiation is inherently high pass, emphasizing noise where there’s not really any signal you would care about, but the boxcar does a passable job of filtering out much of that noise.
The best filter to use in place of the boxcar depends on what we know of the signal we’re trying to detect. For a pure sine wave, the ideal filter would be a narrow bandpass filter tuned to the frequency of the sine wave. For a hammer position which is the second integral of what can look like a spikey acceleration, I would go for some second order lowpass filter, or basically a simple biquad filter.
More later…
luns Will write more when I’m at a computer instead of the phone, but for now, I wanted to point out the (p(0)-p(-4))/4 differentiation is really just the p(0)-p(-1) followed by a length-4 unit-area boxcar.
Aha... I tried various ways of filtering the position signal before differentiating but got better results by, effectively, differentiating the position and low-pass filtering the result. (Am no longer sure which way round @gzpiano was doing it).
luns The best filter to use in place of the boxcar depends on what we know of the signal we’re trying to detect.
For quantisation noise it might be that prevention is better than cure i.e. use higher bit converters. @gzpiano's posted data was quantised to 3-digit decimal but came from a system that will soon be doing 24 bits. OTOH my own plans currently involve a 12-bit converter so I need to work out if that's enough or if I need to switch it up.
JayKominek You'll have plenty of clock cycles for any integer algorithm you could want, but with only M0 cores, I'd expect single precision floating point algorithms to be prohibitive. I think something like an RC filter at 20kHz might be the limit.
Well, unless we prove it otherwise by a big success, the RP2040 is just a test platform for a different ADC, namely one with only 12 bits (less than 9 ENOB) and higher frequency (up to 500kHz), the exact opposite of what @gzpiano is doing (IIRC he uses 24 bits and 8kHz, but check his videos for an authoritative answer).
As such the cost of fully fledged floating point live data processing on the RP2040 is irrelevant. That said, I grew up writing primitive numerical algorithms on a 6502 using just individual bytes. And later watched in awe FRACT386 (later still renamed FRACTINT). If needs be, we can experiment with numerical algorithms with bytes again, so I'll feel younger 🙂 -- the main limitation if one needs speed is that it'd be easier to use fixed point position, but I am wondering why I'm even getting there besides nostalgia, since I just said we won't do that "in production"…
Sorry for the short detour, let's go back to the good discussion @xooorx and @luns are having.
Looks like there is another DIYer... Maybe one of you can contact him and bring him in?
http://forum.pianoworld.com/ubbthreads.php/ubb/showflat/Number/3185913/gonew/1/attempt-to-create-a-midi-controller-for-the-mechanics-of-the.html
vagfilm that is Alexey Schterbakov, I think we exchanged some messages in the past regarding Cybrid, he was very interested in recreating it. He also promoted me on the Pianoteq forum (not knowing that I am a persona non grata there 🤣). You can invite him here, he will be interested.
CyberGene ah, just doing my end of working day forums catch up, PW first 😜and I posted on his entry there with some links.
Totally orthogonal to everything: how does one quote text in this forum? I'm used to having a button that quotes an entire post I'm replying to, which I can then trim down, but best I can figure here, it looks like I would need to manually cut/paste (and corrupt) what I'm quoting here. That feels like putting words into somebody else's mouth, even if the intent that it be their own words.
CyberGene RIP Thanks for the welcome, guys. I imagine I was too terse in trying to make a minor point that it may have come across wrong. I agree that it doesn't fundamentally matter where the sensors are and I have no objections to the sensor being in either location. There was a post I'd seen - likely in a different thread rather than here (things are a little jumbled while I'm still taking everything in) that seemed to emphasize some significance in the hammers moving 5x the speed of the key, and my only point was that it doesn't really matter. I'll just leave it there rather than beat a dead horse. 🙂
xooorx 1) Let-off is set at 4mm (page 10). Possibly slightly wider than a pure acoustic action to accommodate the stop rail? To make the sensing easier?
I imagine this is entirely to allow room for the stop rail to stop the hammers before they hit the strings.
3) From the section on adjusting the sensor height (page 12):
For instances, if the sensor is located at an extremely high position, the position of emitting sound may be higher than the position where the hammer stops by the stop rail. In this state, no electronic sound will be emitted.
On the contrary, if the sensor is located too low, this may cause deterioration of the functionality in detecting consecutive key stroking.So... "too high" = past the stop rail, "too low" = repetition stops working i.e. below the let-off, so it seems that "just right" is indeed limited to the few mm of hammer position between let-off and hammer strike.
I think too low would be going below check. The window of acceptable sensor position is the distance between striking the stop rail and the hammer's check height, minus the distance between the two sensors. The let-off height isn't inherently part of this. I could well be wrong, but my gut feeling is the lower shutter threshold is somewhere below let-off, but of course above check.
luns Totally orthogonal to everything: how does one quote text in this forum?
Select the text you want to quote and a (poorly visible in the case of the dark theme) pop-up allows you to quote if so desired.
luns Thanks for the welcome, guys.
The more we are, the merrier -- both on this project and on the forum in general.
luns I imagine I was too terse in trying to make a minor point that it may have come across wrong.
It didn't for me. I appreciate criticism and different opinions (even though I sometimes do not sound so in my replies, especially if I am typing while being bothered by something, like it is now for the reason stated below). I hope my own reply did not come across wrong!
luns The window of acceptable sensor position is the distance between striking the stop rail and the hammer's check height, minus the distance between the two sensors.
That is actually one of the reasons why I think hammer sensors with ADC are better than comparators or switches at the shank: you can (relatively) sloppily assemble the thing and do an electronic calibration to take the actual positions into account. I have that ready, and I would be playing with it right now, if it weren't for that extremely frustrating issue I mentioned earlier which prevents it from working.
luns The best filter to use in place of the boxcar depends on what we know of the signal we’re trying to detect. For a pure sine wave, the ideal filter would be a narrow bandpass filter tuned to the frequency of the sine wave. For a hammer position which is the second integral of what can look like a spikey acceleration, I would go for some second order lowpass filter, or basically a simple biquad filter.
I should add, the above statements are assuming everything is LTI, but the process we're sensing is not LTI to begin with (ff is not simply an amplitude scaled pp). The simplifying assumption is to keep things tractable. The suggestion of a biquad filter isn't necessarily ideal either, but is also to keep things from getting too complicated.
I've not heard of Savitzky-Golay, but if we're to allow things to get more complicated, elements of what I remember of the Kalman filter seem appropriate. The filter maintains an estimate of the state (for us, the hammer position and velocity) that it can use to predict the input, but also keeps track of how good those estimates are (the covariance). Between the covariance and prediction error, the filter uses the sampled input in varying proportion relative to the estimate from past samples - in effect, there's something of a dynamic averaging time.
This matches with what we'd empirically want to do anyway: to estimate the velocity between two thresholds crossings, we can take the average of the individual step velocities through the entire interval - we'd effectively average few samples for a fast key strike, and many for a slow one.
This is not to say I suggest implementing a Kalman filter: what benefit it might have over the other approaches is likely not enough to be worth the additional complexity, and if that marginal benefit is needed, it's probably more fruitful to invest in higher sampling rates or more bits of an ADC.
The LTI filtering (fixed bandwidth, or equivalently, fixed averaging time) would have to have a wide enough bandwidth to settle in a fast strike, but for a slow strike, this would see more noise. Whether this is a problem depends on how bad the noise is, but of course more bits help. Without a surplus of bits, the time-between-crossings approach (with interpolation to estimate where crossings are between samples) is probably the best.
xooorx Aha... I tried various ways of filtering the position signal before differentiating but got better results by, effectively, differentiating the position and low-pass filtering the result. (Am no longer sure which way round @stem_piano was doing it).
Assuming no noise is introduced along the way, the filtering operations, and the filter coefficients themselves, are commutative. I think @stem_piano convolves the differentiation and the boxcar together into a single FIR which does the same non-adjacent-sample differentiation you did, just looking even farther back than your 4 samples.
xooorx For quantisation noise it might be that prevention is better than cure i.e. use higher bit converters. @stem_piano's posted data was quantised to 3-digit decimal but came from a system that will soon be doing 24 bits. OTOH my own plans currently involve a 12-bit converter so I need to work out if that's enough or if I need to switch it up.
24 bits feels like overkill while 12 feels appropriate to me. Just my gut feeling though - would have to play with the data to really tell.
Thanks!
RIP It didn't for me. I appreciate criticism and different opinions (even though I sometimes do not sound so in my replies, especially if I am typing while being bothered by something, like it is now for the reason stated below). I hope my own reply did not come across wrong!
I think we're all on the same page here.
To pontificate a little more on the hammer vs. key subject though, I think Kawai's shank-butt sensor location may also be motivated by design sharing concerns. From discussion of the CA79/CA99/NV5s/NV10s firmware update discussion, it sounds like these designs share a lot of DNA. It wouldn't surprise me if electrically, they're practically identical, with the NV's sensors designed to mimic the CA's switches
In contrast, the Avant Grand sensors are transmissive grayscale. Presumably they saw enough benefit in the approach to not limit themselves trying to share things with squish-switch designs.
luns I've not heard of Savitzky-Golay, but if we're to allow things to get more complicated, elements of what I remember of the Kalman filter seem appropriate.
The Kalman is the go-to in my biomedical field for smoothing and predicting fast and dynamic data. But my (limited) experience applies to a single data stream. I don't know how fast it would need to be for acting on 88 keys. And this kind of filtering may be overkill for simple rotational movement...
vagfilm I don't know how fast it would need to be for acting on 88 keys.
The designs we DIY'ers are talking about (or at least the ones I've paid attention to 🤣) have at least one microcontrollers for just each dozen of keys or so, not 88. Perhaps even one microcontroller per just a couple of keys (even though in that case it tends to be really wimpy one). It might still be too slow, and also
vagfilm this kind of filtering may be overkill for simple rotational movement...
I was speaking of Kalman on Matlab (the only experience I have...). No idea about implementing Kalman on pico (Python?). Over and out because I am getting out of my depth here. You guys keep the ball and run...
vagfilm No idea about implementing Kalman on pico (Python?). Over and out because I am getting out of my depth here. You guys keep the ball and run...
I'd expect Kalman to be overkill - I only brought it up as a last resort if the data pushes us into that corner, but I don't think we're there. Stopping short of Kalman, we have different simplifications we could make, which end up looking like the other approaches we'd want to try anyway.
luns For a hammer position which is the second integral of what can look like a spikey acceleration, I would go for some second order lowpass filter, or basically a simple biquad filter.
So I played just a little bit with @gzpiano 's data. I didn't even work out the coefficients I'd want for a biquad and just did a simple single pole IIR filter with time constant of about 16 samples. I did this with both the raw data (green), and it quantized to 8 bits (red), both scaled up by 256 for convenience of quantizing. Doesn't look half bad:
There's some quantization noise visible in the velocity curve, but it doesn't seem half bad, and I'd expect that going to the biquad would clean that up some more.
So 8 bits is alright, though it remains to be seen how well we can tolerate reducing the sample rate.
I'll have to pick up with this at another time, but if anybody else wants to pick up where I am, it's really only two lines of matlab:
f=filter([1/16],[1, -15/16],x);
v=filter([1 -1],1,f);
I'll add one more nuance. Just for kicks, here's the same plot but with simple derivatives of the unfiltered raw data and 8 bit quantized (also unfiltered) as well. The derivatives of the filtered raw data was barely visible before and totally covered now, but the filtered 8-bit quantized result which is really close, is still visible.
From this, I can tell the raw data is 10 bits.
I'll also add that the filter really just boils down to per-sample updates of:
out += (in-out)/16
You can think of the filter as taking the prior output as an estimate of where the current position is, then using the difference between the input and that estimate to update the estimate. We don't completely trust the new input - it has its share of noise - so the new estimate is actually 1 part input, 15 parts previous estimate.
Rather than tracking only the position, we can obviously do the same thing with velocity too and include that when making the estimation. This would be the biquad filter I wrote of and will try next when I have the time.
luns 1 part input, 15 parts previous estimate
Heh, this is exactly where I ended up except I chose 1 part to 31:
Getting this out of 10-bit data (and with very simple filtering) has made me a lot happier about my 12-bit converters.
RIP I think we are just trying to use that information to infer what is necessary for the hammers. Speaking of which, The N1/2/3 have hammer sensors right? So perhaps their service manual can help. I just gave it a cursory glance (and it's a service not a design manual) so I could not find the speed of their hammer sensors. However they look like comparators to me a-la original @CyberGene design.
I'm a little surprised by what I found with both this and the Kawai ATX service manuals.
Yamaha has gray-scale optical sensors for the hammer and key. I would have expected the hammer sensor to be used for velocity, and key sensor for note-off sensing (basically damper on/off). However, the schematic seems to indicate that the hammer sensors go through a comparator with programmable DAC output for setting the comparator threshold, and only the key sensors are fed to ADCs.
So, it appears the velocity measurement is done at the key! This would mean the NU1 loud-note design flaw is fundamentally there, and it not showing its head is only a matter of the grand action falling faster relative to the key than the upright does.
I won't rule out the possibility I misread the schematic, but this was jarring enough I would have checked things over at the time, unless I had my attention taken away by something else.
Not to be outdone, Kawai's ATX, while it does have three shutter type optical sensors, actually performs an XOR of two of them right at the sensor board before sending results upstream. So in effect, the three sensors are being used as two! In principle, the two that are XOR'd could even have been combined into one sensor. I really wonder if this was marketing driven, wanting to claim 'triple sensor' implying it's better than two-sensors, when the argument for a third sensor is more typically a description of three comparison thresholds. For squish-switch DP sensors, each sensor only has one threshold, but this isn't so for optical shutters.
Puzzling.
I would not be surprised if the Yamaha hammer sensor exists only to double check if the hammer was really thrown (and avoid the "loud note" events that occur in the models that only have the key sensors). They have all the logic done and tested for calculating velocity based only on the key sensors, so it would make no business sense to alter everything...