gzpiano Any chance you could post some of the low velocity strike data? I've been messing around with the strike data that's on GitHub (and may post something about that soon) but if there's some more problematic data it would be good to see it.
Also, how constrained are you for processing power? Are you doing boxcar + first order just to try the simple stuff first or are you needing to avoid multiplications and divisions?
Sure, next time I have everything put together I will try to remember.
Meanwhile, this link is the place in my algorithm video where I have a plot of the issue:
With respect to processing power, that was not really a factor.
gzpiano this link is the place in my algorithm
Thanks. You've done quite a number of interesting videos.
gzpiano With respect to processing power, that was not really a factor.
I agree. Even with a limited RP2040, I can do about almost one hundred of loops per ms. In each loop I synchronously and sequentially sample the position of three hammers (eventually will do that async with DMA, this is the slowest of everything done in each loop), do some calculations, send data synchronously via I2C (also slow, but done only when needed) and more. Doing a filter like (or even something more complicated) this is nothing.
I am banging my head wrt some stupid details (go https://github.com/davidedelvento/Rpi-pico-i2c-example if you are curious) when those will be solved I'll have a pretty decent 9-key electronics which I can install in the piano and with enough firmware to work (among other things, with some regulation possible via MIDI šļø) and provide my own hammer strike data to further analyze.
Let me go do that right nowā¦
RIP with a limited RP2040, I can do about almost one hundred of loops per ms. In each loop I synchronously and sequentially sample the position of three hammers (eventually will do that async with DMA, this is the slowest of everything done in each loop), do some calculations, send data synchronously via I2C (also slow, but done only when needed) and more. Doing a filter like (or even something more complicated) this is nothing.
You'll have plenty of clock cycles for any integer algorithm you could want, but with only M0 cores, I'd expect single precision floating point algorithms to be prohibitive. I think something like an RC filter at 20kHz might be the limit.
gzpiano Good idea. I convolved a boxcar (moving average, [1 1 1 1 1ā¦ 1]) with a first order numerical derivative [1 -1] to get the taps in that code on github. Rather simple. Where I have seen noise most problematic is in low velocity hammer strikes. Super quiet notes.
The noise problems I saw when playing around with the samples on GitHub turned out to be due to the heavy quantisation of the data. (I see from comments in the Matlab code that you have a fix for this coming).
The numerical differentiation to get the velocity is differentiating the signal and the quantisation noise, which is OK if you've got enough bits but rapidly falls apart if you don't...
Numerically differentiating a 24 bit sine wave:
Numerically differentiating a 16 bit sine wave:
Numerically differentiating a 12 bit sine wave:
Numerically differentiating an 8 bit sine wave:
OK I tried various filters and some of them sort of worked a bit, but nothing worked as well as just doing the differentiation between non-adjacent samples e.g. instead of V = P[0] - P[-1], using V = (P[0] - P[-4])/4.
Cleans up the 8-bit version to this:
Cleans up the 12-bit version to this:
(Also note the plots in the previous post used a fancy 4-point differentiator:
V = (P[-2] - 8P[-1] + 8P[1] - P[2])/12
which is worse than:
V = P[0] - P[-1])
Will write more when Iām at a computer instead of the phone, but for now, I wanted to point out the (p(0)-p(-4))/4 differentiation is really just the p(0)-p(-1) followed by a length-4 unit-area boxcar. The differentiation is inherently high pass, emphasizing noise where thereās not really any signal you would care about, but the boxcar does a passable job of filtering out much of that noise.
The best filter to use in place of the boxcar depends on what we know of the signal weāre trying to detect. For a pure sine wave, the ideal filter would be a narrow bandpass filter tuned to the frequency of the sine wave. For a hammer position which is the second integral of what can look like a spikey acceleration, I would go for some second order lowpass filter, or basically a simple biquad filter.
More laterā¦
luns Will write more when Iām at a computer instead of the phone, but for now, I wanted to point out the (p(0)-p(-4))/4 differentiation is really just the p(0)-p(-1) followed by a length-4 unit-area boxcar.
Aha... I tried various ways of filtering the position signal before differentiating but got better results by, effectively, differentiating the position and low-pass filtering the result. (Am no longer sure which way round @gzpiano was doing it).
luns The best filter to use in place of the boxcar depends on what we know of the signal weāre trying to detect.
For quantisation noise it might be that prevention is better than cure i.e. use higher bit converters. @gzpiano's posted data was quantised to 3-digit decimal but came from a system that will soon be doing 24 bits. OTOH my own plans currently involve a 12-bit converter so I need to work out if that's enough or if I need to switch it up.
JayKominek You'll have plenty of clock cycles for any integer algorithm you could want, but with only M0 cores, I'd expect single precision floating point algorithms to be prohibitive. I think something like an RC filter at 20kHz might be the limit.
Well, unless we prove it otherwise by a big success, the RP2040 is just a test platform for a different ADC, namely one with only 12 bits (less than 9 ENOB) and higher frequency (up to 500kHz), the exact opposite of what @gzpiano is doing (IIRC he uses 24 bits and 8kHz, but check his videos for an authoritative answer).
As such the cost of fully fledged floating point live data processing on the RP2040 is irrelevant. That said, I grew up writing primitive numerical algorithms on a 6502 using just individual bytes. And later watched in awe FRACT386 (later still renamed FRACTINT). If needs be, we can experiment with numerical algorithms with bytes again, so I'll feel younger š -- the main limitation if one needs speed is that it'd be easier to use fixed point position, but I am wondering why I'm even getting there besides nostalgia, since I just said we won't do that "in production"ā¦
Sorry for the short detour, let's go back to the good discussion @xooorx and @luns are having.
Looks like there is another DIYer... Maybe one of you can contact him and bring him in?
http://forum.pianoworld.com/ubbthreads.php/ubb/showflat/Number/3185913/gonew/1/attempt-to-create-a-midi-controller-for-the-mechanics-of-the.html
vagfilm that is Alexey Schterbakov, I think we exchanged some messages in the past regarding Cybrid, he was very interested in recreating it. He also promoted me on the Pianoteq forum (not knowing that I am a persona non grata there š¤£). You can invite him here, he will be interested.
CyberGene ah, just doing my end of working day forums catch up, PW first šand I posted on his entry there with some links.
Totally orthogonal to everything: how does one quote text in this forum? I'm used to having a button that quotes an entire post I'm replying to, which I can then trim down, but best I can figure here, it looks like I would need to manually cut/paste (and corrupt) what I'm quoting here. That feels like putting words into somebody else's mouth, even if the intent that it be their own words.
CyberGene RIP Thanks for the welcome, guys. I imagine I was too terse in trying to make a minor point that it may have come across wrong. I agree that it doesn't fundamentally matter where the sensors are and I have no objections to the sensor being in either location. There was a post I'd seen - likely in a different thread rather than here (things are a little jumbled while I'm still taking everything in) that seemed to emphasize some significance in the hammers moving 5x the speed of the key, and my only point was that it doesn't really matter. I'll just leave it there rather than beat a dead horse. š
xooorx 1) Let-off is set at 4mm (page 10). Possibly slightly wider than a pure acoustic action to accommodate the stop rail? To make the sensing easier?
I imagine this is entirely to allow room for the stop rail to stop the hammers before they hit the strings.
3) From the section on adjusting the sensor height (page 12):
For instances, if the sensor is located at an extremely high position, the position of emitting sound may be higher than the position where the hammer stops by the stop rail. In this state, no electronic sound will be emitted.
On the contrary, if the sensor is located too low, this may cause deterioration of the functionality in detecting consecutive key stroking.So... "too high" = past the stop rail, "too low" = repetition stops working i.e. below the let-off, so it seems that "just right" is indeed limited to the few mm of hammer position between let-off and hammer strike.
I think too low would be going below check. The window of acceptable sensor position is the distance between striking the stop rail and the hammer's check height, minus the distance between the two sensors. The let-off height isn't inherently part of this. I could well be wrong, but my gut feeling is the lower shutter threshold is somewhere below let-off, but of course above check.
luns Totally orthogonal to everything: how does one quote text in this forum?
Select the text you want to quote and a (poorly visible in the case of the dark theme) pop-up allows you to quote if so desired.
luns Thanks for the welcome, guys.
The more we are, the merrier -- both on this project and on the forum in general.
luns I imagine I was too terse in trying to make a minor point that it may have come across wrong.
It didn't for me. I appreciate criticism and different opinions (even though I sometimes do not sound so in my replies, especially if I am typing while being bothered by something, like it is now for the reason stated below). I hope my own reply did not come across wrong!
luns The window of acceptable sensor position is the distance between striking the stop rail and the hammer's check height, minus the distance between the two sensors.
That is actually one of the reasons why I think hammer sensors with ADC are better than comparators or switches at the shank: you can (relatively) sloppily assemble the thing and do an electronic calibration to take the actual positions into account. I have that ready, and I would be playing with it right now, if it weren't for that extremely frustrating issue I mentioned earlier which prevents it from working.
luns The best filter to use in place of the boxcar depends on what we know of the signal weāre trying to detect. For a pure sine wave, the ideal filter would be a narrow bandpass filter tuned to the frequency of the sine wave. For a hammer position which is the second integral of what can look like a spikey acceleration, I would go for some second order lowpass filter, or basically a simple biquad filter.
I should add, the above statements are assuming everything is LTI, but the process we're sensing is not LTI to begin with (ff is not simply an amplitude scaled pp). The simplifying assumption is to keep things tractable. The suggestion of a biquad filter isn't necessarily ideal either, but is also to keep things from getting too complicated.
I've not heard of Savitzky-Golay, but if we're to allow things to get more complicated, elements of what I remember of the Kalman filter seem appropriate. The filter maintains an estimate of the state (for us, the hammer position and velocity) that it can use to predict the input, but also keeps track of how good those estimates are (the covariance). Between the covariance and prediction error, the filter uses the sampled input in varying proportion relative to the estimate from past samples - in effect, there's something of a dynamic averaging time.
This matches with what we'd empirically want to do anyway: to estimate the velocity between two thresholds crossings, we can take the average of the individual step velocities through the entire interval - we'd effectively average few samples for a fast key strike, and many for a slow one.
This is not to say I suggest implementing a Kalman filter: what benefit it might have over the other approaches is likely not enough to be worth the additional complexity, and if that marginal benefit is needed, it's probably more fruitful to invest in higher sampling rates or more bits of an ADC.
The LTI filtering (fixed bandwidth, or equivalently, fixed averaging time) would have to have a wide enough bandwidth to settle in a fast strike, but for a slow strike, this would see more noise. Whether this is a problem depends on how bad the noise is, but of course more bits help. Without a surplus of bits, the time-between-crossings approach (with interpolation to estimate where crossings are between samples) is probably the best.
xooorx Aha... I tried various ways of filtering the position signal before differentiating but got better results by, effectively, differentiating the position and low-pass filtering the result. (Am no longer sure which way round @stem_piano was doing it).
Assuming no noise is introduced along the way, the filtering operations, and the filter coefficients themselves, are commutative. I thinkĀ @stem_pianoĀ convolves the differentiation and the boxcar together into a single FIR which does the same non-adjacent-sample differentiation you did, just looking even farther back than your 4 samples.
xooorx For quantisation noise it might be that prevention is better than cure i.e. use higher bit converters. @stem_piano's posted data was quantised to 3-digit decimal but came from a system that will soon be doing 24 bits. OTOH my own plans currently involve a 12-bit converter so I need to work out if that's enough or if I need to switch it up.
24 bits feels like overkill while 12 feels appropriate to me. Just my gut feeling though - would have to play with the data to really tell.
Thanks!
RIP It didn't for me. I appreciate criticism and different opinions (even though I sometimes do not sound so in my replies, especially if I am typing while being bothered by something, like it is now for the reason stated below). I hope my own reply did not come across wrong!
I think we're all on the same page here.
To pontificate a little more on the hammer vs. key subject though, I think Kawai's shank-butt sensor location may also be motivated by design sharing concerns. From discussion of the CA79/CA99/NV5s/NV10s firmware update discussion, it sounds like these designs share a lot of DNA. It wouldn't surprise me if electrically, they're practically identical, with the NV's sensors designed to mimic the CA's switches
In contrast, the Avant Grand sensors are transmissive grayscale. Presumably they saw enough benefit in the approach to not limit themselves trying to share things with squish-switch designs.
luns I've not heard of Savitzky-Golay, but if we're to allow things to get more complicated, elements of what I remember of the Kalman filter seem appropriate.
The Kalman is the go-to in my biomedical field for smoothing and predicting fast and dynamic data. But my (limited) experience applies to a single data stream. I don't know how fast it would need to be for acting on 88 keys. And this kind of filtering may be overkill for simple rotational movement...
vagfilm I don't know how fast it would need to be for acting on 88 keys.
The designs we DIY'ers are talking about (or at least the ones I've paid attention to š¤£) have at least one microcontrollers for just each dozen of keys or so, not 88. Perhaps even one microcontroller per just a couple of keys (even though in that case it tends to be really wimpy one). It might still be too slow, and also
vagfilm this kind of filtering may be overkill for simple rotational movement...
I was speaking of Kalman on Matlab (the only experience I have...). No idea about implementing Kalman on pico (Python?). Over and out because I am getting out of my depth here. You guys keep the ball and run...