JayKominek I'm doing the Savitzky-Golay filter to compute the derivative of position
I ran the data you posted through Savitzky-Golay (using your python script) and also through "Vf" and "Vff" from my post above (using a spreadsheet). The following shows savgol(13) and savgol(23) in red, Vf and Vff in blue:
I think the difference in peak height is due to savgol doing it more accurately, but in terms of smoothing they're pretty comparable.
Sorry for asking this silly question instead of reading the entire discussion (and it's a shame that it's me asking this question but I've been busy with some other stuff and have not been following the progress of this project in details), what are these filters you're discussing? I mean, what are they for? I'd assume you're actually sampling the hammer through ADC and then smoothing the samples in order to produce a curve of the hammer position, is that why you are using filters? If so, then how do you calculate velocity? By measuring the distance of the curve between predefined points?
RIP At the moment, are you doing this in firmware or on the computer? Do you care to share the code or do you prefer to polish it a bit first?
The firmware knows which sensors are hammers and which are dampers, and knows under what circumstances to compute a "pseudovelocity" and send that out. After that, a PC is picking that pseudovelocity up, scaling it to a range appropriate for a hammer or damper, and then running the result through the equation in the Goebl-Bresin paper to get a floating-point MIDI velocity. I "split" the value on the decimal point, and get the base velocity and the high-res portion.
(Long term, everything the PC does will be done by the "main board" in my design, and it'll just emit MIDI via normal routes. I think you know that, but for the benefit of everyone else reading.)
The relevant code is all in HAL _ ADC _ ConvCpltCallback. (extra spaces to avoid formatting.)
It's gnarled because it is doing this processing on a set of circular buffers.
RIP Before getting there (please folks, do not start an OT on whether or not that's useful: nice conversation, but we need a separate thread since this is too long already), did you verify that you were able to generate all the 127 regular note-on velocities?
I was going to worry about that a bit more when I've got a more realistic calibration procedure in place.
I pounded on a key a bunch, and a saw a lot of different values pop out. Did I see 126 and 127? Nope. Did I try to get 126 and 127? Nope. Can my currently configured scaling produce 126 and 127? Possibly not.
RIP This very thread started on the speculation that the time resolution we have may be inadequate to already generate the 127 normal MIDI values.
Inadequate under the restriction that you're just doing timing between two values being passed.
RIP Also, what virtual instrument are you using to tell the difference, if there is one?
I thought about mentioning that in my first post, but came to the conclusion I refuse to even mention the names of VSTs in pubilc piano forums. I'm still using the one I discussed with you previously, and if you've forgotten, feel free to inquire again on my Github project discussion and I'll answer any questions. It might be more suitable to get into the weeds of my system there, anyways.
CyberGene what are these filters you're discussing? I mean, what are they for? I'd assume you're actually sampling the hammer through ADC and then smoothing the samples in order to produce a curve of the hammer position, is that why you are using filters? If so, then how do you calculate velocity?
The Savitzky-Galoy I'm using can both smooth the data, and compute the derivative in a single FIR filter. So that cleans up any small bit of noise in the ADC signal, and gives me a derivative based on multiple close-by data points, rather than having to compute the slope from just two points spread out in time (like the Cybrid effectively did). That should produce a more instantaneous velocity measurement at time of impact, rather than averaging it out over the last 8-10mm of approach.
CyberGene what are these filters you're discussing?
For the curves shown in my most recent post:
Savitzky-Golay calculates a smooth analytical curve through the raw position samples. That's "analytical" as in it knows the equation of the curve, so velocity is simply and directly the first differential of that equation.
Vff numerically differentiates the raw position samples to go straight to a very noisy and nasty velocity measurement, which is then low pass filtered to get a much smoother velocity measurement.
CyberGene If so, then how do you calculate velocity? By measuring the distance of the curve between predefined points?
The curves above are the calculated velocity as the hammer moves. It's finished and done, the graphs are just showing the results.
RIP this is IR light reflected by nearby hammers, picked up by the "wrong" sensor.
Silly and naive question: if you cover the action with the cabinet, do you still have the "baby strikes"? That gets rid of many of the sunlight reflections and could be enough to make the effect irrelevant.
vagfilm Del Vento this is IR light reflected by nearby hammers, picked up by the "wrong" sensor.
Silly and naive question: if you cover the action with the cabinet, do you still have the "baby strikes"? That gets rid of many of the sunlight reflections and could be enough to make the effect irrelevant.
The data was collected at nighttime with minimal (but still comfortable) light in the room.
I have not tried complete obscurity (it's in my todo list already), but I expect to not make any difference for this issue.
I think these baby peaks are caused by IR-LED light (from either the "right" or "wrong" LED) reflected by the "wrong" hammer and collected by the "wrong" phototransistor when playing the next neighbor note (here by "right" I mean assigned to the note which is being played and by "wrong" I mean assigned to a note semitone higher or lower)
EDIT: nothing is silly and naive, every inquire helps increasing the (collective) understanding of the design, thanks for posting
RIP The data was collected at nighttime with minimal (but still comfortable) light in the room.
In that case, only a long pinhole as you propose will get rid of the reflections, but will probably create additional calibration problems because a narrow pinhole will be aimed at different positions of the hammer head as the hammer travels...
I know you really want to use the velocity at the hammer head and not the shank, but if the sensor was aimed at the shank base (in a place where the total travel would be around 10mm) maybe the effect of IR noise would be smaller.
In any case, I would do a test with lights off...
PNOscan II (optical key position sensing) reduces crosstalk by having you black out the sides of the keys near where they would otherwise reflect into adjacent sensors:
So that could be a solution if you don't mind taking a marker pen to the sides of your hammers.
xooorx Original sampled position: Po
Po differentiated: Vo = Po[t] - Po[t-1]
Vo filtered: Vf = (Vo[t] + 15 x Vf[t-1]) / 16
Vf filtered: Vff = (vf[t] + 7 x Vff[t-1]) / 8
This is a particular case of biquad, in this case overdamped. The group delay is 16+8 samples, while high-frequency attenuation is proportional to 16*8=128. I'm not recommending any specific values, but if you went with 12+12 (critically damped) you'd get slightly better high-frequency attenuation of 12*12 =144 for the same delay.
1/12 isn't a great value to work with nor is the other example I'd give of 1/8/sqrt(2), but my gut feeling is you could get away just fine with both layers of filter being 1/8.
This relationship between delay and high-frequency attenuation can be pushed further by going to an underdamped biquad. In particular, I would want to try out something close to a butterworth filter; it can cut the delay by a factor of sqrt(2) (relative to critically damped), and boosts the wanted signal level by as much around the corner frequency.
xooorx I think the difference in peak height is due to savgol doing it more accurately, but in terms of smoothing they're pretty comparable.
I think the difference in peak goes hand-in-hand with the additional delay shown by your filter. The filter doesn't just attenuate the quantization noise but is also cutting into the wanted signal too. You might be able to narrow the difference by making your filters less aggressive and respond faster to the input.
Nice progress @JayKominek and @RIP I put links to your project on my github page.
For my DIY hybrid, I’ve never seen cross-talk (when playing or in raw samples). In many lighting conditions. Perhaps that is because of what @vagfilm mentioned.
Also, 8Ksps works great for me. Get to 127 MIDI.
Its configurable 500sps to 32Ksps. 32K is for Mozart, 500 is for Van Halen.
Interesting ideas @xooorx.
CyberGene I mean, what are they for? I'd assume you're actually sampling the hammer through ADC and then smoothing the samples in order to produce a curve of the hammer position
Smoothing == filtering
RIP The "baby-strikes" on the right are cross-talks from nearby keys, which could be made smaller with geometric screening (like the 3D printed one I mentioned in another post).
If you hold the key of interest at let-off, how does the cross-talk look then? With the hammer closer to the sensor, I'd expect it to occlude the view of farther neighbours.
xooorx PNOscan II (optical key position sensing) reduces crosstalk by having you black out the sides of the keys near where they would otherwise reflect into adjacent sensors:
That is an option, thanks for mentioning it.
luns I think the difference in peak goes hand-in-hand with the additional delay shown by your filter. The filter doesn't just attenuate the quantization noise but is also cutting into the wanted signal too. You might be able to narrow the difference by making your filters less aggressive and respond faster to the input.
Thanks for analysis @luns -- One thing that I believe in this discussion is getting a bit overlooked is that we do not and should not care about the velocity of the hammer per se. We care about the most accurate velocity estimation at the time right before hitting the virtual strings and the most accurate estimation for that time and both things done in real time.
Who cares if the velocity estimation is jumpy when the hammer is far (and hence its position measurement more noisy)? Nobody should care for that, only the most accurate time and final velocity -- even though velocity estimation from noisy distance measurement is still an interesting intellectual exercise.
gzpiano Nice progress
Thanks!
gzpiano For my DIY hybrid, I’ve never seen cross-talk (when playing or in raw samples).
Have you explicitly looked at it, by comparing isolated strikes with 3-nearby-note-strikes? I so, am surprised to hear you haven't seen it, because I tested the sensors you are using and I did notice that they are responsive to off-axis presence. However I would not surprised to hear that it might be not a big problem.
gzpiano Also, 8Ksps works great for me. Get to 127 MIDI.
Getting 127 is easy. What is (perhaps) hard is getting all possible values from 1 to 127, with no gaps and a reasonable touch. I say perhaps because I'm not there yet, I need at least another weekend, if not more.
luns If you hold the key of interest at let-off, how does the cross-talk look then? With the hammer closer to the sensor, I'd expect it to occlude the view of farther neighbours.
Yes, that is exactly my point with the smaller sensors which are more easily screened by the comparatively larger hammers. Quick sketching indicates that total geometric shadow will happen only when the hammer is very close, probably closer than we need to start caring about for velocity estimation (but I don't know exactly the "how close we care" yet).
RIP Have you explicitly looked at it, by comparing isolated strikes with 3-nearby-note-strikes? I so, am surprised to hear you haven't seen it, because I tested the sensors you are using and I did notice that they are responsive to off-axis presence. However I would not surprised to hear that it might be not a big problem.
@RIP : unless @gzpiano has changed his original design, I think that in his case the sensors are placed really close to the base of the hammer shank, where the angular movement is about 1cm... At that close distance, the off axis effect is reduced (it is 1 cm to the correct hammer base and 2 cms to the neighboring hammers, at an angle that almost precludes IR artifacts). Sensing the base of the hammer is a compromise, but it has its benefits (others being that you can paint the entire hammer in black and leave only a small unpainted patch right beneath the sensor, and that you can easily measure the entire travel in a linear fashion (albeit the travel is short and spatial resolution decreases), while when you sense the hammer head, the hammer at rest is so far that the measurement is very noisy.
As a contribution to the off-axis discussion, here's a plot extracted from that data I posted previously, comparing a sensor being struck, to the adjacent channel where nothing is happening.
Note, the y axis is logarithmic. So while one channel can see an adjacent one moving, that movement barely rises out of the background noise.
Now, that said, I've got CNY70s, which I suspect are a bit more directional than Del Vento's much lower-profile sensors.
vagfilm I think that in his case the sensors are placed really close to the base of the hammer shank, where the angular movement is about 1cm...
Ah, that's right, I forgot about that. Thanks for refreshing my memory. Given that, I am not surprised at all.
JayKominek Note, the y axis is logarithmic. So while one channel can see an adjacent one moving, that movement barely rises out of the background noise.
Nice!
JayKominek Now, that said, I've got CNY70s, which I suspect are a bit more directional than Del Vento's much lower-profile sensors.
I've not done a scientific analysis on this, and I have not installed the CNY70s in the piano (yet), but in my on-the-desk tests waving objects around the sensors, the various sensors seemed similar on-axis and off-axis. If anything the smaller sensors are slightly easier screened by the hammer getting closeby. Speaking of which, it looks like your green channel gets a quite nice screening too when the hammer is close enough. Two questions about it:
JayKominek The firmware knows
JayKominek The relevant code is all in HAL _ ADC _ ConvCpltCallback. (extra spaces to avoid formatting.)
Thank you (and odd that github did not notify me of that commit, I asked to be notified about everything…)
JayKominek It's gnarled because it is doing this processing on a set of circular buffers.
It's actually pretty clear. I don't get 100% of it, but I (think 😅) I get most of it, and you placed quite nice comments, so thanks. If necessary we can talk about some technical details later (perhaps in github is more appropriate than here).
JayKominek Del Vento Before getting there (please folks, do not start an OT on whether or not that's useful: nice conversation, but we need a separate thread since this is too long already), did you verify that you were able to generate all the 127 regular note-on velocities?
I was going to worry about that a bit more when I've got a more realistic calibration procedure in place.
That sounds good. I was just commenting on your "the CC#88 does not make any difference" 🤣
JayKominek Del Vento This very thread started on the speculation that the time resolution we have may be inadequate to already generate the 127 normal MIDI values.
Inadequate under the restriction that you're just doing timing between two values being passed.
Yes, I am coming to the same conclusion, but I have not that as a definite answer. Have you?
JayKominek I thought about mentioning that in my first post, but came to the conclusion I refuse to even mention the names of VSTs in pubilc piano forums.
You don't want to get in trouble eh? 🤣 🤣 🤣
Anyway, thanks for the detailed answer.
I realize how lucky I am since when I started working on my project, and without discussing it with anybody (because nobody would have been interested and familiar enough at the time) I naively assumed that the sequential scanning of all the sensor check points would be so fast that I shouldn't bother much about resolution 🤣 Well, it turned out that was only barely true and only when using the latest Teensy board.
And honestly, if I was to start such a project after reading a thread like this one or the other ones by @RIP and the other Cybrid-like projects, I would simply give up and I'm usually a very obstinate person who isn't easily deterred, etc. I'm also a software engineer and have masters in Engineering Physics... Yet, with all that background I would still find these overly technical talks like in this discussion to introduce too many variables and possibilities for something to not be OK and I would have eventually given up because I usually give up if I think there's reasonable chance for things to not be like I expect them to be.
Ignorance is bliss because I had an apparently overly simplified ideas in my head, based on (almost wrong) assumptions about how things are going to work and it's the lack of any other people competing with me that I just finished it without too much fuss and luckily it managed to work decently 😲🤣
CyberGene I would simply give up
I see your point. To not discourage people who may come to this now, let me instead add: we are nitpicking to make it perfect and very competitive (if not much better) than similar commercial instruments. Getting the "basics" to work is very easy and in fact that is the "goal 2B" of https://github.com/davidedelvento/Mybrid/tree/main/RaspberryPiPico -- In fact that is doable much easier than your Cybrid! Once could even use all commercial parts ("goal 3") and at a low cost.
CyberGene Ignorance is bliss because I had an apparently overly simplified ideas in my head, based on (almost wrong) assumptions about how things are going to work
Well, this is very true! In fact many breakthrough in science came from young people who did not know better. The old school folks knew some theorem or experimental result that proved the new discovery impossible, so they did not even try. The blissful ignorant young, in these cases, were able to discover some loopholes or other workarounds from the theorem and the breakthrough happened. If there is interest, open a separate thread for that, I'm sure @vagfilm and I can come up with some concrete examples of this, besides Cybrid 🤣
RIP blissful ignorant young,
Yes, I have been the three at some time in the past. At this moment I am only 2/3 of that...
RIP have you tried to find out at what distance that screening starts
I just took a look with the multimeter and my fingers wiggling the hammers around. I'm going to say the peaks in the green signal occurs when the purple hammer is about 9-10mm out from the sensor. Literally a hand-waving measurement.
RIP How have you mounted the CNY70 compared to the hammer? LED/Sensor are left/right to the hammer or front/back?
The LED and phototransistor are in-line with the motion of the hammer, modulo some slight twist I applied to the PCB to make them line up a bit better with the bass hammers. The LED is towards the rear of the action, and the phototransistor closer to the front of the action.
Which is odd, I thought I had designed it the other way around, for some reason. Not that I expect it matters a whole lot.
RIP If necessary we can talk about some technical details later (perhaps in github is more appropriate than here).
By all means, I'm happy to take questions on it via Github, or whatever.
RIP Yes, I am coming to the same conclusion, but I have not that as a definite answer. Have you?
Nope, but haven't yet attempted to convince myself one way or the other. I keep hoping one of the other folks will produce a definitive answer. 😀
I think I'm just not going to let the issue slow down construction. There will be enough other issues to resolve, and the materials cost of any boards I might have to scrap is simply small enough I don't care. (My bigger concern will be the hassle of recovering parts which aren't available given the global IC shortage.)
When I think I've got it ready, I'll just hire some university piano students for pizza and beer to come test it. 😅
CyberGene And honestly, if I was to start such a project after reading a thread like this one or the other ones by @Del Vento and the other Cybrid-like projects, I would simply give up
I would have given up without trying if you hadn't shown it could be done.
The extra effort is because I'm trying to make my design field-repairable, and good enough for manufacturers who currently only make acoustics to copy. Or at least give them some more bargaining power.
If I'd just wanted something for myself, I think I would've stayed much closer to your design, and been done much sooner with much less worry. 😁