Scanning speed and velocity mapping of Cybrid

luns 1 part input, 15 parts previous estimate

Heh, this is exactly where I ended up except I chose 1 part to 31:

Getting this out of 10-bit data (and with very simple filtering) has made me a lot happier about my 12-bit converters.

RIP I think we are just trying to use that information to infer what is necessary for the hammers. Speaking of which, The N1/2/3 have hammer sensors right? So perhaps their service manual can help. I just gave it a cursory glance (and it's a service not a design manual) so I could not find the speed of their hammer sensors. However they look like comparators to me a-la original @CyberGene design.

I'm a little surprised by what I found with both this and the Kawai ATX service manuals.

Yamaha has gray-scale optical sensors for the hammer and key. I would have expected the hammer sensor to be used for velocity, and key sensor for note-off sensing (basically damper on/off). However, the schematic seems to indicate that the hammer sensors go through a comparator with programmable DAC output for setting the comparator threshold, and only the key sensors are fed to ADCs.

So, it appears the velocity measurement is done at the key! This would mean the NU1 loud-note design flaw is fundamentally there, and it not showing its head is only a matter of the grand action falling faster relative to the key than the upright does.

I won't rule out the possibility I misread the schematic, but this was jarring enough I would have checked things over at the time, unless I had my attention taken away by something else.

Not to be outdone, Kawai's ATX, while it does have three shutter type optical sensors, actually performs an XOR of two of them right at the sensor board before sending results upstream. So in effect, the three sensors are being used as two! In principle, the two that are XOR'd could even have been combined into one sensor. I really wonder if this was marketing driven, wanting to claim 'triple sensor' implying it's better than two-sensors, when the argument for a third sensor is more typically a description of three comparison thresholds. For squish-switch DP sensors, each sensor only has one threshold, but this isn't so for optical shutters.

Puzzling.

I would not be surprised if the Yamaha hammer sensor exists only to double check if the hammer was really thrown (and avoid the "loud note" events that occur in the models that only have the key sensors). They have all the logic done and tested for calculating velocity based only on the key sensors, so it would make no business sense to alter everything...

luns However, the schematic seems to indicate that the hammer sensors go through a comparator with programmable DAC output for setting the comparator threshold, and only the key sensors are fed to ADCs.

If they can re-set the thresholds fast enough they could be using one comparator to detect 2 or more levels on the same stroke?

luns Not to be outdone, Kawai's ATX, while it does have three shutter type optical sensors, actually performs an XOR of two of them right at the sensor board before sending results upstream. So in effect, the three sensors are being used as two! In principle, the two that are XOR'd could even have been combined into one sensor.

The XOR lets you watch two sensors with a single input pin: It goes high when the first shutter closes and back to low when the second shutter closes, so the length of the pulse is the time between the shutters closing.

Original sampled position: Po
Po differentiated: Vo = Po[t] - Po[t-1]
Vo filtered: Vf = (Vo[t] + 15 x Vf[t-1]) / 16
Vf filtered: Vff = (vf[t] + 7 x Vff[t-1]) / 8

And finally, just to check/visualise what has been retained vs what has been lost from the original data:
Vff integrated: Pc = Pc[t-1] + Vff[t]

xooorx Original sampled position: Po

Thank you to you and all the others who are contributing to this discussion, lots of interesting facts and experimentations here.

One problem I am having with the forum, is that I find its format not ideal for this. I am wondering if we can create a (single, shared -- or each one their own) github repository. There it will be easier to have the plot, the data, the formulas, the code for data collection and for data analysis, all in the same place and update and compare them as needed, rather than having to scroll back and forth and wondering "who said what and when, and is it still their last update and how does it compare with the latest update this other person did and why" -- which is what I am going through now… On the forum we can still post "oh, I updated my repo, and the conclusion is that X works great and Y is useless", with link to the evidence. We might use the https://pianoclack.com/forum/d/276-build-your-own-hybrid-piano-step-three-the-electronics (which is almost a clean slate) or perhaps making a new one for that purpose.

What do you all think? I plan to post everything on github wrt my own data, as soon as I fix an issue I am having with the hardware, which (knock, knock) should happen next weekend.

Thanks!

RIP One problem I am having with the forum, is that I find its format not ideal for this. I am wondering if we can create a (single, shared -- or each one their own) github repository. There it will be easier to have the plot, the data, the formulas, the code for data collection and for data analysis, all in the same place and update and compare them as needed

When I've got code and data it will appear in a GitHub repo, but these current graphs are still at the "thinking out loud" stage. This is @gzpiano's provisional data, with filtering that improves every time @luns comments on it, but all I've got right now really is a spreadsheet where I can mess around with the ideas and make nice plots of them.

I finally sorted out all of my ADC issues. Here's some data I recorded off of two keys last night, for anyone to play with.

https://github.com/jkominek/piano-conversion/blob/master/analyses/jkominek/2022-01-20.forprocessing.txt

Those values are from a 16 bit ADC, at 16kHz. The only processing was to flip the values to match the convention that larger numbers =~ more current from the CNY70.

I'll likely push some more data in there over time, without ceremony. I haven't even looked at the signal off the key sticks, in large part because it was usable even when I had ADC problems.

There's a tiny bit of python next to it which represents what I'm currently thinking of implementing as a signal-to-impact algorithm for hammers. I expect some parameters will be made tweakable per-key things, of course.

Once I get that converted into C, on the controller, dealing with the implementation details of processing from a circular buffer, etc, I'll be sending out pseudo-velocities from the keys. Should reach that point over the weekend.

And here is my first data raw as it come out of the sensors. The code generating the data is here and some helper functions to load it are here

Download above expires in a week. Not putting on my repo because I am refraining to put multi-MB files without first understanding fully what they are. If it turns out to be good, I'll upload it there (it's 2.2MB, not huge, but once in the repo, it will stay there forever in all working copies).

The picture below has been generated from the data with the helper code mentioned above. The flat areas are in order: hammer at rest, hammer at let_off (then there's an obviously-not-flat hammer strike), hammer at drop/backcheck, repetition from there without (and then with) fully releasing the key.

The "baby-strikes" on the right are cross-talks from nearby keys, which could be made smaller with geometric screening (like the 3D printed one I mentioned in another post). I am not worried about them since I plan to use separate damper sensors, but they would be very important to fully characterize if you plan to use a singe sensor like @gzpiano because otherwise your code could mistakenly consider a next-semitone legato playing as a non-release of the previous note.

RIP The "baby-strikes" on the right are cross-talks from nearby keys, which could be made smaller with geometric screening (like the 3D printed one I mentioned in another post). I am not worried about them since I plan to use separate damper sensors[...]

It's worth doing something about that cross-talk if you can. If you play e.g. a cluster of three semitones then the middle note will have its measurements affected by both adjacent hammers which could affect velocity measurement as well as key release.

Its actually a bit worse than it looks because this is raw ADC data. After correcting for the curve in the CNY 70 response those baby-strikes will look somewhat bigger.

xooorx It's worth doing something about that cross-talk if you can. If you play e.g. a cluster of three semitones then the middle note will have its measurements affected by both adjacent hammers which could affect velocity measurement as well as key release.

Thanks for taking a look at the data. Just to make sure other people reading this thread are in the same page, this is IR light reflected by nearby hammers, picked up by the "wrong" sensor. So it's optical cross-talk (not electronic).

I do plan to do something, with a bit of optical/mechanical screening. For key release this looks very important, but in my final design key release will have separate sensors, so that's a non-issue for me.

The cross-talk you see are in fact adjacent notes, and I plan to do some tests as you describe. I will do that after some software improvements that I need for the purpose.

Its actually a bit worse than it looks because this is raw ADC data. After correcting for the curve in the CNY 70 response those baby-strikes will look somewhat bigger.

I am not sure how the curve you posted applies. First, I am using the EAITRCA6, not the CNY70 (for which I expect the problem being worse, because of the geometry of its sensors), but let's assume an identical curve applies. As far as I understand, those curves are for an "infinitely large" object (i.e. much larger than the sensor itself) directly in front of the sensor, whereas the problem here with cross-talk is about a smaller object (nearby hammer), being "on the side".
Moreover, I am not sure I will want to "correct for the response", as in "measure absolute hammer distance". The hammer distance is simply a proxy for (MIDI) velocity, so I plan to go straight there without the intermediate proxy of the distance. That said, your argument still somewhat applies, if hammer position of nearby notes increase the current in the one under measurement, that will skew velocity measurement.

With my "I am not worried" I meant that it looks to be small enough (without any mechanical screening, have you seen my pictures? link below) that it can be easily mitigated. I did not mean "I will just pretend it's not there" 🤣

For example, some extra optical screening can be installed, or just by the "natural" optical screening that happens when the hammer is close enough may be all that is needed (see close-up pictures). This is a big advantage of the small sensors and one of the reasons why I discarded the CNY70 option: the EAITRCA6 is much smaller than the CNY70 -- same would apply if one selects the QRE1113, which might be easier for surface mount industrial soldering, which is how I plan to get it assembled, but I digress….

Link to the pictures: https://pianoclack.com/forum/d/276-build-your-own-hybrid-piano-step-three-the-electronics/3

I'm doing the Savitzky-Golay filter to compute the derivative of position, and then just looking for a peak when the position is "close enough". Apropos of the CNY70 signal shape discussion, by looking at the derivative at its peak, we know we're always at the same physical distance relative to the sensor (that distance being "impacting").

Then on the way back down, I send noteoff when the key stick passes a fixed position. I don't have note off velocity in that video, but I have implemented that since. I feel like it makes things a bit nicer, but it was hardly an A/B test.

After I got in note-off velocity, I went ahead and implemented the CC#88 high res velocities. I can't tell a difference. 🤪

RIP First, I am using the EAITRCA6, not the CNY70

Ah OK I had missed that detail.

RIP Moreover, I am not sure I will want to "correct for the response", as in "measure absolute hammer distance". The hammer distance is simply a proxy for (MIDI) velocity, so I plan to go straight there without the intermediate proxy of the distance.

Even if the finished firmware goes "straight there" it might be useful during development to have at least an idea of where "there" is relative to actual hammer velocity. OTOH if it goes straight to some reasonable function of the actual hammer velocity and that then gets tweaked with a curve that feels right to play, then I suppose that will work too.

JayKominek I'm doing the Savitzky-Golay filter to compute the derivative of position, and then just looking for a peak when the position is "close enough". Apropos of the CNY70 signal shape discussion, by looking at the derivative at its peak, we know we're always at the same physical distance relative to the sensor (that distance being "impacting").

Then on the way back down, I send noteoff when the key stick passes a fixed position. I don't have note off velocity in that video, but I have implemented that since. I feel like it makes things a bit nicer, but it was hardly an A/B test.

At the moment, are you doing this in firmware or on the computer? Do you care to share the code or do you prefer to polish it a bit first?

JayKominek After I got in note-off velocity, I went ahead and implemented the CC#88 high res velocities. I can't tell a difference.

Before getting there (please folks, do not start an OT on whether or not that's useful: nice conversation, but we need a separate thread since this is too long already), did you verify that you were able to generate all the 127 regular note-on velocities? This very thread started on the speculation that the time resolution we have may be inadequate to already generate the 127 normal MIDI values. If that is correct (and I am not saying it is) then adding higher resolution values is obviously a moot point. Also, what virtual instrument are you using to tell the difference, if there is one? That may be important too.

JayKominek I'm doing the Savitzky-Golay filter to compute the derivative of position

I ran the data you posted through Savitzky-Golay (using your python script) and also through "Vf" and "Vff" from my post above (using a spreadsheet). The following shows savgol(13) and savgol(23) in red, Vf and Vff in blue:

I think the difference in peak height is due to savgol doing it more accurately, but in terms of smoothing they're pretty comparable.

Sorry for asking this silly question instead of reading the entire discussion (and it's a shame that it's me asking this question but I've been busy with some other stuff and have not been following the progress of this project in details), what are these filters you're discussing? I mean, what are they for? I'd assume you're actually sampling the hammer through ADC and then smoothing the samples in order to produce a curve of the hammer position, is that why you are using filters? If so, then how do you calculate velocity? By measuring the distance of the curve between predefined points?

RIP At the moment, are you doing this in firmware or on the computer? Do you care to share the code or do you prefer to polish it a bit first?

The firmware knows which sensors are hammers and which are dampers, and knows under what circumstances to compute a "pseudovelocity" and send that out. After that, a PC is picking that pseudovelocity up, scaling it to a range appropriate for a hammer or damper, and then running the result through the equation in the Goebl-Bresin paper to get a floating-point MIDI velocity. I "split" the value on the decimal point, and get the base velocity and the high-res portion.

(Long term, everything the PC does will be done by the "main board" in my design, and it'll just emit MIDI via normal routes. I think you know that, but for the benefit of everyone else reading.)

The relevant code is all in HAL _ ADC _ ConvCpltCallback. (extra spaces to avoid formatting.)

https://github.com/jkominek/piano-conversion/blob/a8b33c9164264ce179180f5ef9f337700e9bbd1d/firmware/adc-board/Core/Src/main.c#L1047

It's gnarled because it is doing this processing on a set of circular buffers.

RIP Before getting there (please folks, do not start an OT on whether or not that's useful: nice conversation, but we need a separate thread since this is too long already), did you verify that you were able to generate all the 127 regular note-on velocities?

I was going to worry about that a bit more when I've got a more realistic calibration procedure in place.

I pounded on a key a bunch, and a saw a lot of different values pop out. Did I see 126 and 127? Nope. Did I try to get 126 and 127? Nope. Can my currently configured scaling produce 126 and 127? Possibly not.

RIP This very thread started on the speculation that the time resolution we have may be inadequate to already generate the 127 normal MIDI values.

Inadequate under the restriction that you're just doing timing between two values being passed.

RIP Also, what virtual instrument are you using to tell the difference, if there is one?

I thought about mentioning that in my first post, but came to the conclusion I refuse to even mention the names of VSTs in pubilc piano forums. I'm still using the one I discussed with you previously, and if you've forgotten, feel free to inquire again on my Github project discussion and I'll answer any questions. It might be more suitable to get into the weeds of my system there, anyways.

CyberGene what are these filters you're discussing? I mean, what are they for? I'd assume you're actually sampling the hammer through ADC and then smoothing the samples in order to produce a curve of the hammer position, is that why you are using filters? If so, then how do you calculate velocity?

The Savitzky-Galoy I'm using can both smooth the data, and compute the derivative in a single FIR filter. So that cleans up any small bit of noise in the ADC signal, and gives me a derivative based on multiple close-by data points, rather than having to compute the slope from just two points spread out in time (like the Cybrid effectively did). That should produce a more instantaneous velocity measurement at time of impact, rather than averaging it out over the last 8-10mm of approach.

CyberGene what are these filters you're discussing?

For the curves shown in my most recent post:

Savitzky-Golay calculates a smooth analytical curve through the raw position samples. That's "analytical" as in it knows the equation of the curve, so velocity is simply and directly the first differential of that equation.

Vff numerically differentiates the raw position samples to go straight to a very noisy and nasty velocity measurement, which is then low pass filtered to get a much smoother velocity measurement.

CyberGene If so, then how do you calculate velocity? By measuring the distance of the curve between predefined points?

The curves above are the calculated velocity as the hammer moves. It's finished and done, the graphs are just showing the results.

RIP this is IR light reflected by nearby hammers, picked up by the "wrong" sensor.

Silly and naive question: if you cover the action with the cabinet, do you still have the "baby strikes"? That gets rid of many of the sunlight reflections and could be enough to make the effect irrelevant.

vagfilm Del Vento this is IR light reflected by nearby hammers, picked up by the "wrong" sensor.

Silly and naive question: if you cover the action with the cabinet, do you still have the "baby strikes"? That gets rid of many of the sunlight reflections and could be enough to make the effect irrelevant.

The data was collected at nighttime with minimal (but still comfortable) light in the room.

I have not tried complete obscurity (it's in my todo list already), but I expect to not make any difference for this issue.

I think these baby peaks are caused by IR-LED light (from either the "right" or "wrong" LED) reflected by the "wrong" hammer and collected by the "wrong" phototransistor when playing the next neighbor note (here by "right" I mean assigned to the note which is being played and by "wrong" I mean assigned to a note semitone higher or lower)

EDIT: nothing is silly and naive, every inquire helps increasing the (collective) understanding of the design, thanks for posting