DIY piano sensors

CyberGene Maybe you've missed it

No, I did not miss it and I am well aware that your code even has the provision for both Teensies (as well as explanation about the changes in pinouts that affect the pedal)... The way I wrote my post was not the best, and it could be interpreted as if this was still untested but, as everybody knows, CG never leaves a stone unturned and of course Teensy 4.1 was tested and implemmented. The only thing I did not remember was if this 4.1 testing was publically discussed on the PW thread or if it was something that was only buried in the GitHub descriptions...

RIP Or are you asking "why do I want to do ADC instead of threshold measurement?" -- if the latter, I am happy to can get into that conversation, but I remember you having that discussion with Jay, and I thought he had convinced you that if one is able to do it (and I think I am), there are advantages to that approach.

These kind of project only get off the ground if they bring any kind of fun and are driven by motivation. So, if you think that ADC is what motivates you, go for it !!! I was not entirely sure what design strategy you wanted to pursue.

I maintain my suspicion (nothing more than a hunch...) that ADC-like continuous detection of hammer/key/damper position is an unnecessary level of detail that will only add latency and complex iterations to the code. The good news is that by sending continuous ADC per hammer/damper/key, if all else fails, you now have the actual voltage values and use them as simple binary thresholds (without the need for ICs for analog voltage comparison, gating, etc). So, even better and simpler 😀.

I wish I could be of any help, but my electronics knowledge is non-existent. I can only serve as an annoying devil's advocate, and raise food for thought... Speaking of which, I would suggest that you think through your proposal of looking at the damper position. If you look at high-speed videos of actions, you will see that the movement of the damper is very erratic, and has high inertia that depends on the weight of the damper head. Moreover, to feed midi signals to a VST I think that is better to differentiate note-off as "key raised" and not as "damper lowered", because maybe that is not the way the VSTs are tipically scripted.

CyberGene Yes, that is possible somehow but it would require additional electronics, maybe on-site ADC conversion and whatnot. Honestly I forgot about everything.

IIRC, you v2 idea was to get away from the potentiometers providing a reference voltage for the comparators and instead use a DAC to provide that value in a digital/programmable way. That will work in the same was as your current design, just that the "turning" can and does happen in software rather than with a screwdriver, and as such it can be programmed according to an automatic-to-execute logic rather than manual process.

CyberGene I've looked into a lot of sensors and solutions, especially other sensors by Vichay. Honestly, the dumbest reason I've selected the CNY70 was that it's bulky which meant it would work perfectly to create a L-shaped sensor board above the hammers

Yes, I remember one post in which you had not even settled yet for an optical sensor! In any case, thanks for explaining this part of your selection process.

vagfilm I maintain my suspicion (nothing more than a hunch...) that ADC-like continuous detection of hammer/key/damper position is an unnecessary level of detail that will only add latency and complex iterations to the code.

If you could have a perfect mechanics and a perfect electronics I'd agree with you. However that ADC level of details will tell you many things (especially in calibration, but not only) that a simple comparator would not. For example, if your comparator detects several triggers in a row, what does it mean? Player vibrating the key? Background flickering light? Electrical noise? Hard to tell. I would expect a few edge cases creating problems like the infamous NU1 "loud note". With an ADC you have the additional information which can help debug these edge cases, if you need to, and plus you could do the same thing as with a DAC (digital threshold selection, which is important e.g. for temperature drift compensation).

Regarding latency, I am not sure if you are talking about the latency of the ADC conversion or the latency of the data analysis. I think both can be kept under control, especially if one uses the right parts (i.e. not the one with which I started this thread, which is obviously too slow).

vagfilm I would suggest that you think through your proposal of looking at the damper position. I

Yeah, I am calling "dampers" but the sensors will be simply looking at the keys. The actual damper action of this grand (and I believe of any grand) is too bulky and too heavy to install and stay within my desired size and weight of the instrument, so I am building a prototype "simulated damper" action which you'll learn about in a couple of weeks (spoiler alert: that "infinite pivot" message of mine is an important detail of this simulated damper action).

vagfilm can only serve as an annoying devil's advocate, and raise food for thought

That's useful and appreciated. In fact I was expecting you chiming in http://forum.pianoworld.com/ubbthreads.php/topics/3175226/sanding-hammers-weird-question.html (sorry guys had to go there, since there are no technicians here) -- also: thanks for this cool video, I had seen similar ones, but not this one

@RIP You need an assistant of some sort ……

Drew-r Be this as it may, you need an assistant

Are you offering again to be one? For a problem assembling the lyre, I need to flip the cabinet upside down (or disassemble too many things which might be better, but more time consuming and tedious). I was thinking of recruiting my daughter for that, but if you offer, I'll be happy to take you. Don't think that "online" will work though…

I have many, many other tasks that need to be done. Woodworking is almost complete (only a hinge remains to be finalized, and that nasty upside-down lyre business). I have other "mechanics" e.g. that simulated damper action (and more). And electronics e.g. going through the list I mentioned earlier in this post and see if there is one with programmable I2C address, and if so what is its sampling rate; then testing the two sensors I know are potential candidates and any third for that list; finding a similar list from other vendors…. And firmware. And software. And more. So if you are offering again to be my assistance, let me know what best interest you and you are good at doing, and I'll gladly recruit you again 😀 Many of these things are more "information" (i.e. trying X and see if it works as imagined or if problems arise), so unlike the "flipping the cabinet" can be easily done remotely.

Thanks everybody!

RIP that ADC level of details will tell you many things (especially in calibration, but not only) that a simple comparator would not.

Ok. I have a sense we are on the same page... I see the obvious advantages of collecting the continuous signal and set in code the threshold detection, over just collecting the time of voltage comparison in which you are blind to who is triggering what. If the sampling is fast enough (and if it is not, it is just a matter of splitting the keyboard into several processing units) that is a far better strategy. And by "latency" I was refering to processing time and not latency of ADC conversions or digital transfer of data...

RIP Are you offering again to be one? For a problem assembling the lyre, I need to flip the cabinet upside down (or disassemble too many things which might be better, but more time consuming and tedious). I was thinking of recruiting my daughter for that, but if you offer, I'll be happy to take you. Don't think that "online" will work though…

I can tell you are a driven person with (possibly too) many irons in the fire, probably don’t get enough sleep or proper nutrition and of course always in need of carving out ample QT for parenting …..and yes, while muscle work is probably among the top 2 to 3 of my best handyman skills , i do not think man or woman or beast has figured out how to do these by Zoom or other virtual means 🙁 so yes, short of me making that big car ride out west that ive often dreamed of , i happily continue to offer my volunteer services and support remotely ….. but in case you are not aware, one of the reasons farmers once had many children was to have ample helpers to do the physical work while Ma and Pa grew older, so there’s that too, if you are careful.

Drew-r I can tell you are a driven person with (possibly too) many irons in the fire

True

Drew-r probably don’t get enough sleep

False

Drew-r proper nutrition

Depends on your definition of "proper", but I am very intentional with it, and while I could eat a little less, the quality of the food I eat is carefully monitored. FWIW I loosely follow the "longevity diet" of Dr. Valter Longo which is sort-of-vegan + fish. Ankle permitting (which is bothering me in these last few weeks), I run dozens of miles per week, which help me get away with the "little less" mentioned above. Should anybody respond to this, please create a separate thread 😀

Drew-r always in need of carving out ample QT for parenting

True

Drew-r one of the reasons farmers once had many children was to have ample helpers to do the physical work

Oh yeah, I know that. I think the scale has tipped the other way in modern times, with kids costing you (monetarily and in time/labor) more than you "extract" from them. I think with mine I am very fortunate compared to other people I see around, but still not at the good 'ole time level.

Drew-r I happily continue to offer my volunteer services and support remotely

Thank you. If you are more specific on your interests and expertise, I'll happily dump tasks on you. These days I am keeping the sight reading app to a hold and I am concentrating on the Cybrid-clone. Interest in that? If so, in what domain? Woodworking? Mechanical design? Simple electronics? Software? Firmware? Simply whatever I don't want to do myself?

Thanks.

RIP drew-r I happily continue to offer my volunteer services and support remotely

Thank you. If you are more specific on your interests and expertise, I'll happily dump tasks on you. These days I am keeping the sight reading app to a hold and I am concentrating on the Cybrid-clone. Interest in that? If so, in what domain? Woodworking? Mechanical design? Simple electronics? Software? Firmware? Simply whatever I don't want to do myself?

If that's not a long enough menu to choose from, I have more: complicated electronics? KiCad and PCB design? Part availability and procurement? Datasheet analysis? Audio testing (as hinted at https://pianoclack.com/forum/d/200-upright-piano-speaker-simulator-addon-idea )? For all of these, I have clear paths and design in mind. I am certainly open to change them, but it's not a clear slate. Yet, if you want to be the creative and provide more ideas and their testing and perhaps implementation in practice, I'm totally open to it!

How about this with a multiplexer? It uses a laser rather than a LED: https://www.vishay.com/docs/84907/vcnl36687s.pdf

I'm having a hard time to understand how many samples it can measure per second (however it does mention that the laser can have a period as short as 8ms which might still be too slow)

In Cybrid with Teensy 4.1 the entire keyboard is scanned in 9 microseconds = 0.009ms. This equates to 112kHz.

CyberGene In Cybrid with Teensy 4.1 the entire keyboard is scanned in 9 microseconds = 0.009ms. This equates to 112kHz.

Nothing like an hard fact to put things into perspective... A simple teensy (one!) can do more than 100 readings per second. While calculating velocities, creating and sending midi messages.

vagfilm can do more than 100 readings per second

A slight correction: 100000 per second 😉 Which is indeed blazing fast and allows for the rather brute force approach of Cybrid: to rely on a fast enough scanning speed for measurement of duration based on consecutive reads of all hammer states.

CyberGene A slight correction: 100000 per second

I wanted to write 100k readings... Considering that hammer travel time is something like 20-40 milliseconds, and that key travel time can be a short as a few milliseconds, 100 readings would not be enough. Definitely...

0.009ms cycle with a 20ms travel time means a 1/2200 accuracy. Far more than the 1/127 you would need with MIDI. You can even use the MIDI HiRes velocity extension.

But excepted Pianoteq, I know no virtual piano which would use them.

Frédéric L

It's a bit different due to the principle of how Cybrid measures duration. You can think of this as the regular triple-sensor digital piano action where you have three on/off switches and you measure the duration between second and the third. However I don't measure the time between the exact activation of each "switch" because that's not possible. Instead I go sequentially through all the switches and check if it has been activated and if it wasn't the last time I checked I mark this as "start". But it means that I may have lost 9 microseconds. For high velocity strikes it means gradual loss of accuracy. I've made calculations that for a MIDI velocity of 127 I have an accuracy of less than ±0.5 if I'm not mistaken which is very good. I can provide the higher resolution but the precision won't be there, so it won't make sense.

BTW, the VPC1 works in the exact same way as Cybrid, calculating duration by measuring duration between switches activation BUT only through sequential scanning, meaning there's also loss of accuracy for higher velocity strikes. I'm not sure how fast their scanning is but I'm pretty sure it would be less than the 600 MHz Teensy. But the hammers in the VPC1 are not moving as fast as a real grand piano action hammers (they don't have 1:5 leverage amplification ratio), so that makes up for the loss of accuracy.

If you count the number of cycles after the first second and before the last, it is like you are measuring a time bit with a (very) little inaccuracy.

I know no other practical way to measure a time

Not sure if I explained it properly. Intuitively you could think of second switch starting a timer and third switch stopping the timer. But that’s not how it works. It could work this way if all switches are simultaneously wired to the CPU and if there were hardware interrupts. Or if the switches had their own timer for each key. But since here we have a single CPU that scans sequentially through switches through multiplexing, you don’t know when exactly the switch was activated. You only know that on previous scan cycle it was off and now it’s on. That means it could have been activated any time between right now and 9 microseconds ago. For low velocities (long duration) 9 microseconds inaccuracy is not significant. For a high velocity strike, hence short duration, 9 microseconds can be already a high enough offset. Well, in my case they lead to 0.5 points of MIDI velocity inaccuracy.

CyberGene Not sure if I explained it properly. Intuitively you could think of second switch starting a timer and third switch stopping the timer. But that’s not how it works. It could work this way if all switches are simultaneously wired to the CPU and if there were hardware interrupts. Or if the switches had their own timer for each key. But since here we have a single CPU that scans sequentially through switches through multiplexing, you don’t know when exactly the switch was activated. You only know that on previous scan cycle it was off and now it’s on. That means it could have been activated any time between right now and 9 microseconds ago. For low velocities (long duration) 9 microseconds inaccuracy is not significant. For a high velocity strike, hence short duration, 9 microseconds can be already a high enough offset. Well, in my case they lead to 0.5 points of MIDI velocity inaccuracy.

Thanks again everybody and particularly @CyberGene for this last clarification. So let's forget for a moment that you have many sensors, and concentrate on one: if my understanding is correct, you are "sampling" the on/off condition of that sensor with a 9us period, which is roughly 3 orders of magnitude faster than what the chip/sensors I got fixated with can do. And that is more or less the "sweet spot" for enough MIDI v1 resolution, so "mine" are too slow -- 'nuff said.

I was partly confused by the fact that the the chip/sensors I was looking have different aspects of the "timing" of measurements and I should not look at things that can be confusing when it's past my bedtime (yes, I go to sleep early, if Flarum tells that). In fact, there is the laser pulse (which can be as short as 50us), then there is the sampling time (which can be at most 8ms, which -- including additional overhead -- gives a mere 67 measurements/sec) and the I2C reading time (which at the highest I2C bus frequencies can be 100us). In any case, this is too slow. Yet, I like this sensor very much: the laser is highly directional, so there is little to no risk of sensing the neighbor key; the laser is pulsed and the pulse is matched in the receiver, so there is a very high rejection of other light sources; and a lot of other functionalities which instead I would need to design and implement myself.

On the other hand, 12-bit ADCs with sampling rate of 500kHz (i.e. sampling time of 2us) are not uncommon, even though if I want to implement (in hw or in sw) some of the stuff that these sensors provide I may need a higher frequency.

So, in conclusion, I will be ordering a few samples of QRE1113, EAITRCA6 and CNY70 run many tests and see which ones I like best. FWIW, the size of the CNY70 is problematic for the way I want to assemble the key sensors. To make my life easier, at this stage I will use a few RPi pico, each of which has 3x 12-bit ADCs @ 500kHz.

Obviously, I will report back here what I find (and probably make a PR on either or both Cybrid and piano-conversion github repos with the code and data which I'll get).

Yes, not exactly, but any time measurement have the same issue. (Any measure has some limited accuracy too).

Even if you hardlink an input to a CPU IRQ, the CPU can memorize the Time Stamp Counter (TSC) when it receives the IRQ, but it won’t know how much time are spent between the last TSC increment and the IRQ. Then it won’t be an exact measure. (With GHz CPU clock you will however have some magnitude more accuracy).

RIP [@CyberGene...] if my understanding is correct, you are "sampling" the on/off condition of that sensor with a 9us period, which is roughly 3 orders of magnitude faster than what the chip/sensors I got fixated with can do. And that is more or less the "sweet spot" for enough MIDI v1 resolution, so "mine" are too slow -- 'nuff said.

The Cybrid, the VPC1, any of the 2/3 squidgy switch based systems, anything with a light gate waiting for a slot to pass through it... none of these systems are measuring velocity (distance/unit-time) directly. They are all measuring colevity (time/unit-distance) and then calculating a velocity by taking the reciprocal of the measured colevity.

(Yes, OK, "colevity" is a word I just made up, but it's still the thing that all these systems are measuring).

7 bits worth of directly measured velocity would give you 128 measurable velocities like this:

But 7 bits of directly measured colevity gives you 128 measurable colevities like this:

And when you calculate 128 velocities from these 128 colevities you get this:

As you can see this gives plenty of velocity resolution down there in the tiny highlighted area at bottom-left, but there's not so much resolution in the middle and there's barely any at the top, and this is why those systems need scan rates so very much faster than you might expect.

With a 9us scan time the Cybrid can "see" ridiculously high velocities: The last value, top-right of the third graph above, represents a hammer speed of many hundreds of miles per hour in the Cybrid. The hammer never goes anything like that fast of course, but it means that the speeds it does go are all clustered down in that bottom-left corner of the graph where the distinguishable velocities are packed close enough together to get the job done.

Your sensors are way too slow to take this approach. But to measure velocity directly instead? To measure the distance moved in a known unit of time, to obtain the first graph above instead of the third? The ones (of "yours") that I looked at were only a bit too slow for doing it that way.

xooorx They are all measuring colevity (time/unit-distance) and then calculating a velocity by taking the reciprocal of the measured colevity.

(Yes, OK, "colevity" is a word I just made up, but it's still the thing that all these systems are measuring).

Very nicely said, thanks. May I nudge you to make a PR with a markdown version of your text and the plots to incorporate in either or both https://github.com/ekumanov/cybrid/ and https://github.com/jkominek/piano-conversion/ ? I think it'll be extremely useful for others, and there is an easier to find place than here. I am sure @CyberGene will agree.

Besides, you could have said in the "traditional" approach the sensors are measuring time (since the distance unit is fixed for all of them) rather than colevity, but that sounds cool 😀

xooorx Your sensors are way too slow to take this approach.

Yes, I finally concluded that on my own, thanks for confirming.

xooorx But to measure velocity directly instead? To measure the distance moved in a known unit of time, to obtain the first graph above instead of the third? The ones (of "yours") that I looked at were only a bit too slow for doing it that way.

Assuming that the hammers could move that distance, which may not be the case: a properly regulated let off could be as close as just 2mm before the "string" strike, so at the highest speed there is simply not enough room for it to travel the fixed amount of time, no? One could (and I definitely will) measure the hammer before let off, but being still attached to the key its speed is "not authoritative" yet, and to infer the final velocity one should integrate the momentum transfer rather than just measuring speed. I'd need to do some math to confirm, but I suspect the error propagation will not make it worth it.

FWIW, I ordered several specimens of those 3 kind of sensors and the boards to control them. I'm now designing some test fixtures and I hope to gather some data during the holidays.