DIY piano sensors

This is mostly a question for @CyberGene but I think making it public could help others, in the same way that your pioneer project inspired many followers (including myself)

I am arriving at the sensor stage of my project and I decided to test a few of them. @CyberGene Pretty much every DIY these days is using the CNY70 sensor which your groundbreaking project selected. I've found your posts on a variety of forums discussing the options you were considering at the time, but I did not find any of the class below.

For example, the same manufacturer of the CNY70 sells many more sophisticated sensors, such as this one which goes "for peanuts". For less than 3 dollars you get one surface mount chip which includes:

• LED emitter (like the CNY70)
• Fototransistor receiver (CNY70)
• 16-bit ADC with offset compensation for temperature drift and ambient light influence
• I2C bus with high speed mode
• other stuff less important
• brews yourself a coffee and serves it in a golden cup

From a cursory look, it might be possible to just put a bunch of these chips on the same I2C bus, and with a simple I2C controller get everything one needs for a DIY piano! Just the sensors you want (176+3 in my case for note on, off and pedals) and a single other board. No potentiometers. Free, industrial grade ADC and bus just there waiting to be used, without the need for one to design and implement them. Back of the envelope math indicate that the bus speed (3.4MHz) and capacity (1008 devices) is more than adequate for the amount of traffic expected, and any GHz grade controller should have enough horsepower to handle all the incoming digital data. But even if not, splitting the keyboard in a few sections with a few controllers would still be easier than all the other approaches I've seen.

I'm really surprised nobody else mentioned these, so I must be missing something? Note that the one I picked is pretty much just a random one among many similar ones Vishay makes -- and I have not looked at other manufacturers yet (but I know that Evec makes the EAITRCA6 which is direct competitor of the CNY70, and the main one which I intended to test side-by-side when I accidentally found these with ADC and I2C integrated in the same die)

So the question is: back in the day, did you evaluate this category of sensors and discarded them for some reasons which at a cursory glance I am missing?

From a quick look, unless I'm missing something, there are two issues here:

1) The maximum 250Hz proximity conversion rate is definitely too slow for hammer sensing (which would have ruled it out for the Cybrid) and probably a bit slow even for key sensing.

2) It's got a fixed I2C slave address so you can't actually hang a bunch of them off the same bus.

And why two sensors per key? Are you thinking of doing hammer (for note on) and key position (for note off) sensing?

Something that was briefly discussed in the old place was to use CNY70s but then have 88 cheap low pin count PIC processors, one per sensor. (You can get a PIC with ADC and I2C for well under a dollar I think). The PICs would do the ADC conversions and perhaps some other processing and then it could all go onto the I2C.

Two short comments:
First: I remember that CG mentioned (public posts? PM? Don't remember now...) that if he redesigned Cybrid (which is not going to happen...) it was possible to get rid of the "trimpots" and have some sort of code/digital adjustment of voltage to do the individual thresholds adjustments of each sensor; maybe CG wants to explain what was his idea...
Second: Cybrid used Teensy 3.6 (max speed of 180MHz), while the newer teensy 4.1 has a max speed of 600MHz. This means that you have 3 times more scanning power, and can scan 176 sensors at even higher rates than the original Cybrid. Moreover, Cybrid uses 3 hammer readings because the hammer drop is the note-off time; if you use a combination of hammer sensor and key sensor, you can have less sequential readings per sensor.
Bottomline: why complicate when the cybrid system is far from its "operational ceiling"?

xooorx 1) The maximum 250Hz proximity conversion rate is definitely too slow for hammer sensing (which would have ruled it out for the Cybrid) and probably a bit slow even for key sensing.

Doh! I totally missed that. That's a total deal breaker for this particular one, I'll have to look at the others. The thing that mislead me was the Proximity frequency in table (391kHz to 3125kHz) which would have been more than adequate.

xooorx 2) It's got a fixed I2C slave address so you can't actually hang a bunch of them off the same bus.

That's also a deal breaker, but the way that I interpreted that section is that such an address is fixed only for host programming and instead in sensing mode it could be changed

xooorx And why two sensors per key? Are you thinking of doing hammer (for note on) and key position (for note off) sensing?

Yes, that's an essential objective of my design. We can discuss why in more detail, but in a nutshell, I want to get the actual damper movement (with simplified damper action, if I am able to scale my prototype accordingly) for both note-off and aftertouch. Making it up that information from the hammer, while possible, goes into many possible edge cases which I don't want to deal with. For example: on soft playing the hammers may not be checked. Moreover, the range of the CNY70 (if I stick with that sensor) is too short to measure hammer rest position (in fact Cybrid measures the shanks and very close to the flange where the total movement is less, but I don't want to do that for other reasons).

xooorx Something that was briefly discussed in the old place was to use CNY70s but then have 88 cheap low pin count PIC processors, one per sensor. (You can get a PIC with ADC and I2C for well under a dollar I think). The PICs would do the ADC conversions and perhaps some other processing and then it could all go onto the I2C.

That is very close to what I plan to do. However that requires an additional board per each (group of) sensor(s), with its design, testing etc etc. The stuff that I've seen for under a dollar per channel is 8 bits and I think I need at least 10, if not 12 (this one had 16). And then you have to do temperature and background offset compensation yourself. For testing the sensors I plan to use two or three Raspberry pico boards (each of which has 3x ADCs, 2x I2C channels, very easy programming and lots of computing power for $4) but I don't think that would be adequate in production.

vagfilm First: I remember that CG mentioned (public posts? PM? Don't remember now...) that if he redesigned Cybrid (which is not going to happen...) it was possible to get rid of the "trimpots"

Yes, I remember that comment too. I'm doing ADC anyway, following Jay's footsteps, so this is moot point for me.

vagfilm Second:

vagfilm why complicate when the cybrid system is far from its "operational ceiling"?

I am not sure I understand the part of the comment which I have not quoted, but I can answer the question pretty easily. Besides having overlooked the two deal-breaker issues that @xooorx mentioned, this would have greatly simplified the design (both the threshold approach -- potentiometers or not -- or the roll-your-own-ADC one), not complicate it by a bit!

Or are you asking "why do I want to do ADC instead of threshold measurement?" -- if the latter, I am happy to can get into that conversation, but I remember you having that discussion with Jay, and I thought he had convinced you that if one is able to do it (and I think I am), there are advantages to that approach.

Thanks a lot to both for your comments!!!

RIP That's a total deal breaker for this particular one

There is a list of all the ones from this vendor at https://www.vishay.com/docs/49495/sg2123_emitters_detectors_sensors.pdf (very last page of the document). I haven't looked at them all yet, but I looked at a few. Unfortunately I haven't found a good way to search by specs on their website: I need to download and read each model's datasheet which is in a slightly different format and not quickly comparable from one to another. In any case, all the ones that I've seen thus far have more or less the deal-breaker problems that @xooorx mentioned: conversion rate too slow and limited number of I2C addresses (at most four addresses, and I need ideally ~200 and at least 45). Some of them support an interrupt mode which works around the conversion rate. I did not examine if the response time and the other operating characteristics of that mode are adequate, because even if they are, the at most 4 hard-coded I2C addresses are still a deal breaker: it'd need to be programmable.

I will look at every single option from that PDF, but I anticipate that the outcome would be "none of them is usable for our purpose". So, this seems the reason why @CyberGene and @JayKominek (not registered here, or at least not yet) have discarded this option and opted for the CNY70 and "roll your own" of all the rest that was needed. I will look if Evec makes anything akin these highly integrated ones and if not I'll be back to my plan of testing only the EAITRCA6 side by side with the CNY70, and roll the ADC myself -- and report back here.

It was too good to be true, but it was nice for a day to think that I could have made all the electronics in a week instead of the several months I am planning…

RIP brews yourself a coffee and serves it in a golden cup

That'd be my sensor. Definitely. Even without the golden cup (but at least some cup, please).

Kidding aside, thanks for these interesting posts. I'm not allowing myself to even think of embarking on such a project (on top of too-many-projects-already), but I'm sure I would immensely enjoy one.

Cheers and happy experiments,

HZ

HZPiano delvento brews yourself a coffee and serves it in a golden cup

That'd be my sensor. Definitely. Even without the golden cup (but at least some cup, please).

What, no pancakes?

@RIP You need an assistant of some sort ……

In a previous thread weeks ago, you mentioned another app (sight-reading?) you want to create and indicated you intend to resume after SAD season is over and may like to have some help, for which i volunteered blank-check to help in any way …. but then ….. well, i looked at that program you linked for me to see about kicking some butt , but …. that program - looks like a bunch of gnarly looking software with levels upon levels of nested menus / submenus - not my wheelhouse of what i’m good at helping in any good way 🙁 . Be this as it may, you need an assistant ….. minder/reminder ….. proofreader …. somebody to dot I’s/cross t’s …. double check specs/ratings …. make sure the coffee ( and pancakes?) is at spec’d temp ….. ‘Nstuff….. best of luck buddy!

RIP So the question is: back in the day, did you evaluate this category of sensors and discarded them for some reasons which at a cursory glance I am missing?

I've looked into a lot of sensors and solutions, especially other sensors by Vichay. Honestly, the dumbest reason I've selected the CNY70 was that it's bulky which meant it would work perfectly to create a L-shaped sensor board above the hammers 🤣 I spent quite some time deciding between one or another approach and to avoid that paralysis I just decided to get the most popular sensor and move from there on. In my daily job as a software developer I work best when under stress and when faced with severe limitations. That is when my creativity boost kicks in, my brain just comes up with solutions to hard problems when strained. I'm not good at long-term planning and choosing proper solutions from scratch. I'm more of a critical situation thinker which is not the best profile for designing a thing such as Cybrid from scratch, ironically...

vagfilm I remember that CG mentioned (public posts? PM? Don't remember now...) that if he redesigned Cybrid (which is not going to happen...) it was possible to get rid of the "trimpots" and have some sort of code/digital adjustment of voltage to do the individual thresholds adjustments of each sensor; maybe CG wants to explain what was his idea...

Yes, that is possible somehow but it would require additional electronics, maybe on-site ADC conversion and whatnot. Honestly I forgot about everything. I haven't been involved with the Cybrid for some time now because it's behind me, I just moved on and have been very busy at work lately and dedicate my little spare time to piano (and Scriabin) and a bit of synths and DAW-s.

vagfilm Second: Cybrid used Teensy 3.6 (max speed of 180MHz), while the newer teensy 4.1 has a max speed of 600MHz. This means that you have 3 times more scanning power, and can scan 176 sensors at even higher rates than the original Cybrid.

Maybe you've missed it but I've replaced Teensy 3.6 with 4.1 the moment it was available (luckily that was right before Covid and I could receive it quickly from the States and release it from customs). Indeed, it is much quicker than 3.6 and improved the detection precision considerably. I think I published some measurements on PW, might be wrong but I think even for the highest velocities the precision was ±0.5 MIDI points (down from 1.5 if I'm not mistaken). In fact, Teensy 4.1 is so quick, I had to increase some delays in the code, so that the pull-up resistors can have time to bring the voltage up before Teensy could read the corresponding sensor value 🙂 It's in the code of GitHub where I have comments describing the corresponding values for either Teensy 3.6 or 4.1. It has the same hardware footprint, so it's swappable. I think only one of the analog inputs was swapped with another one, which requires a one-line change in the code on where the sustain pedal is connected.

CyberGene Maybe you've missed it

No, I did not miss it and I am well aware that your code even has the provision for both Teensies (as well as explanation about the changes in pinouts that affect the pedal)... The way I wrote my post was not the best, and it could be interpreted as if this was still untested but, as everybody knows, CG never leaves a stone unturned and of course Teensy 4.1 was tested and implemmented. The only thing I did not remember was if this 4.1 testing was publically discussed on the PW thread or if it was something that was only buried in the GitHub descriptions...

RIP Or are you asking "why do I want to do ADC instead of threshold measurement?" -- if the latter, I am happy to can get into that conversation, but I remember you having that discussion with Jay, and I thought he had convinced you that if one is able to do it (and I think I am), there are advantages to that approach.

These kind of project only get off the ground if they bring any kind of fun and are driven by motivation. So, if you think that ADC is what motivates you, go for it !!! I was not entirely sure what design strategy you wanted to pursue.

I maintain my suspicion (nothing more than a hunch...) that ADC-like continuous detection of hammer/key/damper position is an unnecessary level of detail that will only add latency and complex iterations to the code. The good news is that by sending continuous ADC per hammer/damper/key, if all else fails, you now have the actual voltage values and use them as simple binary thresholds (without the need for ICs for analog voltage comparison, gating, etc). So, even better and simpler 😀.

I wish I could be of any help, but my electronics knowledge is non-existent. I can only serve as an annoying devil's advocate, and raise food for thought... Speaking of which, I would suggest that you think through your proposal of looking at the damper position. If you look at high-speed videos of actions, you will see that the movement of the damper is very erratic, and has high inertia that depends on the weight of the damper head. Moreover, to feed midi signals to a VST I think that is better to differentiate note-off as "key raised" and not as "damper lowered", because maybe that is not the way the VSTs are tipically scripted.

CyberGene Yes, that is possible somehow but it would require additional electronics, maybe on-site ADC conversion and whatnot. Honestly I forgot about everything.

IIRC, you v2 idea was to get away from the potentiometers providing a reference voltage for the comparators and instead use a DAC to provide that value in a digital/programmable way. That will work in the same was as your current design, just that the "turning" can and does happen in software rather than with a screwdriver, and as such it can be programmed according to an automatic-to-execute logic rather than manual process.

CyberGene I've looked into a lot of sensors and solutions, especially other sensors by Vichay. Honestly, the dumbest reason I've selected the CNY70 was that it's bulky which meant it would work perfectly to create a L-shaped sensor board above the hammers

Yes, I remember one post in which you had not even settled yet for an optical sensor! In any case, thanks for explaining this part of your selection process.

vagfilm I maintain my suspicion (nothing more than a hunch...) that ADC-like continuous detection of hammer/key/damper position is an unnecessary level of detail that will only add latency and complex iterations to the code.

If you could have a perfect mechanics and a perfect electronics I'd agree with you. However that ADC level of details will tell you many things (especially in calibration, but not only) that a simple comparator would not. For example, if your comparator detects several triggers in a row, what does it mean? Player vibrating the key? Background flickering light? Electrical noise? Hard to tell. I would expect a few edge cases creating problems like the infamous NU1 "loud note". With an ADC you have the additional information which can help debug these edge cases, if you need to, and plus you could do the same thing as with a DAC (digital threshold selection, which is important e.g. for temperature drift compensation).

Regarding latency, I am not sure if you are talking about the latency of the ADC conversion or the latency of the data analysis. I think both can be kept under control, especially if one uses the right parts (i.e. not the one with which I started this thread, which is obviously too slow).

vagfilm I would suggest that you think through your proposal of looking at the damper position. I

Yeah, I am calling "dampers" but the sensors will be simply looking at the keys. The actual damper action of this grand (and I believe of any grand) is too bulky and too heavy to install and stay within my desired size and weight of the instrument, so I am building a prototype "simulated damper" action which you'll learn about in a couple of weeks (spoiler alert: that "infinite pivot" message of mine is an important detail of this simulated damper action).

vagfilm can only serve as an annoying devil's advocate, and raise food for thought

That's useful and appreciated. In fact I was expecting you chiming in http://forum.pianoworld.com/ubbthreads.php/topics/3175226/sanding-hammers-weird-question.html (sorry guys had to go there, since there are no technicians here) -- also: thanks for this cool video, I had seen similar ones, but not this one

@RIP You need an assistant of some sort ……

Drew-r Be this as it may, you need an assistant

Are you offering again to be one? For a problem assembling the lyre, I need to flip the cabinet upside down (or disassemble too many things which might be better, but more time consuming and tedious). I was thinking of recruiting my daughter for that, but if you offer, I'll be happy to take you. Don't think that "online" will work though…

I have many, many other tasks that need to be done. Woodworking is almost complete (only a hinge remains to be finalized, and that nasty upside-down lyre business). I have other "mechanics" e.g. that simulated damper action (and more). And electronics e.g. going through the list I mentioned earlier in this post and see if there is one with programmable I2C address, and if so what is its sampling rate; then testing the two sensors I know are potential candidates and any third for that list; finding a similar list from other vendors…. And firmware. And software. And more. So if you are offering again to be my assistance, let me know what best interest you and you are good at doing, and I'll gladly recruit you again 😀 Many of these things are more "information" (i.e. trying X and see if it works as imagined or if problems arise), so unlike the "flipping the cabinet" can be easily done remotely.

Thanks everybody!

RIP that ADC level of details will tell you many things (especially in calibration, but not only) that a simple comparator would not.

Ok. I have a sense we are on the same page... I see the obvious advantages of collecting the continuous signal and set in code the threshold detection, over just collecting the time of voltage comparison in which you are blind to who is triggering what. If the sampling is fast enough (and if it is not, it is just a matter of splitting the keyboard into several processing units) that is a far better strategy. And by "latency" I was refering to processing time and not latency of ADC conversions or digital transfer of data...

RIP Are you offering again to be one? For a problem assembling the lyre, I need to flip the cabinet upside down (or disassemble too many things which might be better, but more time consuming and tedious). I was thinking of recruiting my daughter for that, but if you offer, I'll be happy to take you. Don't think that "online" will work though…

I can tell you are a driven person with (possibly too) many irons in the fire, probably don’t get enough sleep or proper nutrition and of course always in need of carving out ample QT for parenting …..and yes, while muscle work is probably among the top 2 to 3 of my best handyman skills , i do not think man or woman or beast has figured out how to do these by Zoom or other virtual means 🙁 so yes, short of me making that big car ride out west that ive often dreamed of , i happily continue to offer my volunteer services and support remotely ….. but in case you are not aware, one of the reasons farmers once had many children was to have ample helpers to do the physical work while Ma and Pa grew older, so there’s that too, if you are careful.

Drew-r I can tell you are a driven person with (possibly too) many irons in the fire

True

Drew-r probably don’t get enough sleep

False

Drew-r proper nutrition

Depends on your definition of "proper", but I am very intentional with it, and while I could eat a little less, the quality of the food I eat is carefully monitored. FWIW I loosely follow the "longevity diet" of Dr. Valter Longo which is sort-of-vegan + fish. Ankle permitting (which is bothering me in these last few weeks), I run dozens of miles per week, which help me get away with the "little less" mentioned above. Should anybody respond to this, please create a separate thread 😀

Drew-r always in need of carving out ample QT for parenting

True

Drew-r one of the reasons farmers once had many children was to have ample helpers to do the physical work

Oh yeah, I know that. I think the scale has tipped the other way in modern times, with kids costing you (monetarily and in time/labor) more than you "extract" from them. I think with mine I am very fortunate compared to other people I see around, but still not at the good 'ole time level.

Drew-r I happily continue to offer my volunteer services and support remotely

Thank you. If you are more specific on your interests and expertise, I'll happily dump tasks on you. These days I am keeping the sight reading app to a hold and I am concentrating on the Cybrid-clone. Interest in that? If so, in what domain? Woodworking? Mechanical design? Simple electronics? Software? Firmware? Simply whatever I don't want to do myself?

Thanks.

RIP drew-r I happily continue to offer my volunteer services and support remotely

Thank you. If you are more specific on your interests and expertise, I'll happily dump tasks on you. These days I am keeping the sight reading app to a hold and I am concentrating on the Cybrid-clone. Interest in that? If so, in what domain? Woodworking? Mechanical design? Simple electronics? Software? Firmware? Simply whatever I don't want to do myself?

If that's not a long enough menu to choose from, I have more: complicated electronics? KiCad and PCB design? Part availability and procurement? Datasheet analysis? Audio testing (as hinted at https://pianoclack.com/forum/d/200-upright-piano-speaker-simulator-addon-idea )? For all of these, I have clear paths and design in mind. I am certainly open to change them, but it's not a clear slate. Yet, if you want to be the creative and provide more ideas and their testing and perhaps implementation in practice, I'm totally open to it!

How about this with a multiplexer? It uses a laser rather than a LED: https://www.vishay.com/docs/84907/vcnl36687s.pdf

I'm having a hard time to understand how many samples it can measure per second (however it does mention that the laser can have a period as short as 8ms which might still be too slow)

In Cybrid with Teensy 4.1 the entire keyboard is scanned in 9 microseconds = 0.009ms. This equates to 112kHz.

CyberGene In Cybrid with Teensy 4.1 the entire keyboard is scanned in 9 microseconds = 0.009ms. This equates to 112kHz.

Nothing like an hard fact to put things into perspective... A simple teensy (one!) can do more than 100 readings per second. While calculating velocities, creating and sending midi messages.