Del Vento (Jay Kominek)'s DIY hybrid controller (free talk)

CyberGene

I know this will sound like a super silly question that’s way too off for me asking it (but as I said I have lifelong issues remembering stuff I haven’t actively put in my mind recently), so what’s the final working principle of Jay’s solution? Is it by using small ADC chips one for every note (or maybe shared between few of them, how much in total?) and sending the sampled digital proximity values (what’s the resolution, i.e. the sampling frequency for each note?) continuously on a digital bus? And then reading those values from the bus by a central processing unit (Teensy?) that will do some math on the proximities and produce a velocity value from it?

P.S. Spelling it out loud I realize the actual velocity calculation should happen at the ADC level, otherwise the bus won’t cope with 88 sensors continuously sending digital data. So, the bus will only carry the velocity values?

Del Vento

CyberGene o what’s the final working principle of Jay’s solution?

In short, the STM32 have both lots of compute power and I/O, including several ADC channels, IIRC 22 of them without multiplexing per chip. So one chip can monitor 22 hammer or key sensors, and just 8 of them can monitor the whole piano with 176 sensors (for ease of software and flexibility, you do need the key sensor even on the notes which are normally undampened on the piano, so the "note off" is entirely key-based). These boards should be able to read the ADC channels at 10kHz, and do the processing "live" on chip, so each ADC board with one STM32 will be full in charge of 11 keys.

Firmware is still to be finalized, but I expect that out of the ADC boards only note on/off information (not necessarily in MIDI format yet) will get out, directed at the main board. With the exception of calibration and debugging, when more data may be desired.

More details:

https://github.com/jkominek/piano-conversion/blob/master/DESIGN.md

But you are certainly most interested in https://github.com/jkominek/piano-conversion/discussions/31 which has clear and simple graphs comparing this design to others, including your groundbreaking, pioneering work on Cybrid!

PS: if you have comments/questions/rants on the design, I think it's more appropriate to have the conversation there on github or on a separate thread here on pianoclack -- since in this one pianoclack conversation I'm basically only reporting assembly/mechanical/ordering stuff and it's becoming already unwieldy without discussing the design choices…

CyberGene

@Del Vento thanks for the answer. I'll probably forget what you answered to me in 3 months from now which is why I prefer if it stays here rather than on GitHub since ultimately I'm asking just out of curiosity.

I think I understand the design now.

Del Vento These boards should be able to read the ADC channels at 10kHz, and do the processing "live" on chip, so each ADC board with one STM32 will be full in charge of 11 keys.

Do you mean that every single key is scanned at 10kHz or the entire group of 11 keys is scanned at 10kHz?

Because from my notes on Cybrid with the latest Teensy my measurements have 112kHz of scanning which for 85 keys means 1.3kHz of scanning per key and that would be be roughly equivalent to your number of 10kHz for 11 keys. However if it's per key, then it's a clearly better solution indeed.

Not meaning to compete though! Just trying to wrap my head around the solution.

P.S. BTW, the scanning speed of 1.3kHz per key I'm citing is the net scanning speed, i.e. it includes the processing of the events for the key and generating MIDI. Have you calculated how much the delay will be when you include the processing of the data on the ADC boards which AFAIK includes some filtering and math stuff?

Del Vento

CyberGene thanks for the answer.

You're most welcome. The whole point of sharing my efforts here on the forum.

CyberGene since ultimately I'm asking just out of curiosity.

For this purpose, it's good. If it'll become "why don't you do this since it appears to me that's better", then github is the place.

CyberGene Do you mean that every single key is scanned at 10kHz or the entire group of 11 keys is scanned at 10kHz?

I don't remember the exact numbers from the top of my head, but for sure the STM32 ADC clock is usually set around 50MHz, so the 10kHz must be per channel. Plus, since half of the channels are "note off", if necessary and with more complicated software, one may divide by 11 instead of 22, i.e. don't sample note off until a note on is detected (however you do need to continue sampling note-on events also before the corresponding note off is detected otherwise repetitions may have problems, so it's not exactly half).

Moreover, as the github issue #31 say, even at slower clock, the higher spatial resolution of an ADC approach could compensate for "slower" temporal sampling. I think "slower" there means "compared to other options such as dedicated ADCs which can reach up to 500kHz per channel".

CyberGene P.S. BTW, the scanning speed of 1.3kHz per key I'm citing is the net scanning speed, i.e. it includes the processing of the events for the key and generating MIDI. Have you calculated how much the delay will be when you include the processing of the data on the ADC boards which AFAIK includes some filtering and math stuff?

The ADC happens separately in interrupts, there is no "scanning", so there is not time spent on it. Also the STM32 cpus is much more powerful than the other ones in Teensy and similar, so again going in that direction.

CyberGene Not meaning to compete though! Just trying to wrap my head around the solution.

Totally understood. Same reason why I eventually went in this route after having examined a number of options, including yours. Now that I'm settled, I forgot the details (like you) 😃

This has been turned inside out like a glove at https://pianoclack.com/forum/d/289-scanning-speed-and-velocity-mapping-of-cybrid/

dore_m

Just wanted to jump in here and say that I appreciate all of the talk and documentation on this, even though I don't follow it closely. There will come a day when I gather up enough energy to do this, so thanks for the groundbreaking work and documentation.

Del Vento

dore_m Just wanted to jump in here and say that I appreciate all of the talk and documentation on this

You are most welcome and actually the same to you with all the things you do on sampling acoustic pianos and releasing wonderful ready-to-use virtual ones!

scherbakov.al

Guys! You are great fellows and do amazing and very interesting things! Very interesting and useful to follow this. This is incredibly inspiring!

At the beginning, in my project, I used the C language. But later it became curious to learn more about the features of the language. And the C++ language became curious. I went through a couple of steps of some course on C++, read the literature and tried to apply the features of C++ when programming STM32. In C, I tried to optimize the code as much as possible, but using C++, the code began to work 1.5-2 times faster. Although it contains the use of some apparently difficult things for the microcontroller from the standard library. These opportunities for code optimizations in the language were impressive. And at the same time, the code became clearer and simpler.

One of the goals in the project was to use only one inexpensive STM32f411 microcontroller. In my case, its speed is good enough to handle note on events for the entire keyboard (even hi-res midi is obtained at medium and low midi speeds), but not quite enough to handle note on + note off + pedal. Apparently, you will have to transfer to STM32h7 (another curious individual is esp32 with their dual core ..). Or there is still room for optimization the code. (it's very interesting to cram it all into just one f411...)

The work of the code in the interrupt is especially frustrating when a midi message is sent via usb. And the interrupt function code (generated using CubeIDE for usb) contains a massive multiple transfer of data from structure to structure. The job of interrupting to send a message eats up three keyboard scan cycles - too many. And apparently there is a lot of nonsense in the code. Something to think about.

Therefore, it is curious in the future to transfer to some library for STM32 written entirely in C ++ and with the implementation of usb.

I don't know why I posted here, but still..just some moments..

Your project impresses with its scale and sophistication!