A DIY hybrid piano without ADCs (purely digital sensors)

I did some measurements for Everlight elec ITR20001 phototransistor feeding into it a 160kHz 5v pwm signal from an arduino mega:

I'm getting around _800ns fall time of photo transistor output. Which is way less than specified 20microsecond rise/fall time, am I missing something?
This is using 8kOhm resistor as load on emitter side (hooked up as seen in the initial schematic, but without the hc238).

If all sensors are sampled at 20kHz, that gives 50microseconds to switch between the 8, giving 6.25us per sensor time window, 1us of which would be taken up by the transition.

Another way to look at is - if 50us are the time window to sample all 88 sensors, then 8x11 mux using 238 would eat up _8us of that budget in transitions. So this approach might as well work if that is taken into account.

I've upgraded slot-through slit-sensor PCB design with hc238 muxing too.
Now it effectively draw less than 200mA, which means it can easily be directly powered from USB alone.
And requires only 3 resistors instead of 32 previously, only one hc238 IC per 8keys (instead of two shift registers per 8keys), 1 decoupling cap. This reduces part count and simplifies assembly (matters alot if hand placed).

A single Raspberry Pi Pico is wholly sufficient as main USB-MIDI device too, with tinyusb library. RP2040 is 70cents @ 100qty, compatible QPSI flash around 25cents. Ready made original Raspberry Pico boards are 4.2-5€, WeActStudio rp2040 is 3-4€, YD-RP2040 modules go for as low as 1.5-2.5€ a piece.

Total effective electronics BOM of this slot through design ends up being roughly 50-60€, including PCBs, but excluding assembly. No external power supplies necessary either. This is as simple as it gets, but it does require a mechanical element to move through the slit, the only downside of this approach.

I suspect a wholly analog version can also be built in roughly same electronics BOM budget, but it is highly dependent on built in ADCs of stm32 mcus to be "good enough".

My first experiences with programming stm32 via SWD has been thoroughly positive, debugger and breakpoints work great, while also being able to see peripheral registers and their state as machine is running. This is great for troubleshooting. (not using cubeIDE)

stm32 has two vendor specific APIs, stm32 HAL (high level) and stm32 LL (low level). STM32 LL are just thin wrapper over peripheral registers that are memory mapped into flat address space. Interfacing with peripherals is essentially taking 32bit addresses (as pointers) and reading/writing to values to them. Reference manuals have descriptions in detail where each peripheral is in memory space. I find stm32 LL API particularly pleasant and easy to understand.

Other than vendor specific peripherals, both rp2040, stm32g431, stm32h7, teensy4.1, they all have arm cortex m0/m4/m7 cores . RP2040 is a cortex m0 (no floating point unit), teensy4.1 and stm32h7 are cortex m7 (have floating point units), just clocked at 600mhz(teensy) vs 480mhz (stm32h750). They are programmed and debugged in the same way via SWD/JTAG.

I have weact stm32g431 (2x 12bit 0.25us adcs) devboard on hand with weact stlink probe. DMA DAC works well (very useful as signal generator!), sending ADC readings via UART using DMA works well too. ADC conversions thus far seem to work great via DMA too (ADC conversions automatically get written in buffers).

I wrote a small app for streaming and displaying ADC values over UART, works easily up to 1mbit baud rate (haven't tested higher)

(ignore ENOB readings, those are rough estimates if the signal is not "moving")

This is capable of also generating MIDI signals to be routed to Piano VIs. Which means that I can develop and test velocity calculation algorithms fully on PC before porting it over to a MCU.

The "tricky part" is setting up the peripherals/timers/DMA and buffers such that almost nothing is blocking (or calling interrupts as interupts also take cycles to switch in and out of).

For h7 480mhz / 88keys * 20khz sampling = 272 cpu cycles per key. This is only slightly less than teensy at 340 cpu cyles per key.

DavisB ….. 480mhz (stm32h750)

There are stm32h723vgt6 boards with 550mhz processor and built-in full color display for only 10 dollars. I myself switched from stm32h750 to stm32h723. They work great and are inexpensive.

Great to hear that the STM32 are working for you (in fact they are working fantastically well for me too -- but at work where we have a team of specialized engineers each of which can take care of one small but essential aspect of the design and where I never touch them directly).

For a hobbyist like me, almost-alone in this project, the STM32 was putting myself squarely in the "project does not get completed" camp. Like this guy puts it better than I can.

https://www.youtube.com/watch?v=4jgTCayWlwc

Got tinyUSB MIDI working on stm32g4 as well!

Now that I've gotten tinyusb MIDI work with both RP2040 and STM32G4, i'm pretty confident that I'll be able to get it working for any of the supported MCUs on the list.
https://github.com/hathach/tinyusb/blob/master/docs/reference/supported.rst

Of which rp2040 and stm32h750 offer the best value! Honorable mention WCH CH32V203C8T6, RISCV 144mhz, LQFP-48(7x7), 50cents a piece - great MCU for basic MIDI devices with sliders and knobs, etc!

Now I can send MIDI directly from MCU via USB, or send sensor data via UART and generate MIDI on PC via virtual MIDI port!

The first algorithm for analog velocity calculation is very simple - store last N samples, and once current sample crosses some threshold, take current sample and one sample N samples ago (N = 32, 64 or 128, ....) and then take the average of that. This doesn't even require floating point numbers, or division and can be implemented in few cycles with just fixed point math, etc. Then I feel that calculated velocity has to be fitted onto some curve before being translated into MIDI.

(taking current sample and one sample N samples ago is eerily similar to effectively reducing sample rate by N)

Perhaps there's some smart things that can be done with those samples (filtering, rejecting obviously noisy samples, etc), but nothing pops up at the moment! Any ideas?

DavisB Got tinyUSB MIDI working on stm32g4 as well!

Did you got it working on its own, were you able to integrate it with another project (e.g. the ADC example, or your own fully fledged algorithm)? In my experience the former is quite easy, the hard part is the latter. If that is what you have done, big kudos to you!

DavisB Any ideas?

Plenty. Most of the discussion on this topic happened in the thread linked below (and links therein), but many of my multi-part "Build your own DIY hybrid piano - step xxx" have some. And to be clear most of those are not mine, the contributions are from the whole community here!

https://pianoclack.com/forum/d/289-scanning-speed-and-velocity-mapping-of-cybrid

I've built the circuit as seen in this image:

And it doesn't work works only to extremely slow switching speeds, ie. switching only couple of hundred of times a second. After that it works, but only allows to sense one-sensor at a time (at speeds switching sensors 160khz times a second) - if any of the sensors is active, it just reports low 0V for all the other sensors, regardless of their state.

And It's not obvious why is it so... and seems like the only way it could happen is if the transistor starts conducing in reverse for some time after being switched off? (hc238 can both source and sink current).

PCB looks like this, double sided.

Hello! I'm not sure, but it seems to me that the problem may be that the transistor in the sensor takes about 20us to open and close. 20us = 50 kHz period. As stated in the datasheet: Oh yes, it was mentioned above..

I am very impressed that you managed to run the tinyusb library on the stm32! I once tried to make this library work, but apparently I didn't spend enough time studying it and it didn't work out. Nevertheless, it would be extremely interesting to look at the stm32 project with tinyusb running.

scherbakov.​al sorry, I misread your post, never mind 😃(edited my comment)

A couple of years ago, a friend proposed a sensor circuit that used ambient light filtering. It's a cool scheme, I couldn't have come up with such a thing. The LED would turn off, the transistor would be measured, and then the LED would light up. The value of the previous measurement would be subtracted from the current measurement on the transistor. This process seemed to work well, but we ran into a problem when we tried to adjust the frequency of the sensor's stable operation. The frequency was about max 10 kHz when we used CNY70 components.

https://github.com/brrastak/midi_sensor/blob/main/midi_sensor.pdf

I do not know how it works) There's also a need for a control driver for such a sensor.

CyberGene sorry Nothing) I've got a double post above (2 weeks ago. I was writing from the subway and lost contact for a moment).. I tried to delete it, but I couldn't find how. It can be deleted

The odd thing is that the sensors work just fine easily switching up to 20khz as long as they are alone, and don't share the same time-multiplexed analog output signal line at the same time. (or when they operate at very slow switching speeds, then the circuit works as intended)

I've also measured response time of these phototransistors via an oscilloscope individually, which makes me think there's some weird interaction between the sensors themselves as they are switching between each other while sharing the same output line.

The only solution to this issue that I can imagine is putting a signal switching diode on each transistor output...

Alternatively, you can try connecting the outputs from the sensors to another such multiplexer, which would be controlled by the same channels and demultiplexed the outputs into one. It's just that the specifications for the diodes indicate the response time: one is about 200 ns, the other is 50ns, the ultrafast diode is 35ns, and this multiplexer works faster than 20ns.

I did a small prototype on a breadboard switching 3 sensors with arduino nano (arduino nano pins can source and sink up to 20mA)

PORTD |= (1 << PD3);  
PORTD &= ~(1 << PD4); 
PORTD &= ~(1 << PD5); 
delayMicroseconds(6); 
  
PORTD &= ~(1 << PD3); 
PORTD |= (1 << PD4);  
PORTD &= ~(1 << PD5); 
delayMicroseconds(6);

Using essentially the same circuit (but without 238), and shared analog output wire, and it works without any issues.

Now I'm starting to suspect that I have a bad solderjoint somewhere (or one of the 8 sensors is shorted or misbehaving) or some issue of that type, or else it doesn't make sense that this works and the 238 version doesn't. It has to be some sort of silly mistake somewhere.

I've made a hc238 muxing circuit using just through-hole components for purely testing purposes since it easy to attach oscilloscope and see if solder joints are OK. And this time it works as intended!

(trims are there for testing purposes only so i dont have to resolder resistors)

However there are still issues with sensors impacting each others readings. For example if one sensor is reflecting, you get max value 4096, and if all sensors are reflecting, you get only around 3500 as max value, or sensors next to each other. Which isn't optical crosstalk as far as I can tell!

So this kind of muxing probably works just fine as-is for slit sensors, but for purely analog sensors it needs some improvements (such as only blinking leds via hc238 to reduce power consumption and still use analog mux for constantly powered phototransistors?)

The good news is that I've made all the software side on stm32 take zero CPU cycles to do ADC conversions and muxing.
PWM timer with 160khz frequency, 1.25us low (for the transition), 5us high - all easily adjustable.

Every 6.25us timer triggers DMA transfer a bitpattern on relevant GPIO pins (memory to peripheral DMA transfer to BSRR - Bit Set Reset Register)
On the rising edge of 5us high initiates automatic ADC conversion (after the 1.25us transition settling time)
ADC conversion automatically outputs results via another DMA channel out into a result buffer and moves to the next position in a circular buffer.

This uses no interrupts, and effectively no cpu cycles!

You get automatically filled buffers with sensor positions all the time with zero CPU involvement! Sampled at the exact muxing timings I specify!
This happens fully automatically in the background all the time.

What's beautiful about this is that it has the exactly same code and hardware pinouts when muxing using analog switch cd4051 or muxing using hc238!
I took my old PCB that uses analog switches plugged it in - and it just works! Same code & same hardware interface!
All 8 sensors beautifully streamed via UART (up to 5.7ksps each via _921kbit uart)!

Might be a good idea to add another ping-pong DMA buffer (like double buffering)...?! software side really is the easy part with debugger where you can add memory and variable watches and see buffers, periperals and their data changing in real time!

Another cool feature of stm32g4 is VREFBUF (with available values for VREF being 2V, 2.5V, 2.9V). This also works wonderfully.
Built in OPAMP follower also seems to work great. (havented tested built in programmable gain opamp yet!)

If muxing/adc sampling can be made like this to take zero cycles, it feels almost as if single measly stm32g431 (arm32 cortex m4 @ 170mhz) might just be fully sufficient for a full piano...?

Pcbs from JLCPCB came for the hc238 muxed slit version. Seems to work as expected at 20khz per sensor at the moment!
Components per board - 2*8sensors (very fast and easy to solder!), 3resistors(1206), 2 decoupling caps (100nf, 1uf 0805), one sop-16 ic, lots of DNP (do not place) pads, 1.2mm fr4 2layer, one side almost unbroken ground plane (except for where the pcbs join).

Beginner hand-solder friendly (except for two 0805 caps?), the only upgrades to this design that I can think of are:

1. Make it into 2pcbs (11mm * 44keys * 2pcbs=484mm * 2pcbs) instead of 11pcbs (of 11mm * 8keys * 11pcbs=88mm * 11pcbs)
2. Smaller slit sensors, with smaller slit (these have massive 1mm slit where the IR light shines through).

(soldermask is incorrect, it is powered from 3.3v!

Very cost effective design, ~ 4eur (2eur boards, 2eur shipping), for 5pcbs (out of 11 total). Needs one more main board (<100x100mm) with rp2040 and that's it.

DavisB I considered such a design for a long time with those optical interrupt sensors. It makes everything very simple electronic-wise, basically the same as any digital piano or MIDI controller with membrane switches. But it requires careful mechanical regulation and PCB positioning, along with adding some shutters to each hammer and at the time I was afraid that would require way too difficult planning/designing/drawing for the PCB-s to follow the exact hammer footprint and position.

However, from a certain perspective, I think it’s the way to go for a mass produced piano (yeah, Kawai chose that solution, and Yamaha have a mix of this and over complicated key sensors).

Yes, intuition tells me that analog version with ITR8307s (6-7cents a piece) can be built with lower BOM and lower labour/assembly time costs (certainly with SMD PCBA), and overall be less finicky mechanically.

Seems like those will require more than just hc238 ic for muxing.

Something like this maybe:

(completely unbroken ground reference plane, some very thin traces. 1mm thin pcb two layer fr4? Still thinking whether to put a ground pour on top, so this is not a finalized design)

HC238 blinks only leds, CD4051 analog mux for phototransistor output. Phototransistors probably will be always connected and always pulled to ground via resistors (R0-R7). Or optionally they can be pulled to ground AFTER muxing via cd4051 (pulled with R20).

This design has too thin traces for me to build at home, so has to be ordered for an experiment...

Another version to try is to have everything on all the time (and mux via single CD4051) and barely-just-barely try to fit it into 500mA power budget. If it fits in 500mA (or if you don't care about exceeding 500mA a bit), this could be fairly straight forward, simple design too.... (as long as you don't get bitten by lack of analog electrical engineering background)

Comparator method is still not dead in my mind, g431 has a fast dac and a fast comparator.... which could be callibrated automatically programmatically (the threshold points, etc)....

moved to https://pianoclack.com/forum/d/1897-diy-random-talk

Retiring sensor strip from my old narrow key (5.2" octave) prototype:

This is a strip 5pcbs * 8sensorpairs = 40 keys. (5 cables going to main pcb, the 6th one is addr+vcc+gnd). No external power supply required. Whole system runs on less than <150mA. 20khz sample rate.

Bom cost for a whole piano if done like this (with pcbs with mainboard too, here's it's just soldered wire to wire connections on proto-board):

Single ~ 1.1eur yd rp2040 devboard sufficient for whole 88key prototype! BOM cost for the whole system is roughly 41eur all inclusive.

Pinout:

If using such devboard, only 1 pin is left for pedal. This design relies on scmitt-triggers in rp2040 pins (outputs of those slit-sensors are analog!). Main board PCB requires basically almost no circuitry, just direct point-to-point connections (except for address pins, series termination resistors (30-50ohm) would be recommended there.