A DIY hybrid piano without ADCs (purely digital sensors)

DavisB Got tinyUSB MIDI working on stm32g4 as well!

Did you got it working on its own, were you able to integrate it with another project (e.g. the ADC example, or your own fully fledged algorithm)? In my experience the former is quite easy, the hard part is the latter. If that is what you have done, big kudos to you!

DavisB Any ideas?

Plenty. Most of the discussion on this topic happened in the thread linked below (and links therein), but many of my multi-part "Build your own DIY hybrid piano - step xxx" have some. And to be clear most of those are not mine, the contributions are from the whole community here!

https://pianoclack.com/forum/d/289-scanning-speed-and-velocity-mapping-of-cybrid

I've built the circuit as seen in this image:

And it doesn't work works only to extremely slow switching speeds, ie. switching only couple of hundred of times a second. After that it works, but only allows to sense one-sensor at a time (at speeds switching sensors 160khz times a second) - if any of the sensors is active, it just reports low 0V for all the other sensors, regardless of their state.

And It's not obvious why is it so... and seems like the only way it could happen is if the transistor starts conducing in reverse for some time after being switched off? (hc238 can both source and sink current).

PCB looks like this, double sided.

Hello! I'm not sure, but it seems to me that the problem may be that the transistor in the sensor takes about 20us to open and close. 20us = 50 kHz period. As stated in the datasheet: Oh yes, it was mentioned above..

I am very impressed that you managed to run the tinyusb library on the stm32! I once tried to make this library work, but apparently I didn't spend enough time studying it and it didn't work out. Nevertheless, it would be extremely interesting to look at the stm32 project with tinyusb running.

scherbakov.​al sorry, I misread your post, never mind 😃(edited my comment)

A couple of years ago, a friend proposed a sensor circuit that used ambient light filtering. It's a cool scheme, I couldn't have come up with such a thing. The LED would turn off, the transistor would be measured, and then the LED would light up. The value of the previous measurement would be subtracted from the current measurement on the transistor. This process seemed to work well, but we ran into a problem when we tried to adjust the frequency of the sensor's stable operation. The frequency was about max 10 kHz when we used CNY70 components.

https://github.com/brrastak/midi_sensor/blob/main/midi_sensor.pdf

I do not know how it works) There's also a need for a control driver for such a sensor.

CyberGene sorry Nothing) I've got a double post above (2 weeks ago. I was writing from the subway and lost contact for a moment).. I tried to delete it, but I couldn't find how. It can be deleted

The odd thing is that the sensors work just fine easily switching up to 20khz as long as they are alone, and don't share the same time-multiplexed analog output signal line at the same time. (or when they operate at very slow switching speeds, then the circuit works as intended)

I've also measured response time of these phototransistors via an oscilloscope individually, which makes me think there's some weird interaction between the sensors themselves as they are switching between each other while sharing the same output line.

The only solution to this issue that I can imagine is putting a signal switching diode on each transistor output...

Alternatively, you can try connecting the outputs from the sensors to another such multiplexer, which would be controlled by the same channels and demultiplexed the outputs into one. It's just that the specifications for the diodes indicate the response time: one is about 200 ns, the other is 50ns, the ultrafast diode is 35ns, and this multiplexer works faster than 20ns.

I did a small prototype on a breadboard switching 3 sensors with arduino nano (arduino nano pins can source and sink up to 20mA)

PORTD |= (1 << PD3);  
PORTD &= ~(1 << PD4); 
PORTD &= ~(1 << PD5); 
delayMicroseconds(6); 
  
PORTD &= ~(1 << PD3); 
PORTD |= (1 << PD4);  
PORTD &= ~(1 << PD5); 
delayMicroseconds(6);

Using essentially the same circuit (but without 238), and shared analog output wire, and it works without any issues.

Now I'm starting to suspect that I have a bad solderjoint somewhere (or one of the 8 sensors is shorted or misbehaving) or some issue of that type, or else it doesn't make sense that this works and the 238 version doesn't. It has to be some sort of silly mistake somewhere.

I've made a hc238 muxing circuit using just through-hole components for purely testing purposes since it easy to attach oscilloscope and see if solder joints are OK. And this time it works as intended!

(trims are there for testing purposes only so i dont have to resolder resistors)

However there are still issues with sensors impacting each others readings. For example if one sensor is reflecting, you get max value 4096, and if all sensors are reflecting, you get only around 3500 as max value, or sensors next to each other. Which isn't optical crosstalk as far as I can tell!

So this kind of muxing probably works just fine as-is for slit sensors, but for purely analog sensors it needs some improvements (such as only blinking leds via hc238 to reduce power consumption and still use analog mux for constantly powered phototransistors?)

The good news is that I've made all the software side on stm32 take zero CPU cycles to do ADC conversions and muxing.
PWM timer with 160khz frequency, 1.25us low (for the transition), 5us high - all easily adjustable.

Every 6.25us timer triggers DMA transfer a bitpattern on relevant GPIO pins (memory to peripheral DMA transfer to BSRR - Bit Set Reset Register)
On the rising edge of 5us high initiates automatic ADC conversion (after the 1.25us transition settling time)
ADC conversion automatically outputs results via another DMA channel out into a result buffer and moves to the next position in a circular buffer.

This uses no interrupts, and effectively no cpu cycles!

You get automatically filled buffers with sensor positions all the time with zero CPU involvement! Sampled at the exact muxing timings I specify!
This happens fully automatically in the background all the time.

What's beautiful about this is that it has the exactly same code and hardware pinouts when muxing using analog switch cd4051 or muxing using hc238!
I took my old PCB that uses analog switches plugged it in - and it just works! Same code & same hardware interface!
All 8 sensors beautifully streamed via UART (up to 5.7ksps each via _921kbit uart)!

Might be a good idea to add another ping-pong DMA buffer (like double buffering)...?! software side really is the easy part with debugger where you can add memory and variable watches and see buffers, periperals and their data changing in real time!

Another cool feature of stm32g4 is VREFBUF (with available values for VREF being 2V, 2.5V, 2.9V). This also works wonderfully.
Built in OPAMP follower also seems to work great. (havented tested built in programmable gain opamp yet!)

If muxing/adc sampling can be made like this to take zero cycles, it feels almost as if single measly stm32g431 (arm32 cortex m4 @ 170mhz) might just be fully sufficient for a full piano...?

Pcbs from JLCPCB came for the hc238 muxed slit version. Seems to work as expected at 20khz per sensor at the moment!
Components per board - 2*8sensors (very fast and easy to solder!), 3resistors(1206), 2 decoupling caps (100nf, 1uf 0805), one sop-16 ic, lots of DNP (do not place) pads, 1.2mm fr4 2layer, one side almost unbroken ground plane (except for where the pcbs join).

Beginner hand-solder friendly (except for two 0805 caps?), the only upgrades to this design that I can think of are:

1. Make it into 2pcbs (11mm * 44keys * 2pcbs=484mm * 2pcbs) instead of 11pcbs (of 11mm * 8keys * 11pcbs=88mm * 11pcbs)
2. Smaller slit sensors, with smaller slit (these have massive 1mm slit where the IR light shines through).

(soldermask is incorrect, it is powered from 3.3v!

Very cost effective design, ~ 4eur (2eur boards, 2eur shipping), for 5pcbs (out of 11 total). Needs one more main board (<100x100mm) with rp2040 and that's it.

DavisB I considered such a design for a long time with those optical interrupt sensors. It makes everything very simple electronic-wise, basically the same as any digital piano or MIDI controller with membrane switches. But it requires careful mechanical regulation and PCB positioning, along with adding some shutters to each hammer and at the time I was afraid that would require way too difficult planning/designing/drawing for the PCB-s to follow the exact hammer footprint and position.

However, from a certain perspective, I think it’s the way to go for a mass produced piano (yeah, Kawai chose that solution, and Yamaha have a mix of this and over complicated key sensors).

Yes, intuition tells me that analog version with ITR8307s (6-7cents a piece) can be built with lower BOM and lower labour/assembly time costs (certainly with SMD PCBA), and overall be less finicky mechanically.

Seems like those will require more than just hc238 ic for muxing.

Something like this maybe:

(completely unbroken ground reference plane, some very thin traces. 1mm thin pcb two layer fr4? Still thinking whether to put a ground pour on top, so this is not a finalized design)

HC238 blinks only leds, CD4051 analog mux for phototransistor output. Phototransistors probably will be always connected and always pulled to ground via resistors (R0-R7). Or optionally they can be pulled to ground AFTER muxing via cd4051 (pulled with R20).

This design has too thin traces for me to build at home, so has to be ordered for an experiment...

Another version to try is to have everything on all the time (and mux via single CD4051) and barely-just-barely try to fit it into 500mA power budget. If it fits in 500mA (or if you don't care about exceeding 500mA a bit), this could be fairly straight forward, simple design too.... (as long as you don't get bitten by lack of analog electrical engineering background)

Comparator method is still not dead in my mind, g431 has a fast dac and a fast comparator.... which could be callibrated automatically programmatically (the threshold points, etc)....

moved to https://pianoclack.com/forum/d/1897-diy-random-talk

Retiring sensor strip from my old narrow key (5.2" octave) prototype:

This is a strip 5pcbs * 8sensorpairs = 40 keys. (5 cables going to main pcb, the 6th one is addr+vcc+gnd). No external power supply required. Whole system runs on less than <150mA. 20khz sample rate.

Bom cost for a whole piano if done like this (with pcbs with mainboard too, here's it's just soldered wire to wire connections on proto-board):

Single ~ 1.1eur yd rp2040 devboard sufficient for whole 88key prototype! BOM cost for the whole system is roughly 41eur all inclusive.

Pinout:

If using such devboard, only 1 pin is left for pedal. This design relies on scmitt-triggers in rp2040 pins (outputs of those slit-sensors are analog!). Main board PCB requires basically almost no circuitry, just direct point-to-point connections (except for address pins, series termination resistors (30-50ohm) would be recommended there.

All wiring in this design can be completely eliminated by combining all 11pcbs * 88mm * 25mm into one 968mm* ~ 50mm PCB where you plug in yd-rp2040 without any wires required at all. (just usb-type-C into computer, and sustain pedal that's it)

Note that PCBA services usually have board size limitation of 450-500mm. So combining two large pcbs is probably the right thing to do (with some wire jumper connecting the two).

Main board has single 0.96" i2c display for debugging purposes (just shows status animations of all 88 keys and some midi events). This is less finicky than wiring up status LEDs, etc. And very cheap. Highly recommend getting 1.3" i2c displays though. Same resolution, but significantly bigger and price is only +30c +50c or so more.

I bought ten of these yd-2040 boards, thinking that there surely must be some sort of catch, or that they use fake defective parts.
After all, they each cost around ~ 1.1eur each (including shipping)!

rp2040 is ~ 50cents in volume.

Surprisingly enough, they all work perfectly so far. Different patch sometimes uses different flash (sometimes winbond, sometimes zetta, sometimes boyamicro). These 4mbit qspi flashes are usually 8cents a piece or so. But still, it's hard to see how they make almost any money at all on these. Or maybe these are somehow defective and I haven't stepped on the defective peripheral just yet.

I suppose these don't overclock as easily as original pico board (due to use of different flash).

Design wise these are better than original RP2040 pico board, they use usb-type C, LDO instead of RT6150 buck-boost SMPS, seperate boot/rst/user button, rgb led, etc.

I also have original rp pico and weact rp2040 board, but I like these boards the best thus far.

Software wise it's all very simple too (less than 1k loc total, with lots of comments/whitespace, and 30% of it is driving ssd1306)
20khz sampling rate gives a time window of 50us.

Scanning whole piano and adding events to a single-producer single-consumer SCSP ringbuffer takes around 3-4us (or 6us at most, depending on amount of events, etc).
This is implemented using one PIO and one PIO statemachine, 1 dma channel, and interrupt triggered 20k times a second on DMA transfer completion (whole piano scanned). Total interrupt entry-exit is around 50cycles for Arm cortex m0+. CPU ~ 6cycles to test each sensor for events.

And in the main loop, ring-buffer entries (events) are taken out one by one and sent out via USB-MIDI.
This is also where delta time gets converted to MIDI velocity (this can take considerable amount of cycles depending on how it's implemented).

There's 200kb of sram available for lookup tables. Lots of flash space for that too. And plenty of cycles fixed point log implementation, etc.

Second core is completely idle! Or rather in my implementation it drives i2c screen using DMA. So dma transfer has to be started at 30hz. Rest of the time it's idling. Lots of computational power left unused.

rp2040 has absolutely fantastic documentation and SDK, and generally speaking is very pleasant to program for.

The whole piano could be sampled at higher rate too (40khz, etc), but this is limited not by CPU time, but by phototransistor switching time due to hc238 muxing implementation.

At first I was slightly concerned that rp2040 is too weak for this task, but it is absolutely not. The only drawback is that it uses usb 1.1 /fullspeed / 12 mbps 1ms pooling time. The only drawback I can think of thus far.

Great news! As of Pico SDK version 2.1.1 rp2040 has been certified to run at 200mhz!

RP2040 has now been certified to run at a system clock of 200Mhz when using a regulator voltage of at least 1.15 volts.

With this version of the SDK, you can now select a 200Mhz clock for RP2040 simply by setting SYS_CLK_MHZ=200 via preprocessor define. The regulator voltage will automatically be raised for you if necessary.

https://github.com/raspberrypi/pico-sdk/releases/tag/2.1.1

Easy 60% perf gain for nothing! This isn't an speculative overclock either, it's in spec now! This apparently came along with release of the new rp2350.

Seems to works perfectly on the yd-2040 boards, I also had a typo earlier, these boards come with 4mbytes of QSPI flash (32mbit) or 16mbytes of flash(128mbit). Original pico boards have 2mbytes(16mbit) of flash. Not that there's any real use for that much flash memory most of the time, certainly not in a MIDI controller.

Speaking of megabytes, Yamaha P-120 (released in 2002) had total wave rom size of just 32mbytes (for the whole instrument, includes other sounds, not just piano). Yamaha P-515 (released in 2018, 16 years later) had a total rom size of 256mbytes. Yamaha P-525 (released in 2023, 5 years later) has total rom size of..... unspecified amount (for obvious reasons)....

Thanks for the update.

DavisB Retiring sensor strip from my old narrow key

DavisB All wiring in this design can be completely eliminated by combining all 11pcbs * 88mm * 25mm into one 968mm* ~ 50mm PCB

Are you retiring the old ones to replace them with consolidated large ones? Or some other reasons? Are you still working with the slit-sensors? I never tried those myself, but I always thought that the calibration and/or tolerances would be very tough for a DIY project, so I discarded them to begin with. I'm interested to learn your experience with them, if you are willing to share.

DavisB I bought ten of these yd-2040 boards

DavisB each cost around ~ 1.1eur each (including shipping)!

Where did you buy them from?

DavisB At first I was slightly concerned that rp2040 is too weak for this task, but it is absolutely not.

Agreed, it is a very nice little chip. It does have its warts, though, for example this one. If you can stay away from such problems (and for some things I could not), it works great.

DavisB release of the new rp2350.

FWIW, for my "random" projects I switched to this new one (on the official Raspberry board)

I buy these boards from AE (usually when there are major sales (ie. anniversary, blackfriday, 11.11, etc)). There's many stores that sell boards based on yd2040 design such as (https://www.tztstore.com/goods/show-6203.html ).

There are many storefronts that are operated by TZT (or use the same warehouse and sources). Prices sometimes go pretty low.

For regular people I strongly recommend sticking with more reputable sources (LCSC for parts, and for devboards & swd probes WeActStudio, etc). While everything I've ordered pretty much always works for me. Huaqiangbei markets aren't exactly like ordering from Mouser...

Random local small EMS houses make those yd2040, so different batches will have different flash, different silkscreen quality... ie. they lack the consistency and quality you'd get from WeActStudio and similar.

These 0.96"/1.3" inch mono OLEDs are my favorites and along with those 2.8" 240x320 ST7789V SPI color LCDs (as little €2-3 a piece on sale) I stock them and treat them like generic jelly bean parts.