Ok, sooooo…..
JayKominek All assuming your battery is a reasonable voltage to power the RP2040 directly:
I tried with a new CR2032 which was a 3.38V before the test. During the test its voltage dropped to 1.7V and the pico won't power up. After the test the battery raised back to around 2.9V. I guess the whole thing needs more ampere than the battery can supply (and that was before I attached the SD card, whose SPI must consume some more). I guess I should have done some math before, but I didn't 😅 -- I thought to try with two D-cells in series, but I did not have any around that had enough juice to reach more than 3V together. So I tried with (another) new CR2032 connected as I described earlier.
JayKominek VREF is separated from 3V3 by a 200 ohm resistor. I wouldn't expect the battery to prevent noise on VREF unless R7 was removed.
Makes sense. Perhaps it can mitigate such noise a bit, otherwise I am not sure what they mean in the datasheet about that then. In any case, I tried it that way, since that was the only thing I could do (when you only have a hammer…..) -- For the tests below when I say "battery" I mean putting the battery between ADC_VREF and ground, and using it to power LEDs and phototransistors. Everything else is always powered with USB.
JayKominek Has somebody tried this that you're aware of?
Not that I know of.
For reference, here is what a strike looks with USB power. You've seen this already:

And here is what a strike looks like with USB power and PWM forced on like @xooorx suggested (gpio_pull_up(23);
)

And this is with battery (and still forced PWM):

Now, you have certainly seen that the noise is shrinking. Here is a zoom in of the noise with the default USB power:

Pay attention to the vertical scale which is different among these plots. The horizontal scale is instead more or less the same. This is with PWM power -- marginally beter

And finally, here is a zoom for the battery.

Do notice that with the battery the noise is indeed much better, but not as good as it seems at first sight, since the whole signal is somewhat less (compare how the peak does not reach the yellow line in the strike plot), hence the S/N ratio needs to take the smaller signal into account. Yet, it's definitely better, since it's just about 10
ADC values of noise for about 4096-1800 = ~1300
ADC values of signal,
Whereas for the regular / PWM powered cases the signal signal is 4096 - 2400 = ~1600
ADC values and the noise between 30 and 40 ADC values: much more noise for only a bit more signal.
That D-cell tests seems appropriate…. It would be nice to have the code self-sufficient (rather than dependent on the computer issuing commands via minicom as it is now) so one does not need to attach the USB as it is now in my test setup: https://github.com/davidedelvento/no-OS-FatFS-SD-SPI-RPi-Pico
I don't think I'll have the time to do it this weekend.