Trying to create a very patient (and cheap) piano teacher. I realize it will never be a substitute for a real teacher but if it gets me 50% there that's already quite neat. But this is much harder than I thought it would be.
Polyphonic sound decomposition is pretty much an open problem. I have an 'interesting' idea on how to solve it. I'm sure I'll be accused of cheating if I get it to work ;)
Oh, that's true (Melodyne made a plug in that will do it, AFAIK). Can't you do something else, like optically or mechanically read the keys? Do you even need to read them? I guess you want to tell people if they made a mistake, hmm...
Something you could do would be IR LEDs and an IR camera, where the LEDs are only reflected on the shiny keys when they're pressed due to the angle.
Yes, there are many use cases that I have in mind where this is a must. In fact, if I can't crack it then it's a useless project (other than that I will learn from it).
There are all kinds of devices that could give you midi inputs (such as the 'Piano Bar', sadly discontinued), but that's the easy way out. After all, you could also simply use a MIDI enabled keyboard. But I want it to work with a real piano, as well as possibly other instruments.
The best software for this at the moment is Anthemscore.
Hmm, very true. Do you want it to not require modifications to the piano? I guess that would mean that you pretty much need to do sound decomposition...
Yes, zero hardware changes or mods to the piano. I think that would be a dealbreaker for many people. It also means that the barrier to entry would be much lower.
Hmm, yes, if you can get it to work with just sound, that would be trivial to set up. If not, I think the light idea would still be valuable, as technically you wouldn't be changing the piano (just adding a camera and light above it).
I've actually toyed around with that idea, one interesting application would be to convert Rousseau style youtube videos back into midi.
I'm quite sure it is possible but it would still require a lot of set-up. The microphone is such a temptingly simple target that it makes sense to pursue that one first.
That makes sense (although the microphone is only a simple target if you ignore the unknown unknown of deconstructing the sound :P). I'm very much looking forward to your progress!
