Really awesome, what a wonderful way to expand one's horizon on an instrument.
In simple terms, he is making a custom delay-box for a fancy player-piano (the Yamaha Disklavier). For one example he programmed the piano to be symmetric (i.e. any note on one side will nearly immediately play a cognate key from the other side). In another example he has the piano play inversions of arpeggios after he plays them.
There is not much about the code, but a freeze frame at 1' 34" shows a Processing 3 logo [1] which is likely used for generating the real-time graphics for performances, and can also be used to communicate with devices. I think it says great things about the state of programming languages when barriers can be made small enough for professional pianists to make something for themselves!
Barriers have been pretty low for a while now. At IRCAM in the 1980s, visiting composers would typically first be assigned a computer expert to help them, but many of them soon learned to work with e.g. PatchWork or OpenMusic themselves, unassisted (typically programming in Lisp in those days). While I'm happy that this contemporary pianist is able to get the behaviour he wants from his instrument, I don't see how this is particularly knew or innovative when, for example, Philippe Manoury was doing the same three decades ago.
I'm still amazed at the low barrier nowadays - I just sat down this morning with my keyboard and computer. I downloaded Processing and MidiBus and had Tepfer style symmetric piano and delay working in under 30 minutes (though I had a dusty understanding of Java to begin with). [1] I think I'm inspired to try my own Tepfer-piano playing...
Also, the animations are stringly reminiscent of the work of Stephen Malinowski ("smalin"). The one around 3:00 is similar to the visualization he made for a guitar transcription of Clair de Lune, but the "balls" look like Euler's Tonnetz[0] grids.
Edit: I commented before reading TFA, where this is referenced. Silly me.
You don't even need an expensive piano, almost any digital piano could interact with your laptop via MIDI interface. Only thing you'd miss is haptic feedback of keys being pressed automatically.
That said, aside from simple symmetry and echo effects, it would be challenging to build on top of this with a system that would adapt to changes of key, tempo and playing style. As in article, human player is always the one who is adapting to fixed algorithm.
My assumption was that this was going to be more like Google's A.I. Duet. But from the music in the video it seems like hes mostly using deterministic algorithms which is also pretty cool.
Yeah, and this sounds much better. Though the player is much more competent too.
I want to hear an algorithm that analyzes the last however many scale tones and models the tonality in real-time so that the piano can intelligently harmonize with the player. It seems like he's just doing simpler stuff like looping, inverting the tones, etc.
Really cool! Especially how the technology inside the piano keeps the aesthetic of a standard one and only needs a single end connection to a laptop. Previous audio switching and routing equipment I've used has needed a myriad of ports ranging from serial, ethernet, and usb in order to interface with all of them. Combining this into a single system really allows for a user to learn the toolchain quickly and be able to apply algorithms and custom modulation to make the art uniquely their own.
In simple terms, he is making a custom delay-box for a fancy player-piano (the Yamaha Disklavier). For one example he programmed the piano to be symmetric (i.e. any note on one side will nearly immediately play a cognate key from the other side). In another example he has the piano play inversions of arpeggios after he plays them.
There is not much about the code, but a freeze frame at 1' 34" shows a Processing 3 logo [1] which is likely used for generating the real-time graphics for performances, and can also be used to communicate with devices. I think it says great things about the state of programming languages when barriers can be made small enough for professional pianists to make something for themselves!
[1] https://processing.org