On: Notes from Dynamicland: Geokit

[goliatone]

To some extent there is the reactable, which is geared towards making music. I would imagine an extension of the idea could go in the direction you picture. Note that the reactable is tied to the surface to capture some of its interactions

http://reactable.com