This technique might yield a highly expressive MIDI or MPE controller. Silicone is already used in performance instruments as pressure and velocity-sensitive drum pads. MIDI Polyphonic Expression instruments can use five or more separate axes of control. Complex motions like bend, twist, nudge, wiggle, etc could be mapped to these axes.
This is a blatant attempt at recruiting the obvious engineering talents of the OP to the cause of building guitar pedals and synthesizers, which I applaud.
I thought it took way too long to get to the point — why would you want to make a touchable lens? How would you use it? What would it even look like in practice?
Background is great but first give me a reason to get excited about the actual device.
"How would you use it?" is the entire video's point.
Now, what would that look like in practice. I think you should first forget about the form factor. I think the use of a smartphone (at this stage) is mostly convenience. The best lens/sensor/cpu/gpu combos are sold in this package.
When the first webcams came out, I think I would have been skeptical if someone told me "this will be a mouse soon". Today, most mice are basically very VERY low resolution webcams with buttons[1]. Maybe a new type of controller, musical instruments, or an entirely new thing will come out of this... Or nothing much, like the two research paper mentionned. In a mouse form factor i think it could be use for some of the functions of a spacemouse[2].
[2] https://www.youtube.com/watch?v=iHBgNGnTiK4 ( I Made my own Space Mouse for Fusion 360 using Magnets (DIY) ). Just check the intro to have an idea on the use of spacemouse.
Id imagine a flexible skin added to the outside of a robot containing lightpipes or fiberoptics coming back to a camera. This lets you detect collision on the outside of a robot at 30-60fps extremely cheaply. Cheap passive collision detection is missing from robotics and this is a step in the right direction. With multimaterial 3d printing you can make a wide variety of different shells for robotics which can automatically detect failures, collisions, damage and stress for the cost of the plastic and a camera sensor.
Solely the cost the materials is about 2 USD when bought in small quantities, but I am using the least expensive clear silicone. Buying the expensive stuff (Smooth-On's Solaris for example) is about 2x-3x the cost but available in much larger quantities.
I look forward to someone cloning this, driving down the cost, and putting it on crowsupply/sparkfun or somewhere. (I mean that as praise, not malice.)
I assume decent results can be achieved with a super-cheap camera module, if the optics are a good match to the silicone part.
I concur, it really is excellent. I wish I knew the total time investment it took to do and document all of this. It must have taken a very long time, not to mention the proofreading!
Most work was done as a part of my day job. It took about half a year. Significant amount of that time is for writing an academic paper. Same content as the blog post, 10x less readers, 10x more time...
Awesome write-up and effort for a PoC! The problem it’s solving, however, is not immediately obvious. So here’s the relevant (albeit not-so-convincing) passage:
> “but direct, vision-based interaction with soft silicone blobs embedded in a soft robot’s body may be an interesting option.”
Thanks! About the problem: classic academic paper issue. Needs to be framed in a way so it might work for a paper. A lot of the original idea is shaved off, does not work, is not straightforward to explain, etc. and in the end it looks a bit like a solution without a problem.
Original intention was to present a touch-sensitive lens cap for tangible interaction with (old) non-touchscreen devices. Soft robotics is not my research field, but they might be the ones that could use something similar.
The action cam application is truly practical. I have one with three unlabeled buttons, and it's quite a pain to do anything directly with the device UI.
Nice idea in principle but I always try to avoid touching the cam to avoid getting it greasy. Which is the main reason I'm so happy with front fingerprint scanners, on my S8 it happened a lot that I touched the cam which was right beside the scanner.
Really cool idea though. Some brands do the same with the fingerprint scanner which react to swipes.
