Hacker News new | past | comments | ask | show | jobs | submit login
Blob Opera (artsandculture.google.com)
456 points by BerislavLopac on Dec 17, 2020 | hide | past | favorite | 59 comments



There's so much great stuff here:

- The state machine that seems to handle all edge cases, both in things like the tutorial and also in the procedural singing

- The animation, both the rigging (maybe these are bones?) and the spring physics. They blink randomly. Their eyes follow the mouse. Matching the mouths to the vowels and consonants is really well done

- The overall look and feel, including shaders, shadows, DOF, style, UI overlays

- The synthesizing of a human voice with vowels and consonants in a fairly convincing way. Choir synths for music production can run for thousands of dollars

- The real time harmonizing in a seamless way.

- And on top of all that, it works on all device sizes. It actually runs better on my phone than my laptop, probably due to large monitor size (the bane of any WebGL project).

Fantastic


With a little mouse wrangling, you can get them to do a fairly good rendition of counterpoint: https://g.co/arts/feoYyFonGVXicfk47


Even the small things, like the eyes going into a hypnotic trance when the blob is keenly following the blob to their right (our left).


Yep. I dabble in orchestral music, and was floored that this thing had simple articulations. Entry-level chorale libraries sometimes only have bare vowels, and this thing just... works.


There's a small delay while the other singers catch up to the harmony changes of the lead that makes it sounds more natural, it's a nice a touch.


> Choir synths for music production can run for thousands of dollars

Do you have an example? Why are they so expensive, are there no community projects in existence?


I don't know of any "choir synths" that run to thousands of dollars, but there are plenty of sample libraries that cost hundreds of dollars.

I'd guess this is procedurally generated. Different tech.

The graphics are delightful, but the vocals are actually quite crude compared to what's possible, even without any ML.

Two technologies called FOF and PSOLA have been around for decades, and can sound not unlike this.


And it works perfectly on the phone too.


Someone wrote a tool to control the blobs with MIDI. https://mobile.twitter.com/Overlapping/status/13389792569032...


Apparently the Blobpera already listens for MIDI input devices!

https://twitter.com/thealexweaver/status/1339535386444521472


Nice. I've already tried blob Megalovania :) .

Incidentally, does FF have the "pretty print source file" function somewhere that chrome does? The App.js in this site is minified, making it difficult to set breakpoints.


When you have a JS file open in the debugger tab, there's a `{}` button at the bottom of the file tab that prettifies the file.


It would be amazing if someone would create a hosted blob opera midi player.


This is now published as a standalone module https://news.ycombinator.com/item?id=25523521


This is one of many amazing projects by David Li http://david.li/


When you can pull this off on the web and yet people will still insist on making me download an 100mb app to order pizza.


I submitted this yesterday, and it somehow didn’t stick, but here’s a list of the stuff I managed to do with it so. Far:

https://taoofmac.com/space/links/2020/12/16/0904


How do you get them all to sing at the same time?


The lower voices harmonise with the highest singer by default, unless you explicitly mute them.


It does sometimes get itself into a state where that harmonizing doesn't happen, even with nobody muted. I haven't worked out what the trigger is.


You just lead with the leftmost one. Follow the tutorial, it's fun.


Fun. I wonder how Google decides what projects to choose, the artist behind Blob Opera made something similar back in 2018: https://www.adultswim.com/etcetera/choir/


It's the same artist!


Yeah?..

> the artist behind Blob Opera made


I'd love to see the source code on this. At least part of it is a language compiled to WASM (e.g. https://gacembed.withgoogle.com/blob-opera/harmonizermodule....)


My wife was convinced I had suffered some kind of major head injury after I made her watch/listen to this.

I still think it's incredibly cool and cute though.


That's sad :(


this is why I don't show anything like this to mine. But I will show it to my daughter :)


To quote my father, who is a huge opera fan: "I've heard worse!"


Imagine having a company so big you can have an arts and culture division...


Let’s be honest, the arts and culture division is really just a part of marketing. Maybe not in name, or even on the org chart, but in practice.


It's very much a hearts and minds project, with stakeholders in engineering, marketing, press and policy. Each dept uses it to aid their local and regional initiatives. For the first few years it had to fight hard for its existence (originally as a 20% project), scraping budgets together from random sources until stakeholders eventually saw the value.


> until stakeholders eventually saw the value

What's the value if it's gone in 2 years? I wish they'd put similar effort into preserving the app/servers (simple life support) instead of shutting them down.


It's been running in one form or another for 11 years now.


The budgets for arts & culture projects are very small, most companies could afford said division


The automation of opera singers came earlier than expected


Is there a paper or blog post detailing the training process? I could not find any information about that.


This is fantastic. The overall experience is magnificantly done. As a Frontend Developer i am really impressed by the sharpness of the UI and the funcionality the interface makes possible. Does anybody know what the Techstack is or where it is posted how the built this?


Why does need a techstack? It just looks like custom WebGL and custom Web Audio.


Launch one of the Christmas songs, let it play for a few bars, then grab one of the blobs and sing a solo. Amazing!


I enjoyed this, but couldn't get the address bar to disappear on Firefox mobile. Perhaps that's ironic, but I feel like the devs of this have done a great job, and shouldn't be worked about multi browser support.


You can "install" the PWA to launcher homepage. If it's accessed via the homepage shortcut, the address bar would be hidden.


Has anyone reverse engineered the binary format which represents a song?


If anyone else has troubles loading this, It didn't work when I was logged in with my G Suite account. Using an incognito tab works fine though.


Are there machine learning parts here the actual voice generation? Writing the vocal parts, and that note interface don't require ML AFAICT.


Making Blob Opera with David Li: https://www.youtube.com/watch?v=ZfLYuXi6sDI


Reminds of pink trombone



Reminds me of the silicon vocal cords robot "motormouth robot"

https://www.youtube.com/watch?v=Bht96voReEo


Is there any way to get the blobs to sing staccato?


Sort of, if you do it for one at once. The UX guides you the other way.


Really amazing thing and at the same time shame for not embracing this in open source way.


I don't get it! Can someone explain why/how this is special? Seems like it is only a matrix of like 40 states per blob and that the lower blobs just follows with the same state? Of course well executed but..?


The voices are ML generated based on hours of recordings of real opera singers. It seems that the real time synthesis includes details like realistic switching between vowels, realistic trills on long notes, etc. There's a lot going on to make those voices sound natural.


It's cutting edge A.I implemented in a fun, geeky way. It just works effortlessly. What's not to like?!


The lower blobs are creating harmonies based on your blob's state. The fact that it produces results that are both musical and interesting is very impressive to me.


I would like to know what the machine learning part of this is. Not knowing any different, and seeing that you can only drag one character at a time, it seems like it could be entirely done with relatively simple harmony rules.

ML or not I still think it's super cool but it would be great to know what's the ML part.


After experimenting a bit, the harmony appears not to be a simple function of the note you play. Perhaps it's taking into account previous notes, and putting harmony in the context of a 'piece'? (that would be cool) Or maybe it's more simply random.


Addicted in a heartbeat, greatly executed. I'll use before Google decides to remove it, or shutdown the entire Art and Culture department.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: