Hello, HN! Headshot is a tiny side project I've been working on to create better looking profile photos. It started when my wife needed a headshot for work. Instead of just snapping a photo and calling it a day, we were looking for something a little more professional.
There are a ton of native apps that help you remove the background of an image, and after seeing the good work at remove.bg[1] I had a hunch even a script kiddie like myself could get a decent result using client side javascript with an off the shelf image segmentation model.
After a little research I found Google's MediaPipe Selfie Segmentation[2]. It's a derivative of the same tech driving Google Meet. While it is performant on mobile phones, the tradeoff[3] is a less than pixel perfect mask. It performs best when there is clear separation between the person and the background. I went down the rabbit hole in this problem space and found this[4] and this[5] which led me to believe with a little more work I can get a better result. I am completely out of my depth in ML and would most likely need a partner to help me implement better quality image segmentation for high definition photos. I considered using remove.bg's API but with the goal of keeping Headshot free, the unit economics won't work for the time being.
Headshot is built with Vue and Tailwind and hosted on Firebase. It leverages glfx.js[6] letting the user apply image filters to both the foreground and background. With a few filter tweaks you can put together a pretty decent looking profile photo. I compress uploaded images as mobile browsers seem to fall over on raw files taken from an iPhone 13. Final image compositing was pretty straightforward using HTML Canvas and the CanvasRenderingContext2D.globalCompositeOperation. I do apply a bilateral filter[7] to the segmentation mask to smooth out the edges and provide an adjustable blur[8] on the background to create a light ring effect.
The quality of the result really comes down to the quality of the starting photo. When I followed these general tips[9] for taking a good headshot I was able to create something that can pass as a reasonable headshot. I am pretty happy with the result but would love to ultimately improve the segmentation mask under more challenging photographic conditions. The lengths that the Google Meet team went to get a professional result are an interesting technical read[10] that is well above my pay grade.
I hope someone on Hacker News can use Headshot to create a great looking profile photo that they can upload to LinkedIn, Zoom, or Slack.
Happy to answer any questions.
Jason
[1] - https://www.remove.bg/
[2] - https://google.github.io/mediapipe/solutions/selfie_segmenta...
[3] - https://drive.google.com/file/d/1dCfozqknMa068vVsO2j_1FgZkW_...
[4] - https://github.com/anilsathyan7/Portrait-Segmentation
[5] - https://github.com/Volcomix/virtual-background
[6] - https://evanw.github.io/glfx.js/
[7] - https://github.com/m0ose/imageRoutines
[8] - https://github.com/flozz/StackBlur
[9] - https://headshot.page/tips
[10] - https://ai.googleblog.com/2020/10/background-features-in-goo...
Not sure if you have a further plan but could be interesting to have a paid tier / plan / purchase where you use ML to overlay a skeleton and guide people into taking a good headshot from your tips then layer your pre-existing tech as a post process.
Basically an AI photographer. Maybe impossible or very difficult but interesting idea and the profit could be pretty big (a pro photographer is like $200+ / session as you probably know)
https://xkcd.com/1425/