I really have to applaud Apple for doing these things on the device instead of on their cloud servers. They respect your privacy far more than most large tech companies (I would just say “they respect your privacy, others don’t”.)
They discuss photo and video storage on iCloud specifically:
> Apple’s iCloud Photo Library is a cloud-based solution for photo and video storage. However, due to Apple’s strong commitment to user privacy, we couldn’t use iCloud servers for computer vision computations. Every photo and video sent to iCloud Photo Library is encrypted on the device before it is sent to cloud storage, and can only be decrypted by devices that are registered with the iCloud account.
So if I throw out all my Apple devices and forget my current iCloud password, the photos are gone?
Or does Apple store the encryption key for me somewhere as a convenience, so that once I get new decides and reset my iCloud password, I regain access?
That's messed up and makes it certain that I will never use iCloud. Losing my family photos is a much worse outcome than having them accessed by a hacker.
That is what you get when you want real privacy protection, no one but you should be able to access that data. That being said, iCloud is not really a backup, just a File Storage with syncing features, you should get another service (internal or external) to backup your data like Backblaze or a NAS.
It is not capable of identifying individual people, just recognizing faces and facial features in an image. Still quite useful for things like Snapchat's masks.
It was also used to assist blind users in their efforts to use the built in camera app.
>VoiceOver will then use face detection to tell you how many people are in the frame. It will also say if a face is small, and where in the frame a face or faces are located incase you want to try and better center them. When you move, it will tell you the new framing, so you can figure out if you're getting closer to the shot you want.
Does this work better than Google's on-device face detection? I was experimenting with it yesterday and it seems to work pretty well, even in low light.
If the non-NN techniques are faster, but the NN techniques are more accurate, couldn't they be combined to provide both speed and accuracy?
Use the NN to find the faces, and the non-NN to track faces once the NN has found them, and use the NN to periodically check to see that the non-NN is still locked on.
Does that work? A NN might detect half a face, but then how do you switch over to the "find two dots" technique when there's only one eye? This seems susceptible to a lot of flapping.
I'd expect that the non-NN part would be more of a "track movement of this arbitrary blob" thing rather than a "track movement of this face" thing.
Suppose the NN is only 25% of the speed you need to support the frame rate you want. Then every time you get a new face blob list from the NN, the non-NN tracker would to track the blobs for 3 frames. My guess is that in most common photography situations where you need face detection, faces won't move very far or won't change orientation or lighting very much in 3 frames.
I understand this article is about face detection, but on a tangential note, face identification against known faces in the Photos app has been laughable for me on iOS 10 (haven’t checked if this has improved in iOS 11, since I don’t have many new photos with new faces). Even with some manual training, the face identification and name matching seemed quite primitive, getting most of its guesses wrong (for example, matching kids’ faces against adults’ faces).
That said, I do strongly admire and appreciate Apple’s stance on privacy and the software doing these kinds of work on the device and not on the cloud. I can wait for these things to improve.
For me the faces function in ios photos has been pretty amazing. Accurately detecting blurry background faces is surprising. I just checked it and it picked out a picture of me where just my nose and part of a cheek is showing! (aviator sunglasses and scarf covering my face).
I feel like this paper reads part like a paper on AI and like a marketing piece humble brag, “back in 2014 when we were doing X” which seems like its trying to say “we’ve been doing DNN image face before Google and Facebook made it popular”. But if you read papers from Google, FB, MS, et al on AI, they don’t have these kinds of humble brags and they contain far more citations of previous work in the field.
I can be mistaking myself but I read more this article like a technical blog post than a paper. It seems to me it is more akin of the usual technical blog posts written by big tech companies than the writing of a paper formally submitted to an academic journal.
Well, the way Apple publishes their results means that other people always have to refer to the "Apple Journal" when they cite those results. Wheareas Apple does not refer to articles published by Google and Facebook in that way (by naming them). There's no single reference to Google or Facebook on that blog (or journal, as they like to call it).
However, due to Apple’s strong commitment to user privacy, we couldn’t use iCloud servers for computer vision computations. Every photo and video sent to iCloud Photo Library is encrypted on the device before it is sent to cloud storage, and can only be decrypted by devices that are registered with the iCloud account
They could have matter of factly just described their algorithm. How many papers do you read from ACM that start off with a long, marketing oriented set of paragraphs to justify why the paper's algorithm was written?
Let if you're reading a paper on style transfer using DNN, you don't get "Well, we wanted to paint replicas of these paintings, but due to corporate issues, our motto, or our lack of impressionists, we were forced to invent this algorithm..."
I just felt like there was an excess of Apple branding and marketing in what's supposed to be a scientific paper. Yes, Google does this too, but on it's blogs, not in the actual CS papers it publishes.
The papers on Map Reduce or Dremel aren't full of humble brags.
This isn't a research paper. Apple never claimed it to be one. It's not in some scientific journal. It's not peer reviewed. It doesn't contain any level of detail. It lacks any of the sort of norms you would expect from a paper. It's even hosted on Apple.com in a blog style format.
So why are you holding it to a research paper standard ?
Maybe because I expect research to be published not just as marketing fluff, but also to be published as papers too. Apple for a long time was known for blocking their AI researchers from participating at conferences or publishing. They published their first paper only recently in 2016 (https://www.macrumors.com/2016/12/26/apples-ai-team-first-pa...) and it seems like squeezing water from rock.
This behavior rubs me the wrong way. Science is a collaborative community endeavor. Look at how many papers have been published by Google (https://research.google.com/pubs/papers.html) 874 in machine intelligence, arguably Google's main secret sauce that you could argue they should keep as a trade secret for competitive differentiation.
Apple's secrecy in product development is fine, but IMHO, if you're consuming the fruits of community academic and commercial research, and trumpeting your products advancements based on that, it behooves you to publish more openly, the papers, at least on preprint services like ArVix.
Back in 2014 teams at Apple weren't allowed to release papers. If they were you wouldn't be making this comment since you would be including Apple in the list of companies who do this. Not that this is even a paper.
And who cares about Google, Facebook anyway ? They never invented this technology nor were they even the first to incorporate it in shipping products.
Not sure if this was your implication - but this is not capable of identifying people in terms of putting a name to a face. Just identifying where there is a face.
Photos.app is capable of finding faces and then categorizing them though. You put a name on it, and it can even be tied to a contact. Dunno if it’s related to this paper.
