interesting data point on its performance but the reason decision makers might be cloud happy is amazon has the resources to scale this to thousands of RPS and develop the backend codebase up to hundreds of engineer contributions for many years, even as key talent comes and goes. The same might not be true of your prototype especially if the core competency of your business is sales or legal compliance not distributed computing platforms.
Our team was just a general machine learning team with no expertise in compliance issues, and only one or two of us had experience in face detection. We only cared about providing a web service that gave good face detection performance (the downstream consumer was the team that had to then take the face detection output and do business logic with it for compliance).
> amazon has the resources to scale this to thousands of RPS
Actually, this part is mostly straight forward. Not easy, but straight forward, and Amazon already has this part solved for most things. Our ML team did have expertise in making very large-scale services (our company operated a large ecommerce web store, so our traffic was actually very high, top 500 Alexa rank), but our concern over Amazon had nothing to do with whether it could handle enough requests per second.
But this also misses the point. The latency for a single request was too poor for a variety of natural images (crowd scenes, public streets, etc., -- exactly the types of images relevant for many surveillance applications). It was not a throughput issue, I guess unless you expect consumer teams to do something like chopping images up into sub-images, replacing one Rekognition request with a bunch of sub-image requests, and then post-processing to stitch the results together and account for error introduced by the choice of where to split the original image. Yikes.
It's not an issue of "scaling it up" -- it's an issue that the fundamental machine learning algorithm underpinning it suffered too high latency, likely because it was a deep neural network combined with a bunch of post-processing gadgets that are conceptually not good enough for bespoke use cases -- things which engineers on a centralized platform have little incentive to care about.
The real question is whether Amazon has incentives to make this part better, and if so, how? Will a general one-size-fits-all face detector work for customers? Or will they need application-by-application customer-specific tailored models (and if so, why aren't customers savvy enough to realize that the all-in costs to build and operate it in-house are so demonstrably cheaper than Amazon, and might be the only way to meet specific performance goals)?
I hear this so often, that Amazon can "just scale it up" -- but I think if you really work in face detection and have studied the problem deeply, you know this is not right. There's a whole different bucket of problems and issues and cost-effectiveness trade-offs going on.
