On: Show HN: Cortex – Open-source alternative to SageM...

[calebkaiser]

I'm sure others will comment, but in the meantime, some people have written up their experiences using Cortex in production. I'd point you to AI Dungeon: https://medium.com/@aidungeon/how-we-scaled-ai-dungeon-2-to-...

We also have a pretty active Gitter channel: https://gitter.im/cortexlabs/cortex

As for your second question, Cortex uses Docker to containerize models. The rest of Cortex's features (deploying models as microservices, orchestrating an inference cluster, autoscaling, prediction monitoring, etc.) are outside Docker's scope.

[bobosha → calebkaiser]

>orchestrating an inference cluster, autoscaling, prediction monitoring,

Does this approach preclude the need for queuing (a la RabbitMQ) and/or a load balancer?

[calebkaiser → bobosha]

Yep! Cortex deploys load balancers on AWS and manages queueing.

[bobosha → calebkaiser]

This is super-exciting! I didn't know it could be this easy!

How do you handle API authentication? Is there a module that interfaces with AWS API gateway? or external API authentication?

[calebkaiser → bobosha]

Right now, users handle API auth by using AWS API gateway in front of Cortex, but incorporating AWS API Gateway into Cortex to automate this is on our short term roadmap.