While I agree that AI infra startups are hard to build, I strongly disagree with the idea that they are harder than foundational or application layer startups. I think it boils down to what you know and what resources you can muster.
For instance, foundational AI startups are also ridiculously hard to build. You need an insane amount of funding, spend it pretraining models to stay competitive only to find that gains in hardware and model architecture make them obsolete within months plus there's no real guarantee that scaling will keep working.
Application layer startups are hard in a very different way, there's an insane amount of competition and new capabilities are emerging every few weeks. I have worked with a few AI girlfriend startups and they are really struggling with keeping apace and warding off ridiculous amount of competition.
I think it's really just YMMV. Of course, the deeper you get into the stack, the more monopolizing pressure there is. Is it hard to build AI infra startups? Yes 100%. Will there be very few winners? Yes. Is it harder than foundational or application layer startups? Depends on the founders' strengths. Is it Is it a lost cause? I really don't think so.
Author here. Yes, I explicitly called out the danger of thinking application layer startups are easier, because it totally depends on the founding teams' backgrounds and interests.
Just to add- not easily defeated. I am even hosting a bounty for whoever can break this one (the model weights and tokenizer are available on huggingface) :D
Also to clarify- privacy preserving here means protecting from the model inference provider. Sort of like, if openai trained a GPT-4 using this scheme and gave the government the key. Then the gov can use it safely even while it’s hosted on OpenAI’s servers and OpenAI on the other hand does not need to share the model weights with the gov
It is possible to reconstruct the text from an embedding but also more importantly, I believe that by embedding, the OP means sending the computed embedding matrix instead of the input. Which is a simple matrix inversion problem.
The Pagani Utopia is fairly new (launched September 2022) and is completely bereft of screens and gizmos. A complete work of art costing a mere $2.5 million
We are using ellipsis and sweep both for our open source project and they are quite helpful in their own ways. I think selling them as an automated engineer is a little over the top at the moment but once you get the hang of it they can spot common problems in PRs or do small documentation related stuff quite accurately.
Ellipsis caught a bunch of things that would have come up only in code review later. It also got a few things wrong but they are easy to ignore. I like it overall, helpful once you get the hang of it although far from a “junior dev”.
Good catch. The team could add this rule to their Ellipsis config file to make sure that it's always flagged: "Never use magic numbers. Always store the number in a variable and use the variable instead."
> I think selling them as an automated engineer is a little over the top at the moment
Indeed. Amazon originally advertised CodeGuru as being "like having a distinguished engineer on call, 24x7".[^1] That became a punchline at work for a good while.
I can definitely see the value of a tool that helps identify issues and suggest fixes for stuff beyond your typical linter, though. In theory, getting that stuff out of the way could make for more meaningful human reviews. (Just don't overpromise what it can reasonably do.)
As it stands today, Ellipsis isn't sold as an AI software engineer.
One of our largest learnings is that state of the art LLM's aren't good enough to write code autonomously, but they are good enough to be helpful during code review.
Right, I stand corrected, I think I confused the branding of other competing products. I remember really liking the fact that ellipsis does _not_ sell itself as a developer. I’ll edit my comment to reflect that. :)
Not as a “junior dev” that sweeps markets itself as but it is useful in its own ways. For example, one really nifty way I found to effectively use it is to:
- git diff
- gh issue create “sweep: update docs for this file change” for every file changed
It’s not perfect even after that but gives me a good starting point and often just needs a minor change.
For instance, foundational AI startups are also ridiculously hard to build. You need an insane amount of funding, spend it pretraining models to stay competitive only to find that gains in hardware and model architecture make them obsolete within months plus there's no real guarantee that scaling will keep working.
Application layer startups are hard in a very different way, there's an insane amount of competition and new capabilities are emerging every few weeks. I have worked with a few AI girlfriend startups and they are really struggling with keeping apace and warding off ridiculous amount of competition.
I think it's really just YMMV. Of course, the deeper you get into the stack, the more monopolizing pressure there is. Is it hard to build AI infra startups? Yes 100%. Will there be very few winners? Yes. Is it harder than foundational or application layer startups? Depends on the founders' strengths. Is it Is it a lost cause? I really don't think so.
reply