I think Mike’s success is awesome. Sidekiq has created so much value for startups in our industry and can operate up to a scale very few companies have to actually worry about. Other folk have reproduced similar in other ecosystems to success as well (see Hangfire) and there’s another half a dozen ecosystems out there where a motivated person could capture the same value.
The only downside I see is that it’s definitely led to the proliferation of commercializing OSS libs by making them slightly worse on purpose and adding a “go pro!” version. All too common these days. But again, can’t blame anybody on this path - creating good software isn’t easy and people do deserve to be paid for it.
The other place ripe for opportunity is Django-admin style tools. It’s been out for a decade maintained and free and it’s just an awesome tool. There are similar frameworks in Ruby, PHP (part of laravel’s pay to play offering), and Elixir, but not too much outside that. I’m pretty darn surprised there’s no Java or JavaScript world admin framework that can be plugged into with code. (react admin and the like are too low level, and being SPA centric don’t fight the whole battle - getting 95% of the way with no code other than pointing at a model is the real value add). One could also make oodles of cash in .NET here i imagine
> The only downside I see is that it’s definitely led to the proliferation of commercializing OSS libs by making them slightly worse on purpose and adding a “go pro!” version
This is a slightly pessimistic take on open core :)
I'm personally a big fan, as opposed to libraries that are not maintained and die. Sidekiq's competitor, Resque, has a somewhat uncertain situation (it's been dead for an year or so, then new maintainers took over).
The advantage of open core is that nobody prevents devs to create their own plugins, especially for products that are accessible to developers without highly specific knowledge (say, a relational database); this is actually happening with Sidekiq - for example, there is a plugin for a more granular scheduler.
the problem is the opensource part becomes pure marketing. for example the documentation is always about how to do things the "pro" way and becomes useless and even feels sometimes hostile to people searching for alternatives
I think the lack of a dominant JavaScript domain model syntax has limited the availability of admin tools like django-admin. Maybe something driven by OpenAPI/Swagger model definitions?
It is ancient and no longer upgraded (and based on angularjs), but I'd argue that ng-admin (https://github.com/marmelab/ng-admin) is the fastest SPA/JavaScript admin framework out there. It is a bit opinionated about paging and filtering (and painful to do moderately complex UI customization), but beyond that I have seen nothing that has come close to the same level of "quickly get CRUD admin UI available" in react/vue land, largely because (similar to django) it had its own domain entity model.
The same team moved from there to react-admin, and looking at a couple of react-admin experiences I've been involved in I'd argue that it wasn't aiming for the same ease of use.
Regarding django-admin style tools, there are solutions like zk [1] & apache isis [2] for java which are somewhat comparable, though more enterprise oriented.
I'm curious about the ones you like for Elixir! I've been looking for a bit and haven't found one that seems to be maintained and close to feature parity with Django admin
Built on Postgres, it lets your background jobs achieve transactionality and consistency with your database. Its architecture is also well designed to allow it to scale up to about as much as Postgres can handle. The Pro version comes with a great web UI built with Phoenix LiveView that does just about everything one could ask for.
Ash is an amazing framework for Elixir that comes with functionality very equivalent to DRF/filtersets/etc, admin, as well as automatic JSON APIs, GraphQL APIs, plus works great behind LiveView. The maintainer is extremely active and is about to release a much improved website for getting started and learning. I highly suggest joining the discord if you are looking for Django for Elixir.
So glad that Mike has set up a sustainable model for building rock solid infrastructure, however:
I can’t stand that you need to pay for a pro license to avoid completely avoidable data loss. Sidekiq doesn’t use RPOPLPUSH for moving jobs around until you start paying — doing the exact same thing with a different, more correct command. No new infrastructure or configuration is needed. To me this feels sketchy and like a betrayal of the pro-version business model — willfully introducing data loss in order to drive upgrades instead of just removing capabilities or features.
This touches on one of my favorite things about Ruby as a language. The functionality you're asking for becomes pretty easy to monkey patch in yourself without breaking your upgrade path or forcing you to maintain a fork.
IMO that easy of tweaking 3rd party libraries is the reason that Ruby's ecosystem is so polished.
It's really easy to drop in an initializer, make your patch, let it ride in production for a while and then contribute your patch back to the main project than it is to set up a fork and all of the ceremony involved with keeping it up to date.
It's a good example of removing barriers of entry or an idea of Ruby's "sharp knives". Great for when you need it but can also be a disaster when abused.
Yep. Definitely a great power, great responsibility tool.
It's the one thing that keeps Ruby closer to true Aspect Oriented Programming than any other language (in my experience at least). It's probably the biggest thing that I miss from Ruby when working in other languages, even ones that I love like Elixir.
Sidekiq is open source, so couldn't you or someone else make another gem for the change? I know there are gems already that enable sidekiq to replicate some of the paid or enterprise features without paying for those plans.
It surprises me that it is a problem that needs to be solved, and not just a way it should be done from day one. Back in 2011 I wrote a small ruby class that interacts with redis and provides me a reliable method to consume messages from queue.
Back than I just read the redis docs, understands that BRPOPLPUSH is the right method to call, and made a few ruby functions to do so, you could see it here [1], they are plain and simple.
From my perspective, it's just feature starving OSS part, to push customers towards the paid solution.
I mean, it’s also a bug with Resque? Resque predates the introduction of the new command in Redis and hasn’t really been maintained since. “My software is as buggy as 10 year old software unless you pay” seems pretty sketchy to me!
This is really an amazing achievement! It was Mike's IndieHackers interview (https://www.indiehackers.com/interview/how-charging-money-fo...) that really pushed me over the edge to go open-core with Flyway in 2017 and give it a sustainable future. Thanks again Mike, if you're reading this!
Just wanted to say the Prehistory section brought up a lot of nostalgia for me. Those early Ruby projects and articles that Ezra wrote led me to working at his company Engine Yard back in the beginning of 2008. Lots of memories of late nights trying to fix GitHub's deployment (GFS2 nightmares) as they were growing in the community. I remember when Resque came out and we started using it for so many of our customers. Tons of really smart hackers hanging out at the office/apartment on South Park.
Definitely wouldn't have my career without all those early Ruby hackers working together in community. Makes me wonder where in the OSS world that community is thriving just like back then.
Sidekiq is an excellent library, and a great model for open-source monetization.
Avoid hostile/nonfree licensing provisions that restrict commercial use or "require corporations to pay up". Instead, create real value in diversified offerings built upon the OSS project (which is in no way artificially limited). Sidekiq is a gift, Sidekiq Pro & Enterprise are products.
Offer paid features, support, SLAs, etc that really appeal to "enterprise" use-cases. Trust that users will convert as their need for such functionality is justified by their businesses' growth and scale.
I remember when Mike posted about making $1m from sidekiq. It was always such an inspiration and it's amazing to see how far it's come. Kudos to Mike and I hope others find that path successful too.
Sidekiq is the best library i have ever used, i have never ran into any issues using it and Mike has personally answered a bunch of my Stack Overflow questions that involve Sidekiq. He deserves every dollar.
Sidekiq is one of my first answers to the question, “Why would I choose Ruby in 2022?” It is a fantastic library and Mike is a remarkable developer. Congratulations on hitting 10 years and thank you for everything!
This is a great summary of a positive outcome for a hybrid OSS project. I took a look at the other blog posts - has anyone seen writing from the author that covers his experience supporting the pro and enterprise versions? I imagine that would be hard to do alone!
It’s worked out so far. I knew it would be tough to scale up myself as support so I set down as couple of policies:
1. Do it in public if possible via GitHub issues so others can also answer and search in the future.
2. Don’t allow more than N questions per year per customer. A few have drowned me with requests. Nope, you are buying a product, not a trainer or teacher. Read the docs, watch the videos.
3. Pour everything I know into the wiki docs. Note each wiki page will talk about semantics, edge cases, etc. That’s all there to get ahead of the support inquiries.
#3 is one of my favorite things about Sidekiq. Anything I'm trying to do with Sidekiq, I check the wiki first, and it's usually there. Any time I've introduced the library to a team, the docs and wiki are always my recommendation for answering questions. Even Pro/Ent features are covered there, which is so much nicer than having the docs hidden behind a login/paywall.
Thanks for the hard work you've poured into them Mike!
Sidekiq has stayed with me for most of my rails projects. Happy to read this. And I bet I'll continue to remember it as I run into projects that were inspired by it, like Shoryuken (which tries to be "sidekiq for AWS SNS/SQS" https://github.com/ruby-shoryuken/shoryuken).
That's awesome. A significant percentage of Ruby shops are running Sidekiq. I'm happy for his success.
The free version is very capable too; he found a very nice balance between "free version that is still very usable" and "paid version with really useful additional features."
Out of curiosity (I also sell enterprise products) what would make the purchase a no-brainer? Quarterly payments? E.g. I’ve heard that ~$300 (ballpark) is the max amount an individual can spend without a review process. Is that what you mean?
We needed the unique jobs feature, which only part of the enterprise plan.
And since (at that stage) our budget was low, the enterprise plan was too expensive.
I reached out to ask if we can do any other payment scheme, or at least get startup discount to start using, but were declined.
We ended up using an OSS library for that part and even now what we grown, still don't pay for sidekiq. If it was part of Pro, or we had a 1 year discount for startups, we prob just kept on paying even now (that we can afford it).
Sidekiq's a really good library from my experience. I don't use Rails nowadays, but they've had some of the best libraries to work with. ActiveAdmin was another amazing library that saved me on a consulting project.
Off topic: is “open-core” any different than a “Freemium commercial product you can self-host”?
I don’t want this question to sidetrack into a heated debate about open source vs commercial. But I am genuinely curious if there is a difference between the two.
The only downside I see is that it’s definitely led to the proliferation of commercializing OSS libs by making them slightly worse on purpose and adding a “go pro!” version. All too common these days. But again, can’t blame anybody on this path - creating good software isn’t easy and people do deserve to be paid for it.
The other place ripe for opportunity is Django-admin style tools. It’s been out for a decade maintained and free and it’s just an awesome tool. There are similar frameworks in Ruby, PHP (part of laravel’s pay to play offering), and Elixir, but not too much outside that. I’m pretty darn surprised there’s no Java or JavaScript world admin framework that can be plugged into with code. (react admin and the like are too low level, and being SPA centric don’t fight the whole battle - getting 95% of the way with no code other than pointing at a model is the real value add). One could also make oodles of cash in .NET here i imagine