I have my YC interview in 10 minutes and wanted to OSS it before. Agree it lacks a lot of polishing, but that will be done in about 2 weeks :)
JS/TS through Deno is coming in a week, I am currently working on a draft pull request for it.
Really happy to answer all the questions after my interview :)
(I host the demo on a small hetzner node, please be gentle to it, I will scale it up asap :))
PS: Since it's HN: The backend is all in Rust and the feature I am most proud of is automatic parameters and dependency inference for locking the dependency versions and generating the frontend of your app by analyzing the scripts using simple but surprisingly good heuristics :)
has modern python made these less footgun-y? the issue being that the code looks like 'obj' and 'my_list' are as provided in the default arguments every time the function is called, but the defaults are instantiated once and they can be modified inside the function:
def foo(msgs=["hello"]):
print(msgs)
msgs.append("world")
foo() # prints ['hello']
foo() # prints ['hello', 'world']
foo() # prints ['hello', 'world', 'world']
Nonetheless, introspecting on default arguments like that looks like a fine way to turn functions into UI elements if you're going to contrive that those functions aren't going to be re-entered multiple times in the same python runtime.
I read it as a "functions" ("scripts") host similar to AWS Lambda or Azure Functions with a better overall UI especially for notebook/REPL-driven development.
I've been automating a lot of tasks in multiple languages including Python with Azure Functions last few years, and I find this tool interesting to some of my use cases (though not immediately applicable because of existing cloud commitments, as such things go). I have at times thought that it would be great if I had a "Notebook view" of the Python Functions in Azure (preferably with strong git integration) to make some Data Scientist workflows easier. I've also had times where it might be nice when a cache needs to be "kicked" or a cleanup process run if I had a simple automated GUI for certain Function invocations or queue message drops that I could point a Business Analyst to.
In the first use case, I'm aware that Azure offers Notebook workspaces and there is some flow between building a test Notebook environment and then eventually migrating some of the code into a Functions app; but it is definitely not a streamlined all-in-one product and instead requires piecing together a bunch of semi-related ones and hopping back and forth between multiple different Azure resources. It also doesn't help that there seem to be fork between "general Notebook environments" and "ML Notebook environments" that seems more artificial than natural and complicates standing anything up in the first place. (Which is to say currently we aren't really using any of Azure's Notebook tools because they are too complicated and divorced from the use cases we need them. Every now and then I circle back around to explore them and the complexity just makes me angry.) This tool seems to already have some better ideas to approach that workflow and I'm curious to watch it, at the very least if it encourages Microsoft to up their game in competition with it.
As for the other use case, I know Azure Functions has almost all of the reflection metadata it needs to offer better tools here even in Python and Typescript already, but its Code/Test tool is definitely underpowered compared to the metadata it already has. It would be interesting to see if this new project provides competitive fire there.
It makes sense to me (but maybe because I'm building something similar).
You write code and they host it for you and provide a GUI around your code. It's like a crazier (in a good way) version of Kibana or Grafana or Metabase. In Kibana/Grafana/Metabase they provide a hosting platform for queries -- any queries within your internal eng team. This provides a hosting platform for scripts.
Whether or not that is exactly what they are trying to do, it's what I'm trying to do. What I wish I'd had at many prior companies.
I can't find them now while trying to search but it also reminds me of some tools I've seen that generate a GUI from a description of a tool's command line flags.
Yes for sure, I need to improve the docs! I have had issues explaining it in simple terms because it is a system that is quite obvious once you get it but confusing from a first look. I have aimed to make it consistent everywhere to reduce complexity.
Better and simpler docs will be my sole focus for the following weeks + JS/TS support.
But the step-by-step is missing the concepts of variables and resources. If you have any feedback i'd be glad to hear it. I will opensource the website EOD too.
They host "scripts" and "apps" that you can execute against "resources" (i.e. databases etc...). So basically a lightweight, online environment for running code without any deployment?
I suppose an example might be marketing has a DB of prospective customers and this would allow devs to quickly write scripts to clean it, apps to allow people to add to it and automated tasks to trigger events (eg. an email) based on certain metrics etc...
I guess it means being able to rapidly build fairly throwaway systems for internal use.
To me it looked like a low-code tool to automate "processes" like Zapier/ifttt and so on. Basically a glue between apps with a little python thrown in for transformations.
This looks pretty neat! Once the FOSS self-hostable version is out I might try it out for my local hackerspace, it seems like it might be useful for those little things that are too small to warrant standing up infrastructure at our small scale.
If you're up to it, I can send you an email when it's ready to try, likely tomorrow. The Dockerfile is mostly working as is, you just need to setup the DB, pass the connection info and do some manual superadmin rows insertion. I will automate the process asap.
Running the docker on the other hand require to pass a few env variables, among them the connection info to a PG instance. It also require to setup an user by editing directly the database and hashing by hand your password which is super inconvenient. I am working right now on making the process easier, did not expect self-host users so early.
You are right, I will prepare a public docker image and some clear instructions. Things went a bit fast today, I initially planned to do such launch in a few weeks.
I'd use it for basic process automation, airtable data IO, triggering marketing campaigns or slack bots, etc.
It's a heavy weight feature request but if I could lean on it in the way I lean on Prefect.io now for spinning up cloud workers on demand I'd use it for a lot more.
Given the name, and premise of creating flows of python scripts ... is this intended to make people think of Airflow? A lot of what we see in the demo video (on a scheduled basis run a sequence of steps which gather data, produce some summary statistics, share it to slack, parameters which can be defined when a run is triggered or from prior steps, etc etc) is very familiar to users of existing workflow systems.
The UI to create/author flows seems slick, but I also think that for important flows, teams will want to be able to review changes to flows, make cross-cutting updates to several flows when a popular script or connector is updated, etc, in which case being able to view/edit/diff flows as code is quite valuable.
Demo is a bit old (2 months ago), I need to reshoot it.
It has some resemblances with Airflow but one aim is to make it accessible to less technical users that can treat modules like blackbox. Similar to Zapier but the code is inspectable and editable at any point. At the core, it is a scheduler/orchestrator of script execution, so it's similar to Airflow in that sense, but the product is really not meant for the same usage.
I picked the name a bit randomly because it is retro and one of the first automation humans have created. I did not think too much about it but now I love it :)
However I've come across a few other similar products as well over the past year we've been building Airplane, and frequently things appear superficially similar but there's a difference in focus/use case/target customer/etc. For example, Airplane is focused on support, customer success or ops use cases (e.g. "I wrote a script to delete a user record / reset a 2fa token / etc, I can use Airplane to let the customer support team have access in a safe way"). Whereas I've seen other tools in this space target more of a "programmable Zapier" type use case (e.g. "For every new customer, I want to use a Python script to hit the Slack API and generate a custom notification.")
It'll be interesting to see how Windmill evolves over time.
Thanks for the clarification! Also, recently I discovered retool workflows [1], something I think people would use ops, customer success flows too. In general, automation like these solves relevant problem in an org.
Actually I wanted to point out that there is a pyright LSP server interacting with Monaco that would highlight those kinds of error. It is not that unintentional!
I can make python scripts with infinite potential but to invoke them I have to put them behind a CLI, webhook, cron, and then orchestrate the data into these python scripts.
So this would be like the orchestration glue? Write a script, have it conform to a schema and then Windmill can orchestrate it's execution and input/output... with a nice UI for manual invokation?
What I couldn't find in the docs is the following: is it only restricted to an entry-form > some script runs > notifies via service (slack?), or is it also possible to do something like airplane workbooks: a form, then a Python script runs an SQL query, then another form pre-populated with some data from the query, and then another script?
Is there error handling? For example, from the github example, if the email sending fails for some reason can I either select the flow to continue/log or to stop entirely?
Can flows branch? For example after sending an email, can two flows be triggered like a tree and two flows off of each one of those ad infinitum (ad no resourcium)?
>UI is autogenerated from your scripts.Gain velocity, reduce tech debt.
"Reduce tech debt by using this tool that might or might not be maintained in a few years to build a critical part of your business workflow" is a bold take I didn't think I'd see this year, but here we are.
I'm installing this asap tmrw, along with a couple others mentioned in this thread. Tools I didn't know I needed to know ;)
Thanks so much for releasing this useful and time saving software, you da man! Can't wait to see where this goes and I'd like to be part of the community.
Looks really interesting! We have an "eng-b_team" channel which is a place for all of the non-technical folks at my company to collaborate on scripts, SQL queries, etc. They would love to use something like this. How do we set up a self-hosted instance to try it out?
The Dockerfile mostly work by itself, the only thing you will lack is how to generate the super-admin. Sorry I didn't expect to have any announcement today but promised it will be filled by EOD :)
This is brilliant. The world has been needing something like Zapier, but not opaque and locked down. Everything Zapier does is a good idea, but also so easy and the whole model is way more powerful if it's open and extensible.
This is amazing, at one of my previous employers, a colleague had started learning python and was able to automate a lot of the tedious work with python.
This is very interesting. I was actually looking for a cleaner way to write SQL and Python code. I find writing multistring SQL query on Python code messy so looking forward to giving DataStation a shot.
Couple of question-
- Are you building this as a pure Open Source project? I couldn't find the pricing section, and you have built it as an electron project. I am curious to know about your pricing strategy.
- Have you considered incorporating dbt or dbt like syntax query?
> - Are you building this as a pure Open Source project? I couldn't find the pricing section, and you have built it as an electron project. I am curious to know about your pricing strategy.
Long-term plan is that everything currently open source will always be open source (and there's quite a lot I believe). I'd like to release a hosted (paid) SaaS version for teams and an enterprise version also aimed at teams.
But the desktop app that exists now will always be free and open source. And I have no plans to stop adding new databases, file sources, and scripting languages as needed to the open source core.
> Have you considered incorporating dbt or dbt like syntax query?
I think it's already pretty close to this. You can import data within SQL queries (works with SQLite, MySQL, and PostgreSQL) using the `DM_getPanel` syntax. For everything else you can use Jinja-style templates [0]. This allows you to, for example, run a query to select IDs based on some filter in a PostgreSQL database and then pass those IDs to an Elasticsearch filter in another panel -- without writing any code.
I just use the "future" slack sdk for this. It uses deno, rides on the slack auth, and has a little DB in there for storing config and state if you need it. Slack itself provides basic but sufficient UI tools.
I don't know if in practice this is really the same as what OP is providing and it's tied to slack which is abstractly bad. But if you're already working in ts and running slack it's very powerful for how easy it is.
Can you say a bit more about what you mean? Are you talking about this: https://api.slack.com/future? If so, I'm not sure how it relates, but maybe I'm missing something?
I'm more likely I'm missing something, or assuming the main use of windmill is similar to how I use this, when it's just a small overlap.
The problem I needed to solve was how to enable employees outside of engineering to run scripts written by engineers, without them having to set up/interact with a full scripting environment like node. Particularly managing secrets and auth was a problem.
That slack "future" thing makes it very easy to drop a js or ts script into a new "workflow" and push it into the slack org. You can attach it to a slack modal or form, have auto-complete hints, just generally piggy-back on the slack UI that everyone is already familiar with.
It's basically deno with the slack UI over it from the user's perspective.
Again I don't know if this solves any problems people would be using windmill to solve, but if I were using windmill I'd probably be using it for something similar to this.
It's OSS so free if you self-host, otherwise I have a pricing plan on the landing page but it I focused so far only on delivering a beta and not on billing. Hence, right now there is no quota on anything and it is all free for the hosted version. I will be delighted if it provides value to you, get adoption, and I will figure out how to monetize later.
I have my YC interview in 10 minutes and wanted to OSS it before. Agree it lacks a lot of polishing, but that will be done in about 2 weeks :)
JS/TS through Deno is coming in a week, I am currently working on a draft pull request for it.
Really happy to answer all the questions after my interview :)
(I host the demo on a small hetzner node, please be gentle to it, I will scale it up asap :))
PS: Since it's HN: The backend is all in Rust and the feature I am most proud of is automatic parameters and dependency inference for locking the dependency versions and generating the frontend of your app by analyzing the scripts using simple but surprisingly good heuristics :)