.
Open-source developer infrastructure for internal tools. Self-hostable alternative to Airplane, Pipedream, Superblocks and a simplified Temporal with autogenerated UIsm and custom UIs to trigger workflows and scripts as internal apps.
Scripts are turned into UIs and no-code modules, no-code modules can be composed into very rich flows, and script and flows can be triggered from internal UIs made with a low-code builder. The script languages supported are: Python, Typescript, Go, Bash, SQL.
Try it - Docs - Discord - Hub - Contributor's guide
Windmill is fully open-sourced (AGPLv3) and Windmill Labs offers dedicated instance and commercial support and licenses.
main.mp4
- Windmill - Turn scripts into workflows and UIs that you can share and run at scale
-
Define a minimal and generic script in Python, Typescript, Go or Bash that solves a specific task. Here sending a POST request. The code can be defined in the provided Web IDE or synchronized with your own github repo:
-
Your scripts parameters are automatically parsed and generate a frontend.
-
Make it flow! You can chain your scripts or scripts made by the community shared on WindmillHub.
-
Build complex UI on top of your scripts and flows.
Scripts and flows can also be triggered by a cron schedule '_/5 _ * * *' or through webhooks.
You can build your entire infra on top of Windmill!
import * as wmill from "https://deno.land/x/windmill@v1.62.0/mod.ts";
//import any dependency from npm
import cowsay from "npm:cowsay@1.5.0";
export async function main(
a: number,
// unions generate enums
b: "my" | "enum",
// default parameters prefill the field
d = "default arg",
// nested objects work c = { nested: "object" },
// permissioned and typed json
db: wmill.Resource<"postgresql">
) {
const email = Deno.env.get("WM_EMAIL");
// variables are permissioned and by path
let variable = await wmill.getVariable("f/company-folder/my_secret");
const lastTimeRun = await wmill.getState();
// logs are printed and always inspectable
console.log(cowsay.say({ text: "hello " + email + " " + lastTimeRun }));
await wmill.setState(Date.now());
// return is serialized as JSON
return { foo: d, variable };
}We have a powerful CLI to interact with the windmill platform and sync your scripts from local files, github repos and to run scripts and flows on the instance from local commands. See more details.
You can run your script locally easily, you simply need to pass the right
environment variables for the wmill client library to fetch resources and
variables from your instance if necessary. See more:
https://www.windmill.dev/docs/advanced/local_development/.
- Postgres as the database
- backend in Rust with the following highly-available and horizontally scalable
architecture:
- stateless API backend
- workers that pull jobs from a queue in Postgres (and later, Kafka or Redis. Upvote #173 if interested )
- frontend in Svelte
- scripts executions are sandboxed using google's nsjail
- javascript runtime is the deno_core rust library (which itself uses the rusty_v8 and hence V8 underneath)
- typescript runtime is deno
- python runtime is python3
- golang runtime is 1.19.1
Windmill uses nsjail on top of the deno sandboxing. It is production multi-tenant grade secure. Do not take our word for it, take fly.io's one.
There is one encryption key per workspace to encrypt the credentials and secrets stored in Windmill's K/V store.
In addition, we strongly recommend that you encrypt the whole Postgres database. That is what we do at https://app.windmill.dev.
Once a job started, there is no overhead compared to running the same script on the node with its corresponding runner (Deno/Go/Python/Bash). The added latency from a job being pulled from the queue, started, and then having its result sent back to the database is ~50ms. A typical lightweight deno job will take around 100ms total.
We only provide docker-compose setup here. For more advanced setups, like compiling from source or using without a postgres super user, see documentation.
curl https://raw.githubusercontent.com/windmill-labs/windmill/main/docker-compose.yml -o docker-compose.yml
curl https://raw.githubusercontent.com/windmill-labs/windmill/main/Caddyfile -o Caddyfile
curl https://raw.githubusercontent.com/windmill-labs/windmill/main/.env -o .env
docker compose up -d
Go to http://localhost et voilà :)
The default super-admin user is: admin@windmill.dev / changeme.
From there, you can follow the setup app and create other users.
We publish helm charts at: https://github.com/windmill-labs/windmill-helm-charts.
If you do not want, or cannot (for instance, in AWS Aurora or Cloud sql) use a
postgres superuser, you can run ./init-db-as-superuser.sql to init the
required users for Windmill.
To self-host Windmill, you must respect the terms of the AGPLv3 license which you do not need to worry about for personal uses. For business uses, you should be fine if you do not re-expose it in any way Windmill to your users and are comfortable with AGPLv3.
To re-expose any Windmill parts to your users as a feature of your product, or to build a feature on top of Windmill, to comply with AGPLv3 your product must be AGPLv3 or you must get a commercial license. Contact us at ruben@windmill.dev if you have any doubts.
In addition, a commercial license grants you a dedicated engineer to transition your current infrastructure to Windmill, support with tight SLA, and our global cache sync for high-performance/no dependency cache miss of cluster from 10+ nodes to 200+ nodes.
To get the same oauth integrations as Windmill Cloud, mount oauth.json with
the following format:
{
"<client>": {
"id": "<CLIENT_ID>",
"secret": "<CLIENT_SECRET>",
"allowed_domains": ["windmill.dev"] //restrict a client OAuth login to some domains
}
}and mount it at /usr/src/app/oauth.json.
The redirect url for the oauth clients is:
<instance_url>/user/login_callback/<client>
Even if you setup oauth, you will still want to login as admin@windmill.dev / changeme to setup your instance as a super-admin and give yourself admin rights.
The list of all possible "connect an app" oauth clients
To add more "connect an app" OAuth clients to the Windmill project, read the Contributor's guide. We welcome contributions!
You may also add your own custom OAuth2 IdP and OAuth2 Resource provider:
{
"<client>": {
"id": "<CLIENT_ID>",
"secret": "<CLIENT_SECRET>",
// To add a new OAuth2 IdP
"login_config": {
"auth_url": "<auth_endpoint>",
"token_url": "<token_endpoint>",
"userinfo_url": "<userinfo endpoint>",
"scopes": ["scope1", "scope2"],
"extra_params": "<if_needed>"
},
// To add a new OAuth2 Resource
"connect_config": {
"auth_url": "<auth_endpoint>",
"token_url": "<token_endpoint>",
"scopes": ["scope1", "scope2"],
"extra_params": "<if_needed>"
}
}
}For users to receive emails when you invite them to workspaces or add them to the instances using their emails, configure the SMTP env variables in the servers:
SMTP_FROM=noreply@windmill.dev
SMTP_HOST=smtp.gmail.com
SMTP_PORT=587
SMTP_USERNAME=ruben@windmill.dev
SMTP_PASSWORD=yourpasswordapp
You will also want to import all the approved resource types from WindmillHub. A setup script will prompt you to have it being synced automatically everyday.
This will use the backend of https://app.windmill.dev but your own frontend with hot-code reloading.
- Go to
frontend/:npm installnpm run generate-backend-clientnpm run dev
- Et voilà, windmill should be available at
http://localhost:3000/
See the ./frontend/README_DEV.md file for all running options.
- Create a Postgres Database for Windmill and create an admin role inside your
Postgres setup. The easiest way to get a working postgres is running
cargo install sqlx-cli && sqlx migrate run. This will also avoid compile time issue with sqlx'squery!macro - Install nsjail and have it accessible in your PATH
- Install deno and python3, have the bins at
/usr/bin/denoand/usr/local/bin/python3 - Install caddy
- Go to
frontend/:npm install,npm run generate-backend-clientthennpm run dev- In another shell
npm run buildotherwise the backend will not find thefrontend/buildfolder and will crash - In another shell
sudo caddy run --config Caddyfile
- Go to
backend/:DATABASE_URL=<DATABASE_URL_TO_YOUR_WINDMILL_DB> RUST_LOG=info cargo run - Et voilà, windmill should be available at
http://localhost/
Windmill Labs, Inc 2023