Lorem Ipsum
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris
Serverless data pipelines written for Data Engineers.
Frameworkless. No dependencies. Write testable tasks in modern Python.
Compose in YAML and schedule them in serverless Lambda or Airflow. Package them in reusable components for your data analysts to use from the UI.
Fully Open Source. View on GitHub.
Features
Lorem Ipsum
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris
Lorem Ipsum
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris
Lorem Ipsum
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris
Lorem Ipsum
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris
Lorem Ipsum
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris
TYPHOON_DEPLOY_TARGET=typhoon
Serverless deployment with AWS Lambda.
BLOG: Make a serverless telegram bot (20 mins).
Or try it with zero migration risk with robust Airflow transpilation.
# Simply add this to .cfg and a few settings
TYPHOON_DEPLOY_TARGET=airflow
Extend with pure python. Create a function and reference it directly in YAML.
Because they are pure python you can also easily test them with pytest.
name: favorite_authors
schedule_interval: rate(1 day)
tasks:
choose_favorites:
function: typhoon.flow_control.branch
args:
branches:
- J. K. Rowling
- George R. R. Martin
- James Clavell
get_author:
input: choose_favorites
function: functions.open_library_api.get_author
args:
author: !Py $BATCH
Compose elegant YAML DAGs using re-usable hooks and pure python functions just like lego.
VSCode has auto-complete so its really quick to compose any complex task.
Component UI templates can abstract complex workflows for other users.
Make your analytics team self-sufficient by creating data pipeline apps so they can create schedules themselves via the api.
Plugins
Typhoon supports any DB that works with Python DB API 2.0
Python DB API 2.0
Postgres
Local File Storage
Dynamo DB
SQL Alchemy
Snowflake
FTP
Singer Taps (any)
SqLite
Google Big Query
AWS S3
Kafka Producer
My Sql
Google Cloud Storage
Kafka Consumer
Duck DB
HTTP Basic Auth
Elastic Search
Oracle
