Setting up Airflow on AWS
Airflow is one of my favorite tools that I frequently use to setup and manage data science pipelines. The Airflow UI gives us a clear picture of the DAGS and its current status. I may be wrong here but from my experience, I have seen that Airflow on a single machine is not scalable. Thus, to scale Airflow, we can use Kubernetes.
Comments
There aren't any comments yet. Be the first to comment!