본문 바로가기
CS/DataEngineering

Running Airflow 2.0 with Docker in 5 mins

by Diligejy 2022. 5. 26.

https://www.youtube.com/watch?v=aTaytcxy2Ck&ab_channel=DatawithMarc 

 

1. curl -LfO "https://airflow.apache.org/docs/apache-airflow/stable/docker-compose.yaml

2. mkdir ./dags ./plugins ./logs

3. echo -e "AIRFLOW_UID=$(id -u)\nAIRFLOW_GID=0" > .env

4. docker-compose up airflow-init

5. docker-compose up

6. localhost:8080 들어가보기

 

 

Bonus1 (Interact with airflow docker)

- docker exec containerID + airflow command

 

Bonus2 

How to Fix it?

 

-> add environment variable in docker-compose.yaml

AIRFLOW__API__AUTH_BACKEND : 'airflow.api.auth.backend.basic_auth' 

 

After adding the parameter,  Enter "docker-compose down && docker compose up"

 

- curl -X GET --user "airflow:airflow" "http://localhost:8080/api/v1/dags"

 

'CS > DataEngineering' 카테고리의 다른 글

Build An Airflow Data Pipeline To Download Podcasts [Beginner Data Engineer Tutorial]  (0) 2022.06.10
Kubernetes  (0) 2022.06.02
docker  (0) 2022.06.01
Airflow using Docker 삽질기록  (0) 2022.05.24
Airflow 처음 실행해봄  (0) 2022.05.23

댓글