Norway


Hi guys,

New to airflow. I have tried for hours to figure this out and think I have come to the correct process up until Celery Executor. Please see below

When running Airflow in cluster mode, we have workers on various machines that connect back to the scheduler. Airflow uses RabbitMQ to get the messages in a Queue from which the workers can retrieve the message. I just don’t completely understand why we need celery? Does it act as a load balancer?

Any help would be really appreciated.

Thanks



Source link

No tags for this post.

LEAVE A REPLY

Please enter your comment!
Please enter your name here