By default, Airflow tasks push and pull XComs via the metadata database (usually PostgreSQL or MySQL). A simple pattern is:
@task def exclusive_pop(): with r.lock("xcom:my_key", timeout=10): value = r.get("xcom:my_key") r.delete("xcom:my_key") return value airflow xcom exclusive
XComs are a mechanism for between tasks. By default, Airflow tasks push and pull XComs
with DAG('exclusive_xcom_demo', start_date=datetime(2023,1,1), schedule=None) as dag: t1 = PythonOperator(task_id='extract', python_callable=extract) t2 = PythonOperator(task_id='transform', python_callable=transform) t3 = PythonOperator(task_id='load', python_callable=load) schedule=None) as dag: t1 = PythonOperator(task_id='extract'