dabla opened a new issue, #63703:
URL: https://github.com/apache/airflow/issues/63703
### Apache Airflow version
3.1.8
### If "Other Airflow 3 version" selected, which one?
_No response_
### What happened?
The following command was executed:
`airflow db clean --clean-before-timestamp 2026-01-01`
The cleanup process successfully deletes rows from several tables (xcom,
task_instance, dag_run, etc.), but fails when processing dag_version.
```
You have requested that we purge all data prior to 2026-01-01 00:00:00+00:00
for tables ['_xcom_archive', 'asset_event', 'callback_request',
'celery_taskmeta', 'celery_tasksetmeta', 'dag', 'dag_run', 'dag_version',
'deadline', 'import_error', 'job', 'log', 'sla_miss', 'task_instance',
'task_instance_history', 'task_reschedule', 'trigger', 'xcom'].
This is irreversible. Consider backing up the tables first and / or doing a
dry run with option --dry-run.
Enter 'delete rows' (without quotes) to proceed.
delete rows
Checking table import_error
Found 0 rows meeting deletion criteria.
Checking table task_instance_history
Found 0 rows meeting deletion criteria.
Checking table xcom
Found 98 rows meeting deletion criteria.
Performing Delete...
Moving data to table _airflow_deleted__xcom__20260316091330
Finished Performing Delete
Checking table task_instance
Found 43572 rows meeting deletion criteria.
Performing Delete...
Moving data to table _airflow_deleted__task_instance__20260316091330
Finished Performing Delete
Checking table deadline
Found 0 rows meeting deletion criteria.
Checking table dag_run
Found 38877 rows meeting deletion criteria.
Performing Delete...
Moving data to table _airflow_deleted__dag_run__20260316091343
Finished Performing Delete
Checking table dag_version
Found 8 rows meeting deletion criteria.
Performing Delete...
Moving data to table _airflow_deleted__dag_version__20260316091351
Traceback (most recent call last):
File
"/usr/local/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line
1967, in _exec_single_context
self.dialect.do_execute(
~~~~~~~~~~~~~~~~~~~~~~~^
cursor, str_statement, effective_parameters, context
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
)
^
File
"/usr/local/lib64/python3.13/site-packages/sqlalchemy/engine/default.py", line
952, in do_execute
cursor.execute(statement, parameters)
~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
psycopg2.errors.ForeignKeyViolation: update or delete on table "dag_version"
violates foreign key constraint "task_instance_dag_version_id_fkey" on table
"task_instance"
DETAIL: Key (id)=(019a828c-a2ec-7700-bad6-725dd65b53a1) is still referenced
from table "task_instance".
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/local/sbin/airflow", line 10, in <module>
sys.exit(main())
~~~~^^
File "/usr/local/lib/python3.13/site-packages/airflow/__main__.py", line
55, in main
args.func(args)
~~~~~~~~~^^^^^^
File "/usr/local/lib/python3.13/site-packages/airflow/cli/cli_config.py",
line 49, in command
return func(*args, **kwargs)
File "/usr/local/lib/python3.13/site-packages/airflow/utils/cli.py", line
114, in wrapper
return f(*args, **kwargs)
File
"/usr/local/lib/python3.13/site-packages/airflow/utils/providers_configuration_loader.py",
line 54, in wrapped_function
return func(*args, **kwargs)
File
"/usr/local/lib/python3.13/site-packages/airflow/cli/commands/db_command.py",
line 296, in cleanup_tables
run_cleanup(
~~~~~~~~~~~^
table_names=args.tables,
^^^^^^^^^^^^^^^^^^^^^^^^
...<5 lines>...
batch_size=args.batch_size,
^^^^^^^^^^^^^^^^^^^^^^^^^^^
)
^
File "/usr/local/lib/python3.13/site-packages/airflow/utils/session.py",
line 100, in wrapper
return func(*args, session=session, **kwargs)
File
"/usr/local/lib/python3.13/site-packages/airflow/utils/db_cleanup.py", line
541, in run_cleanup
_cleanup_table(
~~~~~~~~~~~~~~^
clean_before_timestamp=clean_before_timestamp,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
...<5 lines>...
batch_size=batch_size,
^^^^^^^^^^^^^^^^^^^^^^
)
^
File
"/usr/local/lib/python3.13/site-packages/airflow/utils/db_cleanup.py", line
372, in _cleanup_table
_do_delete(
~~~~~~~~~~^
query=query,
^^^^^^^^^^^^
...<3 lines>...
batch_size=batch_size,
^^^^^^^^^^^^^^^^^^^^^^
)
^
File
"/usr/local/lib/python3.13/site-packages/airflow/utils/db_cleanup.py", line
262, in _do_delete
raise e
File
"/usr/local/lib/python3.13/site-packages/airflow/utils/db_cleanup.py", line
258, in _do_delete
session.execute(delete)
~~~~~~~~~~~~~~~^^^^^^^^
File
"/usr/local/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line
2351, in execute
return self._execute_internal(
~~~~~~~~~~~~~~~~~~~~~~^
statement,
^^^^^^^^^^
...<4 lines>...
_add_event=_add_event,
^^^^^^^^^^^^^^^^^^^^^^
)
^
File
"/usr/local/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line
2258, in _execute_internal
result = conn.execute(
statement, params or {}, execution_options=execution_options
)
File
"/usr/local/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line
1419, in execute
return meth(
self,
distilled_parameters,
execution_options or NO_OPTIONS,
)
File
"/usr/local/lib64/python3.13/site-packages/sqlalchemy/sql/elements.py", line
527, in _execute_on_connection
return connection._execute_clauseelement(
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
self, distilled_params, execution_options
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
)
^
File
"/usr/local/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line
1641, in _execute_clauseelement
ret = self._execute_context(
dialect,
...<8 lines>...
cache_hit=cache_hit,
)
File
"/usr/local/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line
1846, in _execute_context
return self._exec_single_context(
~~~~~~~~~~~~~~~~~~~~~~~~~^
dialect, context, statement, parameters
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
)
^
File
"/usr/local/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line
1986, in _exec_single_context
self._handle_dbapi_exception(
~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
e, str_statement, effective_parameters, cursor, context
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
)
^
File
"/usr/local/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line
2363, in _handle_dbapi_exception
raise sqlalchemy_exception.with_traceback(exc_info[2]) from e
File
"/usr/local/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line
1967, in _exec_single_context
self.dialect.do_execute(
~~~~~~~~~~~~~~~~~~~~~~~^
cursor, str_statement, effective_parameters, context
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
)
^
File
"/usr/local/lib64/python3.13/site-packages/sqlalchemy/engine/default.py", line
952, in do_execute
cursor.execute(statement, parameters)
~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
sqlalchemy.exc.IntegrityError: (psycopg2.errors.ForeignKeyViolation) update
or delete on table "dag_version" violates foreign key constraint
"task_instance_dag_version_id_fkey" on table "task_instance"
DETAIL: Key (id)=(019a828c-a2ec-7700-bad6-725dd65b53a1) is still referenced
from table "task_instance".
[SQL: DELETE FROM dag_version USING
_airflow_deleted__dag_version__20260316091351 WHERE dag_version.id =
_airflow_deleted__dag_version__20260316091351.id]
(Background on this error at: https://sqlalche.me/e/20/gkpj)
```
### What you think should happen instead?
airflow db clean should:
- delete rows in a safe dependency order, or
- skip dag_version rows that are still referenced, or
- ensure all referencing task_instance rows are removed first.
The command should not crash with a foreign key violation.
### How to reproduce
`airflow db clean --clean-before-timestamp 2026-01-01`
### Operating System
Fedora 5.3
### Versions of Apache Airflow Providers
Latests
### Deployment
Official Apache Airflow Helm Chart
### Deployment details
_No response_
### Anything else?
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
