shivaam commented on issue #62244:
URL: https://github.com/apache/airflow/issues/62244#issuecomment-3988737748
I think this is the bug
**Root cause**
MetastoreBackend.get_connection() (
metastore.py
) calls session.expunge_all() on the shared scoped_session. This gets
triggered during sync_bundles_to_db when _extract_and_sign_template
instantiates S3DagBundle → S3Hook → get_connection('aws_default'). Since both
methods share the same scoped session, expunge_all() destroys the pending
DagBundleModel objects that were just added via session.add().
**How I confirmed it**
I added extra debug logging to sync_bundles_to_db and
MetastoreBackend.get_connection to track session state. The logs show the exact
moment each bundle gets destroyed:
```
# team_alpha_dags added to session
After session.add('team_alpha_dags') — session.new=['team_alpha_dags']
# Next iteration: processing team_beta_dags triggers S3Hook → get_connection
→ expunge_all()
# expunge_all() destroys team_alpha_dags from the SAME session
BEFORE _extract_and_sign_template('team_beta_dags') —
session.new=['team_alpha_dags']
[METASTORE] expunge_all() session id=140126030839936 — Objects expunged:
['team_alpha_dags']
AFTER _extract_and_sign_template('team_beta_dags') — session.new=[]
# team_beta_dags added, then destroyed the same way when shared_dags is
processed
After session.add('team_beta_dags') — session.new=['team_beta_dags']
BEFORE _extract_and_sign_template('shared_dags') —
session.new=['team_beta_dags']
[METASTORE] expunge_all() session id=140126030839936 — Objects expunged:
['team_beta_dags']
AFTER _extract_and_sign_template('shared_dags') — session.new=[]
# Nothing left to commit
sync_bundles_to_db END — session.new=[], session.dirty=[]
```
I can attempt to look into the fix if you confirm the problem.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
