potiuk commented on issue #20442: URL: https://github.com/apache/airflow/issues/20442#issuecomment-998869946
I see. I think this is because the DAG is serialized in the DB with the "NoValueSentinel". Is that possible that you modify the DAG manually (for example add some empty "task" to trigger reserialisation of the DAG? We are going to have a CLI in the future that will trigger reserialisation for such (and similar) cases: https://github.com/apache/airflow/pull/19471 but it will only come in 2.3 (`airflow reserialize`). In the meantime if you have more DAGs like that (and if my guess is confirmed). You could trigger re-serialization via: ``` from airflow.models.serialized_dag import SerializedDagModel from airflow.settings import Session session = Session() session.query(SerializedDagModel).delete() session.commit() ``` CC: @kaxil @collinmcnulty @uranusjr - If I am not mistaken, it seems like this is a new scenario for "accidental" serialization-change impact, showing that re-serialization is needed. I wonder if we could - in the future - have some ways to protect in case we have similar changes (automated reserialization could solve it but I think there is too strong of performance reason against it as we discussed). Seems hard problem to avoid those kind of incompatibilities might easily slip through. What do you think if instead of reserializing everything we simply Delete the serialized DAG that caused serialization issue? That could have at least self-healing potential, but we could treat Serialization Error in a special way and even display a different error to the message ("We have a aproblem this time but check back shortly") WDYT? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
