Dan Davydov created AIRFLOW-3542:
------------------------------------
Summary: next_ds semantics broken for manually triggered runs
Key: AIRFLOW-3542
URL: https://issues.apache.org/jira/browse/AIRFLOW-3542
Project: Apache Airflow
Issue Type: Bug
Components: scheduler
Affects Versions: 1.10.2
Reporter: Dan Davydov
Assignee: Dan Davydov
{color:#222222}next_ds{color}{color:#222222} is useful when you need cron-style
scheduling, e.g. a task that runs for date "X" uses that date for its logic,
e.g. send an email to users saying the run that was supposed to run for date
"X" has completed. The problem is it doesn't behave as expected when it comes
to manually triggered runs as illustrated by the diagrams below.{color}
Using execution_date in a task
*Scheduled Run (works as expected)*
execution_date1 start_date1
\/ \/
*|-----------------------------|*
/\ /\
\_________________/
scheduling_interval
*Manual Run* *(works as expected)*
triggered_date + execution_date + start_date
\/
*|*
Using next_ds in a Task
*Scheduled Run (works as expected)*
next_ds1 + start_date1 next_ds2 + start_date2
\/ \/
*|------------------------------------------------|*
/\ /\
\____________________________/
scheduling_interval
*Manual Run* *(next_ds1 is expected to match triggered_date as in the case for
the manually triggered run that uses the regular execution_date above)*
triggered_date next_ds1 + start_date
\/ \/
*|-------------------------------------------------|*
/\ /\
\____________________________/
0 to scheduling_interval (depending on when the next execution
date is)
Proposal
Have next_ds always set to execution_date for manually triggered runs instead
of the next schedule-interval aligned execution date.
This _might_ break backwards compatibility for some users but it can be argued
that the current functionality is a bug. If it's really desired we can create
new aliases that behave logically although I am against this.
prev_ds should probably also be made consistent with this logic.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
