Yep. I am also for the idea described by Pierre. In short - setting the requirement automatically turn on a DEBUG mode with all possible DEBUG features turned on.
On Thu, Aug 31, 2023 at 6:03 AM Amogh Desai <amoghdesai....@gmail.com> wrote: > Going through the discussions above, I was leaning towards the idea of > removing it completely initially but came across Pierre's idea after. I > like that! > > Thanks & Regards, > Amogh Desai > > On Wed, Aug 30, 2023 at 1:58 PM Jarek Potiuk <ja...@potiuk.com> wrote: > > > Whoa.. I am glad I started it... I see some really good ideas here :). > > > > On Wed, Aug 30, 2023 at 9:20 AM Pierre Jeambrun <pierrejb...@gmail.com> > > wrote: > > > > > Maybe we can introduce a global “DEBUG” config option/env variable. > This > > > could control some more verbose logging but most importantly only when > > this > > > is turned on, we could use the sequential executor, debug executor, > > > _PIP_ADDITIONAL_REQUIREMENTS > > > and any other “debug/development” purpose options. > > > > > > Like for other frameworks this would be documented and user would have > a > > > warning when using it. > > > > > > Then there is nothing more we can do if the user puts in production a > > > cluster in debug mode, this would be deliberate. (Like set the debug > > option > > > to True and then use the sequential executor). > > > > > > This could also automatically start the web server in debug mode and > > other > > > components in a similar way when detected to true. > > > > > > We could even limit the number of workers to 1 or other things that > would > > > make it unsuitable in a production environment. > > > > > > Just an idea. > > > > > > On Wed 30 Aug 2023 at 03:55, Pankaj Koti <pankaj.k...@astronomer.io > > > .invalid> > > > wrote: > > > > > > > I agree with Maciej's rationale here and inclined towards removing it > > > > straight away. > > > > > > > > Also, is it the case that users have discovered this flag themselves > > > > without us documenting it anywhere as a feature to use? > > > > > > > > *We have a leading underscore for the variable and it hints that it's > > for > > > > internal use.* > > > > > > > > I would be up for failing the image without any deprecation/warning > if > > we > > > > have not documented it as a feature to use. > > > > > > > > On Wed, 30 Aug 2023, 03:16 Oliveira, Niko, > <oniko...@amazon.com.invalid > > > > > > > wrote: > > > > > > > > > I'd vote for a period of time with warnings (either in the logs > > and/or > > > in > > > > > the Airflow UI), as a deprecation warning of sorts. Followed by > > > removing > > > > > the feature later on, unless we find that the warnings are enough > to > > > > lower > > > > > the operational load this causes us, but I think that's unlikely. > > > > > > > > > > Cheers, > > > > > Niko > > > > > > > > > > ________________________________ > > > > > From: Jed Cunningham <jedcunning...@apache.org> > > > > > Sent: Tuesday, August 29, 2023 10:05:01 AM > > > > > To: dev@airflow.apache.org > > > > > Subject: RE: [EXTERNAL] [DISCUSS] Preventing users from misusing > > > > > _PIP_ADDITIONAL_REQUIREMENTS ? > > > > > > > > > > CAUTION: This email originated from outside of the organization. Do > > not > > > > > click links or open attachments unless you can confirm the sender > and > > > > know > > > > > the content is safe. > > > > > > > > > > > > > > > > > > > > I also don't like the 10 minute thing. I'd rather we remove it, or > > > > display > > > > > a message like we do sequential executor (we can only do so much, > > this > > > is > > > > > as visible as we can make it really), I think in that order? > > > > > > > > > > > > > > >
