+1 with removal. As long as users are able to continue with the last released version of the operator, I do not see any issues with the removal.
Thanks Andrey for the metrics too. Makes the case stronger. Thanks & Regards, Amogh Desai On Fri, Nov 24, 2023 at 2:58 PM Jarek Potiuk <ja...@potiuk.com> wrote: > Also - with all these discussions we have to remember that we are NOT > forbidding anyone to use Scoop or any other provider that is out there. > They can continue using it for as long they can install released latest > version of the provider. Which means basically forever (or as long as > dependencies of it will not conflict with airflow) . And the worst thing is > - they can also stay in the old version of airflow in case of conflicts > > Removal really removes it from: > > - maintenance in the future (except security fixes maybe) - for example we > will not bump min airflow version and release new versions of the provider > with it > - our code in main > - impacting dependency resolution > - running tests > > If there are no changes to the software itself - we could not be able to do > anything anyway if there are dependency conflicts.. > > > It's not - we are suddenly telling out users to stop using Scoop if they > are using it - they still can and likely for a long time, we just tell them > theu are frozen in time basically. > > > pt., 24 lis 2023, 09:40 użytkownik Andrey Anshin <andrey.ans...@taragol.is > > > napisał: > > > I've tried to look at the download statistics, however it doesn't look > any > > specific. For about 900 downloads per day, some of the providers have the > > same statistics, especially if this provider is very specific and > > unpopular. > > > > PyPi Stats: > > https://pypistats.org/packages/apache-airflow-providers-apache-sqoop > > PePy: > > > > > https://www.pepy.tech/projects/apache-airflow-providers-apache-sqoop?versions=4.*&versions=3.*&versions=2.*&versions=1.* > > Simple analysis from BigQuery PyPi public dataset [1] for the last 90 > days > > (it shows for about 10-20% more daily downloads rather than any public > web > > statistic): https://1drv.ms/x/s!AsekcaVz-qGvxge96Ij2JDNKs4DJ?e=FBkK6O > > > > > > Also seems like spike of downloads [2][3] Airflow, when daily downloads > > raised from 300k to 1M and after slightly go down to 600k, has no effect > to > > particular this one provider package > > > > [1] > > > > > https://packaging.python.org/en/latest/guides/analyzing-pypi-package-downloads/#public-dataset > > [2] https://pypistats.org/packages/apache-airflow > > [3] https://pypistats.org/packages/apache-airflow-providers-amazon > > > > > > > > On Fri, 24 Nov 2023 at 04:42, Ferruzzi, Dennis > <ferru...@amazon.com.invalid > > > > > wrote: > > > > > I'd like to see usage metrics if there are any (number of package > > > downloads or something?) before deciding to drop it, but I'm pretty > > > comfortable that this one is safe to remove given the project's > history. > > > > > > +1 > > > > > > > > > - ferruzzi > > > > > > > > > ________________________________ > > > From: Oliveira, Niko <oniko...@amazon.com.INVALID> > > > Sent: Thursday, November 23, 2023 11:40 AM > > > To: dev@airflow.apache.org > > > Subject: RE: [EXTERNAL] [COURRIEL EXTERNE] [DISCUSS] Suspend/Remove > > Apache > > > Scoop provider > > > > > > CAUTION: This email originated from outside of the organization. Do not > > > click links or open attachments unless you can confirm the sender and > > know > > > the content is safe. > > > > > > > > > > > > AVERTISSEMENT: Ce courrier électronique provient d’un expéditeur > externe. > > > Ne cliquez sur aucun lien et n’ouvrez aucune pièce jointe si vous ne > > pouvez > > > pas confirmer l’identité de l’expéditeur et si vous n’êtes pas certain > > que > > > le contenu ne présente aucun risque. > > > > > > > > > > > > +1 > > > > > > ________________________________ > > > From: Pierre Jeambrun <pierrejb...@gmail.com> > > > Sent: Thursday, November 23, 2023 10:28:52 AM > > > To: dev@airflow.apache.org > > > Subject: RE: [EXTERNAL] [COURRIEL EXTERNE] [DISCUSS] Suspend/Remove > > Apache > > > Scoop provider > > > > > > CAUTION: This email originated from outside of the organization. Do not > > > click links or open attachments unless you can confirm the sender and > > know > > > the content is safe. > > > > > > > > > > > > AVERTISSEMENT: Ce courrier électronique provient d’un expéditeur > externe. > > > Ne cliquez sur aucun lien et n’ouvrez aucune pièce jointe si vous ne > > pouvez > > > pas confirmer l’identité de l’expéditeur et si vous n’êtes pas certain > > que > > > le contenu ne présente aucun risque. > > > > > > > > > > > > +1 > > > > > > Le jeu. 23 nov. 2023 à 17:52, Hussein Awala <huss...@awala.fr> a > écrit : > > > > > > > +1 > > > > > > > > On Thu, Nov 23, 2023 at 11:52 AM Andrey Anshin < > > andrey.ans...@taragol.is > > > > > > > > wrote: > > > > > > > > > Greetings everyone! > > > > > > > > > > Since we began to actively use the mechanism to suspend/remove > > > providers > > > > I > > > > > want to start the discussion about suspend and potential remove > > Apache > > > > > Scoop [1] provider. > > > > > > > > > > Apache Scoop moved into the Attic in July 2021 [2] due to inactive > > > > > development [3] and Apache Scoop PMS was terminated. > > > > > According to the Attic page of Scoop there is no forks of this > > project > > > > > > > > > > Latest releases : > > > > > Scoop1 (stable): 1.4.7 was released on 2018-01-24 > > > > > Sqoop2: 1.99.7 was released on 2016-08-08 > > > > > > > > > > Perhaps the fact that the product is no longer being developed > might > > be > > > > not > > > > > a reason to exclude it from the Airflow providers. > > > > > In that case it would be nice to have a discussion which would use > in > > > > > further cases of suspension/removal providers > > > > > > > > > > [1] https://sqoop.apache.org/ > > > > > [2] https://attic.apache.org/projects/sqoop.html > > > > > [3] https://whimsy.apache.org/board/minutes/Sqoop.html > > > > > > > > > > > > > > >
