Also - with all these discussions we have to remember that we are NOT forbidding anyone to use Scoop or any other provider that is out there. They can continue using it for as long they can install released latest version of the provider. Which means basically forever (or as long as dependencies of it will not conflict with airflow) . And the worst thing is - they can also stay in the old version of airflow in case of conflicts
Removal really removes it from: - maintenance in the future (except security fixes maybe) - for example we will not bump min airflow version and release new versions of the provider with it - our code in main - impacting dependency resolution - running tests If there are no changes to the software itself - we could not be able to do anything anyway if there are dependency conflicts.. It's not - we are suddenly telling out users to stop using Scoop if they are using it - they still can and likely for a long time, we just tell them theu are frozen in time basically. pt., 24 lis 2023, 09:40 użytkownik Andrey Anshin <andrey.ans...@taragol.is> napisał: > I've tried to look at the download statistics, however it doesn't look any > specific. For about 900 downloads per day, some of the providers have the > same statistics, especially if this provider is very specific and > unpopular. > > PyPi Stats: > https://pypistats.org/packages/apache-airflow-providers-apache-sqoop > PePy: > > https://www.pepy.tech/projects/apache-airflow-providers-apache-sqoop?versions=4.*&versions=3.*&versions=2.*&versions=1.* > Simple analysis from BigQuery PyPi public dataset [1] for the last 90 days > (it shows for about 10-20% more daily downloads rather than any public web > statistic): https://1drv.ms/x/s!AsekcaVz-qGvxge96Ij2JDNKs4DJ?e=FBkK6O > > > Also seems like spike of downloads [2][3] Airflow, when daily downloads > raised from 300k to 1M and after slightly go down to 600k, has no effect to > particular this one provider package > > [1] > > https://packaging.python.org/en/latest/guides/analyzing-pypi-package-downloads/#public-dataset > [2] https://pypistats.org/packages/apache-airflow > [3] https://pypistats.org/packages/apache-airflow-providers-amazon > > > > On Fri, 24 Nov 2023 at 04:42, Ferruzzi, Dennis <ferru...@amazon.com.invalid > > > wrote: > > > I'd like to see usage metrics if there are any (number of package > > downloads or something?) before deciding to drop it, but I'm pretty > > comfortable that this one is safe to remove given the project's history. > > > > +1 > > > > > > - ferruzzi > > > > > > ________________________________ > > From: Oliveira, Niko <oniko...@amazon.com.INVALID> > > Sent: Thursday, November 23, 2023 11:40 AM > > To: dev@airflow.apache.org > > Subject: RE: [EXTERNAL] [COURRIEL EXTERNE] [DISCUSS] Suspend/Remove > Apache > > Scoop provider > > > > CAUTION: This email originated from outside of the organization. Do not > > click links or open attachments unless you can confirm the sender and > know > > the content is safe. > > > > > > > > AVERTISSEMENT: Ce courrier électronique provient d’un expéditeur externe. > > Ne cliquez sur aucun lien et n’ouvrez aucune pièce jointe si vous ne > pouvez > > pas confirmer l’identité de l’expéditeur et si vous n’êtes pas certain > que > > le contenu ne présente aucun risque. > > > > > > > > +1 > > > > ________________________________ > > From: Pierre Jeambrun <pierrejb...@gmail.com> > > Sent: Thursday, November 23, 2023 10:28:52 AM > > To: dev@airflow.apache.org > > Subject: RE: [EXTERNAL] [COURRIEL EXTERNE] [DISCUSS] Suspend/Remove > Apache > > Scoop provider > > > > CAUTION: This email originated from outside of the organization. Do not > > click links or open attachments unless you can confirm the sender and > know > > the content is safe. > > > > > > > > AVERTISSEMENT: Ce courrier électronique provient d’un expéditeur externe. > > Ne cliquez sur aucun lien et n’ouvrez aucune pièce jointe si vous ne > pouvez > > pas confirmer l’identité de l’expéditeur et si vous n’êtes pas certain > que > > le contenu ne présente aucun risque. > > > > > > > > +1 > > > > Le jeu. 23 nov. 2023 à 17:52, Hussein Awala <huss...@awala.fr> a écrit : > > > > > +1 > > > > > > On Thu, Nov 23, 2023 at 11:52 AM Andrey Anshin < > andrey.ans...@taragol.is > > > > > > wrote: > > > > > > > Greetings everyone! > > > > > > > > Since we began to actively use the mechanism to suspend/remove > > providers > > > I > > > > want to start the discussion about suspend and potential remove > Apache > > > > Scoop [1] provider. > > > > > > > > Apache Scoop moved into the Attic in July 2021 [2] due to inactive > > > > development [3] and Apache Scoop PMS was terminated. > > > > According to the Attic page of Scoop there is no forks of this > project > > > > > > > > Latest releases : > > > > Scoop1 (stable): 1.4.7 was released on 2018-01-24 > > > > Sqoop2: 1.99.7 was released on 2016-08-08 > > > > > > > > Perhaps the fact that the product is no longer being developed might > be > > > not > > > > a reason to exclude it from the Airflow providers. > > > > In that case it would be nice to have a discussion which would use in > > > > further cases of suspension/removal providers > > > > > > > > [1] https://sqoop.apache.org/ > > > > [2] https://attic.apache.org/projects/sqoop.html > > > > [3] https://whimsy.apache.org/board/minutes/Sqoop.html > > > > > > > > > >
