wasn't involved in this AIP at all, but just wanted to say Congratulations for getting through this!
Howard On Mon, Mar 28, 2022 at 3:49 AM Mateusz Nojek <matno...@gmail.com> wrote: > Hello there! > > Since the main PR (#22311 <https://github.com/apache/airflow/pull/22311>) > with the implementation of AIP-47 > <https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-47+New+design+of+Airflow+System+Tests> > has been merged, I am now closing the lazy consensus voting. The discussion > was conducted in the comments section to the PR. All suggestions were > addressed or implemented. The PR is in the Top #60 of most commented issues > in the whole history of Airflow on GitHub! :) > > The AIP is now also completed, and the next phase is to migrate old system > tests to the new design in compliance with the AIP-47. > It was a busy and exciting journey - it took exactly a month for the AIP > to be accepted and another straight month to be implemented! > Thank you all for help, comments, ideas and support! > > Kind regards, > Mateusz Nojek > > > pon., 21 mar 2022 o 15:09 Mateusz Nojek <matno...@gmail.com> napisał(a): > >> Hello, >> >> After the discussion in the comments section of PR #22311 >> <https://github.com/apache/airflow/pull/22311>, there were several >> updates and changes in the code. The overall hasn't changed significantly, >> but few enhancements were added. Along with the changes in the code, I >> updated the AIP-47 >> <https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-47+New+design+of+Airflow+System+Tests> >> document >> and wanted to list here the modified sections: >> >> - Updated point 7. with watcher import moved closer to its usage in >> "Process of migration in details" >> - Updated point 9. with updated pytest function in "Process of >> migration in details" >> - Updated "Complete test example" with latest changes to the design >> - Updated "Design details"/"Test structure" with new content of >> conftest.py and utils file for system tests >> >> I would also like to thank you for your great suggestions and encourage >> others to familiarize with this design. It is a huge improvement that will >> hopefully increase the quality of testing and drop the number of new bugs >> in the next releases. We are really getting this whole initiative closer to >> perfection. >> >> Kind regards, >> Mateusz Nojek >> >> >> śr., 16 mar 2022 o 15:21 Mateusz Nojek <matno...@gmail.com> napisał(a): >> >>> Hi all, >>> >>> I have a few announcements regarding AIP-47 >>> <https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-47+New+design+of+Airflow+System+Tests> >>> . >>> >>> First of all, after the AIP has been approved on 24th February, we were >>> working hard on preparing the first PR that will include a first batch with >>> migrated system tests and all related changes compliant with the AIP-47. We >>> have prepared code changes with related documentation, new pre-commit >>> hooks, deleted old tests, updated paths and so on... We tried to keep the >>> PR as small as possible (it only covers Google BigQuery operators) but also >>> to have a complete solution so that you can see how it will all work in >>> practice. >>> The PR is available here: https://github.com/apache/airflow/pull/22311 >>> Please take a look and share your comments in the issue. >>> >>> Secondly, we did some changes to the AIP-47 document and I wanted to >>> call for a lazy consensus since these changes were strongly useful for the >>> design - some simplifications and improvements were done. The voting will >>> be running until the first PR is merged to the Airflow main branch. Until >>> then, we can discuss, do other improvements and updates. >>> >>> Some more important changes in the AIP: >>> - DebugExecutor is now contained in conftest.py inside tests/system, >>> not a part of each system test, >>> - if __name__ == "__main__" condition is no longer enclosed in each >>> system test (we decided that running these tests through pytest is enough >>> and we don't need to support the execution via python command), >>> - updated "Process of migration in details" and "Complete test example" >>> with more detailed information on the steps, to make it more clear how to >>> do the migration, >>> - updated "Which users are affected by the change" and added there a >>> number of example DAGs that may need to be migrated to the new design. >>> Also, updated the numbers as of 14th March 2022, >>> - updated "Test structure" with information about modified pytest.ini, >>> which now also recognize file starting with example_* as test files. >>> >>> The diff reflecting changes in the AIP from the moment when it was >>> accepted till today can be found here ( >>> https://cwiki.apache.org/confluence/pages/diffpagesbyversion.action?pageId=199532973&selectedPageVersions=7&selectedPageVersions=11) >>> for your review. >>> >>> The linked PR will hang there as long as there are no concerns and all >>> required approvals are provided. Reading the PR would be probably the best >>> way to familiarize with the changes and the new design. You are also more >>> than welcome to share the news about this AIP and hopefully the rest of the >>> community can join us with migrating system tests to this new design. >>> >>> We have more Google-related operators already covered by new system >>> tests, but we will hold with creating PRs with those changes till the >>> moment where this first PR is merged. >>> >>> Last, but not least, Google has deposited 10,000 $ in credits into the >>> GCP project dedicated for the exclusive use of running CI in Airflow and it >>> is now maintained by the community. It is under preparation to make the new >>> system tests run automatically. More info and the progress of configuring >>> it for the community needs will be announced later. >>> >>> When the mentioned PR <https://github.com/apache/airflow/pull/22311> >>> will be merged, all providers are welcome to start migrating their system >>> tests and decide whether they want to prepare their example DAGs to be also >>> run in the new design (even if they were not run before). >>> >>> Kind regards, >>> Mateusz Nojek >>> >>
