+1 (non-binding) Ran the RC with breeze, ran some example DAGs. Tested my change in #32052, works as expected We also ran example DAGs for astronomer-providers including deferrable operators and they work fine.
Regards, Pankaj Koti *Senior Software Engineer, *OSS Engineering Team. Location: Pune, India Timezone: Indian Standard Time (IST) Email: pankaj.k...@astronomer.io Mobile: +91 9730079985 On Fri, Aug 11, 2023 at 12:40 PM Ephraim Anierobi < ephraimanier...@apache.org> wrote: > Hey fellow Airflowers, > > I have cut Airflow 2.7.0rc1. This email is calling a vote on the release, > which will last at least 72 hours, from Friday, August 11, 2023, at 7:10 am > UTC > until Monday, August 14, 2023, at 7:10 am UTC > < > https://www.timeanddate.com/worldclock/fixedtime.html?msg=8&iso=20230814T0710&p1=1440 > >, > and until 3 binding +1 votes have been received. > > Consider this my (binding) +1. > > Airflow 2.7.0rc1 is available at: > https://dist.apache.org/repos/dist/dev/airflow/2.7.0rc1/ > > *apache-airflow-2.7.0-source.tar.gz* is a source release that comes with > INSTALL instructions. > *apache-airflow-2.7.0.tar.gz* is the binary Python "sdist" release. > *apache_airflow-2.7.0-py3-none-any.whl* is the binary Python wheel "binary" > release. > > Public keys are available at: > https://dist.apache.org/repos/dist/release/airflow/KEYS > > Please vote accordingly: > > [ ] +1 approve > [ ] +0 no opinion > [ ] -1 disapprove with the reason > > Only votes from PMC members are binding, but all members of the community > are encouraged to test the release and vote with "(non-binding)". > > The test procedure for PMC members is described in: > > https://github.com/apache/airflow/blob/main/dev/README_RELEASE_AIRFLOW.md\#verify-the-release-candidate-by-pmc-members > > The test procedure for and Contributors who would like to test this RC is > described in: > > https://github.com/apache/airflow/blob/main/dev/README_RELEASE_AIRFLOW.md\#verify-the-release-candidate-by-contributors > > > Please note that the version number excludes the `rcX` string, so it's now > simply 2.7.0. This will allow us to rename the artifact without modifying > the artifact checksums when we actually release. > > Release Notes: > https://github.com/apache/airflow/blob/2.7.0rc1/RELEASE_NOTES.rst > > *Changes since 2.6.3:* > > *Significant Changes* > > *Remove Python 3.7 support (#30963)* > As of now, Python 3.7 is no longer supported by the Python community. > Therefore, to use Airflow 2.7.0, you must ensure your Python version is > either 3.8, 3.9, 3.10, or 3.11. > > *Old Graph View is removed (#32958)* > The old Graph View is removed. The new Graph View is the default view now. > > > *The "db init", "db upgrade" commands and "[database] > load_default_connections" configuration options are deprecated (#33136).* > Instead, you should use "airflow db migrate" command to create or upgrade > database. This command will not create default connections. > In order to create default connections you need to run "airflow connections > create-default-connections" explicitly, > after running "airflow db migrate". > > *In case of SMTP SSL connection, the context now uses the "default" context > (#33070)* > The "default" context is Python's ``default_ssl_contest`` instead of > previously used "none". The > ``default_ssl_context`` provides a balance between security and > compatibility but in some cases, > when certificates are old, self-signed or misconfigured, it might not work. > This can be configured > by setting "ssl_context" in "email" configuration of Airflow. > > *Setting it to "none" brings back the "none" setting that was used in > Airflow 2.6 and before,* > but it is not recommended due to security reasons ad this setting disables > validation of certificates and allows MITM attacks. > > *Disable default allowing the testing of connections in UI, API and > CLI(#32052)* > For security reasons, the test connection functionality is disabled by > default across Airflow UI, > API and CLI. The availability of the functionality can be controlled by the > ``test_connection`` flag in the ``core`` section of the Airflow > configuration (``airflow.cfg``). It can also be controlled by the > environment variable ``AIRFLOW__CORE__TEST_CONNECTION``. > > The following values are accepted for this config param: > 1. ``Disabled``: Disables the test connection functionality and > disables the Test Connection button in the UI. > > This is also the default value set in the Airflow configuration. > 2. ``Enabled``: Enables the test connection functionality and > activates the Test Connection button in the UI. > > 3. ``Hidden``: Disables the test connection functionality and > hides the Test Connection button in UI. > > For more information on capabilities of users, see the documentation: > > https://airflow.apache.org/docs/apache-airflow/stable/security/security_model.html#capabilities-of-authenticated-ui-users > It is strongly advised to **not** enable the feature until you make sure > that only > highly trusted UI/API users have "edit connection" permissions. > > *The ``xcomEntries`` API disables support for the ``deserialize`` flag by > default (#32176)* > For security reasons, the > ``/dags/*/dagRuns/*/taskInstances/*/xcomEntries/*`` > API endpoint now disables the ``deserialize`` option to deserialize > arbitrary > XCom values in the webserver. For backward compatibility, server admins may > set > the ``[api] enable_xcom_deserialize_support`` config to *True* to enable > the > flag and restore backward compatibility. > > However, it is strongly advised to **not** enable the feature, and perform > deserialization at the client side instead. > > *Change of the default Celery application name (#32526)* > Default name of the Celery application changed from > ``airflow.executors.celery_executor`` to > ``airflow.providers.celery.executors.celery_executor``. > > You should change both your configuration and Health check command to use > the new name: > * in configuration (``celery_app_name`` configuration in ``celery`` > section) use ``airflow.providers.celery.executors.celery_executor`` > * in your Health check command use `` > airflow.providers.celery.executors.celery_executor.app`` > > > *The default value for ``scheduler.max_tis_per_query`` is changed from 512 > to 16 (#32572)* > This change is expected to make the Scheduler more responsive. > > ``scheduler.max_tis_per_query`` needs to be lower than > ``core.parallelism``. > If both were left to their default value previously, the effective default > value of ``scheduler.max_tis_per_query`` was 32 > (because it was capped at ``core.parallelism``). > > To keep the behavior as close as possible to the old config, one can set > ``scheduler.max_tis_per_query = 0``, > in which case it'll always use the value of ``core.parallelism``. > > *Some executors have been moved to corresponding providers (#32767)* > In order to use the executors, you need to install the providers: > > * for Celery executors you need to install > ``apache-airflow-providers-celery`` package >= 3.3.0 > * for Kubernetes executors you need to install > ``apache-airflow-providers-cncf-kubernetes`` package >= 7.4.0 > * For Dask executors you need to install > ``apache-airflow-providers-daskexecutor`` package in any version > > You can achieve it also by installing airflow with ``[celery]``, > ``[cncf.kubernetes]``, ``[daskexecutor]`` extras respectively. > > Users who base their images on the ``apache/airflow`` reference image (not > slim) should be unaffected - the base > reference image comes with all the three providers installed. > > *Improvement Changes* > > *PostgreSQL only improvement: Added index on taskinstance table (#30762)* > This index seems to have great positive effect in a setup with tens of > millions such rows. > > **New Features** > - Add OpenTelemetry to Airflow (`AIP-49 < > > https://github.com/apache/airflow/pulls?q=is%3Apr+is%3Amerged+label%3AAIP-49+milestone%3A%22Airflow+2.7.0%22 > >`_) > - Trigger Button - Implement Part 2 of AIP-50 (#31583) > - Removing Executor Coupling from Core Airflow (`AIP-51 < > > https://github.com/apache/airflow/pulls?q=is%3Apr+is%3Amerged+label%3AAIP-51+milestone%3A%22Airflow+2.7.0%22 > >`_) > - Automatic setup and teardown tasks (`AIP-52 < > > https://github.com/apache/airflow/pulls?q=is%3Apr+is%3Amerged+label%3AAIP-52+milestone%3A%22Airflow+2.7.0%22 > >`_) > - OpenLineage in Airflow (`AIP-53 < > > https://github.com/apache/airflow/pulls?q=is%3Apr+is%3Amerged+label%3AAIP-53+milestone%3A%22Airflow+2.7.0%22 > >`_) > - Experimental: Add a cache to Variable and Connection when called at dag > parsing time (#30259) > - Enable pools to consider deferred tasks (#32709) > - Allows to choose SSL context for SMTP connection (#33070) > - New gantt tab (#31806) > - Load plugins from providers (#32692) > - Add ``ExternalBranchPythonOperator`` (#32787) > - Add option for storing configuration description in providers (#32629) > - Add an option to use a direct DB connection in KEDA when ``pgbouncer`` is > enabled (#32608) > - Introduce Heartbeat Parameter to Allow ``Per-LocalTaskJob`` Configuration > (#32313) > - Add Executors discovery and documentation (#32532) > - Add JobState for job state constants (#32549) > - Add config to disable the 'deserialize' XCom API flag (#32176) > - Show task instance in web UI by custom operator name (#31852) > - Add default_deferrable config (#31712) > - Introducing ``AirflowClusterPolicySkipDag`` exception (#32013) > - Use ``reactflow`` for datasets graph (#31775) > - Add an option to load the dags from db for command tasks run (#32038) > - Add version of ``chain`` which doesn't require matched lists (#31927) > - Use operator_name instead of task_type in UI (#31662) > - Add ``--retry`` and ``--retry-delay`` to ``airflow db check`` (#31836) > - Allow skipped task state task_instance_schema.py (#31421) > - Add a new config for celery result_backend engine options (#30426) > - UI Add Cluster Activity Page (#31123, #32446) > - Adding keyboard shortcuts to common actions (#30950) > - Adding more information to kubernetes executor logs (#29929) > - Add support for configuring custom alembic file (#31415) > - Add running and failed status tab for DAGs on the UI (#30429) > - Add multi-select, proposals and labels for trigger form (#31441) > - Making webserver config customizable (#29926) > - Render DAGCode in the Grid View as a tab (#31113) > - Add rest endpoint to get option of configuration (#31056) > - Add ``section`` query param in get config rest API (#30936) > - Create metrics to track ``Scheduled->Queued->Running`` task state > transition times (#30612) > - Mark Task Groups as Success/Failure (#30478) > - Add CLI command to list the provider trigger info (#30822) > - Add Fail Fast feature for DAGs (#29406) > > **Improvements** > - Configurable health check threshold for triggerer (#33089, #33084) > - add dag_run_ids and task_ids filter for the batch task instance API > endpoint (#32705) > - Ensure DAG-level references are filled on unmap (#33083) > - Add support for arrays of different data types in the Trigger Form UI > (#32734) > - Always show gantt and code tabs (#33029) > - Move listener success hook to after SQLAlchemy commit (#32988) > - Rename ``db upgrade`` to ``db migrate`` and add ``connections > create-default-connections`` (#32810, #33136) > - Remove old gantt chart and redirect to grid views gantt tab (#32908) > - Adjust graph zoom based on selected task (#32792) > - Call listener on_task_instance_running after rendering templates (#32716) > - Display execution_date in graph view task instance tooltip. (#32527) > - Allow configuration to be contributed by providers (#32604, #32755, > #32812) > - Reduce default for max TIs per query, enforce ``<=`` parallelism (#32572) > - Store config description in Airflow configuration object (#32669) > - Use ``isdisjoint`` instead of ``not intersection`` (#32616) > - Speed up calculation of leaves and roots for task groups (#32592) > - Kubernetes Executor Load Time Optimizations (#30727) > - Save DAG parsing time if dag is not schedulable (#30911) > - Updates health check endpoint to include ``dag_processor`` status. > (#32382) > - Disable default allowing the testing of connections in UI, API and CLI > (#32052) > - Fix config var types under the scheduler section (#32132) > - Allow to sort Grid View alphabetically (#32179) > - Add hostname to triggerer metric ``[triggers.running]`` (#32050) > - Improve DAG ORM cleanup code (#30614) > - ``TriggerDagRunOperator``: Add ``wait_for_completion`` to > ``template_fields`` (#31122) > - Open links in new tab that take us away from Airflow UI (#32088) > - Only show code tab when a task is not selected (#31744) > - Add descriptions for celery and dask cert configs (#31822) > - ``PythonVirtualenvOperator`` termination log in alert (#31747) > - Migration of all DAG details to existing grid view dag details panel > (#31690) > - Add a diagram to help visualize timer metrics (#30650) > - Celery Executor load time optimizations (#31001) > - Update code style for ``airflow db`` commands to SQLAlchemy 2.0 style > (#31486) > - Mark uses of md5 as "not-used-for-security" in FIPS environments (#31171) > - Add pydantic support to serde (#31565) > - Enable search in note column in DagRun and TaskInstance (#31455) > - Save scheduler execution time by adding new Index idea for dag_run > (#30827) > - Save scheduler execution time by caching dags (#30704) > - Support for sorting DAGs by Last Run Date in the web UI (#31234) > - Better typing for Job and JobRunners (#31240) > - Add sorting logic by created_date for fetching triggers (#31151) > - Remove DAGs.can_create on access control doc, adjust test fixture > (#30862) > - Split Celery logs into stdout/stderr (#30485) > - Decouple metrics clients and ``validators`` into their own modules > (#30802) > - Description added for pagination in ``get_log`` api (#30729) > - Optimize performance of scheduling mapped tasks (#30372) > - Add sentry transport configuration option (#30419) > - Better message on deserialization error (#30588) > > **Bug Fixes** > - ``Gantt chart:`` Use earliest/oldest ti dates if different than dag run > start/end (#33215) > - Fix ``virtualenv`` detection for Python ``virtualenv`` operator (#33223) > - Correctly log when there are problems trying to ``chmod`` ``airflow.cfg`` > (#33118) > - Pass app context to webserver_config.py (#32759) > - Skip served logs for non-running task try (#32561) > - Fix reload gunicorn workers (#32102) > - Fix future DagRun rarely triggered by race conditions when > ``max_active_runs`` reached its upper limit. (#31414) > - Fix BaseOperator ``get_task_instances`` query (#33054) > - Fix issue with using the various state enum value in logs (#33065) > - Use string concatenation to prepend base URL for log_url (#33063) > - Update graph nodes with operator style attributes (#32822) > - Affix webserver access_denied warning to be configurable (#33022) > - Only load task action modal if user can edit (#32992) > - OpenAPI Spec fix nullable alongside ``$ref`` (#32887) > - Make the decorators of ``PythonOperator`` sub-classes extend its > decorator (#32845) > - Fix check if ``virtualenv`` is installed in ``PythonVirtualenvOperator`` > (#32939) > - Unwrap Proxy before checking ``__iter__`` in is_container() (#32850) > - Override base log folder by using task handler's base_log_folder (#32781) > - Catch arbitrary exception from run_job to prevent zombie scheduler > (#32707) > - Fix depends_on_past work for dynamic tasks (#32397) > - Sort extra_links for predictable order in UI. (#32762) > - Fix prefix group false graph (#32764) > - Fix bad delete logic for dagruns (#32684) > - Fix bug in prune_dict where empty dict and list would be removed even in > strict mode (#32573) > - Add explicit browsers list and correct rel for blank target links > (#32633) > - Handle returned None when multiple_outputs is True (#32625) > - Fix returned value when ShortCircuitOperator condition is falsy and there > is not downstream tasks (#32623) > - Fix returned value when ShortCircuitOperator condition is falsy (#32569) > - Fix rendering of ``dagRunTimeout`` (#32565) > - Fix permissions on ``/blocked`` endpoint (#32571) > - Bugfix, prevent force of unpause on trigger DAG (#32456) > - Fix data interval in ``cli.dags.trigger`` command output (#32548) > - Strip ``whitespaces`` from airflow connections form (#32292) > - Add timedelta support for applicable arguments of sensors (#32515) > - Fix incorrect default on ``readonly`` property in our API (#32510) > - Add xcom map_index as a filter to xcom endpoint (#32453) > - Fix CLI commands when custom timetable is used (#32118) > - Use WebEncoder to encode DagRun.conf in DagRun's list view (#32385) > - Fix logic of the skip_all_except method (#31153) > - Ensure dynamic tasks inside dynamic task group only marks the (#32354) > - Handle the cases that webserver.expose_config is set to > non-sensitive-only instead of boolean value (#32261) > - Add retry functionality for handling process termination caused by > database network issues (#31998) > - Adapt Notifier for sla_miss_callback (#31887) > - Fix XCOM view (#31807) > - Fix for "Filter dags by tag" flickering on initial load of dags.html > (#31578) > - Fix where expanding ``resizer`` wouldn't expanse grid view (#31581) > - Fix MappedOperator-BaseOperator attr sync check (#31520) > - Always pass named ``type_`` arg to drop_constraint (#31306) > - Fix bad ``drop_constraint`` call in migrations (#31302) > - Resolving problems with redesigned grid view (#31232) > - Support ``requirepass`` redis sentinel (#30352) > - Fix webserver crash when calling get ``/config`` (#31057) > > **Misc/Internal** > - Refactor: Simplify code in ``dag_processing`` (#33161) > - For now limit ``Pydantic`` to ``< 2.0.0`` (#33235) > - Refactor: Simplify code in models (#33181) > - Add elasticsearch group to pre-2.7 defaults (#33166) > - Refactor: Simplify dict manipulation in airflow/cli (#33159) > - Remove redundant dict.keys() call (#33158) > - Upgrade ruff to latest 0.0.282 version in pre-commits (#33152) > - Move openlineage configuration to provider (#33124) > - Replace State by TaskInstanceState in Airflow executors (#32627) > - Get rid of Python 2 numeric relics (#33050) > - Remove legacy dag code (#33058) > - Remove legacy task instance modal (#33060) > - Remove old graph view (#32958) > - Move CeleryExecutor to the celery provider (#32526, #32628) > - Move all k8S classes to ``cncf.kubernetes`` provider (#32767, #32891) > - Refactor existence-checking SQL to helper (#32790) > - Extract Dask executor to new daskexecutor provider (#32772) > - Remove atlas configuration definition (#32776) > - Add Redis task handler (#31855) > - Move writing configuration for webserver to main (webserver limited) > (#32766) > - Improve getting the query count in Airflow API endpoints (#32630) > - Remove click upper bound (#32634) > - Add D400 ``pydocstyle`` check - core Airflow only (#31297) > - D205 Support (#31742, #32575, #32213, #32212, #32591, #32449, #32450) > - Bump word-wrap from ``1.2.3 to 1.2.4`` in ``/airflow/www`` (#32680) > - Strong-type all single-state enum values (#32537) > - More strong typed state conversion (#32521) > - SQL query improvements in utils/db.py (#32518) > - Bump semver from ``6.3.0 to 6.3.1`` in ``/airflow/www`` (#32506) > - Bump jsonschema version to ``4.18.0`` (#32445) > - Bump ``stylelint`` from ``13.13.1 to 15.10.1`` in ``/airflow/www`` > (#32435) > - Bump tough-cookie from ``4.0.0 to 4.1.3`` in ``/airflow/www`` (#32443) > - upgrade flask-appbuilder (#32054) > - Support ``Pydantic`` 2 (#32366) > - Limit click until we fix mypy issues (#32413) > - A couple of minor cleanups (#31890) > - Replace State usages with strong-typed ``enums`` (#31735) > - Upgrade ruff to ``0.272`` (#31966) > - Better error message when serializing callable without name (#31778) > - Improve the views module a bit (#31661) > - Remove ``asynctest`` (#31664) > - Refactor sqlalchemy queries to ``2.0`` style (#31569, #31772, #32350, > #32339, #32474, #32645) > - Remove Python ``3.7`` support (#30963) > - Bring back min-airflow-version for preinstalled providers (#31469) > - Docstring improvements (#31375) > - Improve typing in SchedulerJobRunner (#31285) > - Upgrade ruff to ``0.0.262`` (#30809) > - Upgrade to MyPy ``1.2.0`` (#30687) > > **Docs only changes** > - Add links to ``DAGRun / DAG / Task`` in templates-ref.rst (#33013) > - Add docs of how to test for DAG Import Errors (#32811) > - Clean-up of our new security page (#32951) > - Cleans up Extras reference page (#32954) > - Update Dag trigger API and command docs (#32696) > - Add deprecation info to the Airflow modules and classes docstring > (#32635) > - Formatting installation doc to improve readability (#32502) > - Fix triggerer HA doc (#32454) > - Add type annotation to code examples (#32422) > - Document cron and delta timetables (#32392) > - Update index.rst doc to correct grammar (#32315) > - Fixing small typo in python.py (#31474) > - Separate out and clarify policies for providers (#30657) > - Fix docs: add an "apache" prefix to pip install (#30681) > > Cheers, > Ephraim >
