We have asked our security team to re-assess this and provide any guidance in case they feel that there are exploits. If they do share those, I will engage a conversation on the private email list.
On 6/28/21, 3:02 AM, "Jarek Potiuk" <[email protected]> wrote: CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe. Hey everyone, Coming back to the discussion - Josh, Subash, do you have anything more to share regarding the security (again - might be on private@). We've been recently chosen by the ASF to write the blog post about the security practices https://blogs.apache.org/foundation/entry/success-at-apache-security-in and we have the "Airflow security" discussion panel at the Airlfow Summit - https://airflowsummit.org/sessions/2021/panel-airflow-security so I think it would be great to know before that if there is anything we should do, or at least plan about it. We also started to work more closely with Dolev, a security researcher and expert who will take in the panel and who raised a few security issues for Airflow and he is looking more closely at all-things-airflow, so maybe a good time to take a closer look at the plugins/providers/webserver case. J. On Sat, Jun 19, 2021 at 1:47 PM Kaxil Naik <[email protected]> wrote: > > Said differently, Plugins should be reviewed by the team similar to how the admin team would review DAGs. > > But definitely happy to hear about all specific security concerns in as much detail as possible (on private@ though to avoid making those details public risking some less-secure envs). > > Regards, > Kaxil > > On Sat, Jun 19, 2021, 11:56 Kaxil Naik <[email protected]> wrote: >> >> Regarding manipulating/compromising auth via plugins, auth backend is set in airflow.cfg /env vars and is used when the Webserver is started, you can't manipulate it. >> >> If there is any security vector you have discovered I would suggest you to email [email protected] with replication details. >> >> And I agree with what has been already discussed about Dagfile Code and Plugins. What specific risk are we talking about? >> >> Can you go one level deeper in the details please and use private@ if you have reproduction steps. >> >> Regards, >> Kaxil >> >> >> >> On Sat, Jun 19, 2021, 09:21 Ash Berlin-Taylor <[email protected]> wrote: >>> >>> > I would turn your argument the other way around--if you're already in a no-install, serialized model for DAGs why not extend that to all aspects of the webserver such as connections and UI plugins? Seems that would be more consistent. >>> >>> There are more to providers than just defining connection types. >>> >>> For example you can define custom operator links in providers http://airflow.apache.org/docs/apache-airflow/stable/howto/define_extra_link.html and to avoid potential deserialisation attacks the links must be registered with the webserver code before it will create an instance of the class >>> >>> (And links are classes rather than static so that you can do things like generate temporary/presigned S3 URLs) >>> >>> -Ash >>> >>> On 18 June 2021 22:58:29 BST, "Jackson, John" <[email protected]> wrote: >>>> >>>> Plugins, providers, and their associated Python libraries all need to execute code in order to be installed which is a vulnerability. Plugins in particular are often developed/installed by the data engineers and not by system administrators, leading us back to our original problem. >>>> >>>> I would turn your argument the other way around--if you're already in a no-install, serialized model for DAGs why not extend that to all aspects of the webserver such as connections and UI plugins? Seems that would be more consistent. >>>> >>>> On 2021-06-18, 1:36 PM, "Jarek Potiuk" <[email protected]> wrote: >>>> >>>> CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe. >>>> >>>> >>>> >>>>> That would certainly help a bit, but unfortunately it's not just the packages. It's the fact that authentication is tied to Python code that can be patched by anyone with permission to execute code on the web server, which in turn would give them access to packages or any anything else they'd like. >>>> >>>> >>>> But in Airflow 2.0 the code provided by "DAG writers" is not executed >>>> any more. This is entirely gone together with Airflow 1.10. This has >>>> been handled by DAG serialization, which is the only option available >>>> in 2.0. I do not see how the "Users" could add any code if "Admins" >>>> control the packages that are installed in the webserver. Now if >>>> Admin/User is the only problem then I think this is really >>>> misunderstanding coming from the pre-DAG-serialization world of Apache >>>> Airflow. >>>> >>>> J. >>>> -- +48 660 796 129
