Sorry for the delayed reply here. I've been chewing on this one a bit though.
One concern I have is that I highly value having a provider agnostic remote git integration. fsspec, however, has local git or github - no arbitrary remote git support. That means Airflow, in my view, can't just rely on fsspec alone, but will have to also deal with cloning arbitrary repos. Or we expect a "ready to go" cloned repo on local disk, which might be a good tradeoff. This was something we'd have to tackle for AIP-63 anyways, though. Another concern, as I've been considering Airflow 3, is that I see wanting to version "more" than just the DAG folder - possibly a whole venv (or similar). Doing that versioning file-by-file doesn't feel particularly practical to me. Which means that, I think, we could end up using fsspec to get zips/tarballs/"assets", which is still very helpful imo. But it does limit the "slickness" of being sorta behind the scenes like a custom module loader and resource loader would be. I actually see this as also being coupled to Ash's upcoming “Task Execution interface”. I admittedly didn't quite follow the VersionedFS idea you all were discussing. I don't see how it could easily allow Airflows goal of "this version of the dag folder and maybe more" to a backend that is versioned by file. Maybe you can eli5 and connect the dots for me? All that said, my goal this next week is to spend some quality time in this area so we can all start nailing down options for Airflow 3.
