Re: [DISCUSS] New Provider for Gremlin

2025-04-04 Thread Ahmad Farhan
Hey Stephen! I am still waiting for review from code maintainers and they are currently busy with Airflow 3. Also, recently there had been major code changes (after uv PR) that are raising a lot of errors so some work is needed on my side which is WIP. Thanks, Farhan On Fri, 4 Apr 2025 at 1:47 p

Re: [DISCUSS] New Provider for Gremlin

2025-04-04 Thread Stephen Mallette
Hi everyone, I just wanted to check in on the status of the TinkerPop Provider PR: https://github.com/apache/airflow/pull/47446 I don't know if there is any outstanding work left here but I was curious if there was any sense from committers as to when this body of work will be merged and made ava

Re: [DISCUSS] New Provider for Gremlin

2025-03-11 Thread Ahmad Farhan
Hello again! After several days of debugging and branch restructuring I managed to create another PR https://github.com/apache/airflow/pull/47446, this is due to major merge conflicts after the www clean up. The renaming of the provider to apache-tinkerpop is done but I kept the integration testing

Re: [DISCUSS] New Provider for Gremlin

2025-03-08 Thread Paul King
Nice to see the Apache TinkerPop/Gremlin support. As well as using Gremlin with TinkerPop's TinkerGraph, I have also used it with OrientDB, ArcadeDB, and Apache HugeGraph. If anyone is interested, I did a blog post here: https://groovy.apache.org/blog/groovy-graph-databases (Sorry but it's Gro

Re: [DISCUSS] New Provider for Gremlin

2025-03-08 Thread Ahmad Farhan
Hi! I managed to do the last cleanup on the PR, and is ready for thorough review. I will create a `[LAZY CONSENSUS]` thread on Monday/Tuesday. Feel free to review and comment on the PR in the meantime. Farhan On Thu, Mar 6, 2025 at 1:40 PM Stephen Mallette wrote: > Hi Ahmad, thanks for the upd

Re: [DISCUSS] New Provider for Gremlin

2025-03-06 Thread Stephen Mallette
Hi Ahmad, thanks for the updates! I've spent some time looking at the changes and have added some comments and questions. I've pointed out a couple things for future work that I think will be important, but nothing that needs to be changed for this PR in my mind. Looking forward to seeing your resp

Re: [DISCUSS] New Provider for Gremlin

2025-02-26 Thread Ahmad Farhan
I did look into the naming and I thought that it would need to be discussed at some point before the lazy consensus stage after the dev work is done but I guess I was wrong :) I read through some docs regarding the naming and I kept thinking that Apache Gremlin might not be right, so I decided to

Re: [DISCUSS] New Provider for Gremlin

2025-02-26 Thread Jarek Potiuk
Cool. I will let Ahmad comment, but I think we found the **someone** who will help in case there are some future issues with the Tinkerpop/Gremlin provider. While I like Gremlin better (it's just a cool name and I like the logo, tinkerpop has a cool logo as well https://tinkerpop.apache.org/index.h

Re: [DISCUSS] New Provider for Gremlin

2025-02-26 Thread Stephen Mallette
On Wed, Feb 26, 2025 at 12:57 PM Jarek Potiuk wrote: > > In the interest of ASF trademarks, I would suggest it be called > "apache/tinkerpop" with "Gremlin" naming reserved for operators and the > like, as it is now with GremlinOperator. I think this makes sense because > it is connecting to Tink

Re: [DISCUSS] New Provider for Gremlin

2025-02-26 Thread Jarek Potiuk
> In the interest of ASF trademarks, I would suggest it be called "apache/tinkerpop" with "Gremlin" naming reserved for operators and the like, as it is now with GremlinOperator. I think this makes sense because it is connecting to TinkerPop-enabled systems via Gremlin. I would similarly suggest th

Re: [DISCUSS] New Provider for Gremlin

2025-02-26 Thread Stephen Mallette
On 2025/02/26 12:38:02 Jarek Potiuk wrote: > Yeah . `apache/gremlin" seems like a better option then. Does anyone have > anything against it? In the interest of ASF trademarks, I would suggest it be called "apache/tinkerpop" with "Gremlin" naming reserved for operators and the like, as it is

RE: Re: [DISCUSS] New Provider for Gremlin

2025-02-26 Thread Stephen Mallette
fascinating - i was just visiting the list to discuss this very topic having just got started building the very same provider. :) Ahmad, thanks for your work jere. It's about inline with what I was planning. A bit about me, I'm one of the earliest contributors to TinkerPop, a current committer an

Re: [DISCUSS] New Provider for Gremlin

2025-02-26 Thread Ahmad Farhan
I was thinking integration testing might be something to consider for testing this provider, I will have a read through the link and implement that. Thanks Jarek Farhan On Wed, Feb 26, 2025 at 12:40 PM Jarek Potiuk wrote: > Yeah . `apache/gremlin" seems like a better option then. Does anyone ha

Re: [DISCUSS] New Provider for Gremlin

2025-02-26 Thread Jarek Potiuk
Yeah . `apache/gremlin" seems like a better option then. Does anyone have anything against it? I think we are pretty happy with accepting "other apache" projects as providers, so I see no issue with Gremlin - knowing that we can always reach out to our friendly Apache Community in case of any issue

Re: [DISCUSS] New Provider for Gremlin

2025-02-26 Thread Ahmad Farhan
I pushed changes to move the provider into the “apache” directory. After updating the class references across the project, I re-tested and all tests passed. Regarding the use of Gremlin (or another graph query language like Cypher and SPARQL) for a common package approach, here are my thoughts on

Re: [DISCUSS] New Provider for Gremlin

2025-02-24 Thread Ahmad Farhan
I have worked with two different graph database vendors—Azure Cosmos DB and Neo4j. During our migration to Neo4j, we discovered that using the Gremlin language wasn’t possible; we were forced to rewrite all our queries into Cypher, which is the native language for Neo4j and, in my experience, much

[DISCUSS] New Provider for Gremlin

2025-02-21 Thread Ahmad Farhan
Hi Everyone, I’ve created a draft PR (https://github.com/apache/airflow/pull/46977) to introduce and discuss a new provider for using Gremlin—the graph traversal language of Apache TinkerPop (more details here: https://tinkerpop.apache.org/gremlin.html). Gremlin is supported by various graph datab

Re: [DISCUSS] New Provider for Gremlin

2025-02-21 Thread Jarek Potiuk
Without deep looking at the code I love the idea - it's very similar to what we have for common.sql and common.io - and soon common.messaging - I also - long time ago - suggested common.dataframe that someone could submit using Apache Ibis: https://lists.apache.org/thread/qx3yh6h0l6jb0kh3fz9q95b3x5