OK, if I can find some time, I'll have a try. In the long run, it would be better (for me) to have the functionality by default in Solr than to maintain my own piece of code 😀
On Fri, Jan 7, 2022 at 1:42 AM Dennis Gove <dpg...@gmail.com> wrote: > Hi Damiano, > > Yup, that's what I meant. I'd be happy to collaborate with you on this. > > Cheers! > > On Thu, Jan 6, 2022 at 5:09 PM Damiano Albani <damiano.alb...@gmail.com> > wrote: > > > Hi Dennis, > > > > Do you (implicitly) mean by your message that it would be a good idea to > > get the changes you mentioned into the official Solr code base? > > In other words, that a PR implementing this enhancement would be > considered > > by the Solr team? > > > > Regards, > > > > On Wed, Jan 5, 2022 at 1:58 AM Dennis Gove <dpg...@gmail.com> wrote: > > > > > My recollection from working on this code years ago is that other > > > definitions of "equal" can be supported by creating new implementations > > of > > > the Equalitor class ( > > > > > > > > > https://github.com/apache/solr/blob/main/solr/solrj/src/java/org/apache/solr/client/solrj/io/eq/Equalitor.java#L27-L30 > > > ). > > > The purpose of the Equalitor class is not so much to say "these two > > values > > > are the same" but instead "these values can be joined on". Joins were > one > > > of the first streaming expressions created and as such existed before > > > evaluators. The Equalitor class is a bit of an unfortunate holdover > from > > > that initial implementation. Were I doing it again now I'd use > evaluators > > > instead. > > > > > > That said, it may be possible to refactor the Equalitor class as a type > > of > > > Evaluator. An approach like that would, I think, clean up what's > become a > > > confusing holdover of that original implementation and simultaneously > > make > > > it possible to use any evaluator within a join clause. > > > > > > Alternatively, it'd be possible to enhance the join classes to support > > > either Equalitors or Evaluators. Equalitors are constructed with this > > > method - > > > > > > > > > https://github.com/apache/solr/blob/main/solr/solrj/src/java/org/apache/solr/client/solrj/io/stream/expr/StreamFactory.java#L352 > > > - so you could enhance any place that's called from to also support > > > Evaluators. > > > > > > Cheers, > > > Dennis > > > > > > On Tue, Jan 4, 2022 at 5:00 PM Damiano Albani < > damiano.alb...@gmail.com> > > > wrote: > > > > > > > Hello, > > > > > > > > It's the first time that I hear about those Lucene expressions > written > > in > > > > JavaScript. Good to learn about it! > > > > I suppose you're referring to > > > > > > > > > > > > > > https://lucene.apache.org/core/9_0_0/expressions/org/apache/lucene/expressions/js/package-summary.html > > > > ? > > > > I couldn't find much information about how to use it, especially in > > > > combination with Solr. If someone knowledgeable could chime in, that > > > would > > > > be great. > > > > Though what I see on the API documentation page at first impression, > is > > > > that the list of supported functions is pretty limited. > > > > Actually, I think that Solr's decorators provide a similar coverage > of > > > > functions out of the box: > > > > https://solr.apache.org/guide/8_11/stream-evaluator-reference.html. > > > > If I can find some time, I will play with my java() decorator idea > and > > > see > > > > if it is any good. > > > > Especially in terms of performance, where JavaScript-in-Lucene could > > have > > > > the upper hand indeed. > > > > > > > > Regards, > > > > > > > > On Tue, Jan 4, 2022 at 6:41 PM David Smiley <dsmi...@apache.org> > > wrote: > > > > > > > > > I'd prefer to use Lucene's "expressions" module and thus do > > JavaScript. > > > > > This is more accessible to a wider audience, and I believe makes > > > > > safety/security easier (though I have not checked). > > > > > > > > > > ~ David Smiley > > > > > Apache Lucene/Solr Search Developer > > > > > http://www.linkedin.com/in/davidwsmiley > > > > > > > > > > > > > > > On Tue, Jan 4, 2022 at 12:30 PM Eric Pugh < > > > > ep...@opensourceconnections.com > > > > > > > > > > > wrote: > > > > > > > > > > > That looks great! I love how (relatively) simple it all is to > > write > > > > your > > > > > > own logic. > > > > > > > > > > > > One of the reasons that we added packages (bin/solr package) to > > Solr > > > is > > > > > so > > > > > > that if someone wants to add something like a java() evaluator, > > they > > > > can! > > > > > > > > > > > > > On Jan 4, 2022, at 11:40 AM, Damiano Albani < > > > > damiano.alb...@gmail.com> > > > > > > wrote: > > > > > > > > > > > > > > Hi, > > > > > > > > > > > > > > Just a quick note to mention that I've managed to implement > what > > I > > > > > wanted > > > > > > > in terms of non equi-joins. > > > > > > > Should someone be interested, I've put my code on > > > > > > > https://github.com/dalbani/solr-streaming-expressions. > > > > > > > > > > > > > > By the way, I happened to need a startsWith function and I > > > > implemented > > > > > it > > > > > > > quite easily. > > > > > > > But I'm wondering if a very generic -- if not possibly not very > > > safe > > > > -- > > > > > > > java() evaluator could be built. > > > > > > > That would open streaming expressions to the whole Java API > > instead > > > > of > > > > > > > having to write individual evaluators. > > > > > > > For the example of startsWith, it could look like something in > > the > > > > > range > > > > > > of: > > > > > > > > > > > > > >> java(val(Hello), val(World), "arg0.startsWith(arg1)") > > > > > > > > > > > > > > Using say, https://www.javassist.org/, to turn the code > argument > > > > into > > > > > > > bytecode. > > > > > > > What do you think? > > > > > > > > > > > > > > Regards, > > > > > > > > > > > > > > On Wed, Dec 29, 2021 at 12:39 PM Damiano Albani < > > > > > > damiano.alb...@gmail.com> > > > > > > > wrote: > > > > > > > > > > > > > >> Hello, > > > > > > >> > > > > > > >> I'm new to streaming expressions, so I'm trying to understand > > > their > > > > > > >> features and limitations. > > > > > > >> In particular the so-called "stream operators" implementing > join > > > > > > >> operations. > > > > > > >> Like "innerJoin", "leftOuterJoin", etc. > > > > > > >> > > > > > > >> I see that they support a "on" parameter, defining the > > *equality* > > > > > check > > > > > > >> to be performed. > > > > > > >> But, coming from the SQL world, I'm used to being able to use > a > > > > > variety > > > > > > of > > > > > > >> comparison operators in join predicates. That is, not only > > > equality, > > > > > as > > > > > > in > > > > > > >> "equi-joins". > > > > > > >> > > > > > > >> Is there a reason why the current implementation of Solr > > supports > > > > > > >> equi-joins only? Would it be technically possible (and > desired) > > to > > > > > > support > > > > > > >> other comparison operators with joins? > > > > > > >> And maybe somehow allow the use of the available stream > > evaluators > > > > > > >> < > > > https://solr.apache.org/guide/8_11/stream-evaluator-reference.html > > > > >? > > > > > > >> > > > > > > >> To give the context of my question: I'm trying to join 2 sets > of > > > > > > documents > > > > > > >> with a hierarchical relationship. > > > > > > >> My goal is to join them using a "path" field on one side and > > > > > > >> "descendent_path" field on the other side. > > > > > > >> But it looks like that only doc values are accessible (and not > > > > > analyzed > > > > > > >> ones) in streams, so I suppose I'd be left with a join > criteria > > > like > > > > > > this > > > > > > >> pseudo-code: > > > > > > >> > > > > > > >>> on="starts_with(right.path, left.path)" > > > > > > >> > > > > > > >> Where, in this hypothetical example: > > > > > > >> > > > > > > >>> left.path=/categories/category1" > > > > > > >>> > > right.path=/categories/category1/sub-categories/sub-category-a" > > > > > > >> > > > > > > >> > > > > > > >> Or do I completely misunderstand how Solr (streams) work? ;-) > > > > > > >> Thanks for your help! > > > > > > >> > > > > > > >> Regards, > > > > > > >> > > > > > > >> -- > > > > > > >> Damiano Albani > > > > > > >> > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > Damiano Albani > > > > > > > > > > > > _______________________ > > > > > > Eric Pugh | Founder & CEO | OpenSource Connections, LLC | > > > 434.466.1467 > > > > | > > > > > > http://www.opensourceconnections.com < > > > > > > http://www.opensourceconnections.com/> | My Free/Busy < > > > > > > http://tinyurl.com/eric-cal> > > > > > > Co-Author: Apache Solr Enterprise Search Server, 3rd Ed < > > > > > > > > > > > > > > > > > > > > > https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw > > > > > > > > > > > > > > > > > > This e-mail and all contents, including attachments, is > considered > > to > > > > be > > > > > > Company Confidential unless explicitly stated otherwise, > regardless > > > of > > > > > > whether attachments are marked as such. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > Damiano Albani > > > > > > > > > > > > > -- > > Damiano Albani > > > -- Damiano Albani
