I believe this is what substrait.io is trying to accomplish.. Here's some additional info: https://substrait.io/
https://www.youtube.com/watch?v=5JjaB7p3Sjk -----Original Message----- From: Sasha Krassovsky <krassovskysa...@gmail.com> Sent: Wednesday, October 5, 2022 11:29 AM To: dev@arrow.apache.org Subject: Parser for expressions External Email: Use caution with links and attachments Hi everyone, I’ve noticed on the mailing list a few times people asking for a more convenient way to construct an Expression, namely using a string of some sort. I’ve found myself wishing for something like this too when constructing ExecPlans, and so I’ve gone ahead and implemented a parser [0]. I was wondering if anyone had any thoughts about the design of the language? The current implementation parses a lisp-like language. This language has three types of expressions (mirroring the current Expression API): - A call is a normal s-expression, it has the name of the kernel and the list of arguments. Its arguments can be any expression. - A literal (i.e. scalar) starts with a $ and specifies a type and a value, separated by a colon. For example, `$decimal(12,2):10.01` specifies a literal of type decimal(12, 2) and a value of 10.01. - A field_ref starts with a ! and is an identifier in the schema following the DotPath syntax we already have [1]. So for example, the expression (add $int32:1 (multiply !.a !.b)) computes a*b+1 given a batch with columns named a and b. The reason I chose a lisp-like language is that it very directly translates to the current Expression API and that it feels more natural to use a prefix notation for a language where all functions have a name (i.e. no +, -, *, etc.). I’m currently working on a followup PR for specifying ExecPlans from a string (mainly for easier testing), and would like that language to be an extension of this one. Looking forward to hearing everyone’s thoughts! Thanks, Sasha Krassovsky [0] https://urldefense.com/v3/__https://github.com/apache/arrow/pull/14287__;!!KSjYCgUGsB4!enYRTooMrwyJKJzgTlQMdMhpfT7ys3Ol8a8HcHUvxRYRN-a-Up_axLfPGOpUtEDCDs0ee7lHPAzVdz-dooULG_6oZdDk$ <https://urldefense.com/v3/__https://github.com/apache/arrow/pull/14287__;!!KSjYCgUGsB4!enYRTooMrwyJKJzgTlQMdMhpfT7ys3Ol8a8HcHUvxRYRN-a-Up_axLfPGOpUtEDCDs0ee7lHPAzVdz-dooULG_6oZdDk$ > [1] https://urldefense.com/v3/__https://github.com/apache/arrow/blob/master/cpp/src/arrow/type.h*L1726__;Iw!!KSjYCgUGsB4!enYRTooMrwyJKJzgTlQMdMhpfT7ys3Ol8a8HcHUvxRYRN-a-Up_axLfPGOpUtEDCDs0ee7lHPAzVdz-dooULG0GkL0Mn$ <https://urldefense.com/v3/__https://github.com/apache/arrow/blob/master/cpp/src/arrow/type.h*L1726__;Iw!!KSjYCgUGsB4!enYRTooMrwyJKJzgTlQMdMhpfT7ys3Ol8a8HcHUvxRYRN-a-Up_axLfPGOpUtEDCDs0ee7lHPAzVdz-dooULG0GkL0Mn$ > This message may contain information that is confidential or privileged. If you are not the intended recipient, please advise the sender immediately and delete this message. See http://www.blackrock.com/corporate/compliance/email-disclaimers for further information. Please refer to http://www.blackrock.com/corporate/compliance/privacy-policy for more information about BlackRock’s Privacy Policy. For a list of BlackRock's office addresses worldwide, see http://www.blackrock.com/corporate/about-us/contacts-locations. © 2022 BlackRock, Inc. All rights reserved.
