Another aspect to keep in mind is that some tests require internal options to be changed before executing the compute functions (e.g., check overflow, allow NaN comparisons, change validity bits, etc.). Also, there are tests that take randomized inputs and others make use of the min/max values for each specific data type. Certainly, these details can be generalized across languages/testing frameworks but not without careful treatment.
Moreover, each language implementation still needs to test language-specific or internal functions, so having a meta test framework will not necessarily get rid of language-specific tests. ~Eduardo On Fri, May 14, 2021 at 1:56 PM Weston Pace <weston.p...@gmail.com> wrote: > With that in mind it seems the somewhat recurring discussion on coming > up with a language independent standard for logical query plans > ( > https://lists.apache.org/thread.html/rfab15e09c97a8fb961d6c5db8b2093824c58d11a51981a40f40cc2c0%40%3Cdev.arrow.apache.org%3E > ) > would be relevant. Each test case would then be a triplet of (Input > Dataframe, Logical Plan, Output Dataframe). So perhaps tackling this > effort would be to make progress on both fronts. > > On Fri, May 14, 2021 at 7:39 AM Julian Hyde <jhyde.apa...@gmail.com> > wrote: > > > > Do these any of these compute functions have analogs in other > implementations of Arrow (e.g. Rust)? > > > > I believe that as much as possible of Arrow’s compute functionality > should be cross-language. Perhaps there are language-specific differences > in how functions are invoked, but the basic functionality is the same. > > > > If people buy into that premise, then a single suite of tests is a > powerful way to make that happen. The tests can be written in a high-level > language and can generate tests in each implementation language. (For these > purposes, the “high-level language” could be a special text format, could > be a data language such as JSON, or could be a programming language such as > Python; it doesn’t matter much.) > > > > For example, > > > > assertThatCall(“foo(1, 2)”, returns(“3”)) > > > > might actually call foo with arguments 1 and 2, or it might generate a > C++ or Rust test that does the same. > > > > > > Julian > > > > > > > On May 14, 2021, at 8:45 AM, Antoine Pitrou <anto...@python.org> > wrote: > > > > > > > > > Le 14/05/2021 à 15:30, Wes McKinney a écrit : > > >> hi folks, > > >> As we build more functions (kernels) in the project, I note that the > > >> amount of hand-coded C++ code relating to testing function correctness > > >> is growing significantly. Many of these tests are quite simple and > > >> could be expressed in a text format that can be parsed and evaluated. > > >> Thoughts about building something like that to make it easier to write > > >> functional correctness tests? > > > > > > Or perhaps build-up higher level C++ functions if desired? > > > > > > Or even write some of those tests as part of the PyArrow test suite. > > > > > > I'm not sure adding a custom (and probably inflexible) text format is > really a good use of our time. > > > > > > Regards > > > > > > Antoine. > > >
