Re: Gandiva and Threads

2020-08-14 Thread Jacques Nadeau
@ravin...@dremio.com @prav...@dremio.com thoughts? On Tue, Jul 28, 2020 at 3:39 PM Wes McKinney wrote: > Perhaps Gandiva does not handle sliced arrays properly? This would be > worth investigating > > On Mon, Jul 27, 2020 at 7:43 PM Matt Youill > wrote: > > > > Managed to track down the issue

Re: Gandiva and Threads

2020-07-28 Thread Wes McKinney
Perhaps Gandiva does not handle sliced arrays properly? This would be worth investigating On Mon, Jul 27, 2020 at 7:43 PM Matt Youill wrote: > > Managed to track down the issue (sort of). > > I removed a call to set_chunksize on TableBatchReader where the chunk > size was less than the number of

Re: Gandiva and Threads

2020-07-27 Thread Matt Youill
Managed to track down the issue (sort of). I removed a call to set_chunksize on TableBatchReader where the chunk size was less than the number of rows in a table being read. Runs fine now (tested with 100s threads over mils of rows). Strangely, Gandiva fails if I don't call set_chunksize for

Re: Gandiva and Threads

2020-07-27 Thread Wes McKinney
Crashing when running from multiple threads doesn't sound right, perhaps there are some missing synchronizations in internal data structures. Could you open a JIRA issue and show the backtraces of any crashes or other clues about how to reproduce the issues? On Sun, Jul 26, 2020 at 8:12 PM Matt Yo

Gandiva and Threads

2020-07-26 Thread Matt Youill
Hi, Are there any guidelines for using the Gandiva C++ lib from multiple threads? As I increase the number of threads in an application Gandiva begins to raise memory faults (e.g. double free). Sometimes it is on projector creation, sometimes evaluation, seems to happen in multiple unpredicta