Re: Arrow sync call January 18 at 17:00 UTC

2023-01-20 Thread Ian Cook
Beginning this week, we are using Google Docs to capture the notes from the biweekly Arrow sync call. The notes for this and future instances of this call will be captured in this Google Doc: https://docs.google.com/document/d/1xrji8fc6_24TVmKiHJB4ECX1Zy2sy2eRbBjpVJMnPmk/ Going forward, I intend t

Re: Arrow sync call January 4 at 12:00 US/Eastern, 17:00 UTC

2023-01-06 Thread Benson Muite
On 1/7/23 05:54, Ian Cook wrote: >> If a Google Doc is used, can it be configured to send out notifications of > the summary to the list? > > Not as far as I know, but I think we can continue to send a copy of the > notes to the mailing list after each biweekly meeting, copied and pasted > from th

Re: Arrow sync call January 4 at 12:00 US/Eastern, 17:00 UTC

2023-01-06 Thread Ian Cook
> If a Google Doc is used, can it be configured to send out notifications of the summary to the list? Not as far as I know, but I think we can continue to send a copy of the notes to the mailing list after each biweekly meeting, copied and pasted from the Google Doc. On Fri, Jan 6, 2023 at 21:40

Re: Arrow sync call January 4 at 12:00 US/Eastern, 17:00 UTC

2023-01-06 Thread Benson Muite
> Proposal to move sync call meeting notes into a Google Doc > > - Will proposed that we share notes from sync calls in a publicly > viewable Google Doc instead of in emails to the mailing list [2] > - There was a discussion about whether managing edit access to this > Google Doc would be diffic

Re: Arrow sync call January 4 at 12:00 US/Eastern, 17:00 UTC

2023-01-05 Thread Ian Cook
Attendees: Ian Cook Dewey Dunnington Ian Joiner Will Jones David Li Bryce Mecum Rok Mihevc Eduardo Ponce Matthew Topol Jacob Wujciak Discussion: ADBC 0.1.0 release vote - The vote is open [1] - David is looking for more review and votes from PMC members and others Jira to GitHub migration -

Re: Arrow sync call December 21 at 12:00 US/Eastern, 17:00 UTC

2022-12-22 Thread Raúl Cumplido
Hi, Here are the notes from the Arrow sync call from yesterday 21st December: Attendees: - Raúl Cumplido - Bryce Mecum - Rok Mihevc - Matthew Topol - Joris Van den Bossche - Jacob Wujciak Discussion: Migration from Jira to GitHub issues: - Adding new issues on JIRA is closed. Creation of new is

Re: Arrow sync call December 7 at 12:00 US/Eastern, 17:00 UTC

2022-12-08 Thread Ian Cook
Attendees - Ian Cook - Raúl Cumplido - Dewey Dunnington - Ian Joiner - Will Jones - David Li - Rok Mihevc - Antoine Pitrou - Matt Topol - Joris Van den Bossche Discussion Maintenance policy There is a discussion [1] on the mailing list about whether we should define a maintenance policy. - The

Re: Arrow sync call November 23 at 12:00 US/Eastern, 17:00 UTC

2022-11-28 Thread Jacob Wujciak
> The merge script would still be useful to flag issues such as a missing component label, or to ensure the fix version (milestone) is set. It would be possible to turn the current checks for these into a PR check that is obligatory and will block merging until green. [1][2] [1]: https://cwiki.ap

Re: Arrow sync call November 23 at 12:00 US/Eastern, 17:00 UTC

2022-11-28 Thread Antoine Pitrou
Also a note that discussing this in a thread entitled "Arrow sync call November 23" might not raise the attention of all interested parties :-) Le 28/11/2022 à 14:38, Antoine Pitrou a écrit : The merge script would still be useful to flag issues such as a missing component label, or to ens

Re: Arrow sync call November 23 at 12:00 US/Eastern, 17:00 UTC

2022-11-28 Thread Antoine Pitrou
The merge script would still be useful to flag issues such as a missing component label, or to ensure the fix version (milestone) is set. Le 28/11/2022 à 12:09, Joris Van den Bossche a écrit : FYI: Raúl also already opened a PR to update the merge script to work with github issues: https://

Re: Arrow sync call November 23 at 12:00 US/Eastern, 17:00 UTC

2022-11-28 Thread Weston Pace
One thing to note is that you need to have something like "closes #123" in the PR description or a comment in order for GitHub to close the relevant issue when the PR is merged. This isn't too much of a burden to check I think but took a bit of getting used to for me in Substrait where we use the

Re: Arrow sync call November 23 at 12:00 US/Eastern, 17:00 UTC

2022-11-28 Thread Joris Van den Bossche
On Mon, 28 Nov 2022 at 12:09, Joris Van den Bossche wrote: > > FYI: Raúl also already opened a PR to update the merge script to work > with github issues: https://github.com/apache/arrow/pull/14731 (sorry, that PR is to update the github actions workflow (the bot that comments on PRs), not the me

Re: Arrow sync call November 23 at 12:00 US/Eastern, 17:00 UTC

2022-11-28 Thread Joris Van den Bossche
FYI: Raúl also already opened a PR to update the merge script to work with github issues: https://github.com/apache/arrow/pull/14731 Personally I also think that we should consider using the merge button instead of our script (or at least re-evaluate what the script still does better, or might now

Re: Arrow sync call November 23 at 12:00 US/Eastern, 17:00 UTC

2022-11-28 Thread Andrew Lamb
> How do apache/arrow-rs, arrow-datafusion, arrow-julia, et al. handle this? arrow-rs and arrow-datafusion use the squash-and-merge button in the github UI. In general we don't have the same level of curation in commit titles as the main arrow repo. However, I have not heard anyone ask for better

Re: Arrow sync call November 23 at 12:00 US/Eastern, 17:00 UTC

2022-11-25 Thread David Li
This is now configurable, but you have to ask Infra to do it. (You can have GitHub always use the PR title + description.) See https://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/configuring-pull-request-merges/configuring-commit-squashing-for-pull-requests

Re: Arrow sync call November 23 at 12:00 US/Eastern, 17:00 UTC

2022-11-25 Thread Neal Richardson
> - This creates an immediate need to modify the PR merge script; Raúl > opened an issue for this after the call [6]; this also raises the > question of whether we still need the PR merge script or whether > committers can use the "Squash and merge" button in the GitHub web UI > instead > I think

Re: Arrow sync call November 23 at 12:00 US/Eastern, 17:00 UTC

2022-11-24 Thread Ian Cook
Attendees: - Percy T. Aucahuasi - Ian Cook - Raúl Cumplido - James Duong - Todd Farmer - Alenka Frim - Stephanie Hazlitt - Ian Joiner - David Li - Rok Mihevc - Matthew Topol - Joris Van den Bossche - Jacob Wujciak Discussion: Migration from Jira to GitHub issues - ASF Infra has disabled creatio

Re: Arrow sync call November 9 at 12:00 US/Eastern, 17:00 UTC

2022-11-10 Thread Todd Farmer
Hello, As a quick update on the issue tracking migration: - Basic extract from Jira and conversion/import to GitHub issues works well. Some tweaking is needed around content. - Updating source Jira issues now works [1] - The biggest outstanding issue is how to represent related issues [2] - Issue

Re: Arrow sync call November 9 at 12:00 US/Eastern, 17:00 UTC

2022-11-09 Thread Ian Cook
Attendees: - Ian Cook - Raúl Cumplido - Bin Deng - Sean Gallagher - Will Jones - David Li - Ashish Paliwal - Niranda Perera - Matthew Topol Discussion: 10.0.1 release - Primary reason for this patch release is that we do not have PyArrow 10.0.0 wheels for Python 3.11, because the Arrow 10.0.0

Re: Arrow sync call October 26 at 12:00 US/Eastern, 16:00 UTC

2022-10-26 Thread Neal Richardson
Attendees: Vibhatha Abeykoon Raúl Cumplido James Duong Sean Gallagher Ian Joiner Will Jones David Li Antoine Pitrou Neal Richardson Matt Topol Jacob Wujciak Discussion: Release 10.0.0: it's done, and it seemed to go smoothly. Kudos to all who have been working to make releases easier! * But sin

Re: Arrow sync call October 12 at 12:00 US/Eastern, 16:00 UTC

2022-10-13 Thread Ian Cook
Attendees: - Vibhatha Abeykoon - Anja Boskovic - Ian Cook - Will Jones - David Li - Rok Mihevc - Percy Triveño Aucahuasi - Joris Van den Bossche - Jacob Wujciak Discussion: DELTA_BINARY_PACKED encoder for Parquet - Rok looking for help debugging the PR [1] - Tests are failing on some architectu

Re: Arrow sync call September 28 at 12:00 US/Eastern, 16:00 UTC

2022-09-29 Thread Jacob Wujciak
Attendees: - Anja Boskovic - Ashish Paliwal - Bryce Mecum - David Li - Ian Joiner - Jacob Wujciak - Matt Topol - Percy Camilo Triveño Aucahuasi - Raúl Cumplido - Rok Mihevc - Vibhatha Abeykoon - Will Jones Condition of nightly builds - Raúl would like to remind everyone that the great conditions o

Re: Arrow sync call September 14 at 12:00 US/Eastern, 16:00 UTC

2022-09-14 Thread Will Jones
Attendees: - Will Jones - Ian Joiner - Jacob Wujciak - Dhamo - Dewey Dunnington - Sean Gallagher - Ashish Paliwal - Rok Mihevc - James Duong - Bryce Mecum - Anja Boskovic - Matt Topol - David Li Discussion: RLE Progress - Dewey considering implementing

Re: Arrow sync call August 31 at 12:00 US/Eastern, 16:00 UTC

2022-08-31 Thread Ian Cook
Attendees: - Ian Cook - Raúl Cumplido - Dewey Dunnington - James Duong - Ian Joiner - Will Jones - Jonathan Keane - David Li - Rok Mihevc - Niranda Perera - Antoine Pitrou - Gavin Ray - Kae Suarez - Matt Topol - Brent Gardner - Dalton Modlin Discussion: Proposal to switch to C++ 17 as the basel

Re: Arrow sync call August 17 at 12:00 US/Eastern, 16:00 UTC

2022-08-17 Thread Rok Mihevc
Attendees: Matt Topol Will Jones David Li Joris Van den Bosche Eduardo Ponce Atoine Pitrou Jacob Wujciak Ivan Ogasawara Ashish Paliwal Niranda Perera Discussion: - FlightSQL PR reviews David Li is inviting reviewers to two FlightSQL PRs. [1] "ARROW-7744: [Java][FlightRPC] JDBC Driver for Arrow F

Re: Arrow sync call August 3 at 12:00 US/Eastern, 16:00 UTC

2022-08-06 Thread Ian Cook
With apologies for the delay. Attendees: Ian Cook Raúl Cumplido Dewey Dunnington James Duong Todd Farmer Will Jones David Li Rok Mihevc Ivan Ogasawara Ashish Paliwal Niranda Perera Antoine Pitrou Matt Topol Jacob Wujciak Discussion: 9.0.0 release - Vote on RC2 has passed - Post-release tasks a

Re: Arrow sync call July 20 at 12:00 US/Eastern, 16:00 UTC

2022-07-21 Thread Rok Mihevc
> One failing test is the R ubuntu test. Rok noted it is likely unrelated as he is seeing elsewhere [7]. > [7] https://github.com/apache/arrow/runs/7424773120?check_suite_focus=true Dragos is looking into this. It does seem to be an R timezone issue [1]. [1] https://github.com/apache/arrow/comp

Re: Arrow sync call July 20 at 12:00 US/Eastern, 16:00 UTC

2022-07-20 Thread Gavin Ray
Awesome, thanks for the clarification David! On Wed, Jul 20, 2022 at 2:40 PM David Li wrote: > It was pulled out of the ADBC project so you can see an example at [1] > (API changed slightly when ported though). > > Yes, it'll bind one row of values at a time, and your description is > correct. >

Re: Arrow sync call July 20 at 12:00 US/Eastern, 16:00 UTC

2022-07-20 Thread David Li
It was pulled out of the ADBC project so you can see an example at [1] (API changed slightly when ported though). Yes, it'll bind one row of values at a time, and your description is correct. [1]: https://github.com/apache/arrow-adbc/blob/cf43e0cc2ae15ad0ce669b531d475ee218698100/java/driver/jdb

Re: Arrow sync call July 20 at 12:00 US/Eastern, 16:00 UTC

2022-07-20 Thread Gavin Ray
That JDBC PreparedStatement binding utility looks super useful! I had one question about the behavior of it, if that's alright: The doc says: "Each call to next() will bind parameters > from the next row of data, and then the application can execute the > statement, call addBatch(), etc. as desir

Re: Arrow sync call July 20 at 12:00 US/Eastern, 16:00 UTC

2022-07-20 Thread Will Jones
Attendees: - Jacob Wujciak-Jens - James Duong - Rok Mihevc - Raul Cumplido - Eduardo Ponce - Jeremy Parr-Pearson - Will Jones - Joris Van den Bossche Discussion Arrow 9.0.0 Release Increased capacity for crossbow, like 3x, including Macs. Devs are encouraged to use more

Re: Arrow sync call July 6 at 12:00 US/Eastern, 16:00 UTC

2022-07-06 Thread Ian Cook
Attendees: - Ian Cook - Raul Cumplido - James Duong - Will Jones - David Li - Ashish Paliwal - Matt Topol - Jacob Wujciak Discussion: Donation of Flight SQL drivers to Arrow project - The vote to accept the donation of the Flight SQL JDBC driver has passed [1]. We are awaiting some administrati

Re: Arrow sync call June 22 at 12:00 US/Eastern, 16:00 UTC

2022-06-22 Thread Ian Cook
Attendees: - Ian Cook - Will Jones - David Li - Rok Mihevc - Dragoș Moldovan-Grünfeld - Aldrin Montana - Matthew Topol - Jacob Wujciak Discussion: Book about Apache Arrow released this week - Matt Topol's book "In-Memory Analytics with Apache Arrow" goes on sale this Friday June 24 [1] Confere

Re: Arrow sync call June 8 at 12:00 US/Eastern, 16:00 UTC

2022-06-09 Thread Gavin Ray
This is awesome, thanks so much for the comprehensive reply RE: point #9, also holding my breath for data update operations (INSERT/UPDATE/DELETE) to be added to Substrait Have an open issue about it, it needs design work (which I don't think I'm qualified to do) Add Insert/Update/Delete basic fu

Re: Arrow sync call June 8 at 12:00 US/Eastern, 16:00 UTC

2022-06-08 Thread Ian Cook
Hi Gavin, There was no detailed discussion in the meeting about this, just some general comments, but I'll share a few areas of collaboration that I'm aware of: - There is work ongoing to enable the Arrow C++ compute engine (aka "Acero") to consume Substrait plans, change them into ExecPlans, and

Re: Arrow sync call June 8 at 12:00 US/Eastern, 16:00 UTC

2022-06-08 Thread Gavin Ray
Thanks Ian -- can I ask whether there was any discussion of note that happened around Arrow + Substrait stuff? On Wed, Jun 8, 2022 at 5:31 PM Ian Cook wrote: > Attendees: > > Ian Cook > Raúl Cumplido > Alenka Frim > Ian Joiner > Will Jones > Jorge Leitão > David Li > Rok Mihevc > Ashish Paliwal

Re: Arrow sync call June 8 at 12:00 US/Eastern, 16:00 UTC

2022-06-08 Thread Ian Cook
Attendees: Ian Cook Raúl Cumplido Alenka Frim Ian Joiner Will Jones Jorge Leitão David Li Rok Mihevc Ashish Paliwal Matthew Topol Jacob Wujciak Discussion: Recent changes to the merge script for apache/arrow PRs - Now uses a personal access token (PAT) to authenticate to the ASF Jira - Now requ

Re: Arrow sync call May 25 at 12:00 US/Eastern, 16:00 UTC

2022-05-25 Thread Rok Mihevc
Below are the minutes of the call. Best, Rok Present: Dewey Dunnington, Raul Cumplido, Will Jones, Jonathan Keane, Matt Topol, Rok Mihevc, Ian Joiner Agenda: 1. Minimal C++/C interface 2. Naming the C++ Compute Engine 3. Change to bucket creation behaviour in S3FileSystem Notes: 1. Dewey is ask

Re: Arrow sync call May 11 at 12:00 US/Eastern, 16:00 UTC

2022-05-13 Thread Antoine Pitrou
Le 13/05/2022 à 16:30, Alessandro Molina a écrit : I think Arrow should definitely consider adding a DataFrame-like API. There are multiple reasons why exposing Arrow to end users instead of restricting it to developers of framework would be beneficial for the Arrow project itself. A rough ap

Re: Arrow sync call May 11 at 12:00 US/Eastern, 16:00 UTC

2022-05-13 Thread Gavin Ray
I agree with this as well, and I it's also along the lines of what I was trying to propose here: "[RFC] [Java] Higher-level "DataFrame"-like API. Lower barrier to entry, increase adoption/audience and productivity." https://github.com/apache/arrow/issues/12618 It would be really nice if there was

Re: Arrow sync call May 11 at 12:00 US/Eastern, 16:00 UTC

2022-05-13 Thread Alessandro Molina
I think Arrow should definitely consider adding a DataFrame-like API. There are multiple reasons why exposing Arrow to end users instead of restricting it to developers of framework would be beneficial for the Arrow project itself. A rough approximation of DataFrame like API has been growing duri

Re: Arrow sync call May 11 at 12:00 US/Eastern, 16:00 UTC

2022-05-12 Thread Andrew Lamb
Also, it seems as if duckdb[1] is heading in the same direction of adding a dataframe API to their database engine [1] https://github.com/duckdb/duckdb/issues/2000 On Thu, May 12, 2022 at 3:36 PM Andrew Lamb wrote: > For what it is worth, DataFusion has a DataFrame interface[1], that uses > the

Re: Arrow sync call May 11 at 12:00 US/Eastern, 16:00 UTC

2022-05-12 Thread Andrew Lamb
For what it is worth, DataFusion has a DataFrame interface[1], that uses the same underlying `LogicalPlan` structures as the SQL interface. Unsurprisingly it is heavily inspired by pandas. I believe that this interface seems more familiar and popular for DataFusion users who programmatically build

Re: Arrow sync call May 11 at 12:00 US/Eastern, 16:00 UTC

2022-05-12 Thread Wes McKinney
> Discussion about whether the community around Arrow would like to have > DataFrame-like APIs for Arrow in more languages, for example C++ We've discussed this a bit on the mailing list in the past, see https://docs.google.com/document/d/1XHe_j87n2VHGzEbnLe786GHbbcbrzbjgG8D0IXWAeHg/edit#heading

Re: Arrow sync call May 11 at 12:00 US/Eastern, 16:00 UTC

2022-05-11 Thread Ian Cook
Attendees: Joris Van den Bossche Ian Cook Nic Crane Raul Cumplido Ian Joiner David Li Rok Mihevc Dragoș Moldovan-Grünfeld Aldrin Montana Weston Pace Eduardo Ponce Matthew Topol Jacob Wujciak Discussion: Eduardo: Draft PR with a guide showing how to create a new Arrow C++ compute kernel [1] - R

Re: Arrow sync call April 27 at 12:00 US/Eastern, 16:00 UTC

2022-04-27 Thread Benson Muite
Attendees: Ian Joiner Matthew Topol Benson Muite Discussion points: 1) New book on Arrow - covers C++, Python and Go, out in June 2) Building ORC bindings in R would be useful, extensions to parallel R? 3) Comparing ORC and Parquet for IO 4) IO optimization vs SIMD optimization - Parquet seems we

Re: Arrow sync call April 27 at 12:00 US/Eastern, 16:00 UTC

2022-04-27 Thread Ian Cook
Thanks Benson! The Zoom meeting URL for this and other biweekly Arrow sync calls is: https://zoom.us/j/87649033008?pwd=SitsRHluQStlREM0TjJVYkRibVZsUT09 Alternatively, enter this information into the Zoom website or app to join the call: Meeting ID: 876 4903 3008 Passcode: 958092 The Zoom meeting

Re: Arrow sync call April 27 at 12:00 US/Eastern, 16:00 UTC

2022-04-27 Thread David Li
Thanks Benson. If you are able to take notes this week that would be much appreciated. And thanks Joris for the clarification. On Wed, Apr 27, 2022, at 09:34, Joris Van den Bossche wrote: > As a small clarification: the zoom meeting link itself should still work > for anyone to join, it's only t

Re: Arrow sync call April 27 at 12:00 US/Eastern, 16:00 UTC

2022-04-27 Thread Joris Van den Bossche
As a small clarification: the zoom meeting link itself should still work for anyone to join, it's only there is no one from Voltron Data to lead the meeting / take notes (so I also won't be present today). Joris On Wed, 27 Apr 2022 at 13:05, Benson Muite wrote: > Hi, > > Can host if required, t

Re: Arrow sync call April 13 at 12:00 US/Eastern, 16:00 UTC

2022-04-27 Thread Benson Muite
On 4/25/22 2:49 PM, David Li wrote: Following up here: N.B. The Voltron Data folks have a scheduling conflict on 4/27 and will not be able to host the fortnightly sync call. Is anyone available to run the meeting that day? Is anyone available to run the sync call this Wednesday? On Wed, Ap

Re: Arrow sync call April 27 at 12:00 US/Eastern, 16:00 UTC

2022-04-27 Thread Benson Muite
Hi, Can host if required, though the timing is not ideal for me. It may be helpful to vary the timing in future. Benson On 4/25/22 2:49 PM, David Li wrote: Following up here: N.B. The Voltron Data folks have a scheduling conflict on 4/27 and will not be able to host the fortnightly sync c

Re: Arrow sync call April 13 at 12:00 US/Eastern, 16:00 UTC

2022-04-25 Thread David Li
Following up here: > N.B. The Voltron Data folks have a scheduling conflict on 4/27 and will not > be able to host the fortnightly sync call. Is anyone available to run the > meeting that day? Is anyone available to run the sync call this Wednesday? On Wed, Apr 13, 2022, at 12:59, David Li wro

Re: Arrow sync call April 13 at 12:00 US/Eastern, 16:00 UTC

2022-04-18 Thread Will Jones
That sounds great, Weston! Agreed that syncing up with releases seems like a good idea. On Mon, Apr 18, 2022 at 11:51 AM Weston Pace wrote: > I'm happy to provide a quarterly update on C++ engine work but in the > future I'll draft it in PR form so others have a chance to pitch in. > I was inspi

Re: Arrow sync call April 13 at 12:00 US/Eastern, 16:00 UTC

2022-04-18 Thread Weston Pace
I'm happy to provide a quarterly update on C++ engine work but in the future I'll draft it in PR form so others have a chance to pitch in. I was inspired by, and hope to mimic, the Rust community's very cool quarterly roadmap [1][2] as a place to have higher level discussions on what people are hop

Re: Arrow sync call April 13 at 12:00 US/Eastern, 16:00 UTC

2022-04-18 Thread Will Jones
Thanks Weston for providing the update on the C++ compute engine. IMO, it would be very welcome to have that update be a quarterly email to the dev mailing list, and may provide an opportunity to highlight issues in Jira that are good first issues or neglected but important. On Wed, Apr 13, 2022 a

Re: Arrow sync call April 13 at 12:00 US/Eastern, 16:00 UTC

2022-04-13 Thread David Li
Attendees: - David Li - Eduardo Ponce - Gavin Ray - Ian Cook - James Duong - Matthew Topol - Nic - Niranda - Raul Cumplido - Rok - Weston Pace - Will Jones N.B. The Voltron Data folks have a scheduling conflict on 4/27 and will not be able to host the fortnightly sync call. Is anyone available t

Re: Arrow sync call March 30 at 12:00 US/Eastern, 16:00 UTC

2022-03-30 Thread Ian Cook
Attendees: Ian Cook James Duong Ian Joiner Micah Kornfield David Li Rok Mihevc Matt Topol Discussion: C++ helpers for conversion between Arrow columnar formats and row-oriented formats - Discussion at [1]; comments requested - Could this be pursued by factoring out existing code from the CSV

Re: Arrow sync call March 16 at 12:00 US/Eastern, 16:00 UTC

2022-03-16 Thread Rok Mihevc
Sorry to all who will have missed the call because of the Daylight Saving Time change. Here are the notes: Attendees: James Duong, Micah Kornfield, Eduardo Ponce, Will Jones, Rok Mihevc, David Li, Niranda Perera, Alenka Frim, Ian Joiner Discussion: * James Duong - Out of curiosity - does Arrow ha

Re: Arrow sync call March 2 at 12:00 US/Eastern, 17:00 UTC

2022-03-02 Thread David Li
Yeah, things like that would help I believe, to help make it easier to adopt and make sure knowledge of the details is readily available around the community. I have been meaning to tackle ARROW-14698 but haven't gotten to it yet. I think it would also help to have more reviewers, so maybe one

Re: Arrow sync call March 2 at 12:00 US/Eastern, 17:00 UTC

2022-03-02 Thread James Duong
I missed this call (incidentally to present Flight SQL at Subsurface). I'd be interested in helping to champion Flight SQL. What would this entail? Working on extending the documentation, providing example code and cookbook entries? On Wed, Mar 2, 2022 at 10:59 AM Gavin Ray wrote: > Ah got it, t

Re: Arrow sync call March 2 at 12:00 US/Eastern, 17:00 UTC

2022-03-02 Thread Gavin Ray
Ah got it, thank you! =) On Wed, Mar 2, 2022 at 10:33 AM Micah Kornfield wrote: > Hi Gavin, > This was mostly discussing the current discussion on the e-mail thread > titled "Flight/FlightSQL Optimization for Small Results?". The main thing > discussed was whether we think adding the complexity

Re: Arrow sync call March 2 at 12:00 US/Eastern, 17:00 UTC

2022-03-02 Thread Micah Kornfield
Hi Gavin, This was mostly discussing the current discussion on the e-mail thread titled "Flight/FlightSQL Optimization for Small Results?". The main thing discussed was whether we think adding the complexity around negotiation is necessary. The consensus was it probably depends on how big the res

Re: Arrow sync call March 2 at 12:00 US/Eastern, 17:00 UTC

2022-03-02 Thread Gavin Ray
Particularly curious about the small-results FlightSQL optimizations and general FlightSQL developments, if there was anything anyone felt was worth noting outside of the general outline. Thank you =) On Wed, Mar 2, 2022 at 10:12 AM Micah Kornfield wrote: > It was not. Is there anything you wo

Re: Arrow sync call March 2 at 12:00 US/Eastern, 17:00 UTC

2022-03-02 Thread Micah Kornfield
It was not. Is there anything you would like more context on? On Wed, Mar 2, 2022 at 10:10 AM Gavin Ray wrote: > Was this recorded by any chance? No worries if not. > > On Wed, Mar 2, 2022 at 9:58 AM Alessandro Molina < > alessan...@ursacomputing.com> wrote: > > > Attendees: > > > > > > Alessan

Re: Arrow sync call March 2 at 12:00 US/Eastern, 17:00 UTC

2022-03-02 Thread Gavin Ray
Was this recorded by any chance? No worries if not. On Wed, Mar 2, 2022 at 9:58 AM Alessandro Molina < alessan...@ursacomputing.com> wrote: > Attendees: > > > Alessandro Molina > > Micah Kornfield > > David Li > > Joris Van Den Bossche > > > > Discussion: > > > Flight SQL Optimization for Small R

Re: Arrow sync call March 2 at 12:00 US/Eastern, 17:00 UTC

2022-03-02 Thread Alessandro Molina
Attendees: Alessandro Molina Micah Kornfield David Li Joris Van Den Bossche Discussion: Flight SQL Optimization for Small Results - Reference to https://databricks.com/blog/2021/08/11/how-we-achieved-high-bandwidth-connectivity-with-bi-tools.html - Building directly in Flight as flig

Re: Arrow sync call February 16 at 12:00 US/Eastern, 17:00 UTC

2022-02-16 Thread Ian Cook
Attendees: Ian Cook Nic Crane James Duong Alenka Frim Ian Joiner Will Jones Jonathan Keane Rok Mihevc Discussion: Arrow visual identity proposal - If interested, please review the PR [1] arrow vcpkg port 7.0.0 update PR [2] - To maintain ORC functionality, the orc vcpkg port must be updated [3

Re: Arrow sync call February 2 at 12:00 US/Eastern, 17:00 UTC

2022-02-02 Thread Ian Cook
Attendees: Ian Cook Ian Joiner Will Jones Jon Keane Micah Kornfield Rok Mihevc Alessandro Molina Antoine Pitrou Jacob Quinn Krisztián Szűcs Matt Topol Joris Van den Bossche Discussion: Recent ORC writer work - Ian Joiner is working on several follow-up issues [1] 7.0.0 release current state - K

Re: Arrow sync call January 19 at 12:00 US/Eastern, 17:00 UT

2022-01-19 Thread Ian Cook
Attendees: Colin Alworth Ian Cook Nic Crane Alenka Frim Will Jones David Li Rok Mihevc Dragoș Moldovan-Grünfeld Niranda Perera Antoine Pitrou Matt Topol Joris Van den Bossche Discussion: 7.0.0 release - Krisztián has a PR open for RC0 [1] - Joris has identified a regression [2] so there will li

Re: Arrow sync call January 5 at 12:00 US/Eastern, 17:00 UTC

2022-01-05 Thread Eduardo Ponce
Hi all, With respect to what examples/information may be relevant to add/improve in documentation, I find that browsing GitHub issues [1] is a good place to identify some cases on how users are using Arrow. Moreover, many of the GH issues related to code examples, contain snippets of code in the r

Re: Arrow sync call January 5 at 12:00 US/Eastern, 17:00 UTC

2022-01-05 Thread Rok Mihevc
Attendees Nic Crane, Micah Kornfeld, Eduardo Ponce, Will Jones, Rok Mihevc, David Li, Niranda Perera, Benson Muite Agenda - Discussion about the new columnar memory layout - Preparing for 7.0.0 release - 2nd or 3rd week of January - Documentation improvement - Support for table like structures

Re: Arrow sync call December 22 at 12:00 US/Eastern, 17:00 UTC

2021-12-22 Thread Benson Muite
On 12/22/21 11:04 PM, Ian Cook wrote: Discussion of how best to use this meeting in 2022 - Consider changing the day/time? How can we best accommodate time zones, people doing Arrow dev work in day jobs vs. on evenings and weekends, etc? Further discussion about this is welcome Mining the Arrow

Re: Arrow sync call December 22 at 12:00 US/Eastern, 17:00 UTC

2021-12-22 Thread Ian Cook
Attendees Ian Cook James Duong Pradeep Garigipati Ian Joiner David Li Rok Mihevc Eduardo Ponce Discussion Upcoming Arrow 7.0.0 release [1] - Expected in first half of January - Please plan to complete PRs for 7.0.0 in time - Please bump Fix Version from 7.0.0 to 8.0.0 for Jira issues not expecte

Re: Arrow sync call December 8 at 12:00 US/Eastern, 17:00 UTC

2021-12-08 Thread Ian Cook
Attendees Ian Cook Alenka Frim Andrew Lamb David Li Antoine Pitrou Discussion Proposal from Joris Van den Bossche to split the Sphinx docs project into multiple projects [1] - This is mostly a technical proposal but it affects how we maintain the docs - Feedback and questions welcome Proposal

Re: Arrow sync call November 24 at 12:00 US/Eastern, 17:00 UTC

2021-11-24 Thread Nic
Attendees: Eduardo Ponce Keith Kraus Joris Van den Bossche Rok Mihevc Nic Crane James Duong Jonathan Keane Discussion Eduardo brought up the mini-app discussion from the mailing list discussion[1] - In short, this is a scaled down version of Arrow with some functionality and validation checks etc

Re: Arrow sync call November 10 at 12:00 US/Eastern, 17:00 UTC

2021-11-10 Thread Jonathan Keane
Meeting notes: # Participants Nic Weston David Eduardo Benson Rok Antoine Alenka James Matt Micah # 6.0.1 patch release The RC1 for 6.0.1 is on its way and will have a vote shortly # Flight SQL David wanted to talk about Flight SQL from Dremio. We are close, would like someone to revi

Re: Arrow sync call October 27 at 12:00 US/Eastern, 16:00 UTC

2021-10-27 Thread Nic
Meeting notes below Attendees: Nic Crane Jonathan Keane Eduardo Ponce Niranda Perera Benson Muite Micah Kornfield Joris Van den Bossche Discussion: - Update on release - the vote passed and post-release tasks are in progress. The R package has been submitted to CRAN and resubmitting shortly as

Re: Arrow sync call October 13 at 12:00 US/Eastern, 16:00 UTC

2021-10-14 Thread Ian Cook
Attendees: Ian Cook Micah Kornfield Rok Mihevc Benson Muite Antoine Pitrou Matt Topol Discussion: 6.0.0 release [1] - RC0 expected in the next several days - Krisztián will serve as release manager Bumping versions of bundled dependencies [2] - Micah raised concerns that this is potentially ri

Re: Arrow sync call September 29 at 12:00 US/Eastern, 16:00 UTC

2021-09-29 Thread Neal Richardson
Attendees: Phillip Cloud Alenka Frim Jörn Horstman Jonathan Keane Micah Kornfield Rok Mihevc Benson Muite Jacques Nadeau Eduardo Manuel Ponce Mojica Weston Pace Neal Richardson Joris Van den Bossche Discussion: * C++ query engine update (see previous mailing list message) * 6.0 release is coming

Re: Arrow sync call September 15 at 12:00 US/Eastern, 16:00 UTC

2021-09-15 Thread Nate Bauernfeind
Meeting notes for arrow-sync on 09/15/2021. Attendees: - Nate Bauernfeind - Nic Crane - Alenka Frim - Rok Mihevc - Niranda Perera There was no discussion this week; all attendees were here to lurk, listen, and be-a-fly-on-the-wall. See you in two weeks. -- On Wed, Sep 15, 2021 at 8:08 AM Ian

Re: Arrow sync call September 1 at 12:00 US/Eastern, 16:00 UTC

2021-09-15 Thread Ian Cook
Meeting notes (with apologies for the delay in sending these out): Attendees: - Nate Bauernfeind - Phillip Cloud - Ian Cook - Nic Crane - David Li - Rok Mihevc - Eduardo Ponce - Micah Kornfield - Niranda Perera Discussion: - Deephaven built a asynchronous event-driven framework for Arrow Flight

Re: Arrow sync call August 18 at 12:00 US/Eastern, 16:00 UTC

2021-08-18 Thread Ian Cook
Meeting notes: Attendees: Ian Cook Nic Crane Alenka Frim David Li Rok Mihevc Niranda Perera Eduardo Ponce Discussion: - Voltron Data update - Ursa Computing now part of Voltron Data [1] - Ursa Labs now Voltron Labs - Mission to support Apache Arrow project and community continues - Sugges

Re: Arrow sync call August 3 at 12:00 US/Eastern, 16:00 UTC

2021-08-04 Thread Jonathan Keane
Notes for the meeting, it was relatively short and sparsely attended this fortnight: Attendees: * David Li * Jonathan Keane * Nic Crane * Neal Richardson Topics discussed * Compute IR proposal: There's been some discussion, check it out * CRAN resubmission, we have the fixes we need, will send th

Re: Arrow sync call July 21 at 12:00 US/Eastern, 16:00 UTC

2021-07-22 Thread Ian Cook
Meeting notes: Attendees: Nate Bauernfeind Ian Cook Nic Crane Jonathan Keane Rok Mihevc Weston Pace Eduardo Ponce Discussion: - 5.0.0 release is underway this week - Thanks to all who have contributed! - No blocking Jira issues[1] - Krisztián is managing this release and is currently prep

Re: Arrow sync call July 7 at 12:00 US/Eastern, 16:00 UTC

2021-07-07 Thread Ian Cook
Thanks to all who attended. Meeting notes: Attendees: Nate Bauernfeind Ian Cook Nic Crane Alenka Frim Micah Kornfield Jorge Leitao Alessandro Molina Weston Pace Eduardo Ponce Pol Santamaria Discussion: - Arrow 5.0.0 release - Goal: release end of next week or worst case the third week of July

Re: Arrow sync call July 7 at 12:00 US/Eastern, 16:00 UTC

2021-07-07 Thread Nate Bauernfeind
Is this still happening today? On Tue, Jul 6, 2021 at 11:07 AM Ian Cook wrote: > Hi all, > > Our biweekly sync call is tomorrow at > https://meet.google.com/vtm-teks-phx. All are welcome to join. Notes > will be shared with the mailing list afterward. > > Ian > --

Re: Arrow sync call July 7 at 12:00 US/Eastern, 16:00 UTC

2021-07-07 Thread Ian Cook
Update: For the meeting starting now, please us this Google Meet URL: https://meet.google.com/ebp-tczo-xjn Ian On Tue, Jul 6, 2021 at 12:07 PM Ian Cook wrote: > > Hi all, > > Our biweekly sync call is tomorrow at > https://meet.google.com/vtm-teks-phx. All are welcome to join. Notes > will be sh

Re: Arrow sync call May 26 at 12:00 US/Eastern, 16:00 UTC

2021-06-08 Thread Neal Richardson
Belated notes from the call last time: Attendees: Nate Bauernfeind Ian Cook Nic Crane James Duong Tiffany Lam Jorge Cardoso Leitão Rok Mihevc Gyan Prakash Neal Richardson Discussion: - 4.0.1 patch release: vote passed, doing the post release tasks - FlightSQL: James and Tiffany picking back up

Re: Arrow sync call May 12 at 12:00 US/Eastern, 16:00 UTC

2021-05-12 Thread Neal Richardson
Attendees: Jim Apple Ian Cook Nic Crane Prem Sagar Gali Jonathan Keane Micah Kornfield David Li Rok Mihevc Niranda Perera Eduardo Ponce Gyan Prakash Neal Richardson Aster Rosa Naman Udasi Discussion: * Interval type: Micah is working on implementations in C++ and Java. Note that Parquet uses uns

Re: Arrow sync call April 28 at 12:00 US/Eastern, 16:00 UTC

2021-04-28 Thread Neal Richardson
Attendees: Nate Bauernfeind Ian Cook Nic Crane Jonathan Keane Micah Kornfield Andrew Lamb Rok Mihevc Ruan Pearce-Authers Prudhvi Porandla Neal Richardson Ying Zhou Discussion: * 4.0 release: done! Watch out for issues that might merit a patch release, raise on the mailing list. Note that Rust al

Re: Arrow sync call April 14 at 12:00 US/Eastern, 16:00 UTC

2021-04-14 Thread Ian Cook
Thanks everyone who attended. Meeting notes: Attendees: Colin Alworth Nate Bauernfeind Ian Cook Nic Crane Andy Grove Jonathan Keane Micah Kornfield David Li Kirill Lykov Roc Mihevc Alessandro Molina Weston Pace Antoine Pitrou Neal Richardson Krisztián Szűcs Naman Udasi Discussion: - Arrow 4.0.0

Re: Arrow sync call March 31 at 12:00 US/Eastern, 16:00 UTC

2021-03-31 Thread Jonathan Keane
Thank you everyone who attended, here are the notes. Attendees: Jonathan Keane Colin Alworth David Sanders Micah Kornfield Rok Mihevc Projjal Chanda Eduardo Ponce Krill Lykov Discussion: - 4.0 release - zstd compression for the java library (has PR that is approved but ne

Re: Arrow sync call March 31 at 12:00 US/Eastern, 16:00 UTC

2021-03-31 Thread Wes McKinney
It does, but I would suggest that someone volunteer to host the call each week and send out a Zoom link for that week's call On Wed, Mar 31, 2021 at 11:11 AM Antoine Pitrou wrote: > > > I'm fine with Zoom. But doesn't need it a host as well? > > > Le 31/03/2021 à 18:09, Wes McKinney a écrit : >

Re: Arrow sync call March 31 at 12:00 US/Eastern, 16:00 UTC

2021-03-31 Thread Antoine Pitrou
I'm fine with Zoom. But doesn't need it a host as well? Le 31/03/2021 à 18:09, Wes McKinney a écrit : The Google Meet link is on dremio.com, so there must not be someone from the org to let people in. What do folks think about moving to Zoom for future meetings (which shouldn't have this pro

Re: Arrow sync call March 31 at 12:00 US/Eastern, 16:00 UTC

2021-03-31 Thread Wes McKinney
The Google Meet link is on dremio.com, so there must not be someone from the org to let people in. What do folks think about moving to Zoom for future meetings (which shouldn't have this problem)? On Wed, Mar 31, 2021 at 11:07 AM Jonathan Keane wrote: > > I'm experiencing the same here. > > On We

Re: Arrow sync call March 31 at 12:00 US/Eastern, 16:00 UTC

2021-03-31 Thread Jonathan Keane
I'm experiencing the same here. On Wed, Mar 31, 2021 at 11:06 AM Kirill Lykov wrote: > Hi, > > I don't know about the others but I cannot join because someone needs to > let me in. > Might be it the problem also for other people? > > On Tue, Mar 30, 2021 at 5:53 PM Neal Richardson < > neal.p.ric

Re: Arrow sync call March 31 at 12:00 US/Eastern, 16:00 UTC

2021-03-31 Thread Kirill Lykov
Hi, I don't know about the others but I cannot join because someone needs to let me in. Might be it the problem also for other people? On Tue, Mar 30, 2021 at 5:53 PM Neal Richardson wrote: > Hi all, > Our biweekly call is coming up tomorrow at > https://meet.google.com/vtm-teks-phx. All are we

Re: Arrow sync call March 3 at 12:00 US/Eastern, 16:00 UTC

2021-03-17 Thread Neal Richardson
Attendees: Ian Cook Andy Grove Jonathan Keane Micah Kornfield David Li Rok Mihevc Prudhvi Porandla Neal Richardson Discussion: * 4.0 release: coming in about 3 weeks (April). Some outstanding issues to resolve: * Bintray: the service is being turned off on May 1, so we can't use it anymore. We

Re: Arrow sync call February 17 at 12:00 US/Eastern, 17:00 UTC

2021-02-18 Thread Neal Richardson
Apologies for the delay in sending the notes out. Attendees: Ian Cook Andy Grove Jonathan Keane Micah Kornfield Hoi Link Tiffany Lam Andrew Lamb Jorge Cardoso Leitão Kirill Lykov Rok Mihevc Weston Pace Ruan Pearce-Authers Neal Richardson Naman Udasi Dmitry Unknown Discussion: * PR/JIRA tool: An

Re: Arrow sync call February 17 at 12:00 US/Eastern, 17:00 UTC

2021-02-17 Thread Antoine Pitrou
Le 17/02/2021 à 12:07, Andrew Lamb a écrit : > *Proposal*: Allow Rust and other implementations release additional point > / maintenance versions at a different cadences, out of lockstep with the > major arrow releases. We could still release the Rust library as part of > the major Arrow releas

  1   2   3   >