Re: [DISCUSS] Acero roadmap / philosophy

ting new > >> vectors > >>>>> between function calls. Usage of selection vectors to avoid > >>>> materializing > >>>>> filter results. General avoidance of allocation and preference for > >>>> thread > >>>>> local data. >

Re: [DISCUSS] Acero roadmap / philosophy

g block for those that are exploring unique execution models or whose workloads don't cleanly fit into an SQL query. Acero has been used effectively for academic research. Acero has been valuable for me personally as a sort of "reference implementation" for a Substrait consumer as w

Re: [DISCUSS] Acero roadmap / philosophy

; > 4. Acero does no query optimization. Hopefully Substrait > optimizers > > > will > > > > emerge to fill this gap. In the meantime, this remains a significant > > gap > > > > when comparing Acero to most other execution engines. > > &

Re: [DISCUSS] Acero roadmap / philosophy

ke > > > sure they can. In fact, I would be in favor of explicitly abandoning > the > > > morsel-batch model and focusing on larger batch sizes in the spirit of > > > simplicity. > > > > > > This does not mean that I want to abandon Acero. Acero is val

Re: [DISCUSS] Acero roadmap / philosophy

and focusing on larger batch sizes in the spirit of > > simplicity. > > > > This does not mean that I want to abandon Acero. Acero is valuable for a > > number of users who don't need that last 20% of performance and would > > rather not introduce a new library.

Re: [DISCUSS] Acero roadmap / philosophy

been used effectively for academic research. Acero has been valuable for me personally as a sort of "reference implementation" for a Substrait consumer as well as being a reference engine for connectivity and decentralization in general. ## My roadmap Over the next year I plan on tran

[DISCUSS] Acero roadmap / philosophy

models or whose workloads don't cleanly fit into an SQL query. Acero has been used effectively for academic research. Acero has been valuable for me personally as a sort of "reference implementation" for a Substrait consumer as well as being a reference engine for connectivity and dec

Re: [Rust] Using GitHub milestones for roadmap and release planning

of communicating our roadmap rather than maintaining a separate > document somewhere else. > > Here is the milestone for DataFusion 15.0.0, for example [1]. > > Would others find this helpful? The plan would be to release 15.0.0 on the > target date even if there are unfinished

[Rust] Using GitHub milestones for roadmap and release planning

I've started experimenting with GitHub milestones in the DataFusion project. I find it helpful to have a prioritized list of issues that I am working on for the next release, and the milestone description can be used as a way of communicating our roadmap rather than maintaining a separate doc

Re: Datafusion's vision and roadmap?

Datafusion is, but does not really > give a vision going forward. A few months ago we tried a "what is everyone > thinking of working on" type approach [1] to create a roadmap. While that > was insightful, I agree having a single unified (even if vague) goal would > be very helpful

Re: Datafusion's vision and roadmap?

*what* Datafusion is, but does not really give a vision going forward. A few months ago we tried a "what is everyone thinking of working on" type approach [1] to create a roadmap. While that was insightful, I agree having a single unified (even if vague) goal would be very helpful I would welcome

Datafusion's vision and roadmap?

Hi, This is regarding my question about the datafusion's vision and roadmap. As a new contributor, I wonder what would be a vision and roadmap that most of the contributors can/already have be aligned upon. Maybe due to my lack of prior context I might have missed such discussion, or maybe

[Rust] Crowdsourced Arrow Rust 4.0 Roadmap

As discussed on the Rust call today, please find below a link to a shared document where anyone who plans to contribute to the 4.0 release of the Arrow Rust implementation is welcomed to share what they plan to work on: https://docs.google.com/document/d/1qspsOM_dknOxJKdGvKbC1aoVoO0M3i6x1CIo58mmN2

Re: Collecting Arrow critique and our roadmap on that

red our thoughts > > > > there, we can put them in the appropriate places. > > > > > > > > I think that some of the result will go into the FAQ, some into > > > > documentation (maybe more "how-to" and "getting started" gu

Re: Collecting Arrow critique and our roadmap on that

tting started" guides in the > > > respective language docs, as well as some "how to share Arrow data > > > from X to Y"), and other things that we haven't yet done should go > > > into a sort of Roadmap document on the main website. We have some v

Re: Collecting Arrow critique and our roadmap on that

ot; guides in the > > respective language docs, as well as some "how to share Arrow data > > from X to Y"), and other things that we haven't yet done should go > > into a sort of Roadmap document on the main website. We have some very > > outdated content rela

Re: Collecting Arrow critique and our roadmap on that

-to" and "getting started" guides in the > respective language docs, as well as some "how to share Arrow data > from X to Y"), and other things that we haven't yet done should go > into a sort of Roadmap document on the main website. We have some very > outdated

Re: Collecting Arrow critique and our roadmap on that

f the result will go into the FAQ, some into documentation (maybe more "how-to" and "getting started" guides in the respective language docs, as well as some "how to share Arrow data from X to Y"), and other things that we haven't yet done should go into a sort of Roadm

Collecting Arrow critique and our roadmap on that

Hello, there has been a lot of public discussions lately with some mentions of actually informed, valid critique of things in the Arrow project. From my perspective, these things include "there is not STL-native C++ Arrow API", "the base build requires too much dependencies", "the pyarrow packa

Re: Assembling an initial development roadmap for Arrow in R

), but we are on our way! thanks Wes On Thu, Sep 6, 2018 at 6:49 PM Wes McKinney wrote: > > I'm going to transfer the document over to Confluence soon and start > creating JIRA issues. If anyone has further comments about the > roadmap, please add them to the document or request e

Re: Assembling an initial development roadmap for Arrow in R

I'm going to transfer the document over to Confluence soon and start creating JIRA issues. If anyone has further comments about the roadmap, please add them to the document or request edit access. Thanks! On Tue, Aug 28, 2018 at 5:16 PM Wes McKinney wrote: > > hi folks, > > In

Assembling an initial development roadmap for Arrow in R

ess let me know and I'll add you This should eventually be transferred to the dev wiki (https://cwiki.apache.org/confluence/display/ARROW) but started this as a Google doc to make collaboration a little lighter weight as we get started. After some comments collect, I suggest we move the roadm

Re: Housing longer-term Arrow development, design, and roadmap documents

gt; wrote: > >> > Hi, > >> > > >> >> https://cwiki.apache.org/confluence/display/ARROW > >> >> > >> >> If any PMC members would like to be administrators of the space, > >> >> please let me know your Confluence usernam

Re: Housing longer-term Arrow development, design, and roadmap documents

se let me know your Confluence username. You have to create a >> >> separate account (it does not appear to be linked to JIRA accounts) >> > >> > Can you add me? I've created "kou" account on Confluence. >> > >> > >> > Thanks, >

Re: Housing longer-term Arrow development, design, and roadmap documents

et me know your Confluence username. You have to create a > >> separate account (it does not appear to be linked to JIRA accounts) > > > > Can you add me? I've created "kou" account on Confluence. > > > > > > Thanks, > > -- > > kou >

Re: Housing longer-term Arrow development, design, and roadmap documents

you add me? I've created "kou" account on Confluence. > > > Thanks, > -- > kou > > In > "Re: Housing longer-term Arrow development, design, and roadmap documents" > on Tue, 26 Jun 2018 11:27:50 -0400, > Wes McKinney wrote: > >> GitH

Re: Housing longer-term Arrow development, design, and roadmap documents

e? I've created "kou" account on Confluence. Thanks, -- kou In "Re: Housing longer-term Arrow development, design, and roadmap documents" on Tue, 26 Jun 2018 11:27:50 -0400, Wes McKinney wrote: > GitHub wiki pages lack collaboration features like commenting. It wi

Re: Housing longer-term Arrow development, design, and roadmap documents

be linked to JIRA accounts) >>> >>> Thanks >>> >>> On Sun, Jun 24, 2018 at 1:14 PM, Uwe L. Korn wrote: >>>> Hello, >>>> >>>> I would prefer Confluence over GitHub pages because I would hope that one >>>> c

Re: Housing longer-term Arrow development, design, and roadmap documents

, >>>> >>>> I would prefer Confluence over GitHub pages because I would hope that one >>>> can integrate the ASF JIRA via widgets into the wiki pages. The vast >>>> amount of issues should all be categorizable into some topic. Once these >>>

Re: Housing longer-term Arrow development, design, and roadmap documents

Hub pages because I would hope that one >>> can integrate the ASF JIRA via widgets into the wiki pages. The vast amount >>> of issues should all be categorizable into some topic. Once these are >>> triaged, they should pop up in the respective wiki pages that could form

Re: Housing longer-term Arrow development, design, and roadmap documents

amount >> of issues should all be categorizable into some topic. Once these are >> triaged, they should pop up in the respective wiki pages that could form a >> roadmap. That way, newcomers should get a better start to find the things to >> work on for a certain topic. >&g

Re: Housing longer-term Arrow development, design, and roadmap documents

pic. Once these are triaged, > they should pop up in the respective wiki pages that could form a roadmap. > That way, newcomers should get a better start to find the things to work on > for a certain topic. > > Cheers > Uwe > > On Sun, Jun 24, 2018, at 7:02 PM, Antoine Pit

Re: Housing longer-term Arrow development, design, and roadmap documents

form a roadmap. That way, newcomers should get a better start to find the things to work on for a certain topic. Cheers Uwe On Sun, Jun 24, 2018, at 7:02 PM, Antoine Pitrou wrote: > > Hi Wes, > > I wonder if GitHub wiki pages would be an easier-to-approach alternative?

Re: Housing longer-term Arrow development, design, and roadmap documents

ages and new areas of > functionality, I'd like to discuss how we could better accommodate > longer-term asynchronous discussions and stay organized about the > development roadmap. > > At any given time, there could be 10 or more initiatives ongoing, and > the number of

Housing longer-term Arrow development, design, and roadmap documents

hi folks, Since the scope of Apache Arrow has grown significantly in the last 2.5 years to encompass many programming languages and new areas of functionality, I'd like to discuss how we could better accommodate longer-term asynchronous discussions and stay organized about the development ro

Arrow JS tasks and roadmap

Brian Hulette and I have outlined this list of tasks/improvements for the expanded Arrow JS implementation: https://docs.google.com/document/d/142dek89oM2TVI2Yql106Zo8IB1Ff_9zDg_EG6jPWS0M/edit?usp=sharing

[jira] [Created] (ARROW-1668) [JS] Draft roadmap / planning document for JS library development

Wes McKinney created ARROW-1668: --- Summary: [JS] Draft roadmap / planning document for JS library development Key: ARROW-1668 URL: https://issues.apache.org/jira/browse/ARROW-1668 Project: Apache Arrow

[jira] [Created] (ARROW-975) [Website] Publish Roadmap section on website

Wes McKinney created ARROW-975: -- Summary: [Website] Publish Roadmap section on website Key: ARROW-975 URL: https://issues.apache.org/jira/browse/ARROW-975 Project: Apache Arrow Issue Type

Re: Roadmap

hi Edmon, There is no official roadmap yet. For now, JIRA is the point of record: https://issues.apache.org/jira/browse/ARROW-64?jql=project%20%3D%20arrow If you have a well-defined feature or a bug with any of the code, please create an issue there. If you are concerned about whether something

Roadmap

Is there a development/feature roadmap yet for Arrow releases? If so, can we put it on Github/project wiki?