[FlightRPC][Java] HTTP cookie support

2020-10-29 Thread James Duong
Hi all, I've created a PR for changing the Java FlightClient to have the ability to send/receive cookie headers: https://github.com/apache/arrow/pull/8554 I ran into a few apparent bugs in the JDK HttpCookie.parse method though: - The HttpCookie class treats Max-Age=-1 as a sentinel to say th

Re: Serializing nested pandas dataframes

2020-10-29 Thread Adam Lippai
This is what I want to extend for multiple tables: https://issues.apache.org/jira/browse/ARROW-10045?focusedCommentId=17207790&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17207790 I would need to come up with custom binary wrapper for multiple serialized pyarrow

Re: Serializing nested pandas dataframes

2020-10-29 Thread Adam Lippai
If I have a DataFrame with columns Date, Category, Value and group by Category I'll have multiple DataFrames with Date, Value columns. The result of the groupby is DataFrameGroupBy, which can't be serialized. This is why I tried to assemble a nested DataFrame instead (like the one in the SO link pr

Re: Serializing nested pandas dataframes

2020-10-29 Thread Joris Van den Bossche
Can you give a more specific example of what kind of hierarchical data you want to serialize? (eg the output of a groupby operation in pandas typically is still a dataframe that can be converted to pyarrow and serialized). In general, for hierarchical data we have the nested data types (eg struct

Serializing nested pandas dataframes

2020-10-29 Thread Adam Lippai
Hi, is there a way to serialize (IPC) hierarchical tabular data (eg. output of pandas groupby) in python? I've tried to call pa.ipc.serialize_pandas() on this example, but it throws error: https://stackoverflow.com/questions/51505504/pandas-nesting-dataframes Best regards, Adam Lippai

[NIGHTLY] Arrow Build Report for Job nightly-2020-10-29-0

2020-10-29 Thread Crossbow
Arrow Build Report for Job nightly-2020-10-29-0 All tasks: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-10-29-0 Failed Tasks: - conda-win-vs2017-py38: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-10-29-0-azure-conda-win-vs2017-py38 - ga