It was probably lost in the wall of text, but the point I was trying to
convey was that fetching all rows via protobuf crashed PQS, while fetching
it via JSON didn't. Basically, the only way I can get protobuf
serialization to work is to 1. bump up the PQS memory and/or 2. fetch in
batches, while I
Manoj,
re: #2 Please start by reading:
https://plumbr.io/outofmemoryerror/gc-overhead-limit-exceeded
This should give you a thorough explanation of what this error means.
You need to increase the memory footprint and/or JVM GC properties for
PQS to address your issue. When you request all of
Thanks for the pointers Josh.
Here’s a set of instructions to demonstrate the performance issues I’m
seeing when using protobuf serialization. Hopefully these are comprehensive
enough to reproduce the issue. If not, please let me know and I’m happy to
provide clarifications.
Basic outline of the
On 11/5/18 10:10 PM, Manoj Ganesan wrote:
Thanks for the pointers Josh. I'm working on getting a representative
concise test to demonstrate the issue.
Meanwhile, I had one question regarding the following:
You are right that the operations in PQS should be exactly the same,
regardles
Thanks for the pointers Josh. I'm working on getting a representative
concise test to demonstrate the issue.
Meanwhile, I had one question regarding the following:
You are right that the operations in PQS should be exactly the same,
> regardless of the client you're using -- that is how this arch
Is the OOME issue regardless of using the Java client (sqlline-thin) and
the Python client? I would like to know more about this one. If you can
share something that reproduces the problem for you, I'd like to look
into it. The only suggestion I have at this point in time is to make
sure you se
Thanks Josh for the response!
I would definitely like to use protobuf serialization, but I'm observing
performance issues trying to run queries with a large number of results.
One problem is that I observe PQS runs out of memory, when its trying to
(what looks like to me) serialize the results in
I would strongly suggest you do not use the JSON serialization.
The JSON support is implemented via Jackson which has no means to make
backwards compatibility "easy". On the contrast, protobuf makes this
extremely easy and we have multiple examples over the past years where
we've been able to
Thanks, Dimitri. A pull request would be great. Looks like the outstanding
items are:
- Fixing licensing
- Deciding on package structure. FWIW, the phoenix-spark module has scala
code, so this is a little bit of precedence for non Java code.
- Tweaking our release process/scripts to handle Python a
First off, I think we need to figure out which version of "phoenixdb" we
want to include. Technically, I don't know which would be preferred: DB
API 2.0 or SQLAlchemy. Do we have to choose just one? I can only say
that I know of users have been using Lukas' variant but I don't know if
that's be
Hi James,
How would you prefer the integration into the Phoenix project to be
handled? I'm more than happy to do it, but I don't see a clear path.
Licensing is not an issue, since it's using the same license as
Phoenix itself. I guess I'd just need to sign the CLA?
Adding it to the Phoenix sourc
Thanks, Dimitri! If you're interested, I think the community would welcome
this as a contribution to Apache Phoenix so it can appear in our regular
distributions.
Regards,
James
On Thu, Mar 23, 2017 at 5:21 PM, Dimitri wrote:
> Hi,
>
> I make some update on my fork of Lukáš Lalinský phoenix pyt
Awesome, Lukas! Thanks so much - this is great!
On Thu, Nov 3, 2016 at 2:14 PM, Lukáš Lalinský wrote:
> Hello,
>
> I have released new version of the Python client library for working with
> Phoenix. Thanks to Mark Heppner it now uses the Protocol Buffers interface
> instead of JSON, which fixes
13 matches
Mail list logo
