. We had more issues with storm 2,
> mostly because of way how we are using and configured it and we ended with
> staying with Storm 1.2.3.
> I wonder why only we observed this issue. Like you, we also have custom
> kryo serializers used.
>
> Best regards,
> Michal
>
> ---
Hi Julien,
Unfortunately I have no fix for that. We had more issues with storm 2, mostly
because of way how we are using and configured it and we ended with staying
with Storm 1.2.3.
I wonder why only we observed this issue. Like you, we also have custom kryo
serializers used.
Best regards
)
~[kryo-3.0.3.jar:?] at
com.esotericsoftware.kryo.io.Input.readUtf8_slow(Input.java:575)
~[kryo-3.0.3.jar:?] at
com.esotericsoftware.kryo.io.Input.readUtf8(Input.java:553)
~[kryo-3.0.3.jar:?] at
com.esotericsoftware.kryo.io.Input.readString(Input.java:483)
~[kryo-3.0.3.jar:?]*
whereas others are
orm/blob/master/storm-client/src/jvm/org/apache/storm/messaging/DeserializingConnectionCallback.java#L39
>
> Looks quite good for me. There should be one Kryo instance per thread.
>
>
> But I'm less sure about other serialization / deserialization parts:
>
> https://githu
There should be one Kryo instance per thread.
But I'm less sure about other serialization / deserialization parts:
https://github.com/apache/storm/search?p=1&q=KryoValuesDeserializer&unscoped_q=KryoValuesDeserializer
<https://github.com/apache/storm/search?q=KryoValue
I don't think we've changed Kryo stuff much since 1.0.0. Last version
upgrade was https://github.com/apache/storm/pull/1255.
Maybe it could be a threading issue in the Netty messaging implementation
in
https://github.com/apache/storm/tree/master/storm-client/src/jvm/org/apache/storm
com.esotericsoftware.kryo.util.DefaultClassResolver.readName(DefaultClassResolver.java:156)
~[kryo-3.0.3.jar:?] at
com.esotericsoftware.kryo.util.DefaultClassResolver.readClass(DefaultClassResolver.java:133)
~[kryo-3.0.3.jar:?] at com.esotericsoftware.kryo.Kryo.readClass(Kryo.java:670)
~[kryo-3.0.3
com.esotericsoftware.kryo.util.DefaultClassResolver.readName(DefaultClassResolver.java:156)
~[kryo-3.0.3.jar:?] at
com.esotericsoftware.kryo.util.DefaultClassResolver.readClass(DefaultClassResolver.java:133)
~[kryo-3.0.3.jar:?] at com.esotericsoftware.kryo.Kryo.readClass(Kryo.java:670)
~[kryo-3.0.3
n't answer your question, but if you've written a serializer
>>> that could potentially be used by others I encourage you to add it to
>>> this repository: https://github.com/magro/kryo-serializers
>>>
>>> On Mon, Nov 9, 2015 at 1:07 PM, Sean Bol
; On 11/09/2015 10:13 AM, Stephen Powis wrote:
>> Sorry I can't answer your question, but if you've written a serializer
>> that could potentially be used by others I encourage you to add it to
>> this repository: https://github.com/magro/kryo-serializers
>>
>> O
f you've written a serializer
> that could potentially be used by others I encourage you to add it to
> this repository: https://github.com/magro/kryo-serializers
>
> On Mon, Nov 9, 2015 at 1:07 PM, Sean Bollin wrote:
>> Hi, we've written a custom Kryo serializer. As far
Sorry I can't answer your question, but if you've written a serializer
that could potentially be used by others I encourage you to add it to
this repository: https://github.com/magro/kryo-serializers
On Mon, Nov 9, 2015 at 1:07 PM, Sean Bollin wrote:
> Hi, we've written a cust
Hi, we've written a custom Kryo serializer. As far as I can tell,
Storm doesn't use serialization if you're running it in local mode.
Yet, we need to be able to test our custom Kryo serializer, preferably locally.
Is it possible to test the serializer locally to ensure that it actually works?
up to a certain point, and the Kryo underflow error
would start throwing in the second bolt.
The implementation:
Bolt2.java
public void execute(Tuple tuple, BasicOutputCollector collector) {
if (tuple.size() > 0) {
try {
SimplePojo sources = new SimplePojo((SimplePojo) tuple.getValu
g), I get this
> buffer underflow exception in Bolt3 when storm deserializes.
> I also just discovered that this does not happen if I put the additional
> string field in the middle and not at the end.
>
> Does that help? My sense is that this is probably a kryo bug.
>
> Appreciate
erflow exception in Bolt3 when storm deserializes.
I also just discovered that this does not happen if I put the additional
string field in the middle and not at the end.
Does that help? My sense is that this is probably a kryo bug.
Appreciate you taking the time.
On Thu, Mar 19, 2015 at 5:0
gt;
> On Thu, Mar 19, 2015 at 4:20 PM Vladimir Protsenko
> wrote:
> Hi, I've got once the same error when in deserialization was reading the
> wrong number of bytes that were sent in serialization code.
>
> 2015-03-19 19:01 GMT+04:00 Luke Rohde :
> Hi, I'm see
ere sent in serialization code.
>
> 2015-03-19 19:01 GMT+04:00 Luke Rohde :
>
>> Hi, I'm seeing this exception from kryo deep inside storm and I haven't
>> been able to figure it out. Forums/docs on kryo seem to mention that this
>> can happen if you use a kryo insta
Hi, I've got once the same error when in deserialization was reading the
wrong number of bytes that were sent in serialization code.
2015-03-19 19:01 GMT+04:00 Luke Rohde :
> Hi, I'm seeing this exception from kryo deep inside storm and I haven't
> been able to figure it out
Hi, I'm seeing this exception from kryo deep inside storm and I haven't
been able to figure it out. Forums/docs on kryo seem to mention that this
can happen if you use a kryo instance in multiple threads concurrently, but
I'm not doing that (and this appears to be coming from sto
Yes, it uses kryo. When you emit something in storm it will use kryo
underneath as well. If you cannot solve these by today lets meet Thursday.
Unfortunately i'm traveling right now and cannot meet you tomorrow.
Thanks,
Supun..
On Jan 27, 2015 9:12 AM, "hjh" wrote:
> Hi
Hi,
When using Utils.serialize/deserialize to serialize/deserialize
tuples does it use kryo? If so what serializer it uses? And when
registering serialization in the config, is it actually register the
class in the kryo? Thank you very much!!
Sorry my head was in a total mess after getting into this problem.
After hours' working, I found out that it was caused by a bug in my
code. Thank you anyway!!
On 01/23/2015 03:38 PM, Curtis Allen wrote:
Says right in your stack trace. Kryo - 2.21
On Fri, Jan 23, 2015 at 11:28 AM
Says right in your stack trace. Kryo - 2.21
On Fri, Jan 23, 2015 at 11:28 AM, hjh wrote:
> Hi Dose anyone know what version of kryo storm currently use? Because I
> got exception when use kryo to serialize class.
>
> 18736 [Thread-30-pose_array_receiver] ERROR
> backtype.storm.
Hi Dose anyone know what version of kryo storm currently use? Because I
got exception when use kryo to serialize class.
18736 [Thread-30-pose_array_receiver] ERROR
backtype.storm.daemon.executor -
com.esotericsoftware.kryo.KryoException: Encountered unregistered class
ID: 11
Serialization
Hi,
I use kryo to serialize and deserialize class defined by myself(all
registered in kryo), but when I run my topology, I got this exception as
shown below. I googled for this issue it seems to be kryo version that
caused this problem. I checked the version of the kryo I use, it is
2.21
> wrote:
>
>> Your problem was answered in Stack Overflow
>> http://stackoverflow.com/questions/24244689/kryo-compatibility-in-storm-0-9-1and-titan-graph-database-0-4-2-with-apache-cass/24246845#24246845
>>
>> Titan 0.4.2 + Storm 0.9.2 are compatible.
>> Storm 0.9.2
I saw that on SO and wondered how folks had solved the issue, did you run
into any issues with 2.22 on the classpath?
Nick
On Thu, Oct 9, 2014 at 4:10 PM, Curtis Allen
wrote:
> Your problem was answered in Stack Overflow
> http://stackoverflow.com/questions/24244689/kryo-compatibility-in
Your problem was answered in Stack Overflow
http://stackoverflow.com/questions/24244689/kryo-compatibility-in-storm-0-9-1and-titan-graph-database-0-4-2-with-apache-cass/24246845#24246845
Titan 0.4.2 + Storm 0.9.2 are compatible.
Storm 0.9.2 is *not* compatible with Titan 0.5.0
To fix this
We just started using 0.5, so
It was a non-prod environment, and we reloaded the data.
(I¹m sure that is no help at all)
Definitely check your classpath for the different kryo versions though.
Titan has a kryo as a dependency and it conflicts with Storm¹s.
The classpath may be different when
ored data with 0.4, and you are trying to read it with 0.5
> (from Storm), you may run into Kryo version issues.
>
> -brian
>
> ---
>
> Brian O'Neill
>
> Chief Technology Officer
>
>
> *Health Market Science*
>
> *The Science of Better Results*
&g
We ran into the same problem.
What version of Titan was used to load the graph database?
If you¹ve stored data with 0.4, and you are trying to read it with 0.5 (from
Storm), you may run into Kryo version issues.
-brian
---
Brian O'Neill
Chief Technology Officer
Health Market Scienc
Hi all,
I have a peculiar problem...
I'm using a kafka storm combo to feed updates to my Titan Graph db. The
topology runs quite happily when local but as soon as I try to run it on a
cluster I run into Kryo errors.
2014-10-09 18:41:47 b.s.util [ERROR] Async loop
You might find this interesting
http://stackoverflow.com/questions/24244689/kryo-compatibility-in-storm-0-9-1and-titan-graph-database-0-4-2-with-apache-cass/24246845#24246845
On Wed, Oct 8, 2014 at 10:16 AM, Nathan Leung wrote:
> 0.9.2 release upgrades to kryo 2.21.
>
> On Wed, Oct 8
0.9.2 release upgrades to kryo 2.21.
On Wed, Oct 8, 2014 at 12:13 PM, Klausen Schaefersinho <
klaus.schaef...@gmail.com> wrote:
> Hi,
>
> Every time I want to test a simple topology I get the following error
> which seems to be caused by different versions of the Kryo depe
Hi,
Every time I want to test a simple topology I get the following error which
seems to be caused by different versions of the Kryo dependency. I have a
dependency on my code on kryo (2.17), which I thought is the same version
as Storm. But obviously not...
Here is the exception
36 matches
Mail list logo
