Thanks for the update Steve! This is very helpful and I find the blog is a good read too! Appreciate your contribution to the community.
Guozhang On Thu, Aug 27, 2020 at 11:06 AM Steve Jones <jones.ste...@gmail.com> wrote: > Ok so an update here, it's just a learning thing alongside the day job so > it took a little longer than expected. I upgraded to 2.6 but the same > thing happened, the reason is that rocksDB doesn't include the .so for the > Raspberry Pi platform, hence the linker error. Solution was to build the > JNI jar on the Raspberry PI. > > Blogged it so there is a record in case someone else has the challenge. > One 'interesting' piece was that although the application had never > successfully run there was a record of the state somewhere in the broker so > once the linker issue was the resolved it failed because it couldn't access > the RocksDB database in the state that it thought existed, changed the > application.id and it all worked fine. Not sure that counts as a defect > on > the Kafka side and I didn't have time to track down the root cause. > > > http://service-architecture.blogspot.com/2020/08/getting-rocksdb-working-on-raspberry-pi.html > > On Mon, 24 Aug 2020 at 20:53, John Roesler <vvcep...@apache.org> wrote: > > > Ah, yes, before messing with rocksjni. As Sophie said, you'll probably > > just want to try using 2.6.0. It should work "out of the box". > > > > Thanks, > > -John > > > > On Mon, Aug 24, 2020, at 21:08, Steve Jones wrote: > > > Version: *kafka_2.12-2.5.0* > > > > > > RocksDB compiles fine, Kafka compiles fine, I'm just trying to work out > > > where the linking is done and if that can be changed to a static rather > > > than a dynamic link. Any hints appreciated, and I'll document out what > > > I've found. Its very stable on the Raspberry Pi as a broker, and for > > > streams if the streams is run on another machine. > > > > > > Steve > > > > > > On Mon, 24 Aug 2020 at 17:49, Sophie Blee-Goldman <sop...@confluent.io > > > > > wrote: > > > > > > > Yeah, if you weren't already running it, try upgrading Streams to > 2.6. > > They > > > > recently > > > > added support for "all" platforms to rocksdb and we updated the > > dependency > > > > to > > > > get this fix in 2.6. See KAFKA-9225 > > > > <https://issues.apache.org/jira/browse/KAFKA-9225> > > > > > > > > If you already were running 2.6, then, that's unfortunate. You might > > have > > > > some luck > > > > asking the rocksdb folks if all else fails > > > > > > > > On Mon, Aug 24, 2020 at 5:46 PM John Roesler <vvcep...@apache.org> > > wrote: > > > > > > > > > Hi Steve, > > > > > > > > > > Which version of Streams is this? I vaguely recall that we updated > > to a > > > > > version of Rocks that’s compiled for ARM, and I think some people > > have > > > > used > > > > > it on ARM, but I might be misremembering. > > > > > > > > > > I’m afraid I can’t be much help in debugging this, but maybe some > > others > > > > > on the list have more context. If all else fails, you can probably > > narrow > > > > > it down to the Java RocksDB library. If you create a standalone > java > > > > > program using the same rocks dependency that we do, then you can > > pretty > > > > > confidently raise it with the Rocks folks. > > > > > > > > > > If you want to side-step the issue while debugging this, plugging > in > > a > > > > > different store implementation (like the in-memory one) would > > probably > > > > > work, although I’m sure memory is scarce on a Raspberry Pi. > > > > > > > > > > I hope this helps, > > > > > -John > > > > > > > > > > On Mon, Aug 24, 2020, at 19:26, Steve Jones wrote: > > > > > > I'm trying to install Kafka Streams on a Raspberry PI, it works > > fine > > > > as a > > > > > > broker, works fine as both a producer and consumer, but when I > try > > and > > > > > run > > > > > > streams on the PI rather than on the Mac there is a linker issue: > > > > > > > > > > > > Exception in thread > > > > > > "main-broker-f53264a1-0c70-445f-bf3f-bf634a9a1ed2-StreamThread-1" > > > > > > java.lang.UnsatisfiedLinkError: > > > > > /tmp/librocksdbjni15158764823832728522.so: > > > > > > /tmp/librocksdbjni15158764823832728522.so: cannot open shared > > object > > > > > file: > > > > > > No such file or directory (Possible cause: can't load IA 32 .so > on > > a > > > > ARM > > > > > > platform) > > > > > > > > > > > > at java.base/java.lang.ClassLoader$NativeLibrary.load0(Native > > Method) > > > > > > > > > > > > at > > > > > > > java.base/java.lang.ClassLoader$NativeLibrary.load(ClassLoader.java:2452) > > > > > > > > > > > > at > > > > > > > > > > > > > > > > > > java.base/java.lang.ClassLoader$NativeLibrary.loadLibrary(ClassLoader.java:2508) > > > > > > > > > > > > at > > java.base/java.lang.ClassLoader.loadLibrary0(ClassLoader.java:2704) > > > > > > > > > > > > at > > java.base/java.lang.ClassLoader.loadLibrary(ClassLoader.java:2637) > > > > > > > > > > > > > > > > > > I've recompiled rocksdb on the RaspberryPI and added that to the > > > > loadpath > > > > > > but the same error occurs. I've done the Google search but not > > found > > > > > > anything around streams on ARM (Raspberry PI) and what needs to > be > > > > > > recompiled/linked for this to work. > > > > > > > > > > > > > > > > > > Help appreciated. > > > > > > > > > > > > > > > > > > Steve Jones > > > > > > > > > > > > > > > > > > > > > -- -- Guozhang
