Re: Writing to Avro from pyflink

2021-04-26 Thread Edward Yang
Hi Dian, Thanks for trying it out, it ruled out a problem with the python code. I double checked the jar path and only included the jar you referenced without any luck. However, I tried creating a python 3.7 (had 3.8) environment for pyflink and the code worked without any errors! On Sun, Apr 25

Re: Writing to Avro from pyflink

2021-04-25 Thread Dian Fu
Hi Eddie, I have tried your program with the following changes and it could execute successfully: - Replace `rf"file:///{os.getcwd()}/lib/flink-sql-avro-1.12.2.jar;file:///{os.getcwd()}/lib/flink-avro-1.12.2.jar;file:///{os.getcwd()}/lib/avro-1.10.2.jar”` with rf`"file:///Users/dianfu/code/src

Re: Writing to Avro from pyflink

2021-04-25 Thread Edward Yang
Hi Dian, I tried your suggestion but had the same error message unfortunately. I also tried file:/ and file:// with the same error, not sure what's going on, I assume writing to avro works fine in java and scala? Eddie On Sat, Apr 24, 2021 at 10:03 PM Dian Fu wrote: > I guess you only need fil

Re: Writing to Avro from pyflink

2021-04-24 Thread Dian Fu
I guess you only need file:///{os.getcwd()}/lib/flink-sql-avro-1.12.2.jar . Could you remove flink-avro-1.12.2.jar and avro-1.10.2.jar and try again? Regards, Dian > 2021年4月24日 上午8:29,Edward Yang 写道: > > I've been trying to write to the avro format with pyflink 1.12.2 on ubuntu, > I've test

Writing to Avro from pyflink

2021-04-23 Thread Edward Yang
I've been trying to write to the avro format with pyflink 1.12.2 on ubuntu, I've tested my code with an iterator writing to csv and everything works as expected. Reading through the flink documentation I see that I should add jar dependencies to work with avro. I downloaded three jar files that I b