Hi everyone,
I need a suggestion regarding usage of the side input pattern and sliding
window, especially while replaying old kafka logs/offsets.
FYI: I am running beam 2.19 on google dataflow.
I have a use case where I read a continuous stream of data from Kafka and
need to calculate one score (
Thank you all for attending today's session! Here is the YT recording:
https://www.youtube.com/watch?v=ZCV9aRDd30U
And link to the slides:
https://github.com/aijamalnk/beam-learning-month/blob/master/Unlocking%20the%20Power%20of%20Apache%20Beam%20with%20Apache%20Flink.pdf
On Tue, May 26, 2020 at 8
Hello,
I am trying to read from, process and write data to HDFS with beam pipelines in
python. Using Flink Runner. Beam version 2.19.0. Flink 1.9.2
My initial code (pasted below my sign) to make sure I can read and write, works
fine on Local Runner. However, I get the following error message (p
Further to that:
At the Flink Job/Task Manager end, I configured/setup the following:
HADOOP_CONF_DIR
HADOOP_USER_NAME
hadoop jars copied under $FLINK_HOME/lib
And made sure a pyflink script is able to read and write into the hdfs system.
Should I setup / configure anything at the Job Server?
I
