Unsubscribe
This is resolved by the first approach I mentioned.
Thanks Team
On Mon, 30 Aug 2021 at 12:35, tarun joshi <1985.ta...@gmail.com> wrote:
> An update on this , I see that `IndexedRecord` is part of Avro Library.
> Please correct me If I am wrong in assuming that the "Pojo'
rom AWS Kinesis directly into
Avro.
- *Or* convert those GSON parsed POJOs into Avro compatible POJOs at
Stream-time.
Please let me know if anyone has a better way to do this.
On Mon, 30 Aug 2021 at 10:13, tarun joshi <1985.ta...@gmail.com> wrote:
> Hey all,
> I am trying to writ
Hey all,
I am trying to write a simple pipeline to read
Read Stringified JSON from Kinesis -> parsed to POJO -> converted to Avro
-> for the purpose of writing Parquet files to AWS S3.
1) This is my SimpleMapper
public class SimpleMapper extends RichMapFunction {
private static final GsonBui
Hey All,
I am running flink in docker containers (image Tag
:flink:scala_2.11-java11) on EC2.
I am able to connect to a Kinesis Connector but nothing is being consumed.
My command to start Jobmanager and TaskManager :
*docker run \--rm \--volume /root/:/root/ \--env
JOB_MANAGER_RPC_ADDR
ths, specifically when the Flink FileSystem class is used.
> Since you are using Hadoop directly you are side-stepping the plugin
> mechanism.
>
> So you have to make sure that Hadoop + Hadoop's S3 filesystem is available
> to the client.
>
> On 06/08/2021 08:02, tarun joshi wr
Hey All,
I am running flink in docker containers (image Tag
:flink:scala_2.11-java11) on EC2 and getting exception as I am trying to
submit a job through the local ./opt/flink/bin
*org.apache.flink.client.program.ProgramInvocationException: The main
method caused an error: No FileSystem for schem
