I'm writing a Flink processor that will read a bunch of JSON records from
Kafka and then write them to S3 in parquet format using the FileSink. I've
got most things in place, the only thing I haven't been able to figure out
is how to change the compression codec used by the writer. Is there any
rec
ate). This will
> automatically checkpoint the state. In case of a failure Flink will always
> resume from the latest successfully completed checkpoint.
>
> Cheers,
> Till
>
> On Thu, Apr 29, 2021 at 12:25 PM Tim Josefsson
> wrote:
>
>> Thanks for the suggestions! I&#
ent a trigger which
> fires when it sees a certain event or after some time.
>
> [1]
> https://ci.apache.org/projects/flink/flink-docs-stable/dev/stream/operators/windows.html#triggers
>
> Cheers,
> Till
>
> On Wed, Apr 28, 2021 at 5:25 PM Tim Josefsson
> wrote:
>
>>
Hello!
I'm trying to figure out how to implement a window that will emit events at
regular intervals or when a specific event is encountered.
A bit of background. I have a stream of events from devices that will send
events to our system whenever a user watches a video. These events include
a uni
0
transaction.state.log.replication.factor=1
transaction.state.log.min.isr=1
On Mon, 16 Nov 2020 at 14:05, Tim Josefsson
wrote:
> Hello!
>
> I'm having some problems with my KafkaProducer that I've been unable to
> find a solution to.
>
> I've set up a simple Flink Job
ally set on the Kafka
brokers as well to rule out any mistakes there.
Best,
Tim
On Thu, 12 Nov 2020 at 14:46, Tim Josefsson
wrote:
> Also realized I had a typo in the config dump I did in the previous email
> (the one from the 10th). If I don't do
> Properties p
60 and not 6 as
I had written.
On Thu, 12 Nov 2020 at 13:37, Tim Josefsson
wrote:
> Sure, I've attached it to this email. The process seems to restart once
> the TimeoutException happens so it's repeated a couple of times.
>
> Thanks for looking at it!
>
> /T
leads to the
> TimeoutException?
>
> Best,
> Aljoscha
>
> On 10.11.20 17:54, Tim Josefsson wrote:
> > Hey Aljoscha,
> >
> > I'm setting the transaction.timeout.ms when I create the
> FlinkKafkaProducer:
> >
> > I create a Properties object and the
4e33125ef858-1
value.serializer = class
org.apache.kafka.common.serialization.ByteArraySerializer
So I imagine the Producer is picking up the change but it still returns
errors when running the job.
Best regards,
Tim
On Tue, 10 Nov 2020 at 16:14, Aljoscha Krettek wrote:
> On 10.11.20 11:53, Tim Josefsson wrote:
> > Also when checking my
