Greetings,
I'm trying to do following:
I'm streaming messages from PubSub with encoded json which contain DB
fields like "primary key" as well as "operation type", which can have
values: insert, update, delete to reflect db operations.
Goal is to produce one file per day (with 30 minute updates)
>
>
>> That is fine. We can ignore the timestamp as possible suspect for
>> debugging this. Using custom timestamps from records is normal.
>>
>>
> Could you clarify of what you meant with "withTimestampFn() was never
> meant to
> be public"? I am using it to attach the right timestamp to an elemen
Hi Beamers:
Time to sync with the community on all the awesome stuff we've been doing!
*Add the highlights from October to now (or planned events and talks) that
you want to share by 11/14 11:59 p.m. PDT.*
We will collect the notes via Google docs but send out the final version
directly to the u
RabbitMqIO wasn't merged into Beam until after the 2.8.0 release was cut.
The first release it will appear in is 2.9.0.
Andrew
On Thu, Nov 8, 2018 at 1:10 PM Jeroen Steggink | knowsy
wrote:
> Hi guys,
>
> I tried getting the new RabbitMqIO, however, it's not present in Maven
> Central (http://c
Hello,
RabbitMQ was merged into master after the 2.8.0 release, so you will
have to wait until 2.9.0 is released, or compile/package it by
yourself.
Regards,
Ismaël
On Thu, Nov 8, 2018 at 10:10 PM Jeroen Steggink | knowsy
wrote:
>
> Hi guys,
>
> I tried getting the new RabbitMqIO, however, it's
Hi guys,
I tried getting the new RabbitMqIO, however, it's not present in Maven
Central (http://central.maven.org/maven2/org/apache/beam/), while other
2.8.0 sdks are.
Any reason why?
Best,
Jeroen
Posted my answer on SO, reposting here for visibility:
Note that the HBaseCoderProviderRegistrar[1] already registers the
HBaseMutationCoder for the Mutation type automatically already.
Using the maven-shade plugin without handling service files contained in
META-INF/ inside your output jar is a
to user@beam.
-- Forwarded message -
From: Pramod Rao
Date: Wed, Nov 7, 2018 at 5:59 PM
Subject: Re: [external-thread] at-least once with job changes on Beam
KinesisIO
To: Fei Xue
Cc: , , Parviz Deyhim <
dey...@google.com>, Ryan McDowell
I see that the KinesisIO is checkpoin
On Wed, Nov 7, 2018 at 6:49 PM Raghu Angadi wrote:
>
> On Wed, Nov 7, 2018 at 5:04 AM Kaymak, Tobias
> wrote:
>
>>
>> On Tue, Nov 6, 2018 at 6:58 PM Raghu Angadi wrote:
>>
>>> You seem to be reading from multiple topics and your timestamp policy is
>>> too simplistic (withTimestampFn() was neve
Hi，
I am running into an issue using HBaseIO with FlinkRunner.
The exception is below:
Exception in thread "main" java.lang.IllegalStateException: Unable to
return a default Coder for
ParDo(HBaseProfile)/ParMultiDo(HBaseProfile).output [PCollection]. Correct
one of the following root causes:
No
10 matches
Mail list logo
