Now it runs so well guys, what I have done is read the stream payload with a KafkaIO but using the parameters withoutMetadata and applying Values.String.create() which will return a PCollection<String> and then I can process my payload as simple records.
thanks so much On Wed, May 1, 2019 at 1:38 PM Juan Carlos Garcia <[email protected]> wrote: > It looks correct to me, you can try adding a log4j.properties in your test > resources in order to have some output in your console while run it from > within the IDE. > > Are you sure you can access the Kafka from your machine? > > JC > > On Wed, May 1, 2019 at 5:27 PM Andres Angel < > [email protected]> wrote: > >> Let me update my code guys, here is the new version: >> https://pastebin.com/UdT4D7VW >> >> >> >> On Wed, May 1, 2019 at 11:10 AM Juan Carlos Garcia <[email protected]> >> wrote: >> >>> Hi Andres, >>> >>> You are missing the call to pipeline method *run()* >>> >>> JC >>> >>> On Wed, May 1, 2019 at 4:35 PM Andres Angel < >>> [email protected]> wrote: >>> >>>> Hello everyone, >>>> >>>> Guys I'm trying to consume a kafka topic within my job pipeline, the >>>> main idea is firs read the payload from the kafka topic using KafkaIO, this >>>> read will return a PCollection<KafkaRecord<Long,String>> then I want to >>>> turn this into a PCollection<KV<String,String>> where later I might be able >>>> to read the V and turn it into a PCollection<String>. >>>> >>>> I have designed a first sample code : https://pastebin.com/UdT4D7VW >>>> and at the line 18 I added a print hoping visualize the value payload in >>>> order to create later my final PCollection<String>, however when I run the >>>> pipeline it doesnt print anything so I dont know if is really even >>>> consuming my topic. >>>> >>>> Question: >>>> * How should I debug this to know where I'm? I'm using IntelliJ , Java >>>> 8 and maven 3 >>>> * Am I right on my assumptions to read >>>> the PCollection<KafkaRecord<Long,String>> into PCollection<String> ? >>>> * There is any other better way? >>>> >>>> thanks so much >>>> AU >>>> >>> >>> >>> -- >>> >>> JC >>> >>> > > -- > > JC > >
