Thanks Marton
I will use this code to implement my testing.
Best regards
Hawin
On Wed, Jun 10, 2015 at 1:30 AM, Márton Balassi <balassi.mar...@gmail.com>
wrote:
> Dear Hawin,
>
> You can pass a hdfs path to DataStream's and DataSet's writeAsText and
> writeAsCsv methods.
> I assume that you are running a Streaming topology, because your source is
> Kafka, so it would look like the following:
>
> StreamExecutionEnvironment env =
> StreamExecutionEnvironment.getExecutionEnvironment();
>
> env.addSource(PerisitentKafkaSource(..))
> .map(/* do you operations*/)
>
> .wirteAsText("hdfs://<namenode_name>:<namenode_port>/path/to/your/file");
>
> Check out the relevant section of the streaming docs for more info. [1]
>
> [1]
> http://ci.apache.org/projects/flink/flink-docs-master/apis/streaming_guide.html#connecting-to-the-outside-world
>
> Best,
>
> Marton
>
> On Wed, Jun 10, 2015 at 10:22 AM, Hawin Jiang <hawin.ji...@gmail.com>
> wrote:
>
>> Hi All
>>
>>
>>
>> Can someone tell me what is the best way to write data to HDFS when Flink
>> received data from Kafka?
>>
>> Big thanks for your example.
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> Best regards
>>
>> Hawin
>>
>>
>>
>
>
