Wouldnt 2 core-site and hdfs-site xmls need to be provided in this case then ?
Thanks Deepak On Tue, May 22, 2018, 19:34 Raul Valdoleiros < raul.valdoleiros.olive...@gmail.com> wrote: > Hi Kien, > > Thanks for you reply. > > Your goal is to store the checkpoints in one hdfs cluster and the data in > other hdfs cluster. > > So the flink should be able to connect to two different hdfs clusters. > > Thanks > > 2018-05-22 15:00 GMT+01:00 Kien Truong <duckientru...@gmail.com>: > >> Hi, >> >> If your cluster are not high-availability clusters then just use the full >> path to the cluster. >> >> For example, to refer to directory /checkpoint on cluster1, use >> hdfs://namenode1_ip:port/checkpoint >> >> Like wise, /data on cluster2 will be hdfs://namenode2_ip:port/data >> >> >> If your cluster is a HA cluster, then you need to modify the >> hdfs-site.xml like section 1 of this guide >> >> >> https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.4/bk_administration/content/distcp_between_ha_clusters.html >> >> Then use the full path to the cluster hdfs://cluster1ha/checkpoint & >> hdfs://cluster2ha/data >> >> Regards, >> Kien >> >> >> On 5/21/2018 9:19 PM, Raul Valdoleiros wrote: >> >>> Hi, >>> >>> I want to store my data in one hdfs and the flink checkpoints in another >>> hdfs. I didn't find a way to do it, anyone can point me a direction? >>> >>> Thanks in advance, >>> Raul >>> >> >
