Hi again,
try to do this without ignite, looks like there's a problem on hive side.
<property>
<name>fs.defaultFS</name>
<value>hdfs://localhost:9000</value>
</property>
this means that you uses Hadoop directly, to make hive work with IGFS
you need to change hdfs to igfs:
<property>
<name>fs.defaultFS</name>
<value>igfs://localhost:9000</value>
</property>
Thanks,
Mikhail.
On Wed, Aug 2, 2017 at 12:35 PM, csumi <[email protected]> wrote:
> Ignite-hadoop accelerator is configured as
> https://apacheignite.readme.io/v1.0/docs/hadoop-
> accelerator#section-secondary-file-system.
>
> Idea is to run hive queries on IGFS but looks like its not working that
> way.
> Not sure how to confirm if hive is connecting to HDFS or IGFS. Also, I
> think
> creating partition by passing comma separated string to fs.append is not
> creating partition correctly which is resulting in no result on hive query.
>
> Below are some of the configurations I have. Please let me know if I need
> to
> share any more details.
>
> core-site.xml is having these properties:
>
> <property>
> <name>fs.defaultFS</name>
> <value>hdfs://localhost:9000</value>
> </property>
>
>
> <property>
> <name>fs.igfs.impl</name>
> <value>org.apache.ignite.hadoop.fs.v1.
> IgniteHadoopFileSystem</value>
> </property>
> <property>
> <name>fs.AbstractFileSystem.igfs.impl</name>
> <value>org.apache.ignite.hadoop.fs.v2.
> IgniteHadoopFileSystem</value>
> </property>
> <property>
>
> <name>dfs.client.block.write.replace-datanode-on-failure.policy</name>
> <value>NEVER</value>
> </property>
>
> ignite's default-config has below configuration
>
> <property name="secondaryFileSystem">
> <bean
> class="org.apache.ignite.hadoop.fs.IgniteHadoopIgfsSecondaryFileSystem">
> <property name="fileSystemFactory">
> <bean
> class="org.apache.ignite.hadoop.fs.CachingHadoopFileSystemFactory">
> <property name="uri"
> value="hdfs://localhost:9000/"/>
>
> <property name="configPaths">
>
> <list>
>
> <value>D:/hadoop/etc/hadoop/core-site.xml</value>
>
> </list>
>
> </property>
> </bean>
> </property>
> </bean>
> </property>
>
>
>
>
>
>
>
> --
> View this message in context: http://apache-ignite-users.
> 70518.x6.nabble.com/how-to-append-data-to-IGFS-so-that-
> data-gets-saved-to-Hive-partitioned-table-tp15725p15887.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>
--
Thanks,
Mikhail.
