Hi Gyula,
will the out of core backend be in a separate maven module? If so, you can
include the module only in the "hadoop-2" profile.
As you can see in the main pom.xml, "flink-yarn" and "flink-fs-tests" are
also "hadoop2" only modules:
<profile>
<id>hadoop-2</id>
<activation>
<property>
<!-- Please do not remove the 'hadoop2' comment. See
./tools/generate_specific_pom.sh -->
<!--hadoop2--><name>!hadoop.profile</name>
</property>
</activation>
<properties>
<hadoop.version>${hadoop-two.version}</hadoop.version>
<shading-artifact.name>flink-shaded-hadoop2</shading-artifact.name>
<shading-artifact-module.name>flink-shaded-hadoop2</shading-artifact-module.name>
</properties>
<modules>
<module>flink-yarn</module>
<!-- Include the flink-fs-tests project only for HD2.
The HDFS minicluster interfaces changed between the two versions.
-->
<module>flink-fs-tests</module>
</modules>
</profile>
If the backend is not in a separate maven module, you can use reflection.
Check out the RollingSink#reflectHflushOrSync() method. It calls "hflush"
only if the method is available ;)
On Fri, Jan 22, 2016 at 10:23 PM, Gyula Fóra <[email protected]> wrote:
> Hi,
>
> While developing the out-of-core state backend that will store state
> directly to hdfs (either TFiles or BloomMapFiles), I realised that some
> file formats and features I use are hadoop 2.x only.
>
> What is the suggested way to handle features that use hadoop 2.x api? Can
> these be excluded from the travis build for the hadoop 1 profile somehow?
>
> Thanks,
> Gyula
>
