Okay, I solved the other issue with viewing logs which proved that correct,
non-null values are being loaded. I believe I have a different issue
altogether so will create a separate thread for that. Thanks for the help
Chesnay!
On Thu, Oct 22, 2020 at 11:30 AM Manas Kale <manaskal...@gmail.com> wrote:
> Hi Chesnay,
> The Config reader has everything static, so I tried using
>
> Config.class.getClassLoader().getResourceAsStream("pipeline.properties");
>
> Here's the .pom file for that file:
>
> <build>
>
> <resources>
> <resource>
> <directory>src/main/resources</directory>
> <includes>
> <include>pipeline.properties</include>
> <include>log4j.properties</include>
> </includes>
> </resource>
> </resources>
>
> ...
>
> </build>
>
>
> I can see the pipeline.properties packaged in my JAR at the root level
> (using maven package command).
> However, this works on my IDE minicluster but loads null values when
> submitted to the cluster.
> The Config class is not at the package root, rather it is a few levels
> deep. Does that make a difference and cause the getClassLoader() to treat
> an inner package as root?
>
>
>
> On Wed, Oct 21, 2020 at 6:06 PM Chesnay Schepler <ches...@apache.org>
> wrote:
>
>> You could bundle said file in the jar and retrieve it via
>> getClass().getClassLoader().getResource("<file name>").
>>
>> On 10/21/2020 2:24 PM, Manas Kale wrote:
>> > Hi,
>> > I have a Flink job that I am packaging as a JAR that is submitted to
>> > the Flink cluster runtime. However, this JAR reads a few configuration
>> > values from a .properties file.
>> > What is the recommended way to package this properties file when
>> > submitting to a cluster? Do I have to copy it to a folder in my flink
>> > cluster installation?
>> >
>> > My own attempt is a somewhat convoluted method that is not working.
>> > Basically I set an environment variable that points to the properties
>> > file, and I use that at runtime to read configuration values. This
>> > works when I run it in my IDE as a minicluster but fails when I submit
>> > it to the cluster. I'm kind of stuck debugging this as for some reason
>> > I am not able to see the logs from the configuration reader class
>> > (asked a question about that in a separate thread).
>>
>>
>>
