Hi Yang,
I've just tried your suggestions, but, unfortunately, in yarn per job mode
it doesn't work, both commands return null.
I double checked that file is shipped to yarn container, but I feel that it
happens later in process.
At the moment I'm reading file with File interface, instead of getting it
as resource, which I do in local mode.
Regards,
Maxim.
On Mon, Feb 17, 2020 at 3:03 PM Yang Wang <danrtsey...@gmail.com> wrote:
> Hi Maxim,
>
> I have verified that the following two ways could both work.
>
> getClass().getClassLoader().getResource("lib/job.properties")
> getClass().getClassLoader().getResource("job.properties")
>
>
> Best,
> Yang
>
> Maxim Parkachov <lazy.gop...@gmail.com> 于2020年2月17日周一 下午6:47写道：
>
>> Hi Yang,
>>
>> thanks, this explains why classpath behavior changed, but now I struggle
>> to
>> understand how I could overwrite resource, which is already shipped in
>> job jar.
>>
>> Before I had job.properties files in JAR in under
>> resources/lib/job.properties
>> for local development and deploying on cluster it was overwritten
>> with environment specific settings in lib/job.properties of flink
>> distribution.
>> Now this doesn't seem to work. I'm using:
>>
>> getClass.getClassLoader.getResource("lib/job.properties")
>>
>> to get file. Could it be the problem ?
>>
>> Thanks,
>> Maxim.
>>
>> On Mon, Feb 17, 2020 at 4:12 AM Yang Wang <danrtsey...@gmail.com> wrote:
>>
>>> Hi Maxim Parkachov,
>>>
>>> The users files also have been shipped to JobManager and TaskManager.
>>> However, it
>>> is not directly added to the classpath. Instead, the parent directory is
>>> added to the
>>> classpath. This changes are to make resource classloading work. You
>>> could check more
>>> information here[1].
>>>
>>>
>>> [1]. https://issues.apache.org/jira/browse/FLINK-13127
>>>
>>>
>>> Best,
>>> Yang
>>>
>>> Maxim Parkachov <lazy.gop...@gmail.com> 于2020年2月15日周六 上午12:58写道：
>>>
>>>> Hi everyone,
>>>>
>>>> I'm trying to run my job with flink 1.10 with YARN cluster per-job
>>>> mode. In the previous versions all files in lib/ folder were automatically
>>>> included in classpath. Now, with 1.10 I see only *.jar files are included
>>>> in classpath. but not "other" files. Is this deliberate change or bug ?
>>>>
>>>> Generally, what is recommended way to include custom files in classpath
>>>> and ship it during start to all containers ?
>>>>
>>>> Thank
>>>>
>>>
