Hi
I just want to let you know, that I did implement a version for wrapping
properties in the cluster:
object PropertiesWrapper {
def getFromConfig(configFile: String): ParameterTool = {
val prop: Properties = new Properties()
var input: InputStream = null
try {
input = PropertiesWrapper.getClass.getResourceAsStream("/" + configFile);
// load a properties file
prop.load(input)
} finally {
if (input != null) {
input.close();
}
}
ParameterTool.fromMap(prop.asInstanceOf[Map[String,String]])
}
def mergeArgsWithProperties(configuration:Properties, args:
Array[String]):Properties = {
configuration
}
}
Have a nice day
--Simon
> On 24 May 2016, at 01:51, Bajaj, Abhinav <[email protected]> wrote:
>
> I was gonna post the exact question and noticed this thread.
>
> It will be great if we can have a method in parameter tool to load from
> resources.
>
> Thanks Simon :)
>
> Abhinav Bajaj
> Senior Engineer
> HERE Predictive Analytics
> Office: +12062092767
> Mobile: +17083299516
> HERE Seattle
> 701 Pike Street, #2000, Seattle, WA 98101, USA
> 47° 36' 41" N. 122° 19' 57" W
> HERE Maps
>
>
>
>
>
>
>
>
>
>
>
> On 5/23/16, 8:54 AM, "simon peyer" <[email protected]> wrote:
>
>> Hi Max
>>
>> Thanks a lot for your helpful answer.
>> It now works on the cluster.
>> It would be great to have a method for loading from resources.
>>
>> -Cheers
>> Simon
>>
>>
>>> On 23 May 2016, at 17:52, Maximilian Michels <[email protected]> wrote:
>>>
>>> Hi Simon,
>>>
>>> AFAIK this is the way to go. We could add a method to the
>>> ParameterTool which loads from a resource to make it more convenient.
>>>
>>> Cheers,
>>> Max
>>>
>>> On Mon, May 23, 2016 at 4:42 PM, simon peyer <[email protected]> wrote:
>>>> Hi
>>>>
>>>> @Max
>>>> So for each file in the src/main/resources folder, I first have to create a
>>>> new file, copy the file from the resources folder to this new file and then
>>>> I'm able to parse it?
>>>>
>>>> @Stefano
>>>> I think the files in src/main/resources are integrated automatically
>>>> right?
>>>> Or am I missing something?
>>>>
>>>> Cheers
>>>> Simon
>>>>
>>>>
>>>> On 23 May 2016, at 16:30, Stefano Baghino <[email protected]>
>>>> wrote:
>>>>
>>>> Are you using Maven to package your project? I believe the resources
>>>> plugin[1] can suit your needs.
>>>>
>>>> [1]:
>>>> http://maven.apache.org/plugins/maven-resources-plugin/examples/include-exclude.html
>>>>
>>>>
>>>> On 23 May 2016, at 16:28, Maximilian Michels <[email protected]> wrote:
>>>>
>>>> Hi Simon,
>>>>
>>>> You'll have to write the property file to disk first to load it using
>>>> the ParameterTool.fromPropertiesFile method.
>>>>
>>>> For example:
>>>>
>>>> // copy config from Java resource to a file
>>>> File configOnDisk = new File("/path/to/config.properties");
>>>> Files.copy(getClass.getClassLoader.getResourceAsStream("config.properties"),
>>>> configOnDisk.toPath());
>>>> // load the new file
>>>> ParameterTool.fromPropertiesFile(configOnDisk);
>>>>
>>>>
>>>> Cheers,
>>>> Max
>>>>
>>>>
>>>>
>>>> On Mon, May 23, 2016 at 3:56 PM, simon peyer <[email protected]>
>>>> wrote:
>>>>>
>>>>> Hi together
>>>>>
>>>>> Currently I'm using flink on a docker cluster on AWS.
>>>>> I would like to use property files with the integrated
>>>>> ParameterTool.fromPropertiesFile function of Flink.
>>>>>
>>>>> Locally this version works absolutely fine:
>>>>> val configuration =
>>>>> ParameterTool.fromPropertiesFile("src/main/resources/config.properties")
>>>>>
>>>>> But on the cluster this didn't work, so we introduced this version, which
>>>>> also doesn't work:
>>>>>
>>>>> val baseParameters =
>>>>> ParameterTool.fromPropertiesFile(getClass.getClassLoader.getResource("config.properties").getFile)
>>>>>
>>>>> gives
>>>>>
>>>>> java.io.FileNotFoundException: Properties file
>>>>> file:/tmp/flink-web-upload-57bcc912-bc98-4c89-b5ee-c5176155abd5/992186c1-b3c3-4342-a5c8-67af133155e4pipeline-0.1.0-all.jar!/config.properties
>>>>> does not exist
>>>>>
>>>>> The property file is located in src/main/resources.
>>>>> Do you have any idea of how to integrate such property files into the jar
>>>>> package?
>>>>>
>>>>> -Thanks
>>>>> Simon
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> BR,
>>>> Stefano Baghino
>>>>
>>>> Software Engineer @ Radicalbit
>>>>
>>>>
>>
