Ken, this is specifically about running integration tests and not a users
main().
Note, that PipelineOptions JSON format was used because it was a convenient
serialized form that is easy to explain to people what is required.
Using a different string tokenizer and calling
PipelineOptionsFactory.fromArgs() with the parsed strings seems like it
would be better.
These are the supported formats for fromArgs():
* --project=MyProject (simple property, will set the "project"
property to "MyProject")
* --readOnly=true (for boolean properties, will set the "readOnly"
property to "true")
* --readOnly (shorthand for boolean properties, will set the
"readOnly" property to "true")
* --x=1 --x=2 --x=3 (list style simple property, will set the "x"
property to [1, 2, 3])
* --x=1,2,3 (shorthand list style simple property, will set the "x"
property to [1, 2, 3])
* --complexObject='{"key1":"value1",...} (JSON format for all other
complex types)
Using a string tokenizer that minimizes the number of required escape
characters would be good so we could use newline characters as our only
token. I would avoid ',\t ' as tokens since they are more likely to appear.
On Mon, Jan 8, 2018 at 10:33 AM, Kenneth Knowles <k...@google.com> wrote:
> We do have a plain command line syntax, and whoever writes the
> main(String[]) function is responsible for invoking the parser. It isn't
> quite as nice as standard arg parse libraries, but it isn't too bad. It
> would be great to improve, though.
>
> Jackson is for machine-to-machine communication or other situations where
> command line parsing doesn't work so well.
>
> Are we using these some other way?
>
> Kenn
>
> On Sun, Jan 7, 2018 at 7:21 AM, Romain Manni-Bucau <rmannibu...@gmail.com>
> wrote:
>
>>
>>
>> Le 7 janv. 2018 12:53, "Jean-Baptiste Onofré" <j...@nanthrax.net> a écrit :
>>
>> Hi Romain,
>>
>> I guess you are assuming that pipeline options are flat command line like
>> argument, right ?
>>
>> Actually, theoretically, pipeline options can be represented as json,
>> that's why we use jackson.
>> The pipeline options can be serialized/deserialized as json.
>>
>>
>> Yes but if users (or dev ;)) start to use that then it is trivial to
>> break the cli handling and fromArgs parsing or, if not, break the user
>> experience. So at the end it is a kind of "yes but no", right?
>>
>> PS: already see some advanced users having a headache trying to pass
>> pipeline options in json so using the plain command line syntax can be more
>> friendly too.
>>
>>
>> The purpose is to remove the jackson dependencies ?
>>
>>
>> Later yes, seeing core dep tree I identify a lot of dep which can
>> conflict in some env and are not really needed or bring much being in the
>> core - like avro as mentionned in another thread. Can need a sanitizing
>> round. Short term it was really a "why is it that complicated" ;).
>>
>>
>>
>> Regards
>> JB
>>
>>
>> On 01/07/2018 11:38 AM, Romain Manni-Bucau wrote:
>>
>>> Hi guys,
>>>
>>> not sure i fully get why jackson is used to parse pipeline options in
>>> the testing integration
>>>
>>> why not using a token parser like [1] which allows to map 1-1 with the
>>> user interface (command line) the options?
>>>
>>> [1] https://github.com/Talend/component-runtime/blob/master/comp
>>> onent-server/src/main/java/org/talend/sdk/component/server/l
>>> ang/StringPropertiesTokenizer.java
>>>
>>> Romain Manni-Bucau
>>> @rmannibucau <https://twitter.com/rmannibucau> | Blog <
>>> https://rmannibucau.metawerx.net/> | Old Blog <
>>> http://rmannibucau.wordpress.com> | Github <
>>> https://github.com/rmannibucau> | LinkedIn <
>>> https://www.linkedin.com/in/rmannibucau>
>>>
>>
>> --
>> Jean-Baptiste Onofré
>> jbono...@apache.org
>> http://blog.nanthrax.net
>> Talend - http://www.talend.com
>>
>>
>>
>
