Part of the motivation to use JSON for more complex options was that
it avoids having to define (and document, test, have users learn, ...)
yet another format for expressing lists, maps, etc.
On Mon, Jan 8, 2018 at 11:19 AM, Lukasz Cwik <lc...@google.com> wrote:
> Ken, this is specifically about running integration tests and not a users
> main().
>
> Note, that PipelineOptions JSON format was used because it was a convenient
> serialized form that is easy to explain to people what is required.
> Using a different string tokenizer and calling
> PipelineOptionsFactory.fromArgs() with the parsed strings seems like it
> would be better.
>
> These are the supported formats for fromArgs():
> * --project=MyProject (simple property, will set the "project" property
> to "MyProject")
> * --readOnly=true (for boolean properties, will set the "readOnly"
> property to "true")
> * --readOnly (shorthand for boolean properties, will set the "readOnly"
> property to "true")
> * --x=1 --x=2 --x=3 (list style simple property, will set the "x"
> property to [1, 2, 3])
> * --x=1,2,3 (shorthand list style simple property, will set the "x"
> property to [1, 2, 3])
> * --complexObject='{"key1":"value1",...} (JSON format for all other
> complex types)
>
> Using a string tokenizer that minimizes the number of required escape
> characters would be good so we could use newline characters as our only
> token. I would avoid ',\t ' as tokens since they are more likely to appear.
>
> On Mon, Jan 8, 2018 at 10:33 AM, Kenneth Knowles <k...@google.com> wrote:
>>
>> We do have a plain command line syntax, and whoever writes the
>> main(String[]) function is responsible for invoking the parser. It isn't
>> quite as nice as standard arg parse libraries, but it isn't too bad. It
>> would be great to improve, though.
>>
>> Jackson is for machine-to-machine communication or other situations where
>> command line parsing doesn't work so well.
>>
>> Are we using these some other way?
>>
>> Kenn
>>
>> On Sun, Jan 7, 2018 at 7:21 AM, Romain Manni-Bucau <rmannibu...@gmail.com>
>> wrote:
>>>
>>>
>>>
>>> Le 7 janv. 2018 12:53, "Jean-Baptiste Onofré" <j...@nanthrax.net> a écrit :
>>>
>>> Hi Romain,
>>>
>>> I guess you are assuming that pipeline options are flat command line like
>>> argument, right ?
>>>
>>> Actually, theoretically, pipeline options can be represented as json,
>>> that's why we use jackson.
>>> The pipeline options can be serialized/deserialized as json.
>>>
>>>
>>> Yes but if users (or dev ;)) start to use that then it is trivial to
>>> break the cli handling and fromArgs parsing or, if not, break the user
>>> experience. So at the end it is a kind of "yes but no", right?
>>>
>>> PS: already see some advanced users having a headache trying to pass
>>> pipeline options in json so using the plain command line syntax can be more
>>> friendly too.
>>>
>>>
>>> The purpose is to remove the jackson dependencies ?
>>>
>>>
>>> Later yes, seeing core dep tree I identify a lot of dep which can
>>> conflict in some env and are not really needed or bring much being in the
>>> core - like avro as mentionned in another thread. Can need a sanitizing
>>> round. Short term it was really a "why is it that complicated" ;).
>>>
>>>
>>>
>>> Regards
>>> JB
>>>
>>>
>>> On 01/07/2018 11:38 AM, Romain Manni-Bucau wrote:
>>>>
>>>> Hi guys,
>>>>
>>>> not sure i fully get why jackson is used to parse pipeline options in
>>>> the testing integration
>>>>
>>>> why not using a token parser like [1] which allows to map 1-1 with the
>>>> user interface (command line) the options?
>>>>
>>>> [1]
>>>> https://github.com/Talend/component-runtime/blob/master/component-server/src/main/java/org/talend/sdk/component/server/lang/StringPropertiesTokenizer.java
>>>>
>>>> Romain Manni-Bucau
>>>> @rmannibucau <https://twitter.com/rmannibucau> | Blog
>>>> <https://rmannibucau.metawerx.net/> | Old Blog
>>>> <http://rmannibucau.wordpress.com> | Github
>>>> <https://github.com/rmannibucau>
>>>> | LinkedIn <https://www.linkedin.com/in/rmannibucau>
>>>
>>>
>>> --
>>> Jean-Baptiste Onofré
>>> jbono...@apache.org
>>> http://blog.nanthrax.net
>>> Talend - http://www.talend.com
>>>
>>>
>>
>
