We may be able to modify the handler to just concatenate the arguments
again with a comma. I'll look into that tomorrow.
On 25.04.2018 11:02, Dongwon Kim wrote:
Hi Chesnay,
I already modified our application to use semicolon as delimiter and
now I can run a job using rest API via dispatcher.
Nevertheless, though I understand the difficulty of modifying API, it
needs to be adjusted as many users are highly likely to use a list of
brokers as an argument of their streaming applications and use comma
to parse the list.
Otherwise, it must be written on the document to warn users to use
other characters as delimiter for program arguments.
Best,
- Dongwon
2018. 4. 25. 오후 5:28, Chesnay Schepler <ches...@apache.org
<mailto:ches...@apache.org>> 작성:
Currently I don't see a way to circumvent the splitting. You will
have to use a different delimiter, I guess a semi-colon could work?
The code is rather optimistic in that it assumes commas to not occur
within a parameter value, and doesn't support any kind of escaping or
quoting. (And this is a rabbit hole I'd rather avoid)
Ultimately I would love to change this call to send the parameters as
JSON instead (then you wouldn't have to deal with escaping
characters...), but we can't do that until the API versioning is in
place (no ETA).
On 20.04.2018 12:37, Dongwon Kim wrote:
Hi,
I'm trying to run a program by sending POST requests.
I've already spawned dispatcher in YARN and uploaded a jar file to
the dispatcher.
I want to execute my application using the following arguments
(--topic and --kafkaParams):
--topic gps-topic --kafkaParams
bootstrap.servers=dacoe2:20245,group.id
<http://group.id/>=trajectory-tracker
As you can see, there's a comma in the value of kafkaParams.
When I'm sending my application using the following command from
Bash (%20 is a space and $2C is a comma),
bash> curl -X POST
'http://dacoe4.weave.local:45097/jars/7b6880d7-b899-4243-8b9b-a01ad7f8a854_Tmap-1.0-SNAPSHOT.jar/run?entry-class=com.skt.tmap.trajectorytracker.TrajectoryTracker&program-args=--topic%20gps-topic%20--kafkaParams%20bootstrap.servers=dacoe2:20245%2Cgroup.id
<http://group.id/>=trajectory-tracker'
I get the following response from the dispatcher:
{
"errors": [
"Expected only one value [--topic gps-topic --kafkaParams
bootstrap.servers=dacoe2:20245, group.id
<http://group.id/>=trajectory-tracker]."
]
}
What I found from the source code is that
org.apache.flink.runtime.rest.messages.MessageQueue tries split the
value of program-ages using comma as a delimiter.
I think I could modify my program to get arguments in a different
way but it is not going to be intuitive to use different characters
for a delimiter instead of comma.
How you guys think?
Or there's a way to avoid this behavior of splitting the value of
program-args into multiple pieces?
best,
- Dongwon
