Hi Chesnay, I already modified our application to use semicolon as delimiter and now I can run a job using rest API via dispatcher. Nevertheless, though I understand the difficulty of modifying API, it needs to be adjusted as many users are highly likely to use a list of brokers as an argument of their streaming applications and use comma to parse the list. Otherwise, it must be written on the document to warn users to use other characters as delimiter for program arguments.
Best, - Dongwon > 2018. 4. 25. 오후 5:28, Chesnay Schepler <ches...@apache.org> 작성: > > Currently I don't see a way to circumvent the splitting. You will have to use > a different delimiter, I guess a semi-colon could work? > > The code is rather optimistic in that it assumes commas to not occur within a > parameter value, and doesn't support any kind of escaping or quoting. (And > this is a rabbit hole I'd rather avoid) > > Ultimately I would love to change this call to send the parameters as JSON > instead (then you wouldn't have to deal with escaping characters...), but we > can't do that until the API versioning is in place (no ETA). > > On 20.04.2018 12:37, Dongwon Kim wrote: >> Hi, >> >> I'm trying to run a program by sending POST requests. >> I've already spawned dispatcher in YARN and uploaded a jar file to the >> dispatcher. >> >> I want to execute my application using the following arguments (--topic and >> --kafkaParams): >> --topic gps-topic --kafkaParams bootstrap.servers=dacoe2:20245,group.id >> <http://group.id/>=trajectory-tracker >> As you can see, there's a comma in the value of kafkaParams. >> >> When I'm sending my application using the following command from Bash (%20 >> is a space and $2C is a comma), >> bash> curl -X POST >> 'http://dacoe4.weave.local:45097/jars/7b6880d7-b899-4243-8b9b-a01ad7f8a854_Tmap-1.0-SNAPSHOT.jar/run >> >> <http://dacoe4.weave.local:45097/jars/7b6880d7-b899-4243-8b9b-a01ad7f8a854_Tmap-1.0-SNAPSHOT.jar/run>?entry-class=com.skt.tmap.trajectorytracker.TrajectoryTracker&program-args=--topic%20gps-topic%20--kafkaParams%20bootstrap.servers=dacoe2:20245%2Cgroup.id >> <http://group.id/>=trajectory-tracker' >> >> I get the following response from the dispatcher: >> { >> "errors": [ >> "Expected only one value [--topic gps-topic --kafkaParams >> bootstrap.servers=dacoe2:20245, group.id >> <http://group.id/>=trajectory-tracker]." >> ] >> } >> >> What I found from the source code is that >> org.apache.flink.runtime.rest.messages.MessageQueue tries split the value of >> program-ages using comma as a delimiter. >> >> I think I could modify my program to get arguments in a different way but it >> is not going to be intuitive to use different characters for a delimiter >> instead of comma. >> >> How you guys think? >> >> Or there's a way to avoid this behavior of splitting the value of >> program-args into multiple pieces? >> >> best, >> >> - Dongwon >
