Hi, I tried to search Flink Jira for this but couldn't find a ticket to match. If there's no ticket yet, did I understand correctly though, that you would be open to support GET as an alternative method for all of Flink's REST operations?
Now that 1.5 was released, it's a pity that this problem kind of prevents us from upgrading – or spend time on creating a workaround. On Thu, Apr 5, 2018 at 9:10 PM, Till Rohrmann <trohrm...@apache.org> wrote: > This improvement is unfortunately out of scope for the 1.5 release since > the feature freeze is already quite some time ago. But I hope that this > improvement will make it into the 1.6 release. > > Cheers, > Till > > On Thu, Apr 5, 2018 at 4:45 PM, Juho Autio <juho.au...@rovio.com> wrote: > >> Thanks for the answer. Wrapping with GET sounds good to me. You said next >> version; do you mean that Flink 1.5 would already include this improvement >> when it's released? >> >> On Thu, Apr 5, 2018 at 2:40 PM, Till Rohrmann <trohrm...@apache.org> >> wrote: >> >>> Hi Juho, >>> >>> you are right that due to a limitation in the Yarn proxy [1] we cannot >>> directly contact the cluster through the Yarn proxy. >>> >>> The way it works at the moment is that the Flink client retrieves the >>> AM's hostname through the ApplicationReport and then directly talks to the >>> AM. This of course requires that one can reach the respective Yarn >>> container. This is a limitation, though not a regression, which we want to >>> improve with the next version of Flink. An idea would be to wrap the REST >>> calls in a GET call to make them pass through the Yarn proxy. >>> >>> [1] https://issues.apache.org/jira/browse/YARN-2084 >>> >>> Cheers, >>> Till >>> >>> On Wed, Apr 4, 2018 at 4:31 PM, Fabian Hueske <fhue...@gmail.com> wrote: >>> >>>> Hi Juho, >>>> >>>> Thanks for raising this point! >>>> >>>> I'll add Chesnay and Till to the thread who contributed to the REST API. >>>> >>>> Best, Fabian >>>> >>>> 2018-04-04 15:02 GMT+02:00 Juho Autio <juho.au...@rovio.com>: >>>> >>>>> I just learned that Flink savepoints API was refactored to require >>>>> using HTTP POST. >>>>> >>>>> That's fine otherwise, but makes life harder when Flink is run on top >>>>> of YARN. >>>>> >>>>> I've added example calls below to show how POST is declined by >>>>> the hadoop-yarn-server-web-proxy*, which only supports GET and PUT. >>>>> >>>>> Can you think of any solution to this? If I would be able to determine >>>>> the actual host & port for Flink UI, I could use that instead of the proxy >>>>> address. But that would probably require opening at least one more port, >>>>> so >>>>> it's not the optimal solution either. Ideally I would have Flink REST API >>>>> completely accessible with GET and PUT methods. >>>>> >>>>> To me it seems like AWS EMR will also hit this issue as soon as they >>>>> start supporting Flink 1.5, because they seem to run Flink as a YARN app. >>>>> >>>>> >>>>> *) https://github.com/apache/hadoop/blob/trunk/hadoop-yarn-p >>>>> roject/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web >>>>> -proxy/src/main/java/org/apache/hadoop/yarn/server/webproxy/ >>>>> WebAppProxyServlet.java#L296-L306 >>>>> >>>>> >>>>> $ http POST http://10.0.10.71:20888/proxy/ >>>>> application_1522844153347_0001/jobs/652e207f8578574d4a322e23 >>>>> d4f8b908/checkpoints >>>>> >>>>> HTTP/1.1 405 HTTP method POST is not supported by this URL >>>>> Cache-Control: must-revalidate,no-cache,no-store >>>>> Content-Length: 1523 >>>>> Content-Type: text/html; charset=iso-8859-1 >>>>> Date: Wed, 04 Apr 2018 12:48:05 GMT >>>>> Date: Wed, 04 Apr 2018 12:48:05 GMT >>>>> Pragma: no-cache >>>>> Pragma: no-cache >>>>> >>>>> <html> >>>>> <head> >>>>> <meta http-equiv="Content-Type" content="text/html; >>>>> charset=ISO-8859-1"/> >>>>> <title>Error 405 HTTP method POST is not supported by this URL</title> >>>>> </head> >>>>> <body><h2>HTTP ERROR 405</h2> >>>>> <p>Problem accessing /proxy/application_15228441533 >>>>> 47_0001/jobs/652e207f8578574d4a322e23d4f8b908/checkpoints. Reason: >>>>> <pre> HTTP method POST is not supported by this URL</pre></p><hr >>>>> /><i><small>Powered by Jetty://</small></i><br/> >>>>> <br/> >>>>> <br/> >>>>> <br/> >>>>> <br/> >>>>> <br/> >>>>> <br/> >>>>> <br/> >>>>> <br/> >>>>> <br/> >>>>> <br/> >>>>> <br/> >>>>> <br/> >>>>> <br/> >>>>> <br/> >>>>> <br/> >>>>> <br/> >>>>> <br/> >>>>> <br/> >>>>> <br/> >>>>> >>>>> </body> >>>>> </html> >>>>> >>>>> >>>>> $ http PUT http://10.0.10.71:20888/proxy/ >>>>> application_1522844153347_0001/jobs/652e207f8578574d4a322e23 >>>>> d4f8b908/checkpoints >>>>> >>>>> HTTP/1.1 404 Not Found >>>>> Access-Control-Allow-Origin: * >>>>> Cache-Control: no-cache >>>>> Content-Length: 25 >>>>> Content-Type: application/json; charset=UTF-8 >>>>> Date: Wed, 04 Apr 2018 12:48:09 GMT >>>>> Date: Wed, 04 Apr 2018 12:48:09 GMT >>>>> Expires: Wed, 04 Apr 2018 12:48:09 GMT >>>>> Expires: Wed, 04 Apr 2018 12:48:09 GMT >>>>> Pragma: no-cache >>>>> Pragma: no-cache >>>>> >>>>> { >>>>> "errors": [ >>>>> "Not found." >>>>> ] >>>>> } >>>>> >>>>> ^ expected to get this from flink because there's no PUT /checkpoints. >>>>> >>>> >>>> >>> >> >
