Hi Austin,
Thanks for the recommendations! After internal discussion I have decided to
park this FLIP for now until I have more capacity to commit to it.
Jeremy
On 5/17/22, 10:26 AM, "Austin Cawley-Edwards" <austin.caw...@gmail.com> wrote:
CAUTION: This email originated from outside of the organization. Do not
click links or open attachments unless you can confirm the sender and know the
content is safe.
Hey Jeremy,
Thanks for kicking off this discussion. As a Flink user, I too struggled
with the lack of HTTP support and rolled my own with AsyncIO. Reading
through the FLIP, I just have a few general questions and comments.
* It is not clear to me if multiple HTTP methods are supported or not? It's
listed in "Limitations" that only POSTs are allowed, but the constructor
accepts a "method" argument.
* More of a nit, the FLIP contains a lot of code, making it feel a bit more
like a PR already. It would be easier to understand the proposed interfaces
alone, and keep the implementation POC as a separate link IMO.
Since there are so many different types of HTTP APIs, and many different
ways of using them, I think the proposal would benefit from taking a more
general approach to both request building and response handling. For
instance, some APIs may return hints for retry that are not contained in
the status code alone (e.g., a "retry-after" header or such + a 429 status
code). Can we already think about how to more generally expose these two
concepts? For the retries, it might be too idealistic, but standardizing on
a retry interface like the one proposed in FLIP-232[1] would make these
aysnc/http APIs feel much more aligned.
wdyt?
Austin
[1]:
https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=211883963
On Tue, May 17, 2022 at 10:21 AM Ber, Jeremy <jd...@amazon.com.invalid>
wrote:
> Hi there, We would like to start a discussion thread on FLIP-233:
> Introduce HTTP Connector<
>
https://cwiki.apache.org/confluence/display/FLINK/FLIP-233%3A+Introduce+HTTP+Connector>
> where we propose to create a connector for delivering arbitrary data
> packets from Apache Flink to an HTTP Endpoint. This connector will give
> users flexibility to deliver data to any destination which supports REST
> endpoints. This includes REST APIs, Amazon Lambda, users internal or
> external consumers, among other options.
>
> Looking forward to your feedback.
>
> Thank you,
> Jeremy Ber
>
>
