Hi Joe, Thank you for your feedback! I don't think this is going to touch existing protobuf based throughput. But we will do what you have suggested.
- Sijie On Thu, May 14, 2020 at 6:49 AM Joe Francis <j...@verizonmedia.com.invalid> wrote: > We do have concerns, and we have implemented throttles on even our existing > REST APIs in the broker. > So we would like to be able to turn this off, and validation that the > implementation changes do not degrade existing protobuf based throughput > > Joe > > On Thu, May 14, 2020 at 12:35 AM Sijie Guo <guosi...@gmail.com> wrote: > > > On Wed, May 13, 2020 at 10:31 PM Matteo Merli <matteo.me...@gmail.com> > > wrote: > > > > > +1 > > > > > > The REST API is more simple and intuitive for developers compared to > > > WebSocket. > > > > > > While it's easy to get started with Websocket API, the biggest problem > > > for applications is how to handle errors and corner cases. Eg: > > > re-establishing a producer session while maintaining a queue of > > > pending messages. This is overly complicated and that's what the > > > regular Pulsar client lib already solves. > > > > > > Couple of comments on the proposal: > > > 1. I'd like to see the option, for example based on different > > > Content-Types to have simplified body. eg: `curl -XPOST $URL -d > > > @file.txt` could be able to just publish a raw bytes payload on the > > > given topic. > > > > > > > We can keep the option available. > > > > > > > 2. >>> Because consumers are stateful, any consumer created with the > > > REST API are tied to a specific broker. <<< > > > I'd avoid this requirement since it makes app devs life much more > > > complicated. > > > > > > > People can use the HOST header for targeting to a specific broker. > > > > If applications don't use the HOST header, it will fall back to the > > redirection approach as what we did for other REST requests. So the > request > > goes to the owner broker for consuming and acknowledging messages. > > it works with both brokers and proxies and it is the same as other > restful > > requests. > > > > > > > > > 3. >>> Note that this request must be made to the specific broker > > > holding the consumer instance. <<< > > > That's not necessary to go on the same host. A shared > > > subscription dispatcher will be able to apply the ack even if it's > > > received from a different consumer (and it will clean up the pending > > > acks). > > > > > > > It is yes for shared subscriptions. But it doesn't work for other types > of > > subscriptions. > > We would like to start with this approach to make it work seamlessly for > > different types of subscriptions. > > We can look into how to optimize it as it goes. > > > > > > > > > > -- > > > Matteo Merli > > > <matteo.me...@gmail.com> > > > > > > On Wed, May 13, 2020 at 6:19 PM Jia Zhai <zhaiji...@gmail.com> wrote: > > > > > > > > +1 > > > > > > > > Best Regards. > > > > > > > > > > > > Jia Zhai > > > > > > > > Beijing, China > > > > > > > > Mobile: +86 15810491983 > > > > > > > > > > > > > > > > > > > > On Wed, May 13, 2020 at 6:25 PM Sijie Guo <si...@apache.org> wrote: > > > > > > > > > Hi all, > > > > > > > > > > Some of the people have been asking for a REST-proxy like > equivalent > > > > > component in Pulsar. Pulsar already has a web service that provides > > the > > > > > management API over a Pulsar cluster. We don't actually need a > > separate > > > > > component or service for supporting streaming events in and out of > > > Pulsar > > > > > using HTTP. We can just add endpoints to the existing web server to > > > support > > > > > those functionalities. > > > > > > > > > > I'd like to start a discussion on adding REST endpoints to the > > current > > > web > > > > > server for producing, consuming and reading messages. So people can > > use > > > > > HTTP for streaming events in and out of Pulsar without setting up > > > > > additional components. > > > > > > > > > > The full proposal is written at > > > > > > > > > > > > > > > > https://github.com/apache/pulsar/wiki/PIP-64:-Introduce-REST-endpoints-for-producing,-consuming-and-reading-messages > > > > > . > > > > > > > > > > Feedback and comments are welcome! > > > > > > > > > > - Sijie > > > > > > > > > > >
