Hi Ming, Please check my comment above. The reason we don't add it as a separate service is that Pulsar already has an HTTP server and a lot of applications are already programming over it. Adding another component or a separate service that is overlapped with existing Pulsar functionalities doesn't resolve the problem. Instead, it confuses Pulsar users. We are trying to provide Pulsar users with a seamless integration experience rather than installing many different services and components.
- Sijie On Wed, May 13, 2020 at 3:36 PM Ming Luo <ming....@kafkaesque.io.invalid> wrote: > Hi, > > Thanks for Sijie's proposal. I resonate Rajan's comment that this project > should become a peripheral to Pulsar because of the stateless nature of > HTTP and scaling up a web service inside Pulsar might be affecting > its current design. For that reason, I developed a project with an HTTP > interface to Pulsar and webhook notification that pushes messages to > consumers. > > It is written in Go. The current repo is at > https://github.com/kafkaesque-io/pulsar-beam It can be deployed in > standalone or inside a Pulsar cluster. I know someone has started using it > in their production trial. It makes sense for the HTTP interface to > become an optional add-on to Pulsar. I would like to contribute this back > to Apache Software Foundation as a standalone repo like Pulsar Manager. > What do you think? Let me know how I can help. > > Cheers > Ming > > On Wed, 13 May 2020 at 17:00, Rajan Dhabalia <rdhaba...@apache.org> wrote: > > > *Hi,I would like to add little history about Websocket vs REST API > > implementation in Pulsar. Before open-sourcing Pulsar and WebSocket proxy > > in Yahoo, we had implemented and evaluated both WebSocket and REST-HTTP > > APIs options and then decided to not introduce REST-API and kept > Websocket > > proxy solution in Pulsar. Let me add a few points to understand why we > > should avoid adding REST API in Pulsar. 1. One of the advantages of REST > > architecture is statelessness which means that any server can handle any > > request. However, it’s true for Websocket protocol as well but Websoket > is > > a bi-directional protocol with full duplex communication and > client-server > > can communicate on a single TCP connection which avoids different > overheads > > such as authentication on every request that needs to be done on > REST-Http > > api.2. If client doesn’t use persist connection correctly on REST-HTTP > then > > every produce/consume request can go to a different server and it will > > create a lot of complexity to manage the control flow, producer/consumer > > queuing at REST-server which internally manages pulsar-client, clean up > > unused resources, etc.3. Performance and scalability: Websocket is highly > > scalable and provides higher performance and throughput compare to > > REST-HTTP.4. Almost all programming languages and web-browsers > > <https://caniuse.com/#feat=websockets> support WebSocket.5. Pulsar is > > known > > for performance and scalability, so, we try to select solutions that work > > with scale and provide better performance.6. Pulsar broker gives an > option > > to start WebSocket proxy along with broker and that doesn’t require extra > > component to manage. So, if one doesn’t want to manage additional > component > > then one can start with the broker process as well. 7. Other messaging > > system also moving away from their legacy REST-HTTP api to WebSocket > > because of performance and scalability concerns.* > > *So, I think let’s not add this feature in Pulsar but it can be added a > > utility project outside of pulsar if it is needed for specific usecases.* > > > > *Thanks,* > > > > *Rajan* > > > > On Wed, May 13, 2020 at 3:25 AM Sijie Guo <si...@apache.org> wrote: > > > > > Hi all, > > > > > > Some of the people have been asking for a REST-proxy like equivalent > > > component in Pulsar. Pulsar already has a web service that provides the > > > management API over a Pulsar cluster. We don't actually need a separate > > > component or service for supporting streaming events in and out of > Pulsar > > > using HTTP. We can just add endpoints to the existing web server to > > support > > > those functionalities. > > > > > > I'd like to start a discussion on adding REST endpoints to the current > > web > > > server for producing, consuming and reading messages. So people can use > > > HTTP for streaming events in and out of Pulsar without setting up > > > additional components. > > > > > > The full proposal is written at > > > > > > > > > https://github.com/apache/pulsar/wiki/PIP-64:-Introduce-REST-endpoints-for-producing,-consuming-and-reading-messages > > > . > > > > > > Feedback and comments are welcome! > > > > > > - Sijie > > > > > > > > -- > Powered by Pulsar > Engineering, Kafkaesque >
