Re: [DISCUSS] PIP-1 Pulsar proxy component

2017-07-05 Thread Matteo Merli
On Fri, Jun 30, 2017 at 5:09 PM, Dave Fisher wrote: > One thought is make the proxy into a WebSocket frontend but then pass > through to the broker using the TCP protocol. > > WebSocket would fit a pattern that everyone is used to scaling. It could > be setup through Tomcat with connections throu

Re: [DISCUSS] PIP-1 Pulsar proxy component

2017-07-05 Thread Matteo Merli
I have updated the Wiki page with a section about TLS : https://github.com/apache/incubator-pulsar/wiki/PIP-1#tls-encryption-and-authentication Also, I just posted a PR with the implementation: https://github.com/apache/incubator-pulsar/pull/548 -- Matteo Merli On Fri, Jun 30, 2017 at 5:09 P

Re: [DISCUSS] PIP-1 Pulsar proxy component

2017-06-30 Thread Dave Fisher
> On Jun 30, 2017, at 4:39 PM, Matteo Merli wrote: > > On Fri, Jun 30, 2017 at 12:41 PM, Dave Fisher wrote: > >> So I am clear the problem is having an SSL endpoint that authenticates the >> client and allows the messages to flow through to the correct broker. >> > > > The main problem this

Re: [DISCUSS] PIP-1 Pulsar proxy component

2017-06-30 Thread Matteo Merli
On Fri, Jun 30, 2017 at 12:41 PM, Dave Fisher wrote: > So I am clear the problem is having an SSL endpoint that authenticates the > client and allows the messages to flow through to the correct broker. > The main problem this proposal is trying to solve is how to expose Pulsar, which is a state

Re: [DISCUSS] PIP-1 Pulsar proxy component

2017-06-30 Thread Matteo Merli
On Fri, Jun 30, 2017 at 2:46 PM, Maurice Barnum wrote: > it's a bit unfortunate the proxy would have to decode enough of the wire > protocol to find the lookup commands and responses to build the "routing > map". i'd still be tempted to do it with nginx and a lua module. if you're > careful, you

Re: [DISCUSS] PIP-1 Pulsar proxy component

2017-06-30 Thread Maurice Barnum
it's a bit unfortunate the proxy would have to decode enough of the wire protocol to find the lookup commands and responses to build the "routing map".   i'd still be tempted to do it with nginx and a lua module. if you're careful, you can avoid deserializing responses unless you have an outstand

Re: [DISCUSS] PIP-1 Pulsar proxy component

2017-06-30 Thread Dave Fisher
> On Jun 30, 2017, at 12:12 PM, Matteo Merli wrote: > > On Thu, Jun 29, 2017 at 6:33 PM, Dave Fisher wrote: > >> I mean do you think it would meet the needs of a proxy including SSL? >> >> I'll look into this more as this proxy design intrigues me. >> > > > So, ZooKeeper is more of a distr

Re: [DISCUSS] PIP-1 Pulsar proxy component

2017-06-30 Thread Matteo Merli
On Thu, Jun 29, 2017 at 6:33 PM, Dave Fisher wrote: > I mean do you think it would meet the needs of a proxy including SSL? > > I'll look into this more as this proxy design intrigues me. > So, ZooKeeper is more of a distributed coordination service and it doesn't really work as a proxy. We use

Re: [DISCUSS] PIP-1 Pulsar proxy component

2017-06-29 Thread Dave Fisher
Sent from my iPhone > On Jun 29, 2017, at 5:19 PM, Matteo Merli wrote: > >> On Thu, Jun 29, 2017 at 4:06 PM, Dave Fisher wrote: >> >> Does Apache Zookeeper meet these needs? >> > > Dave, do you mean with respect to SSL? I'm asking as a developer and not as a mentor. I mean do you think i

Re: [DISCUSS] PIP-1 Pulsar proxy component

2017-06-29 Thread Matteo Merli
On Thu, Jun 29, 2017 at 5:18 PM, Maurice Barnum wrote: > this seems like it should be a solved problem.from a quick look, for > example, it seems like nginx can do this already. > https://www.nginx.com/resources/admin-guide/tcp-load-balancing/ > That would only work for the initial topic lookup.

Re: [DISCUSS] PIP-1 Pulsar proxy component

2017-06-29 Thread Matteo Merli
On Thu, Jun 29, 2017 at 4:06 PM, Dave Fisher wrote: > Does Apache Zookeeper meet these needs? > Dave, do you mean with respect to SSL? -- Matteo Merli

Re: [DISCUSS] PIP-1 Pulsar proxy component

2017-06-29 Thread Maurice Barnum
this seems like it should be a solved problem.from a quick look, for example, it seems like nginx can do this already. https://www.nginx.com/resources/admin-guide/tcp-load-balancing/ On Thursday, June 29, 2017, 4:54:29 PM PDT, Rajan Dhabalia wrote: >> or if people prefer we can create a GitHu

Re: [DISCUSS] PIP-1 Pulsar proxy component

2017-06-29 Thread Rajan Dhabalia
>> or if people prefer we can create a GitHub issue to have the discussion on the proposal I think github issue would be more helpful for discussion. so, my vote to github-issue. Thanks, Rajan On Thu, Jun 29, 2017 at 1:51 PM, Matteo Merli wrote: > I have created the wiki page with a first propo

Re: [DISCUSS] PIP-1 Pulsar proxy component

2017-06-29 Thread Dave Fisher
Hi Matteo, Does Apache Zookeeper meet these needs? Regards, Dave > On Jun 29, 2017, at 2:33 PM, Matteo Merli wrote: > > Good point, > > I think we should support SSL, though it might get terminated at the proxy > layer. > > Whether the connection between proxy and broker it's encrypted would

Re: [DISCUSS] PIP-1 Pulsar proxy component

2017-06-29 Thread Matteo Merli
Good point, I think we should support SSL, though it might get terminated at the proxy layer. Whether the connection between proxy and broker it's encrypted would be kind of a separated issue, but in any case we cannot just forward the encrypted connection because that would break the SSL certifi

Re: [DISCUSS] PIP-1 Pulsar proxy component

2017-06-29 Thread Joe Francis
 SSL will be supported? Joe On Thursday, June 29, 2017 1:51 PM, Matteo Merli wrote: I have created the wiki page with a first proposal (as we discussed earlier). https://github.com/apache/incubator-pulsar/wiki/PIP-1 Please add feedback in this thread (or if people prefer we can create

[DISCUSS] PIP-1 Pulsar proxy component

2017-06-29 Thread Matteo Merli
I have created the wiki page with a first proposal (as we discussed earlier). https://github.com/apache/incubator-pulsar/wiki/PIP-1 Please add feedback in this thread (or if people prefer we can create a GitHub issue to have the discussion on the proposal). Other than the proposal itself, it wou