Hi Sijie, Matteo and Joe ,
Thanks for your kind and professional opinions .
For performance concern from Joe, what I think is like this, different log
levels for different detail outputs and let dev decide which is good for them,
and I also mentioned in PIP about we can disable this feature by parameter in
some critical scenarios.
For protocol implement from Matteo and Joe, we could try to implement it
for we need to get from the proxy, like what it did in other proxies, proxysql
and twemproxy , or any others.
The point for this feature is we could have more details for our traffic,
not only for performances, and also resources usage and msg details for debug
and audit, like client issues, proxy issue or broker issues or ...
Thanks.
On Wednesday, January 30, 2019, 11:26:07 AM PST, Joe F
<joefranc...@gmail.com> wrote:
I run Pulsar proxy in production, and the same concern here.
I don't think we can get any of these metrics unless we start parsing
protocol, and definitely its going to make everything slower, and create
additional memory and GC pressures.
Joe
On Wed, Jan 30, 2019 at 11:08 AM Matteo Merli <matteo.me...@gmail.com>
wrote:
> Missed to comment on this :)
>
> One issue might arise from the fact that proxy is not actually parsing
> each and every request.
>
> The proxy only "speaks" Pulsar protocol initial Connect/Connected
> handshake, in which the proxy forwards the client credentials and
> route it through the appropriate broker.
>
> After the initial handshake, the proxy is essentially degrading itself
> into a 1-1 TCP proxy, patching 2 TCP connections through
> without checking the commands anymore. That's the reason we only have
> metrics around bytes/s and "operation/s" where
> operation maps to a "buffer" we're getting from a socket (with no
> direct relation to IP packets).
>
>
> --
> Matteo Merli
> <matteo.me...@gmail.com>
>
> On Wed, Jan 30, 2019 at 9:19 AM Sijie Guo <guosi...@gmail.com> wrote:
> >
> > Yao,
> >
> > Thank you for your proposal! The proposal looks good to me +1.
> >
> > In general, the ASF is using lazy consensus for a lot of things, like
> > adopting PIPs. basically, if there is no objection coming up within a
> > period (typically 1~2 days), you are good to pull the trigger and send
> PRs
> > :-)
> >
> > - Sijie
> >
> > On Sun, Jan 27, 2019 at 11:39 AM sun yao <foreversun...@yahoo.com
> .invalid>
> > wrote:
> >
> > >
> > >
> > > Hi folks,
> > > Pulsar Proxy is almost a gateway for all pulsar requests, it
> would
> > > be helpful if it can record more details for the traffic, like source,
> > > target, session id, response time(different stage) for each request,
> even
> > > for message body. I am proposing an improve for pulsar proxy, for
> > > more information : PIP:
> > >
> https://github.com/apache/pulsar/wiki/PIP-28%3A-Improve-pulsar-proxy-log-messages
> > > Feel free to let me know any ideas or suggestions for
> this
> > > feature. Thanks.
> > >
>
