First, sorry for sluggish response time .. >>>E.g. I need latency histograms, but this seems unsupported (benchmark >>>results can >>>only have avg/min/max/stddev). For me, this isn't "nice to have", but >>>essential. >>>Throughput is one thing. Constistent low latency a completely different. The >>>latter is >>>much much harder.
>>Codespeed is terrible. But this is not one of the ways in which it is >>terrible. Codespeed doesn't care if you label your measurement "latency". I >>think you've just noticed that what the existing benchmarks measure is mostly >>(entirely?) throughput. If you wanted to >>write a latency benchmark, I >>don't think anything's stopping you. >I believe Tobias was not saying "codespeed can't have a measurement called >'latency'" but rather "codespeed can't do histograms of measurements, which we >need for measurement of latency and you don't need for measurement of >throughput". Is that accurate? I don't know if there's a histogram feature >hidden in the UI somewhere. Yes, exactly. That's what I meant. As an example of the kind of analysis and visualization I am after, please have a look here: http://www.brendangregg.com/HeatMaps/latency.html In particular, latency heatmaps are an incredible useful visualization. http://www.brendangregg.com/perf.html#HeatMaps >It would be nice to at least try a little bit to contribute things (like a >histogram feature) to codespeed before charging off in a completely different >direction. It is Django and canvas for gfx. The former I have no know-how and no use/motivation for. Rendering HTML on the server isn't something we do anymore (WebSocket talking WAMP to Crossbar.io, anything else is just static HTML/JS/CSS/Images). For the latter, there is D3 (http://d3js.org/) which is awesome and vector graphics. For heatmaps, canvas _might_ be fine, but for virtually anything chart like, D3 has a lot to bring to the table. In general, I can follow the argument of "contributing instead of reinventing" and "paying back", but in this particular case, I can't justify sinking time into this (codespeed). Cheers, /Tobias _______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
