Thank's Jason. These are all good ideas: On Wednesday 13 November 2024 at 02:06:56 UTC Jason E. Aten wrote:
Idea 0: Can you just click on the graphviz boxes in the pprof http page for your section and then just show the source for only those? Might give you a quick idea of which are the hottest spots only among those func of interest. Yes. This was my original thought. We can "focus" on the function that's at the start of the subtree I'm interested in. However, the statistics don't scale. In other words, I'd like the root node of the subtree to represent a 100% of cumulative CPU time in the profile. Idea 1: Can you write a test or benchmark that just exercises the important section? I suspect you could then run the profiler on your benchmark/test. If not, it might be worth refactoring your code to allow this. Otherwise you could write an alternative "main" as a harness that just calls the important section. Maybe. However, the code that is outside of the subtree is important because of the data that is used by the subtree. The problem is that the path to the subtree is complex and also compute heavy. I suppose I could create a suitable test harness for this purpose, but it's really live information I'm interested in here. But I'll give this some more thought. Idea 2: referring to the StartCPUProfile docs: > On Unix-like systems, StartCPUProfile does not work by default for Go code built > with -buildmode=c-archive or -buildmode=c-shared. StartCPUProfile relies on the > SIGPROF signal, but that signal will be delivered to the main program's SIGPROF > signal handler (if any) not to the one used by Go. To make it work, call os/signal.Notify > for syscall.SIGPROF, but note that doing so may break any profiling being done by the main program. This makes it sounds like the SIGPROF signal is used to do the sampling. So you might be able to manipulate it (e.g. ignore it when not in your code). Seems like a hack, but might be worth experimenting with. I had a quick look at the pprof package to see if there was anything obvious I could do. I didn't go into runtime package however, which I suspect I would need to. I'd probably get out of my depth very quickly once I was in there. Idea 3: post processing. You could probably take apart the profile log after it is recorded and discard the samples that are irrelevant. I think this is probably the best and most sustainable solution. A post-processing filter to create a profile containing just the subtree and scaled statistics would be good. Idea 4: simple manual timing. Insert time.Now() and time.Since() calls at strategic points, and measure the improvement as you tweak your code. I would normally go for something like this but the subtree is quite complex. Manual timing points wouldn't necessarily reveal what I want to see - and I would need to know where the (possible) issues are before I added the timing calls! I suppose I could use something like this to bisect where a problem might be, but that sounds like it would take a long time and not be repeatable. But as I said, these are all great ideas. Thanks for taking the time to outline them. As I was writing this reply it occurred to me that maybe a continuous profiling solution might be good for my case. Does anyone have any recommendations for continuous profiling with the kind of detail offered by pprof? Thanks Stephen -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion visit https://groups.google.com/d/msgid/golang-nuts/7411e568-fa93-4229-b94f-67a17b2d01fbn%40googlegroups.com.
