Re: [lttng-dev] babletrace2 graph performance considerations

2020-03-30 Thread Rocky Dunlap via lttng-dev
Each graph would parse the whole stream? What if I only connect one of the outputs from the source.ctf.fs? I would think it would only access that one stream file. Rocky On Mon, Mar 30, 2020 at 4:27 PM Simon Marchi wrote: > On 2020-03-30 1:30 p.m., Rocky Dunlap wrote: > > In my case I have CT

Re: [lttng-dev] bt2 python - accessing component instances added to graph

2020-03-30 Thread Rocky Dunlap via lttng-dev
(Adding the list back in - I meant to reply-all before...) Simon, Yes, the obj solution should work just fine - I was wondering about that parameter and the approach makes sense to me. Thanks! Rocky On Mon, Mar 30, 2020 at 2:38 PM Simon Marchi wrote: > On 2020-03-30 4:24 p.m., Rocky Dunlap w

Re: [lttng-dev] babletrace2 graph performance considerations

2020-03-30 Thread Rocky Dunlap via lttng-dev
Simon, Thanks - see below, On Mon, Mar 30, 2020 at 7:32 AM Simon Marchi wrote: > On 2020-03-30 12:10 a.m., Rocky Dunlap via lttng-dev wrote: > > A couple of questions on performance considerations when setting up bt2 > processing graphs. > > > > 1. Do parts of the p

[lttng-dev] bt2 python - accessing component instances added to graph

2020-03-29 Thread Rocky Dunlap via lttng-dev
I noticed that when I add a component to a bt2 graph, e.g.: sink = g.add_component(SinglePETSink, "sink"+str(idx)) The return value "sink" is not actually an instance of SinglePETSink (which has been defined as a local Python class). How do I get to the instance of SinglePETSink that was created

[lttng-dev] babletrace2 graph performance considerations

2020-03-29 Thread Rocky Dunlap via lttng-dev
A couple of questions on performance considerations when setting up bt2 processing graphs. 1. Do parts of the processing graph that can execute concurrently do so? Does the user have any control over this, e.g., by creating threads in sink components? 2. It looks like you cannot connect one out

Re: [lttng-dev] Babeltrace2 - compilation error with intel18

2020-03-23 Thread Rocky Dunlap via lttng-dev
Simon, See below... On Mon, Mar 23, 2020 at 11:24 AM Simon Marchi wrote: > On 2020-03-23 12:56 p.m., Rocky Dunlap wrote: > > Simon, > > > > Success! This change worked and I was able to build and install with > icc. Thank you! > > Ok, thanks. Out of curiosity, did run a "make check"? That w

Re: [lttng-dev] Babeltrace2 - compilation error with intel18

2020-03-23 Thread Rocky Dunlap via lttng-dev
Simon, Success! This change worked and I was able to build and install with icc. Thank you! I still get a lot of warnings during the build (see below) which seems like configure should work these out ahead of time. They are easy enough to ignore, but if support for intel is added, it would be n

Re: [lttng-dev] Babeltrace2 - compilation error with intel18

2020-03-23 Thread Rocky Dunlap via lttng-dev
Simon, Yes, I will be happy to give this a try. What's the easiest way to get this patch? (Sorry, I'm less familiar with Gerrit...) Rocky On Mon, Mar 23, 2020 at 9:44 AM Simon Marchi wrote: > On 2020-03-20 11:12 p.m., Simon Marchi via lttng-dev wrote: > > So since distutils really wants to c

Re: [lttng-dev] Babeltrace2 - compilation error with intel18

2020-03-20 Thread Rocky Dunlap via lttng-dev
Simon, Thanks for this - so after I updated to gcc 9.2, the shared libraries command ran correctly and the build finished. All the tests pass and I was able to read in a CTF trace from a python script. The issue was that the system gcc was gcc 4.X.X, so too old. So, I had to manually add the new

Re: [lttng-dev] Babeltrace2 - compilation error with intel18

2020-03-20 Thread Rocky Dunlap via lttng-dev
system I can make this change to force it to use what's sets as CC instead of defaulting to GCC? Rocky On Fri, Mar 20, 2020 at 3:47 PM Simon Marchi wrote: > On 2020-03-20 5:10 p.m., Rocky Dunlap via lttng-dev wrote: > > I am trying to compile BT2 with Python bindings using Intel

[lttng-dev] Babeltrace2 - compilation error with intel18

2020-03-20 Thread Rocky Dunlap via lttng-dev
I am trying to compile BT2 with Python bindings using Intel18. I receive the following error during the build: In file included from py-common.c(31): /apps/intel/intelpython3/include/python3.6m/Python.h(149): error #193: zero used for undefined preprocessing identifier "_MSC_VER" #if _MSC_VER

Re: [lttng-dev] Large number of stream files in CTF trace -- too many file handles

2020-03-16 Thread Rocky Dunlap via lttng-dev
Jonathan, Increasing the soft FD limit worked great, both for the command line and the Python script. Thanks for the help! Rocky On Mon, Mar 16, 2020 at 8:51 AM Jonathan Rajotte-Julien < jonathan.rajotte-jul...@efficios.com> wrote: > Hi, > > > If this is not the right approach, how should I pr

[lttng-dev] Large number of stream files in CTF trace -- too many file handles

2020-03-13 Thread Rocky Dunlap via lttng-dev
I am attempting to use babeltrace2 to read a CTF trace that has ~2000 stream files. This is a custom trace collected from an MPI application on an HPC platform. In this case, each MPI process opens and writes to its own stream file, so you end up with one file per MPI task. When I attempt to rea

[lttng-dev] bt2 Python quick start/examples?

2020-03-11 Thread Rocky Dunlap via lttng-dev
I see that the bt2 Python documentation is not yet ready: https://babeltrace.org/docs/v2.0/python/bt2/ In lieu of that, where is the best place to find at least a few examples of using the bt2 Python API for basic trace processing tasks? Thanks, Rocky _