Very cool. Would explicitly setting the byte order for each field be a short term workaround?
Sent from my BlackBerry - the most secure mobile device - via the Rogers Network From: eeppelitel...@gmail.com Sent: March 13, 2019 8:30 AM To: matthew.khou...@ericsson.com Cc: jonathan.rajotte-jul...@efficios.com; dschae...@blackberry.com; tracecompass-...@eclipse.org; lttng-dev@lists.lttng.org Subject: Re: [lttng-dev] [tracecompass-dev] Scalability On Wed, Mar 13, 2019 at 8:13 AM Matthew Khouzam <matthew.khou...@ericsson.com> wrote: > > Hi Doug & Efficios, > > > I suspect the issue is with libbabeltrace-ctf. When opening the trace in > Trace Compass, it failed saying the sequence was 2gb big. When opening in > babeltrace, it OOMed with 2gb allocations. When opening with a hex editor, I > saw 2gb allocations. Yes, byte order issues. Here's the fix: <https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_efficios_babeltrace_pull_103&d=DwIFaQ&c=yzoHOc_ZK-sxl-kfGNSEvlJYanssXN3q-lhj0sp26wE&r=NrrbvTHWa2Nbp_kAN0Hl1o3lM1WAwSes64uBjxjNhMc&m=mro2UhDCczUi0qeroLQR1acWtiyKnR0DeQlLZcufM9M&s=0QPaQrHAx605SVgAPeaZnF264NFCA44uOkcXdbV5dpA&e=>. Phil > > > On another subject, fixing the trace made it scale well in both babeltrace > and Trace Compass. > > > I don't know why, data_length writes the wrong data. Hope this helps all > parties. > > > BR, > > Matthew > > ________________________________ > From: tracecompass-dev-boun...@eclipse.org > <tracecompass-dev-boun...@eclipse.org> on behalf of Doug Schaefer > <dschae...@blackberry.com> > Sent: Tuesday, March 12, 2019 2:50:29 PM > To: jonathan.rajotte-jul...@efficios.com > Cc: lttng-dev@lists.lttng.org; tracecompass-...@eclipse.org > Subject: Re: [tracecompass-dev] [lttng-dev] Scalability > > I have attached the C file for a generator that creates a trace that > reproduces the memory issue with babeltrace. The same issue happens with a > custom program in the call to add a trace to a context. > > I am using babeltrace that I get from my Ubuntu 18.04 distro. It has the > version number 1.5.5. > > Thanks for the help! > > On Tue, 2019-03-12 at 15:47 +0000, Doug Schaefer wrote: > > In the thread on the tracecompass list, I also mentioned that the babeltrace > utility itself ran out of memory when reading the file. That's actually my > biggest worry. I got around the generator issue by flushing every 100K events. > > I'll create a quick generator that shows the same issue and post it in a bit. > > Doug. > > On Tue, 2019-03-12 at 11:34 -0400, Jonathan Rajotte-Julien wrote: > > On Tue, Mar 12, 2019 at 03:23:23PM +0000, Doug Schaefer wrote: > > It should be easy to generate a random one with the class I mentioned below > > and fill it with 16 million events. > > > Not if the problem is in the generator code ;). > > > BTW, two of the int fields are 5 and 10 > > bits respectively, but I'm not sure that matters (or at least it shouldn't). > > > Make sure to check the endian-ness of those fields. > > > What class? Not sure I see any reference to any. > > > Since you are generating CTF the error can be in the generator code. Please > > provide either a trace reproducing the issue or a base reproducer. It will be > > much easier overall than us trying to reproduce it blindly. > > > Cheers. > > > _______________________________________________ > > lttng-dev mailing list > > lttng-dev@lists.lttng.org > > https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.lttng.org_cgi-2Dbin_mailman_listinfo_lttng-2Ddev&d=DwICAg&c=yzoHOc_ZK-sxl-kfGNSEvlJYanssXN3q-lhj0sp26wE&r=NrrbvTHWa2Nbp_kAN0Hl1o3lM1WAwSes64uBjxjNhMc&m=DWwlFfl5FD5R8-HPY3GfbG6l6m7y_o2gXPER8IkQstk&s=bPK6X3vufUtEAemGpslNIlIL4UslBsgK-SWnraL-e9g&e= > > > --------------------------------------------------------------------- > This transmission (including any attachments) may contain confidential > information, privileged material (including material protected by the > solicitor-client or other applicable privileges), or constitute non-public > information. Any use of this information by anyone other than the intended > recipient is prohibited. If you have received this transmission in error, > please immediately reply to the sender and delete this information from your > system. Use, dissemination, distribution, or reproduction of this > transmission by unintended recipients is not authorized and may be unlawful. > _______________________________________________ > lttng-dev mailing list > lttng-dev@lists.lttng.org > https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.lttng.org_cgi-2Dbin_mailman_listinfo_lttng-2Ddev&d=DwIFaQ&c=yzoHOc_ZK-sxl-kfGNSEvlJYanssXN3q-lhj0sp26wE&r=NrrbvTHWa2Nbp_kAN0Hl1o3lM1WAwSes64uBjxjNhMc&m=mro2UhDCczUi0qeroLQR1acWtiyKnR0DeQlLZcufM9M&s=0dykfQKNunY0J3PjBwLhAqvF-WUIJ-gk74D-duicVKI&e=
_______________________________________________ lttng-dev mailing list lttng-dev@lists.lttng.org https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
