[lttng-dev] [PATCH urcu] Cleanup: missing sign compare fixes
Signed-off-by: Michael Jeanson
---
tests/benchmark/test_perthreadlock.c | 2 +-
tests/benchmark/test_urcu_gc.c | 4 ++--
tests/benchmark/test_urcu_qsbr_gc.c | 4 ++--
3 files changed, 5 insertions(+), 5 deletions(-)
diff --git a/tests/benchmark/test_perthreadlock.c
b/tests/benchmark/test_perthreadlock.c
index 512b1fc..13fc820 100644
--- a/tests/benchmark/test_perthreadlock.c
+++ b/tests/benchmark/test_perthreadlock.c
@@ -207,7 +207,7 @@ void *thr_reader(void *data)
void *thr_writer(void *data)
{
unsigned long wtidx = (unsigned long)data;
- long tidx;
+ unsigned long tidx;
printf_verbose("thread_begin %s, tid %lu\n",
"writer", urcu_get_thread_id());
diff --git a/tests/benchmark/test_urcu_gc.c b/tests/benchmark/test_urcu_gc.c
index 1e50e4c..34cf141 100644
--- a/tests/benchmark/test_urcu_gc.c
+++ b/tests/benchmark/test_urcu_gc.c
@@ -55,7 +55,7 @@ static unsigned long wdelay;
static struct test_array *test_rcu_pointer;
-static unsigned int reclaim_batch = 1;
+static long reclaim_batch = 1;
struct reclaim_queue {
void **queue; /* Beginning of queue */
@@ -449,7 +449,7 @@ int main(int argc, char **argv)
printf("SUMMARY %-25s testdur %4lu nr_readers %3u rdur %6lu wdur %6lu "
"nr_writers %3u "
"wdelay %6lu nr_reads %12llu nr_writes %12llu nr_ops %12llu "
- "batch %u\n",
+ "batch %ld\n",
argv[0], duration, nr_readers, rduration, wduration,
nr_writers, wdelay, tot_reads, tot_writes,
tot_reads + tot_writes, reclaim_batch);
diff --git a/tests/benchmark/test_urcu_qsbr_gc.c
b/tests/benchmark/test_urcu_qsbr_gc.c
index 11f0359..26bc352 100644
--- a/tests/benchmark/test_urcu_qsbr_gc.c
+++ b/tests/benchmark/test_urcu_qsbr_gc.c
@@ -57,7 +57,7 @@ static unsigned long duration;
/* read-side C.S. duration, in loops */
static unsigned long rduration;
-static unsigned int reclaim_batch = 1;
+static long reclaim_batch = 1;
struct reclaim_queue {
void **queue; /* Beginning of queue */
@@ -450,7 +450,7 @@ int main(int argc, char **argv)
printf("SUMMARY %-25s testdur %4lu nr_readers %3u rdur %6lu wdur %6lu "
"nr_writers %3u "
"wdelay %6lu nr_reads %12llu nr_writes %12llu nr_ops %12llu "
- "batch %u\n",
+ "batch %ld\n",
argv[0], duration, nr_readers, rduration, wduration,
nr_writers, wdelay, tot_reads, tot_writes,
tot_reads + tot_writes, reclaim_batch);
--
2.17.1
___
lttng-dev mailing list
lttng-dev@lists.lttng.org
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
Re: [lttng-dev] [PATCH urcu] Cleanup: missing sign compare fixes
Merged with modifications. test_perthreadlock compares the iterator against 0
with a decrementing iterator, so we need to work on long.
Thanks,
Mathieu
- On Dec 19, 2018, at 11:44 AM, Michael Jeanson mjean...@efficios.com wrote:
> Signed-off-by: Michael Jeanson
> ---
> tests/benchmark/test_perthreadlock.c | 2 +-
> tests/benchmark/test_urcu_gc.c | 4 ++--
> tests/benchmark/test_urcu_qsbr_gc.c | 4 ++--
> 3 files changed, 5 insertions(+), 5 deletions(-)
>
> diff --git a/tests/benchmark/test_perthreadlock.c
> b/tests/benchmark/test_perthreadlock.c
> index 512b1fc..13fc820 100644
> --- a/tests/benchmark/test_perthreadlock.c
> +++ b/tests/benchmark/test_perthreadlock.c
> @@ -207,7 +207,7 @@ void *thr_reader(void *data)
> void *thr_writer(void *data)
> {
> unsigned long wtidx = (unsigned long)data;
> - long tidx;
> + unsigned long tidx;
>
> printf_verbose("thread_begin %s, tid %lu\n",
> "writer", urcu_get_thread_id());
> diff --git a/tests/benchmark/test_urcu_gc.c b/tests/benchmark/test_urcu_gc.c
> index 1e50e4c..34cf141 100644
> --- a/tests/benchmark/test_urcu_gc.c
> +++ b/tests/benchmark/test_urcu_gc.c
> @@ -55,7 +55,7 @@ static unsigned long wdelay;
>
> static struct test_array *test_rcu_pointer;
>
> -static unsigned int reclaim_batch = 1;
> +static long reclaim_batch = 1;
>
> struct reclaim_queue {
> void **queue; /* Beginning of queue */
> @@ -449,7 +449,7 @@ int main(int argc, char **argv)
> printf("SUMMARY %-25s testdur %4lu nr_readers %3u rdur %6lu wdur %6lu "
> "nr_writers %3u "
> "wdelay %6lu nr_reads %12llu nr_writes %12llu nr_ops %12llu "
> - "batch %u\n",
> + "batch %ld\n",
> argv[0], duration, nr_readers, rduration, wduration,
> nr_writers, wdelay, tot_reads, tot_writes,
> tot_reads + tot_writes, reclaim_batch);
> diff --git a/tests/benchmark/test_urcu_qsbr_gc.c
> b/tests/benchmark/test_urcu_qsbr_gc.c
> index 11f0359..26bc352 100644
> --- a/tests/benchmark/test_urcu_qsbr_gc.c
> +++ b/tests/benchmark/test_urcu_qsbr_gc.c
> @@ -57,7 +57,7 @@ static unsigned long duration;
>
> /* read-side C.S. duration, in loops */
> static unsigned long rduration;
> -static unsigned int reclaim_batch = 1;
> +static long reclaim_batch = 1;
>
> struct reclaim_queue {
> void **queue; /* Beginning of queue */
> @@ -450,7 +450,7 @@ int main(int argc, char **argv)
> printf("SUMMARY %-25s testdur %4lu nr_readers %3u rdur %6lu wdur %6lu "
> "nr_writers %3u "
> "wdelay %6lu nr_reads %12llu nr_writes %12llu nr_ops %12llu "
> - "batch %u\n",
> + "batch %ld\n",
> argv[0], duration, nr_readers, rduration, wduration,
> nr_writers, wdelay, tot_reads, tot_writes,
> tot_reads + tot_writes, reclaim_batch);
> --
> 2.17.1
--
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com
___
lttng-dev mailing list
lttng-dev@lists.lttng.org
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
[lttng-dev] [PATCH lttng-tools] Fix: don't start tracing twice for UST applications
The change introduced by commit 0498a00cbc "Fix: sessiond: don't
allocate buffers and files for inactive sessions" adds a
ust_app_global_update() for each application without removing the
pre-existing ust_app_start_trace(), which effectively attempts to start
tracing twice for each application already registered prior to the
"start" command.
Signed-off-by: Mathieu Desnoyers
---
src/bin/lttng-sessiond/ust-app.c | 9 -
1 file changed, 9 deletions(-)
diff --git a/src/bin/lttng-sessiond/ust-app.c b/src/bin/lttng-sessiond/ust-app.c
index 0b0a6253..feb59d80 100644
--- a/src/bin/lttng-sessiond/ust-app.c
+++ b/src/bin/lttng-sessiond/ust-app.c
@@ -5018,7 +5018,6 @@ end:
*/
int ust_app_start_trace_all(struct ltt_ust_session *usess)
{
- int ret = 0;
struct lttng_ht_iter iter;
struct ust_app *app;
@@ -5044,14 +5043,6 @@ int ust_app_start_trace_all(struct ltt_ust_session
*usess)
ust_app_global_update(usess, app);
}
- cds_lfht_for_each_entry(ust_app_ht->ht, &iter.iter, app, pid_n.node) {
- ret = ust_app_start_trace(usess, app);
- if (ret < 0) {
- /* Continue to next apps even on error */
- continue;
- }
- }
-
rcu_read_unlock();
return 0;
--
2.11.0
___
lttng-dev mailing list
lttng-dev@lists.lttng.org
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
[lttng-dev] tracing multithread user program and API support for enabling/disabling events and for adding/removing context fields
We are experimenting LTTng for tracing multi-threaded program, it works very well for us. Thank you for having this great tool. But we have some concerns about the overhead and scalability of the tracing. Could you share some insight of the following questions? 1. The session domain communicates with the user application via Unix domain socket, from LTTng document. is the communication frequent, such as each event requires communication, or the communication just happens at the beginning to configure user space tracing? 2. For the consumer domain, is the consumer domain has a thread per CPU/channel to write to disk or relay the traces, or it is a single threaded-process handling all the channels and ring buffers, which could become a bottleneck if we have large number of user threads all feeding traces? 3. In the one channel/ring buffer per CPU setting, if a user thread migrates from one CPU to another, are the traces generated by that user thread fed to the two channels/buffers for the two CPUs? 4. So far, the events for tracing can be enabled and disabled from command line, are you considering to have runtime options (APIs) to enable or disable certain events? Or this is the feature that already in or can be implemented in different way? 5. For context field, from the document, context fields cannot be removed from a channel once you add it. I would like to request a feature to allow removing context fields in the user program. Thank you very much. Yonghong ___ lttng-dev mailing list lttng-dev@lists.lttng.org https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
[lttng-dev] [PATCH lttng-tools] Fix: session list lock must be held on session put operation
Signed-off-by: Jonathan Rajotte --- src/bin/lttng-sessiond/rotation-thread.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/bin/lttng-sessiond/rotation-thread.c b/src/bin/lttng-sessiond/rotation-thread.c index 8e63e160f..6669372dd 100644 --- a/src/bin/lttng-sessiond/rotation-thread.c +++ b/src/bin/lttng-sessiond/rotation-thread.c @@ -761,9 +761,9 @@ int handle_job_queue(struct rotation_thread_handle *handle, * possible for a job targeting that session to have * already been queued before it was destroyed. */ - session_unlock_list(); free(job); session_put(session); + session_unlock_list(); continue; } @@ -832,7 +832,6 @@ int handle_condition(const struct lttng_condition *condition, goto end; } session_lock(session); - session_unlock_list(); ret = unsubscribe_session_consumed_size_rotation(session, notification_thread_handle); @@ -861,6 +860,7 @@ int handle_condition(const struct lttng_condition *condition, end_unlock: session_unlock(session); session_put(session); + session_unlock_list(); end: return ret; } -- 2.17.1 ___ lttng-dev mailing list lttng-dev@lists.lttng.org https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
Re: [lttng-dev] tracing multithread user program and API support for enabling/disabling events and for adding/removing context fields
Hi Yonghong, - On Dec 19, 2018, at 1:19 PM, Yonghong Yan wrote: > We are experimenting LTTng for tracing multi-threaded program, it works very > well for us. Thank you for having this great tool. But we have some concerns > about the overhead and scalability of the tracing. Could you share some > insight > of the following questions? > 1. The session domain communicates with the user application via Unix domain > socket, from LTTng document. is the communication frequent, such as each event > requires communication, or the communication just happens at the beginning to > configure user space tracing? This Unix socket is only for "control" of tracing (infrequent communication). The high-throughput tracing data goes through a shared memory map (per-cpu buffers). > 2. For the consumer domain, is the consumer domain has a thread per > CPU/channel > to write to disk or relay the traces, or it is a single threaded-process > handling all the channels and ring buffers, which could become a bottleneck if > we have large number of user threads all feeding traces? Each consumer daemon is a single thread at the moment. It could be improved by implementing a multithreaded design in the future. It should help especially in NUMA setups, where having the consumer daemon on the same NUMA node as the ring buffer it reads from would minimize the amount of remote NUMA accesses. Another point is cases where I/O is performed to various target locations (different network interfaces or disks). When all I/O goes through the same interface, the bottleneck becomes the block device or the network interface. However, for scenarios involving many network interfaces or block devices, then multithreading the consumer daemon could become useful. This has not been a priority for anyone so far though. > 3. In the one channel/ring buffer per CPU setting, if a user thread migrates > from one CPU to another, are the traces generated by that user thread fed to > the two channels/buffers for the two CPUs? The event generated will belong to the per-cpu buffer on which the "sched_getcpu()" invocation occurs for the event. It is only saved into a single per-cpu buffer, even if the thread is migrated to a different CPU before it completes writing the event. This effectively creates infrequent situations where threads write into other cpu's per-cpu buffers. Note that the "reserve" and "commit" operations are smp-safe in lttng-ust for that reason. > 4. So far, the events for tracing can be enabled and disabled from command > line, > are you considering to have runtime options (APIs) to enable or disable > certain > events? Or this is the feature that already in or can be implemented in > different way? We already expose a public LGPLv2.1 API for this. See lttng-tools: include/lttng/event.h: lttng_enable_event() It is implemented by liblttng-ctl.so > 5. For context field, from the document, context fields cannot be removed > from a > channel once you add it. I would like to request a feature to allow removing > context fields in the user program. That's unfortunately not that simple. The channel context field belongs to the channel, which maps to the "stream" description in the resulting CTF metadata in the output trace. That stream description is invariant once it has been created. So currently the only way to remove a context would be to destroy your tracing session and create a new one. Thanks for your interest in LTTng! Mathieu > Thank you very much. > Yonghong > ___ > lttng-dev mailing list > lttng-dev@lists.lttng.org > https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev -- Mathieu Desnoyers EfficiOS Inc. http://www.efficios.com ___ lttng-dev mailing list lttng-dev@lists.lttng.org https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
Re: [lttng-dev] tracing multithread user program and API support for enabling/disabling events and for adding/removing context fields
Mathieu, Thank you for your response. see inline ... On Wed, Dec 19, 2018 at 4:20 PM Mathieu Desnoyers < mathieu.desnoy...@efficios.com> wrote: > Hi Yonghong, > > - On Dec 19, 2018, at 1:19 PM, Yonghong Yan wrote: > > We are experimenting LTTng for tracing multi-threaded program, it works > very well for us. Thank you for having this great tool. But we have some > concerns about the overhead and scalability of the tracing. Could you share > some insight of the following questions? > 1. The session domain communicates with the user application via Unix > domain socket, from LTTng document. is the communication frequent, such as > each event requires communication, or the communication just happens at the > beginning to configure user space tracing? > > This Unix socket is only for "control" of tracing (infrequent > communication). The high-throughput tracing data goes through a shared > memory map (per-cpu buffers). > > 2. For the consumer domain, is the consumer domain has a thread per > CPU/channel to write to disk or relay the traces, or it is a single > threaded-process handling all the channels and ring buffers, which could > become a bottleneck if we have large number of user threads all feeding > traces? > > Each consumer daemon is a single thread at the moment. It could be > improved by implementing a multithreaded design in the future. It should > help especially in NUMA setups, where having the consumer daemon on the > same NUMA node as the ring buffer it reads from would minimize the amount > of remote NUMA accesses. > > Another point is cases where I/O is performed to various target locations > (different network interfaces or disks). When all I/O goes through the same > interface, the bottleneck becomes the block device or the network > interface. However, for scenarios involving many network interfaces or > block devices, then multithreading the consumer daemon could become useful. > > This has not been a priority for anyone so far though. > That makes sense. > 3. In the one channel/ring buffer per CPU setting, if a user thread > migrates from one CPU to another, are the traces generated by that user > thread fed to the two channels/buffers for the two CPUs? > > The event generated will belong to the per-cpu buffer on which the > "sched_getcpu()" invocation occurs for the event. It is only saved into a > single per-cpu buffer, even if the thread is migrated to a different CPU > before it completes writing the event. This effectively creates infrequent > situations where threads write into other cpu's per-cpu buffers. Note that > the "reserve" and "commit" operations are smp-safe in lttng-ust for that > reason. > so after the migration, new events will still be written to another CPU's buffer? > 4. So far, the events for tracing can be enabled and disabled from command > line, are you considering to have runtime options (APIs) to enable or > disable certain events? Or this is the feature that already in or can be > implemented in different way? > > We already expose a public LGPLv2.1 API for this. See lttng-tools: > > include/lttng/event.h: lttng_enable_event() > > It is implemented by liblttng-ctl.so > Great and will try that. > 5. For context field, from the document, context fields cannot be removed > from a channel once you add it. I would like to request a feature to allow > removing context fields in the user program. > > That's unfortunately not that simple. The channel context field belongs to > the channel, which maps to the "stream" description in the resulting CTF > metadata in the output trace. That stream description is invariant once it > has been created. > That makes senses. We use babeltrace to process trace records. So far, it presents a single-ordered event sequence and we have to use a dedicated event record filed to separate the traces what are from different channels/threads. Is there library or API (C or python-based) that allows us to process traces channel by channel? I saw the traces are saved in files of channels. Thank you. Yonghong > > So currently the only way to remove a context would be to destroy your > tracing session and create a new one. > > Thanks for your interest in LTTng! > > Mathieu > > > > Thank you very much. > Yonghong > > > > ___ > lttng-dev mailing list > lttng-dev@lists.lttng.org > https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev > > > -- > Mathieu Desnoyers > EfficiOS Inc. > http://www.efficios.com > ___ lttng-dev mailing list lttng-dev@lists.lttng.org https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
Re: [lttng-dev] tracing multithread user program and API support for enabling/disabling events and for adding/removing context fields
Got another question about lttng_enable_event(): Using this API will impact per-user-thread tracepoint or the whole-user-process? I am thinking of the whole process, but want to confirm. Yonghong On Wed, Dec 19, 2018 at 4:20 PM Mathieu Desnoyers < mathieu.desnoy...@efficios.com> wrote: > Hi Yonghong, > > - On Dec 19, 2018, at 1:19 PM, Yonghong Yan wrote: > > We are experimenting LTTng for tracing multi-threaded program, it works > very well for us. Thank you for having this great tool. But we have some > concerns about the overhead and scalability of the tracing. Could you share > some insight of the following questions? > 1. The session domain communicates with the user application via Unix > domain socket, from LTTng document. is the communication frequent, such as > each event requires communication, or the communication just happens at the > beginning to configure user space tracing? > > This Unix socket is only for "control" of tracing (infrequent > communication). The high-throughput tracing data goes through a shared > memory map (per-cpu buffers). > > 2. For the consumer domain, is the consumer domain has a thread per > CPU/channel to write to disk or relay the traces, or it is a single > threaded-process handling all the channels and ring buffers, which could > become a bottleneck if we have large number of user threads all feeding > traces? > > Each consumer daemon is a single thread at the moment. It could be > improved by implementing a multithreaded design in the future. It should > help especially in NUMA setups, where having the consumer daemon on the > same NUMA node as the ring buffer it reads from would minimize the amount > of remote NUMA accesses. > > Another point is cases where I/O is performed to various target locations > (different network interfaces or disks). When all I/O goes through the same > interface, the bottleneck becomes the block device or the network > interface. However, for scenarios involving many network interfaces or > block devices, then multithreading the consumer daemon could become useful. > > This has not been a priority for anyone so far though. > > 3. In the one channel/ring buffer per CPU setting, if a user thread > migrates from one CPU to another, are the traces generated by that user > thread fed to the two channels/buffers for the two CPUs? > > The event generated will belong to the per-cpu buffer on which the > "sched_getcpu()" invocation occurs for the event. It is only saved into a > single per-cpu buffer, even if the thread is migrated to a different CPU > before it completes writing the event. This effectively creates infrequent > situations where threads write into other cpu's per-cpu buffers. Note that > the "reserve" and "commit" operations are smp-safe in lttng-ust for that > reason. > > 4. So far, the events for tracing can be enabled and disabled from command > line, are you considering to have runtime options (APIs) to enable or > disable certain events? Or this is the feature that already in or can be > implemented in different way? > > We already expose a public LGPLv2.1 API for this. See lttng-tools: > > include/lttng/event.h: lttng_enable_event() > > It is implemented by liblttng-ctl.so > > 5. For context field, from the document, context fields cannot be removed > from a channel once you add it. I would like to request a feature to allow > removing context fields in the user program. > > That's unfortunately not that simple. The channel context field belongs to > the channel, which maps to the "stream" description in the resulting CTF > metadata in the output trace. That stream description is invariant once it > has been created. > > So currently the only way to remove a context would be to destroy your > tracing session and create a new one. > > Thanks for your interest in LTTng! > > Mathieu > > > > Thank you very much. > Yonghong > > > > ___ > lttng-dev mailing list > lttng-dev@lists.lttng.org > https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev > > > -- > Mathieu Desnoyers > EfficiOS Inc. > http://www.efficios.com > ___ lttng-dev mailing list lttng-dev@lists.lttng.org https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
