Sorry for the much delayed response. I have not had enough time lately!
* Nitish Saboo <nitish.sabo...@gmail.com> [190508 05:48]:
> "Do you mean "log/syslog" from the standard library? What does
> initialize do?"
>
> >>I have installed syslog-ng parser on my Linux box and I am planning you
> use syslog-ng parser and wanted to initialise it's engine for parsing the
> data.
There are two general aspects to your problem that are unclear. First,
what do you mean by "syslog-ng engine" or "syslog-ng parser" and how
does it run from within the C code? Second, what underlying problem are
you trying to solve that you believe running two of these "engines" on
different CPUs will solve?
In previous messages, you had just said "syslog engine" and gave no
indication that non-Go code was involved. Clarifying in your last
message that you are using C code with cgo is a good first step, but we
need more information.
I was thinking of three different things you might mean by "syslog
engine". The first is something that accepted messages from Go code in
this program and sent them to an external syslog daemon to be logged.
The second is something running in this program that acted as a
replacement for a standard syslog daemon. The third is something that
takes the output of a syslog daemon (the log files) and parses them to
allow analysis. A fourth possibility comes to mind, now that you have
specified that you are using syslog-ng through C code: a syslog-ng
plugin that is somehow called by an external syslog-ng daemon during its
handling of messages to be logged.
This fourth possibility now seems to be the most likely, based on your
last message, so I will focus my questions in this direction. Does the
initialize_engine function return after the initialization completes,
before any actual processing occurs, or does it keep running until that
"engine" has completed all its processing? If the former, how does the
"engine" ask your program to perform some processing? Is the
communication with the syslog-ng daemon hidden in a library linked to
your program, and the library invokes callbacks on a C function that you
provided, or does the library give you an open socket that you read and
write to handle processing requests from the daemon? Is the library
multi-thread aware? Does it create its own threads?
Regardless of which of the four possibilities (or others I haven't
thought of) is appropriate, the important question is how does the code
in initialize_engine arrange for processing of requests? Does it spawn
a new process? Does it create a new thread? Does it use C callback
functions? Does it use file descriptors, such as Unix sockets?
The second aspect, _why_ are you trying to run each engine on its own
CPU, is one that others in this thread have asked about, but your answer
has been "Yes, I want to run each engine on a separate CPU", which does
not give the type of information that helps us give you a good answer to
your question. You might be trying to provide fail-over, or load
balancing, or something else. Do you really need to lock each "engine"
on its own CPU, or is having two concurrently running "engines", letting
the Go scheduler take care of the details of OS threads and CPUs,
sufficient?
Keep in mind that when the function invoked as the go routine
terminates, Go no longer "knows" about the go routine. Any stack
allocated by Go for that go routine will be freed, and no further
scheduling of that sequence of instructions will occur. If the go
routine only calls Go code, this is simple enough, but when the go
routine calls C code which creates its own threads, you need to have a
deeper understanding of both cgo and the Go scheduler.
So, the LockOSThread call in the following might be completely
superfluous:
go func() {
runtime.LockOSThread()
C.initialize_engine(some, arguments)
}()
It will lock the go routine onto a single OS thread for the duration of
initialize_engine, but it may not have any effect on whatever "engine"
was started by initialize_engine after initialize_engine returns.
In fact, if initialize_engine, as part of creating the "engine", creates
a new OS thread, then the go routines themselves may be completely
superfluous, and something like this sequential code:
C.initialize_engine(some, arguments) // start first engine
C.initialize_engine(other, different_arguments) // start second engine
may be perfectly suitable.
In summary, for us to help you, you need to answer these two questions:
1. What problem are you trying to solve by locking each "engine" to its
own CPU (or thread or go routine)? (Hint: "running two instances of
engine" is not a sufficient answer.)
2. How does the code in initialize_engine change the execution state
(e.g. threads and processes) and how does the Go program communicate
with the "engine" after it is started?
...Marvin
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/golang-nuts/20190517163448.lw5yrxwrztbcx7ez%40basil.wdw.
For more options, visit https://groups.google.com/d/optout.
