On 2016-01-07 20:03, Richard Gaskin wrote:
I'm just far enough into Robert Love's "Linux System Programming" that
I think the solution to FastCGI may be much simpler than I'd
previously thought.
I think you need to read a bit more about fork ;)
I think we need a new command that launches a specified process but in
a way that uses a call to "fork" to pass file descriptors (which
include sockets and other I/O info) to the child process.
The 'fork()' system call is a very low-level primitive which is the
basis of executing other processes on UNIX based systems. You have to
use it very carefully - indeed, pretty much any use of fork will almost
immediately be followed by a call to some variant of 'exec', which
basically runs a different executable in the current process, replacing
the original one.
In many ways it would work very similarly to the existing "open
process", but allow params to give the child process access to things
like socket connections, pipes, files, etc. the parent process has
access to at the time the child process is launched.
Well 'gives access to' is slightly misleading here as it suggests that
you can use them all, which isn't really true. When you fork the kernel:
1) clones the task structure (the thing the kernel uses to represent a
process)
2) marks all memory pages as copy on write
3) increases the reference count on every thing attached to a file
descriptor
This means that when the child process starts running its memory image
is identical to the parent, and it has *exactly the same* file
descriptors as the parent.
Critically, any state which the original process has related to
connections to things (e.g. databases or display servers) is the same in
parent and child, which means they have the same 'state' on the other
end of the connection. This means that any usage of them in either after
the fork will cause things to break, probably horrendously as the two
processes run asynchronously.
This means that you need to engineer things so that the child gets
appropriate fd's and the parent keeps appropriate fd's. The typical
thing you do here is faff around with pipes. For example, here is the
critical bit of code from the engine's open process command:
int tochild[2];
int toparent[2];
int4 index = MCnprocesses;
if (pipe(tochild) == 0)
{
if (pipe(toparent) == 0)
{
MCU_realloc((char **)&MCprocesses, MCnprocesses,
MCnprocesses + 1, sizeof(Streamnode));
MCprocesses[MCnprocesses].name = strclone("shell");
MCprocesses[MCnprocesses].mode = OM_NEITHER;
MCprocesses[MCnprocesses].ohandle = NULL;
MCprocesses[MCnprocesses].ihandle = NULL;
if ((MCprocesses[MCnprocesses++].pid = fork()) == 0)
{
close(tochild[1]);
close(0);
dup(tochild[0]);
close(tochild[0]);
close(toparent[0]);
close(1);
dup(toparent[1]);
close(2);
dup(toparent[1]);
close(toparent[1]);
execl(MCshellcmd, MCshellcmd, "-s", NULL);
_exit(-1);
}
MCS_checkprocesses();
close(tochild[0]);
char *str = path2utf(ep.getsvalue().clone());
write(tochild[1], str, strlen(str));
delete str;
write(tochild[1], "\n", 1);
close(tochild[1]);
close(toparent[1]);
MCS_nodelay(toparent[0]);
if (MCprocesses[index].pid == -1)
{
if (MCprocesses[index].pid > 0)
MCS_kill(MCprocesses[index].pid, SIGKILL);
MCprocesses[index].pid = 0;
MCeerror->add(EE_SHELL_BADCOMMAND, 0, 0,
ep.getsvalue());
return IO_ERROR;
}
}
else
{
close(tochild[0]);
close(tochild[1]);
MCeerror->add(EE_SHELL_BADCOMMAND, 0, 0,
ep.getsvalue());
return IO_ERROR;
}
}
But for those of you more familiar with Linux system programming, do I
misunderstand the difficulty involved?
Yes - fork() isn't what you are looking for. It isn't magical - it does
*precisely* what it says it does which is insufficient for what you are
proposing (and isn't what it is really used for anywhere).
Forking seems so common in other tools, and not having it appears to
be the one detail standing between where we are now and having not
just FastCGI, but also being able to build truly excellent application
servers on par with Node.js and other similar systems.
I'd say forking (except the purpose of implementing the equivalent of
shell or open process) is actually very rare. Any language which offers
bindings to system libraries and such will likely have os.fork() or some
such for completeness - however this doesn't mean it is actually used
very much (nor should it - there are few patterns around fork which are
safe and generally useful - shell and open process pretty much cover 99%
of them).
LiveCode is a great language, and if we had the ability to fork we
should be able to build a wide range of powerful, scalable, efficient
systems, breaking far beyond the limitations of CGI we're limited to
now.
Using LiveCode in the contexts you are talking about doesn't require
fork().
LiveCode can almost be used as a Node.js type server already - start a
-ui standalone engine running which 'accepts connections' and dispatches
requests for appropriate protocols out to whatever you need (the missing
bit, as I've mentioned before is that some 'long' running things such as
db access block, rather than do things asynchronously). Like other
Node.js style setups, I'd imagine you'd need some sort of
'load-balancing' magic to farm the Node.js-like processes which farm the
requests - I'm sure others know more about this than me.
Adding FastCGI support to LiveCode server would also be a similar
solution (where the blocking nature of some aspects of LiveCode wouldn't
matter so much). When you run this through something such as mod_fcgi,
you get the 'farming' of processes for free, so in LiveCode you only
have to worry about servicing the requests, rather than how processes
get started up and shutdown and used.
Basically, managing collections of processes to fulfil CGI-like requests
(which is what fork() - because it is what you use to do open process -
is used for in these contexts) is already a solved problem - there is no
need for it to be solved again.
If all we need is a new command to wrap the Linux "fork" call, after I
finish Love's book I may brush up on my C skills and give it a go.
No need to 'brush up' on your C skills. Something along the lines of the
following in LiveCode Builder should work:
library module org.livecode.fork
foreign handler _Fork() returns CInt binds to "fork"
public handle Fork()
return _Fork()
end handler
end module
Although, as mentioned above, it won't get you very far for the reasons
outlined above.
Warmest Regards,
Mark.
--
Mark Waddingham ~ m...@livecode.com ~ http://www.livecode.com/
LiveCode: Everyone can create apps
_______________________________________________
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
