Re: live cd
Oops, didn't send this to the ML... On 25 February 2011 21:44, Oz wrote: > Debian Gnu/Hurd seems to be complicated to install from what i > understand because 1 its not a live cd or install cd. 2-you need qemu > virtulization. I want to run it natevly as the sole operating system > on one netbook i have. i dont know or understand much i am a ubuntu > gnu/linux user and have been using gnu/linux for a couple of years and > am truly a noob compared to you guys but want to use a fully free > system and support the gnu project. but i need the easiesest route > possible :) thats why i was think about the arch hurd live cd. > If you're going to use the graphical one, be warned that that's pretty much just a proof-of-concept. It's a pet project one of the devs is working on and is pretty slow/unstable. Also, the official installation CD is pretty unstable, too. -- Michael Walker (http://www.barrucadu.co.uk) Arch Hurd Developer; GNU Webmaster; FSF member #8385 http://www.archhurd.org http://www.gnu.org http://www.fsf.org
XMLFS for GSoC
Hi, I'd like to apply to work on xmlfs for GSoC this summer. I've been going through the code over the past few days and working on it a bit, and have come up with a list of things to implement: * Fix all gcc warnings [done] This was really just so I'd be able to see which functions were where, this was all very trivial. * Support for changes to the backing store [begun] Not so trivial, as I haven't really done anything with ports before, which I'm using to monitor changes to the backing store. However, I'm making good progress with this and I think I'll be done within the next day or two. * Support for editing nodes * Support for adding and deleting nodes * Support for XSLT transformations * Document all functions thoroughly (their purpose, parameters, return values, and any changes to globals) As this is my first experience of translator programming, I haven't decided to attempt the complementary translator, as I don't think I would be able to do that in the time available. My goal for xmlfs is a translator that can be used to easily parse and modify XML files via a shell script, which is document thoroughly enough so that someone with little or no translator programming experience can understand and use it as a reference and help for their own work. The only thing I anticipate any particular difficulty with is the adding/deleting nodes and XSLT transformations, as I haven't worked with libxml2 before and so would need to read up on that. I'm a first year computer science student, and have been using C for a few years now. I'm pretty familiar with using the Hurd (due to Arch Hurd), but not so much the programming side of things, and this is the first time I've done anything with translators or ports. However, I'm picking it up as I go along (albeit with a lot of googling for appropriate functions and examples). Finally, even if this doesn't get picked as a GSoC task (after I send off my application), I still intend to do this over the summer to better learn Hurd programming, and so either way there will be an improved xmlfs this year :) -- Michael Walker (http://www.barrucadu.co.uk) Arch Hurd Developer; GNU Webmaster; FSF member #8385 http://www.archhurd.org http://www.gnu.org http://www.fsf.org
PATCH 1/2 - fix all compiler warnings. (was XMLFS for GSoC)
{ 0, 0, 0, 0, 0, 0 }
};
static const char args_doc[] = "XML-DOC";
@@ -52,37 +54,39 @@ static const char doc[] =
"\vThis translator appears like a directory which tries to match the XML"
" tree in XML-DOC as closely as possible.";
+error_t parse_opt (int key, char *arg, struct argp_state *state)
+{
+ switch (key)
+ {
+case 'd':
+ debug = fopen (arg, "w");
+ setbuf (debug, NULL);
+ break;
+case ARGP_KEY_ARG:
+ if (state->arg_num == 0)
+xmlfilename = arg;
+ else
+return ARGP_ERR_UNKNOWN;
+ break;
+default:
+ return ARGP_ERR_UNKNOWN;
+}
+
+return 0;
+}
+
int
main (int argc, char **argv)
{
mach_port_t bootstrap, underlying_node;
io_statbuf_t underlying_stat;
file_t xmlfile;
- char *xmlfilename = NULL;
error_t err;
+ xmlfilename = NULL;
debug = NULL;
- error_t parse_opt (int key, char *arg, struct argp_state *state)
-{
- switch (key)
- {
- case 'd':
- debug = fopen (arg, "w");
- setbuf (debug, NULL);
- break;
- case ARGP_KEY_ARG:
- if (state->arg_num == 0)
- xmlfilename = arg;
- else
- return ARGP_ERR_UNKNOWN;
- break;
- default:
- return ARGP_ERR_UNKNOWN;
- }
- return 0;
-}
- struct argp argp = { options, parse_opt, args_doc, doc };
+ struct argp argp = { options, parse_opt, args_doc, doc, 0, 0, 0 };
asprintf ((char **) &argp_program_version, "%s %s",
netfs_server_name, netfs_server_version);
@@ -104,9 +108,9 @@ main (int argc, char **argv)
if (!xmlfilename)
/* Try to open the underlying node, which is incidently
our default XML file. */
-xmlfile = openport (underlying_node, O_READ);
+xmlfile = (file_t) openport (underlying_node, O_READ);
else
-xmlfile = open (xmlfilename, O_READ);
+xmlfile = (file_t) open (xmlfilename, O_READ);
xmlfs = malloc (sizeof (struct xmlfs));
@@ -114,7 +118,7 @@ main (int argc, char **argv)
netfs_root_node->nn_stat = underlying_stat;
netfs_root_node->nn_stat.st_mode =
-S_IFDIR | (underlying_stat.st_mode & ~S_IFMT & ~S_ITRANS);
+S_IFDIR | (underlying_stat.st_mode & (unsigned int) ~S_IFMT &
(unsigned int) ~S_ITRANS);
if (err)
error (1, err, "Cannot create filesystem");
diff --git a/xmlfs.h b/xmlfs.h
index e3c0af6..d47613e 100644
--- a/xmlfs.h
+++ b/xmlfs.h
@@ -26,6 +26,7 @@
#include
#include
#include
+#include
#include
@@ -61,7 +62,12 @@ extern struct xmlfs *xmlfs;
error_t xmlfs_create (file_t, struct xmlfs *);
+/* Parse an option from the argv array */
+error_t parse_opt (int key, char *arg, struct argp_state *state);
+
+extern char *xmlfilename;
+
extern FILE *debug;
-#define DEBUG(format, ...) if (debug) fprintf (debug, format, ## __VA_ARGS__)
+#define DEBUG(...) if (debug) fprintf (debug, __VA_ARGS__)
#endif /* __XMLFS_H__ */
--
Michael Walker (http://www.barrucadu.co.uk)
Arch Hurd Developer; GNU Webmaster; FSF member #8385
http://www.archhurd.org http://www.gnu.org http://www.fsf.org
PATCH 2/2 - monitor backing store changes and update. (was XMLFS for GSoC)
fs.c
+++ b/xmlfs.c
@@ -20,6 +20,7 @@
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111, USA.
*/
+#include "monitor.h"
#include "xmlfs.h"
#include "version.h"
@@ -75,13 +76,30 @@ error_t parse_opt (int key, char *arg, struct
argp_state *state)
return 0;
}
+void
+rebuild_xmlfs (void* param)
+{
+ file_t xmlfile;
+ error_t err;
+
+ (void) param;
+
+ DEBUG ("NOTICE: Reloading XML file\n");
+
+ xmlfile = (file_t) open (xmlfilename, O_READ);
+
+ err = xmlfs_create (xmlfile, xmlfs);
+ if (err)
+error (1, err, "Cannot update filesystem");
+}
+
int
main (int argc, char **argv)
{
- mach_port_t bootstrap, underlying_node;
+ mach_port_t bootstrap, underlying_node, xmlport;
io_statbuf_t underlying_stat;
- file_t xmlfile;
error_t err;
+ file_t xmlfile;
xmlfilename = NULL;
debug = NULL;
@@ -108,10 +126,16 @@ main (int argc, char **argv)
if (!xmlfilename)
/* Try to open the underlying node, which is incidently
our default XML file. */
-xmlfile = (file_t) openport (underlying_node, O_READ);
+{
+ xmlfile = (file_t) openport (underlying_node, O_READ);
+ xmlport = MACH_PORT_NULL;
+}
else
-xmlfile = (file_t) open (xmlfilename, O_READ);
-
+{
+ xmlfile = (file_t) open (xmlfilename, O_READ);
+ xmlport = file_name_lookup (xmlfilename, 0, 0);
+}
+
xmlfs = malloc (sizeof (struct xmlfs));
err = xmlfs_create (xmlfile, xmlfs);
@@ -123,6 +147,16 @@ main (int argc, char **argv)
if (err)
error (1, err, "Cannot create filesystem");
+ /* Monitor changes on the underlying XML file */
+ DEBUG ("INFO: Starting file change monitor\n");
+
+ if (xmlport != MACH_PORT_NULL)
+{
+ err = set_file_monitor (xmlport, &rebuild_xmlfs, NULL);
+ if (err)
+error (1, err, "Cannot start file change monitor\n");
+}
+
DEBUG ("INFO: Entering main loop\n");
netfs_server_loop ();
diff --git a/xmlfs.h b/xmlfs.h
index d47613e..fa5aeb2 100644
--- a/xmlfs.h
+++ b/xmlfs.h
@@ -62,6 +62,9 @@ extern struct xmlfs *xmlfs;
error_t xmlfs_create (file_t, struct xmlfs *);
+/* Rebuild the XMLFS data structure */
+void rebuild_xmlfs (void* param);
+
/* Parse an option from the argv array */
error_t parse_opt (int key, char *arg, struct argp_state *state);
--
Michael Walker (http://www.barrucadu.co.uk)
Arch Hurd Developer; GNU Webmaster; FSF member #8385
http://www.archhurd.org http://www.gnu.org http://www.fsf.org
Re: XMLFS for GSoC
> So the idea is that one would launch an XSLT translator, with a XSLT > style sheet as input, on top of an xmlfs directory, and the result would > be a file representing the output of the XSLT processor? Yes, that's what I was thinking. Though, being able to run it on top of an XML file should be pretty simple, too. > BTW, I fear that the scope of your proposal is a bit excessive. The > summer tends to be over pretty quickly -- unless you are extremely fast, > I don't think it's likely you will get past making xmlfs and perhaps > unxmlfs fully functional... I did originally just plan to do xmlfs, but then I realised that most of the nontrivial code in the xslt translator and in unxmlfs would be the same - the directory-tree-to-XML parser, which shouldn't really be too hard. The only other nontrivial part of the xslt code is the libnetfs-based translator component, but as that's just presenting the transformed XML, that could probably just be copied straight from xmlfs. Thus, I don't think much extra work will really be needed for either unxmlfs or xslt. Additionally, I did go from zero Hurd programming knowledge to implementing a working file change monitor in a couple of days, I do tend to pick things up pretty quickly. I think I'll have a go at the directory-tree-to-XML parser tonight and see how I feel about the proposal after that, and revise it if necessary. Thanks for the comments. -- Michael Walker (http://www.barrucadu.co.uk) Arch Hurd Developer; GNU Webmaster; FSF member #8385 http://www.archhurd.org http://www.gnu.org http://www.fsf.org
Re: PATCH 1/2 - fix all compiler warnings. (was XMLFS for GSoC)
> That's always a good start! Where is this repository that your patch is > against? http://cvs.savannah.gnu.org/viewvc/xmlfs/?root=hurdextras > Your editor / mailer broke the lines, so this patch won't even apply. > Either you instruct your editor / mailer, or let git send-email take care > of that, or attach the patches instead of publishing them inline. Heh, oops. I forgot claws-mail strictly enforces line length limits. I'll keep that in mind > Hrm. Not sure whether we'd like all of these. But you're (to the best > of my knowledge) to only person working on xmlfs these days, and we have > no general template, so I'd let you decide. If any of them turn out to be a problem, or overly complicate the code (as pretty much everything needs to be explicitly stated with those flags) I'll consider removing some of them. > If we really want this, wouldn't it be better to use ``__attribute__ > ((unused))'' with each of the functions' parameters? In Hurd code, we > can unconditionally use GNU C / GCC features. I didn't know of that attribute, and I think I will start using GCC/GNU stuff more (dropping -pedantic), as GCC is the compiler used (as antrik pointed out in another email) > I don't know the surrounding code yet, but should gsize perhaps by a > size_t instead? Probably, a lot of the 'problems' could undoubtedly have been fixed in other ways. > memcpy with length of one? Why not replace that with: > > data[size++] = '\n'; > > (Your cast of data doesn't look right anyway; do you understand and > agree?) I should have a look at the whole function -- it still looks a > bit suspicious: what will come after the '\n' charater; is a '\0' > expected there? Yes, that would be a much better solution. I didn't really read the code thoroughly enough when fixing the warnings, usually going for the most obvious solution (which may turn out to have caused some issues later down the line, I suppose). And yes, the case of data looks completely wrong *facepalm* > Why move these functions? Typically, all definitions should be as tight > in scope as possible. Warnings about nested functions, though if I'm going to drop -pedantic and use GCC extensions, that doesn't matter. > That doesn't look sane. According to netfs.h these indeed aren't const, > but why? They're used only in libnetfs/io-version.c. I'm not sure. I assumed there was a reason for libnetfs not having them as consts. > Both openport and open return an int file descriptor, so why is xmlfile a > file_t (which is another name for a mach_port_t)? Looks like xmlfs_create (in fs.c) is using a file_t as a file descriptor. As that's clearly wrong I'll change that. >> @@ -114,7 +118,7 @@ main (int argc, char **argv) >> >> netfs_root_node->nn_stat = underlying_stat; >> netfs_root_node->nn_stat.st_mode = >> - S_IFDIR | (underlying_stat.st_mode & ~S_IFMT & ~S_ITRANS); >> + S_IFDIR | (underlying_stat.st_mode & (unsigned int) ~S_IFMT & >> (unsigned int) ~S_ITRANS); > > What's the warning here? Implicit casting to unsigned int. Thanks for the comments. -- Michael Walker (http://www.barrucadu.co.uk) Arch Hurd Developer; GNU Webmaster; FSF member #8385 http://www.archhurd.org http://www.gnu.org http://www.fsf.org
Re: PATCH 2/2 - monitor backing store changes and update.
>> all: $(OBJS)
>> - $(CC) -o $(BINARY) $(OBJS) $(LDFLAGS)
>> + @$(CC) -o $(BINARY) $(OBJS) $(LDFLAGS)
>>
>> .c.o:
>> - $(COMPILE) -c $<
>> + @$(COMPILE) -c $<
>>
>> clean:
>> - rm -f *.o core *.obj *~ $(BINARY)
>> + @rm -f *.o core *.obj *~ $(BINARY) fs_notify.c
>
> This change is not related to the purpose of the patch... Please put it
> in an extra patch :-)
Ok. I sometimes include multiple small logical changes in one commit,
I'll have to get out of that habit.
>> -
>> +
>
> Avoid such gratuitous changes please...
Oops.
>> new file mode 100644
>> index 000..1a441ca
>> --- /dev/null
>> +++ b/monitor.c
>> @@ -0,0 +1,86 @@
>> +#include "monitor.h"
> [...]
>
> Missing Copyright/License header.
Ok, I wasn't entirely sure what to put as the xmlfs code I started
with had HurdFR stuff, so I decided to leave it until I learned what
the copyright situation for the xmlfs code is.
>> +int
>> +notice_change (mach_msg_header_t *inp, mach_msg_header_t *outp)
>> +{
>
> IIRC most Hurd code has a copy of the function documentation in the .c
> file... Don't know about the existing xmlfs code though?
Hmm, I definitely need to look up commenting/documentation in the GNU
coding style.
>> + if (handler != NULL)
>> + (*handler) (params);
>
> I don't think we should ever get into a situation where the notification
> is set up but the handler not? So I guess that should rather be an
> assert()?
Yes, this should be an assert.
>> +/* Only works with one file for now. TODO: work with multiple files */
>> +error_t
>> +set_file_monitor (file_t thefile, void (*thehandler) (void*), void
>> *theparams)
>
> Err... Why would we need to monitor multiple files in xmlfs?
I originally wrote the monitoring code as a standalone program,
intending to extend it. Forgot to remove some of the irrelevant
code/comments, it seems.
>> +{
>> + mach_port_t notify;
>> + error_t err;
>> + notice_t noticedata;
>> + cthread_t noticethread;
>> +
>> + if (thefile == MACH_PORT_NULL)
>> + error (1, 0, "Null file port given\n");
>
> Again, shouldn't that rather be an assert()?
Yes. I don't really use asserts for some reason, so I'll have to get
into the habit.
>> + if (err)
>> + return err;
>
> I think this will also leak bucket and class in the error case?
>
Will fix.
>> + char filename[1024]; /* Hard filename length limit of 1024, TODO:
>> better solution */
>
> Augh, augh, augh! :-)
>
> I think you should fix this before the patch is commited...
Argh, argh, argh. Note to self: before committing, grep for "TODO"
>
>> + xmlfile = (file_t) open (xmlfilename, O_READ);
>> +
>> + err = xmlfs_create (xmlfile, xmlfs);
>
> I believe this will leak a lot of stuff on each file change?
I'll definitely need to have a look at that. Almost certainly
(developing seems so much harder with no valgrind to run after every
change :P)
>> - mach_port_t bootstrap, underlying_node;
>> + mach_port_t bootstrap, underlying_node, xmlport;
>
> I'm somewhat confused by the xmlport/xmlfile duality... Perhaps you
> could add a comment explaining it?
Yes, I'll add a comment explaining that. Basically, it's because I
need to pass a port to file_notice_changes.
--
Michael Walker (http://www.barrucadu.co.uk)
Arch Hurd Developer; GNU Webmaster; FSF member #8385
http://www.archhurd.org http://www.gnu.org http://www.fsf.org
Re: XMLFS for GSoC
> I did originally just plan to do xmlfs, but then I realised that most > of the nontrivial code in the xslt translator and in unxmlfs would be > the same - the directory-tree-to-XML parser, which shouldn't really be > too hard. The only other nontrivial part of the xslt code is the > libnetfs-based translator component, but as that's just presenting the > transformed XML, that could probably just be copied straight from > xmlfs. Thus, I don't think much extra work will really be needed for > either unxmlfs or xslt. Ok, I've had a go at making an xmlfs-directory-structure to XML converter, and thought in more detail about what needs to be implemented for each translator and what can be shared, and I've concluded that I probably can get this done in the time available. Shared nontrivial code: * XML to libnetfs read-only translator: partially done Partial compliance to XML spec (problem being the use of, eg "foo0" rather than "foo[0]") * xmlfs tree representation to libxml2 representation: partially done Can descend down a directory tree and identify whether everything encountered is a tag name, attribute, or text node. Dies with an error if anything else is encountered. All that remains is having it build a libxml2 representation of the tree. * Watch a file/directory for changes and run a function when there is one: done Unshared nontrivial code: * XML writing Not needed for xslt because, as far as I'm aware, XSLT is a one-way mapping. I may be wrong however, so I'll look into this. All that needs to be done for this is to update the libxml2 representation of the XML document. This will be pretty tricky, as I'll have to learn to use libxml2 properly, but completely doable. * Read support in unxmlfs unxmlfs takes an xmlfs-compatible directory tree and turns it into an XML file, thus reading will be done via libtrivfs, with no write support (I see no need/use for that). I haven't used libtrivfs before, but hurdextras should provide more than enough examples for me to learn. The translators themselves will be implemented out of the following components: * xmlfs: * XML to libnetfs read-only translator * Watch a file/directory for changes and run a function when there is one * Update the presented directory hierarchy * XML writing * unxmlfs: * Watch a file/directory for changes and run a function when there is one * Update the presented XML file * xmlfs tree representation to libxml2 representation * Read support in unxmlfs * xslt * Watch a file/directory for changes and run a function when there is one * Update the presented directory hierarchy * xmlfs tree representation to libxml2 representation * Perform an XSLT transformation * XML to libnetfs read-only translator I think that this is perfectly achievable in the time available. -- Michael Walker (http://www.barrucadu.co.uk) Arch Hurd Developer; GNU Webmaster; FSF member #8385 http://www.archhurd.org http://www.gnu.org http://www.fsf.org
Re: XMLFS for GSoC
> Well, if you need the exact same code, this is a pretty good indication > that it probably shouldn't be in the XSLT translator at all... Working > on top of a directory tree would only make sense if it would actually > use the provided structure directly, rather than first serializing it. > > Or do you mean just the part that reads the directory structure and > constructs an internal DOM tree from it? Yes, that part. > I don't think the XSLT translator should present a directory tree as > output. Nothing in XSLT requires the output to be XML. Aha, I had a feeling I was misunderstanding something somewhere. In that case, xslt would be even simpler - the code would be pretty much the same as unxmlfs (turn an xmlfs directory tree into an internal DOM representation and present it as output via trivfs), but just with a transformation performed. In fact, if that's the case maybe even just a "--xslt=FILE" option to unxmlfs would do the trick. However, XSLT is commonly used to turn one XML document into another, so perhaps there should be an option to try and present it as a directory tree, but default to presenting just a single file. I need to think about that. In any case, that's not adding much complexity (probably simplifying things in some areas, in fact). -- Michael Walker (http://www.barrucadu.co.uk) Arch Hurd Developer; GNU Webmaster; FSF member #8385 http://www.archhurd.org http://www.gnu.org http://www.fsf.org
