Re: [PATCH v2] Explicitly check for linking format to use for efiemu64 module

2013-11-26 Thread Andrey Borzenkov
On Tue, Nov 26, 2013 at 11:53 AM, Beeblebrox  wrote:
> OK, I'll try it. But I need clarification:
>
> * Should I copy the repo and do a git rollback on the copy?

Just setup separate tree to perform build. The actual state does not
really matter; you can use current master, ignore doc build error for
now.

> * Easier to copy only grub/docs to another folder, but how do I start
> the build then? The Makefile in grub/docs will fail just as when run
> from top-level.
>
>>> just copy grub.texi over
> From my current repo, or an older ver?

git clone grub-repo
git bisect start
git bisect bad
git bisect good OLD-COMMIT-ID
cp docs/grub.texi /path/to/build/tree/docs/grub.texi
(cd /path/to/build/tree; make)

git bisect good|bad
cp docs/grub.texi /path/to/build/tree/docs/grub.texi
(cd /path/to/build/tree; make)

...

repeat until you find the first bad commit.man git-bisect.

___
Grub-devel mailing list
Grub-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/grub-devel


Re: Configure grub for pxe boot and nfs-mounted root

2013-11-26 Thread Beeblebrox
Since I got grub's pxe set to go, I have started to try the ideas & solutions.
* Answer to my first question from previous post (I don't see a way
where tftp-server would be able to serve-up the grub menu if it were
in say /data/tftp path?): This seems not possible. TFTPD cannot chroot
(-s) unless the folder structure is inside the jail. Setting to
different NFS path gives "folder not found" error from tftpd. (posted
for informational purposes)
* grub-mknetdir creates 700 permission folder structure. It needs to
be changed to 755 in order for tftpd to serve "core.0" (744 is
insufficient).
* grub-mknetdir folder is /data/amd64//boot/grub. I created a grub.cfg
under there AND under i386-pc. The client shows "welcome to grub"
message, then falls to "file not found". Was I supposed to pass the
grub.cfg location when I ran grub-mknetdir?
* Which insmod need for NFS exported folders? There's no nfs*.mod?
* Since I have 3 separate NFS exports which grub will choose from on
the menu, is it possible to create a "device.map" file with something
like:
(hd0,0) 192.168.2.1:/data/amd64
(hd0,1) 192.168.2.1:/data/i386
(hd0,2) 192.168.2.1:/data/isos
That way, I could do and change  set root='hd0,1 or 2 or 3' for each
menu entry? Or is the device.map deprecated?

Thanks for the input

___
Grub-devel mailing list
Grub-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/grub-devel


Re: Configure grub for pxe boot and nfs-mounted root

2013-11-26 Thread Vladimir 'φ-coder/phcoder' Serbinenko
On 26.11.2013 11:01, Beeblebrox wrote:
> Since I got grub's pxe set to go, I have started to try the ideas & solutions.
> * Answer to my first question from previous post (I don't see a way
> where tftp-server would be able to serve-up the grub menu if it were
> in say /data/tftp path?): This seems not possible. TFTPD cannot chroot
> (-s) unless the folder structure is inside the jail. Setting to
> different NFS path gives "folder not found" error from tftpd. (posted
> for informational purposes)
This doesn't seem to be a GRUB problem at all. I use tftpd which serves
from /var/tftpd
> * grub-mknetdir creates 700 permission folder structure. It needs to
> be changed to 755 in order for tftpd to serve "core.0" (744 is
> insufficient).
I'll think how to handle it sanely. Only mknetdir needs this change
> * grub-mknetdir folder is /data/amd64//boot/grub. I created a grub.cfg
> under there AND under i386-pc. The client shows "welcome to grub"
> message, then falls to "file not found". Was I supposed to pass the
> grub.cfg location when I ran grub-mknetdir?
boot/grub ($prefix) is where grub.cfg goes. Judging from your first
question, is it possible you serve from a different folder than you
think you do?
> * Which insmod need for NFS exported folders? There's no nfs*.mod?
You're right there isn't any. Right now we have only tftp and http.
Patches are welcome
> * Since I have 3 separate NFS exports which grub will choose from on
> the menu, is it possible to create a "device.map" file with something
> like:
> (hd0,0) 192.168.2.1:/data/amd64
> (hd0,1) 192.168.2.1:/data/i386
> (hd0,2) 192.168.2.1:/data/isos
> That way, I could do and change  set root='hd0,1 or 2 or 3' for each
> menu entry? Or is the device.map deprecated?
> 
No device.map is not for this at all. For once it's not used by runtime
at all.
> Thanks for the input
> 
> ___
> Grub-devel mailing list
> Grub-devel@gnu.org
> https://lists.gnu.org/mailman/listinfo/grub-devel
> 




signature.asc
Description: OpenPGP digital signature
___
Grub-devel mailing list
Grub-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/grub-devel


Re: Configure grub for pxe boot and nfs-mounted root

2013-11-26 Thread Beeblebrox
>> This doesn't seem to be a GRUB problem at all. I use tftpd which serves
from /var/tftpd
You are right, it is not a GRUB problem; it's a folder structure error
in the way I approached the problem.

>> boot/grub ($prefix) is where grub.cfg goes. Judging from your first
>> question, is it possible you serve from a different folder than you think 
>> you do?
No, the dhcp-tftp chain is correct. Because /data/amd64 is a jail
(like a chroot) and the chroot is where dhcp and tftpd are running, I
thought maybe the correct sytax should be "grub-mknetdir
--net-directory=/ --subdir=boot/grub", then copy the files to
boot/grub under the chroot folder. No difference, however and client
drops to "grub rescue". /var/log/xferlog shows:
in.tftpd[41762]: RRQ from 192.168.2.34 filename i386-pc/core.0
in.tftpd[41762]: tftp: client does not accept options
in.tftpd[41763]: RRQ from 192.168.2.34 filename i386-pc/core.0
in.tftpd[41764]: RRQ from 192.168.2.34 filename /boot/grub/i386-pc/normal.mod
in.tftpd[41764]: sending NAK (1, File not found) to 192.168.2.34

grub rescue> set
cmdpath=(tftp,192.168.2.1)i386-pc
prefix=(tftp,192.168.2.1)/boot/grub
root=tftp,192.168.2.1
net_pxe_boot_file=i386-pc/core.0

I changed the dhcp-tftpd configuration from prefix /boot/grub/i386-pc,
to test whether the problem lay in tftpd's inability to rad giles from
folders one level up (cd ..). Same result.

___
Grub-devel mailing list
Grub-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/grub-devel


Re: Configure grub for pxe boot and nfs-mounted root

2013-11-26 Thread Vladimir 'φ-coder/phcoder' Serbinenko
On 26.11.2013 12:26, Beeblebrox wrote:
>>> This doesn't seem to be a GRUB problem at all. I use tftpd which serves
> from /var/tftpd
> You are right, it is not a GRUB problem; it's a folder structure error
> in the way I approached the problem.
> 
>>> boot/grub ($prefix) is where grub.cfg goes. Judging from your first
>>> question, is it possible you serve from a different folder than you think 
>>> you do?
> No, the dhcp-tftp chain is correct. Because /data/amd64 is a jail
> (like a chroot) and the chroot is where dhcp and tftpd are running, I
> thought maybe the correct sytax should be "grub-mknetdir
> --net-directory=/ --subdir=boot/grub", then copy the files to
> boot/grub under the chroot folder. No difference, however and client
> drops to "grub rescue". /var/log/xferlog shows:
> in.tftpd[41762]: RRQ from 192.168.2.34 filename i386-pc/core.0
> in.tftpd[41762]: tftp: client does not accept options
> in.tftpd[41763]: RRQ from 192.168.2.34 filename i386-pc/core.0
> in.tftpd[41764]: RRQ from 192.168.2.34 filename /boot/grub/i386-pc/normal.mod
> in.tftpd[41764]: sending NAK (1, File not found) to 192.168.2.34
> 
You misconfigured your server --net-directory is where root of tftp is.
--subdir is subdirectory of it. Yet your server has root in /boot/grub
and not subdirectory
> grub rescue> set
> cmdpath=(tftp,192.168.2.1)i386-pc
> prefix=(tftp,192.168.2.1)/boot/grub
> root=tftp,192.168.2.1
> net_pxe_boot_file=i386-pc/core.0
> 
> I changed the dhcp-tftpd configuration from prefix /boot/grub/i386-pc,
> to test whether the problem lay in tftpd's inability to rad giles from
> folders one level up (cd ..). Same result.
> 
> ___
> Grub-devel mailing list
> Grub-devel@gnu.org
> https://lists.gnu.org/mailman/listinfo/grub-devel
> 




signature.asc
Description: OpenPGP digital signature
___
Grub-devel mailing list
Grub-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/grub-devel


Re: [PATCH v2] Support to disable reed-solomon codes

2013-11-26 Thread Vladimir 'φ-coder/phcoder' Serbinenko
On 26.11.2013 16:48, Jonathan McCune wrote:

> >  This redundancy may be cumbersome if attempting
> > +to cryptographically validate the contents of the bootloader
> embedding
> > +area, or in more modern systems with GPT-style partition tables
> > +(@pxref{BIOS installation}) where GRUB does not reside in any
> > +unpartitioned space outside of the MBR.  Disable the Reed-Solomon
> What's the reasonning behind GPT part?
> 
> 
> I looked at these archived threads discussing the reasoning behind
> including RS codes in the first place:
> http://lists.gnu.org/archive/html/grub-devel/2010-09/msg00218.html
> http://lists.gnu.org/archive/html/grub-devel/2010-09/msg00205.html
> ... and the motivation appeared to prioritize tolerating bad behavior
> from proprietary software over actual disk errors.  I'm not aware of
> weird proprietary software stealing blocks from a GPT BIOS boot partition.
>  
Perhaps we should contact Adobe to ask for an upgrade...
> Sure, changed in v3.  I left the actual option as --no-rs-codes, and it
> changes an option variable from its default of 1 to 0.
>  
Yes, that's what I meant.
> Okay, added __attribute__ ((unused)) and a comment where it gets passed
> a 0 on sparc64.  The way setup.c is written it would be more invasive to
> actually drop the parameter.
>  
Ok.
> 
> Okay, dropped.  Not sure if the way I #defined a NO_RS_CODES_KEY -1 is
> the right way to do an option with no short form.
>  
No, it should be enum and start at 0x100




signature.asc
Description: OpenPGP digital signature
___
Grub-devel mailing list
Grub-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/grub-devel


grub-mkimage and other utils documentation

2013-11-26 Thread Andrey Borzenkov
I started to clean --pubkey in docs, hit grub-mkimage reference, hit
"grub-install is just a shell script" ... it really needs cleanup. Is
grub-install terse description intentional? If not, I'm going to
actually document all utilities and all options not defined as hidden.
We probably need to mention common options between various utilities
as well.

Or should some tools be skipped? I'm fine to document them in
grub-devel though, but I'd like to have them documented /somewhere/.

___
Grub-devel mailing list
Grub-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/grub-devel


Re: grub-mkimage and other utils documentation

2013-11-26 Thread Bruce Dubbs

Andrey Borzenkov wrote:

I started to clean --pubkey in docs, hit grub-mkimage reference, hit
"grub-install is just a shell script" ... it really needs cleanup. Is
grub-install terse description intentional? If not, I'm going to
actually document all utilities and all options not defined as hidden.
We probably need to mention common options between various utilities
as well.

Or should some tools be skipped? I'm fine to document them in
grub-devel though, but I'd like to have them documented /somewhere/.


Things that can be run by a user or admin should be in the main 
documentation.  Those things run by a developer for 
building/testing/debugging, etc should be in grub-devel.


I'd put any program that is installed in the file system
/{,usr/}{,s}bin  directories in the category of admin.

  -- Bruce


___
Grub-devel mailing list
Grub-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/grub-devel


Re: grub-mkimage and other utils documentation

2013-11-26 Thread Andrey Borzenkov
В Tue, 26 Nov 2013 09:02:09 -0800
Jonathan McCune  пишет:

> On Tue, Nov 26, 2013 at 8:40 AM, Andrey Borzenkov wrote:
> 
> > I started to clean --pubkey in docs, hit grub-mkimage reference, hit
> > "grub-install is just a shell script" ... it really needs cleanup.
> 
> 
> I think the grub-mkimage reference can be dropped now that the native-code
> grub-install is in place and supports --pubkey itself.

It is far more than --pubkey. grub-install section itself starts with
reference to grub-mkimage.

>   Sorry, at the time
> those were written it was hard not to reveal some internals.  Thanks for
> taking the initiative to clean them up.
> 
> -Jon
> 
> 
> 
> > Is
> > grub-install terse description intentional? If not, I'm going to
> > actually document all utilities and all options not defined as hidden.
> > We probably need to mention common options between various utilities
> > as well.
> >
> > Or should some tools be skipped? I'm fine to document them in
> > grub-devel though, but I'd like to have them documented /somewhere/.
> >
> > ___
> > Grub-devel mailing list
> > Grub-devel@gnu.org
> > https://lists.gnu.org/mailman/listinfo/grub-devel
> >


___
Grub-devel mailing list
Grub-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/grub-devel


Re: [Xen-devel] pvgrub2 is merged

2013-11-26 Thread Fabio Fantoni


Il 25/11/2013 20:35, M A Young ha scritto:

On Mon, 25 Nov 2013, Fabio Fantoni wrote:


I did a test following informations on one of post before:
git clone git://git.sv.gnu.org/grub.git # commit 
61e1b9a49d48035bde52784abb54c3212b647fc8

./autogen.sh
./configure --target=x86_64 --with-platform=xen
mkdir -p boot/grub/
cat > boot/grub/grub.cfg <
You may want to adapt this script to your circumstances. I ended up with
cat > boot/grub/grub.cfg <./grub-mkstandalone --grub-mkimage=./grub-mkimage -o pvgrub2.xen -O 
x86_64-xen -d grub-core/ boot/grub/grub.cfg
I also suggest export pkgdatadir=. before this so it looks in the grub 
source rather than the installed version.


Thanks for reply.
Seems not working:
export pkgdatadir=. && ./grub-mkstandalone --grub-mkimage=./grub-mkimage 
-o pvgrub2.xen -O x86_64-xen -d grub-core/ boot/grub/grub.cfg
./grub-mkstandalone: warning: cannot open directory 
`/usr/local/share/locale': File o directory non esistente.


I also added the partition mods but Sid domU still unable to boot :(

I have also another question:
Is possible specify multiple path where search the grub.cfg for support 
all mainly distributions and add a custom cfg path support taking it 
from arguments?


Thanks for any reply and sorry for my bad english.



Of course this may not help your current problem, though I can boot a 
domU guest with grub configured as above via the hvc0 interface with 
vnc enabled.


Michael Young



___
Grub-devel mailing list
Grub-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/grub-devel


Re: [Xen-devel] pvgrub2 is merged

2013-11-26 Thread Andrey Borzenkov
В Tue, 26 Nov 2013 18:58:47 +0100
Fabio Fantoni  пишет:

> 
> I have also another question:
> Is possible specify multiple path where search the grub.cfg for support 
> all mainly distributions and add a custom cfg path support taking it 
> from arguments?
> 

You can do something like

if search --set root --file /boot/grub2/grub.cfg ; then
  configfile /boot/grub2/grub.cfg
elif search --set root --file /boot/grub/grub.cfg ; then
  configfile /boot/grub/grub.cfg
elif ...
  ...
fi

If xen provides way to pass arguments to kernel, it sure could be
implemented as arguments to grub. Actually someone asked for a way to
pass arguments to grub on EFI, so this could share implementation.

___
Grub-devel mailing list
Grub-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/grub-devel


Re: 2.02 Release roadmap

2013-11-26 Thread Andrey Borzenkov
В Mon, 25 Nov 2013 18:58:09 +0100
Vladimir 'φ-coder/phcoder' Serbinenko  пишет:

> Hello, all. It's time to start gearing towards 2.02 release
> - 2.01 number will be skipped in order to follow odd/even convention for
> release/git.
> On 17th December will be the feature freeze, after it only bugfixes and
> documentation will be committed.
> Release will be when we're confifent enough in absence of major bugs.
> Features I expect to go in before freeze:
> - Leif Lindholm's arm64 port: it's pretty complete from what I saw.
> - multiboot2 extension to skip teminating boot services
> - mac HFS+ install
> - yeeloong 3A support (almost done)
> - grub-file
> - truecrypt
> - Andrey's inline inode patch
> 
> In case anyone has unreviewed patches please inform me as soon as
> possible (mail could have been lost or I might have forgotten some of them)
> 

What is the state of new-autogen branch? That's something I'd
definitely welcome as autogen is not present in all distros which makes
maintaining patches that touch build system rather unpleasant.


signature.asc
Description: PGP signature
___
Grub-devel mailing list
Grub-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/grub-devel


Re: 2.02 Release roadmap

2013-11-26 Thread Vladimir 'φ-coder/phcoder' Serbinenko
On 26.11.2013 19:49, Andrey Borzenkov wrote:
> В Mon, 25 Nov 2013 18:58:09 +0100
> Vladimir 'φ-coder/phcoder' Serbinenko  пишет:
> 
>> Hello, all. It's time to start gearing towards 2.02 release
>> - 2.01 number will be skipped in order to follow odd/even convention for
>> release/git.
>> On 17th December will be the feature freeze, after it only bugfixes and
>> documentation will be committed.
>> Release will be when we're confifent enough in absence of major bugs.
>> Features I expect to go in before freeze:
>> - Leif Lindholm's arm64 port: it's pretty complete from what I saw.
>> - multiboot2 extension to skip teminating boot services
>> - mac HFS+ install
>> - yeeloong 3A support (almost done)
>> - grub-file
>> - truecrypt
>> - Andrey's inline inode patch
>>
>> In case anyone has unreviewed patches please inform me as soon as
>> possible (mail could have been lost or I might have forgotten some of them)
>>
> 
> What is the state of new-autogen branch? That's something I'd
> definitely welcome as autogen is not present in all distros which makes
> maintaining patches that touch build system rather unpleasant.
> 
Please start new threads for such requests
> 
> 
> ___
> Grub-devel mailing list
> Grub-devel@gnu.org
> https://lists.gnu.org/mailman/listinfo/grub-devel
> 




signature.asc
Description: OpenPGP digital signature
___
Grub-devel mailing list
Grub-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/grub-devel


Re: [Xen-devel] pvgrub2 is merged

2013-11-26 Thread Andrew Cooper
On 26/11/13 18:12, Andrey Borzenkov wrote:
> В Tue, 26 Nov 2013 18:58:47 +0100
> Fabio Fantoni  пишет:
>
>> I have also another question:
>> Is possible specify multiple path where search the grub.cfg for support 
>> all mainly distributions and add a custom cfg path support taking it 
>> from arguments?
>>
> You can do something like
>
> if search --set root --file /boot/grub2/grub.cfg ; then
>   configfile /boot/grub2/grub.cfg
> elif search --set root --file /boot/grub/grub.cfg ; then
>   configfile /boot/grub/grub.cfg
> elif ...
>   ...
> fi
>
> If xen provides way to pass arguments to kernel, it sure could be
> implemented as arguments to grub. Actually someone asked for a way to
> pass arguments to grub on EFI, so this could share implementation.

The way PV guests get a command line from the toolstack is via the
start_info.cmd_line, which is up to 1024 bytes.

This will be available to anything which gets its hand on the start info
page.

~Andrew

___
Grub-devel mailing list
Grub-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/grub-devel


[PATCH 2/3] Show file name in error messages from AutogenParser.

2013-11-26 Thread Colin Watson
---
 gentpl.py | 37 +
 1 file changed, 21 insertions(+), 16 deletions(-)

diff --git a/gentpl.py b/gentpl.py
index f0f4493..5bee43a 100644
--- a/gentpl.py
+++ b/gentpl.py
@@ -119,14 +119,15 @@ class AutogenState:
  need_idx, need_rbracket, indx_name, have_value, done) = range(12)
 
 class AutogenParseError(Exception):
-def __init__(self, message, line):
+def __init__(self, message, path, line):
 super(AutogenParseError, self).__init__(message)
+self.path = path
 self.line = line
 
 def __str__(self):
 return (
 super(AutogenParseError, self).__str__() +
-" at line %d" % self.line)
+" at file %s line %d" % (self.path, self.line))
 
 class AutogenDefinition(list):
 def __getitem__(self, key):
@@ -161,6 +162,7 @@ class AutogenParser:
 self.def_stack = [("", self.definitions)]
 self.curdef = None
 self.new_name = None
+self.cur_path = None
 self.cur_line = 0
 
 @staticmethod
@@ -172,6 +174,9 @@ class AutogenParser:
 def is_value_name_char(c):
 return c in ":^-_" or c.isalnum()
 
+def error(self, message):
+raise AutogenParseError(message, self.cur_file, self.cur_line)
+
 def read_tokens(self, f):
 data = f.read()
 end = len(data)
@@ -210,15 +215,13 @@ class AutogenParser:
 while True:
 offset += 1
 if offset >= end:
-raise AutogenParseError(
-"EOF in quoted string", self.cur_line)
+self.error("EOF in quoted string")
 if data[offset] == "\n":
 self.cur_line += 1
 if data[offset] == "\\":
 offset += 1
 if offset >= end:
-raise AutogenParseError(
-"EOF in quoted string", self.cur_line)
+self.error("EOF in quoted string")
 if data[offset] == "\n":
 self.cur_line += 1
 # Proper escaping unimplemented; this can be filled
@@ -260,21 +263,19 @@ class AutogenParser:
 yield AutogenToken.var_name, s
 offset = end_name
 else:
-raise AutogenParseError(
-"Invalid input character '%s'" % c, self.cur_line)
+self.error("Invalid input character '%s'" % c)
 yield AutogenToken.eof, None
 
 def do_need_name_end(self, token):
 if len(self.def_stack) > 1:
-raise AutogenParseError(
-"Definition blocks were left open", self.cur_line)
+self.error("Definition blocks were left open")
 
 def do_need_name_var_name(self, token):
 self.new_name = token
 
 def do_end_block(self, token):
 if len(self.def_stack) <= 1:
-raise AutogenParseError("Too many close braces", self.cur_line)
+self.error("Too many close braces")
 new_name, parent_def = self.def_stack.pop()
 parent_def.append((new_name, self.curdef))
 self.curdef = parent_def
@@ -292,7 +293,7 @@ class AutogenParser:
 def do_indexed_name(self, token):
 self.new_name = token
 
-def read_definitions(self, f):
+def read_definitions_file(self, f):
 self.curdef = self.definitions
 self.cur_line = 0
 state = AutogenState.init
@@ -367,12 +368,17 @@ class AutogenParser:
 if handler is not None:
 handler(token)
 else:
-raise AutogenParseError(
+self.error(
 "Parse error in state %s: unexpected token '%s'" % (
-state, token), self.cur_line)
+state, token))
 if state == AutogenState.done:
 break
 
+def read_definitions(self, path):
+self.cur_file = path
+with open(path) as f:
+self.read_definitions_file(f)
+
 defparser = AutogenParser()
 
 #
@@ -791,8 +797,7 @@ parser = OptionParser(usage="%prog DEFINITION-FILES")
 _, args = parser.parse_args()
 
 for arg in args:
-with open(arg) as f:
-defparser.read_definitions(f)
+defparser.read_definitions(arg)
 
 rules("module", module)
 rules("kernel", kernel)
-- 
1.8.4.3

___
Grub-devel mailing list
Grub-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/grub-devel


[PATCH 0/3] Eliminate the use of Autogen

2013-11-26 Thread Colin Watson
I already mentioned this on IRC, but I thought it'd be helpful to send
it here as well.

This patch series (also in the cjwatson/new-autogen git branch)
eliminates the use of Autogen and instead generates Automake input
directly from gentpl.py.

Autogen's definitions files were useful, and I've maintained
compatibility with the relevant subset of them, but the template
language was cumbersome; beyond a certain level of complexity it became
necessary to use the Guile extensions, and the impedance mismatch
between those and the Python template generation was considerable.  It's
simpler, much faster, and very much less confusing to generate
Makefile.*.am directly.

I've checked that this generates almost identical output, even when the
modules from grub-extras are in use.  The differences amount to a few
unimportant things such as removal of trailing newlines, and the
movement of a declarations block for libgnulib.a a bit further up the
file; they are short enough to be easily reviewable and not significant
enough to be worth contorting the generator code to avoid.

Colin Watson (3):
  Generate Makefile.*.am directly from gentpl.py, eliminating the use of
Autogen.  The Autogen definitions files remain intact as they offer
a useful abstraction.
  Show file name in error messages from AutogenParser.
  Handle #if/#endif and C-style comments in AutoGen definitions files.

 .gitignore   |1 -
 ChangeLog|6 +
 INSTALL  |1 -
 autogen.sh   |   11 +-
 conf/Makefile.common |   15 +-
 conf/Makefile.extra-dist |1 -
 gentpl.py| 1050 --
 7 files changed, 651 insertions(+), 434 deletions(-)

-- 
1.8.4.3

___
Grub-devel mailing list
Grub-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/grub-devel


[PATCH 1/3] Generate Makefile.*.am directly from gentpl.py, eliminating the use of Autogen. The Autogen definitions files remain intact as they offer a useful abstraction.

2013-11-26 Thread Colin Watson
---
 .gitignore   |1 -
 ChangeLog|6 +
 INSTALL  |1 -
 autogen.sh   |   11 +-
 conf/Makefile.common |   15 +-
 conf/Makefile.extra-dist |1 -
 gentpl.py| 1007 +++---
 7 files changed, 608 insertions(+), 434 deletions(-)

diff --git a/.gitignore b/.gitignore
index dc93061..2292cc9 100644
--- a/.gitignore
+++ b/.gitignore
@@ -159,7 +159,6 @@ GPATH
 GRTAGS
 GSYMS
 GTAGS
-Makefile.tpl
 compile
 depcomp
 mdate-sh
diff --git a/ChangeLog b/ChangeLog
index 279cc23..88b3c83 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,9 @@
+2013-11-26  Colin Watson  
+
+   Generate Makefile.*.am directly from gentpl.py, eliminating the use
+   of Autogen.  The Autogen definitions files remain intact as they
+   offer a useful abstraction.
+
 2013-11-26  Vladimir Serbinenko  
 
Add PCI command activation to all PCI drivers as required for coreboot
diff --git a/INSTALL b/INSTALL
index e81a095..cb9df30 100644
--- a/INSTALL
+++ b/INSTALL
@@ -55,7 +55,6 @@ need the following.
 * Python 2.5.2 or later
 * Autoconf 2.60 or later
 * Automake 1.10.1 or later
-* Autogen 5.10 or later
 
 Prerequisites for make-check:
 
diff --git a/autogen.sh b/autogen.sh
index 10fe68f..7424428 100755
--- a/autogen.sh
+++ b/autogen.sh
@@ -8,8 +8,6 @@ unset LC_ALL
 find . -iname '*.[ch]' ! -ipath './grub-core/lib/libgcrypt-grub/*' ! -ipath 
'./build-aux/*' ! -ipath './grub-core/lib/libgcrypt/src/misc.c' ! -ipath 
'./grub-core/lib/libgcrypt/src/global.c' ! -ipath 
'./grub-core/lib/libgcrypt/src/secmem.c'  ! -ipath 
'./util/grub-gen-widthspec.c' ! -ipath './util/grub-gen-asciih.c' |sort > 
po/POTFILES.in
 find util -iname '*.in' ! -name Makefile.in  |sort > po/POTFILES-shell.in
 
-autogen --version >/dev/null || exit 1
-
 echo "Importing unicode..."
 python util/import_unicode.py unicode/UnicodeData.txt 
unicode/BidiMirroring.txt unicode/ArabicShaping.txt grub-core/unidata.c
 
@@ -32,10 +30,7 @@ for x in mpi-asm-defs.h mpih-add1.c mpih-sub1.c mpih-mul1.c 
mpih-mul2.c mpih-mul
 ln -s generic/"$x" grub-core/lib/libgcrypt-grub/mpi/"$x"
 done
 
-echo "Creating Makefile.tpl..."
-python gentpl.py | sed -e '/^$/{N;/^\n$/D;}' > Makefile.tpl
-
-echo "Running autogen..."
+echo "Generating Automake input..."
 
 # Automake doesn't like including files from a path outside the project.
 rm -f contrib grub-core/contrib
@@ -59,8 +54,8 @@ for extra in contrib/*/Makefile.core.def; do
   fi
 done
 
-cat $UTIL_DEFS | autogen -T Makefile.tpl | sed -e '/^$/{N;/^\n$/D;}' > 
Makefile.util.am
-cat $CORE_DEFS | autogen -T Makefile.tpl | sed -e '/^$/{N;/^\n$/D;}' > 
grub-core/Makefile.core.am
+python gentpl.py $UTIL_DEFS > Makefile.util.am
+python gentpl.py $CORE_DEFS > grub-core/Makefile.core.am
 
 for extra in contrib/*/Makefile.common; do
   if test -e "$extra"; then
diff --git a/conf/Makefile.common b/conf/Makefile.common
index 9f0d1d6..d887df1 100644
--- a/conf/Makefile.common
+++ b/conf/Makefile.common
@@ -131,20 +131,15 @@ EXTRA_DIST =
 CLEANFILES =
 BUILT_SOURCES =
 
-# Rules for autogen definition files
-
-.PRECIOUS: $(top_srcdir)/Makefile.tpl
-$(top_srcdir)/Makefile.tpl: $(top_srcdir)/gentpl.py
-   python $< | sed -e '/^$$/{N;/^\\n$$/D;}' > $@.new || (rm -f $@.new; 
exit 1)
-   mv $@.new $@
+# Rules for Automake input
 
 .PRECIOUS: $(top_srcdir)/Makefile.util.am
-$(top_srcdir)/Makefile.util.am: $(top_srcdir)/Makefile.util.def 
$(top_srcdir)/Makefile.utilgcry.def $(top_srcdir)/Makefile.tpl
-   cat $(top_srcdir)/Makefile.util.def $(top_srcdir)/Makefile.utilgcry.def 
| autogen -T $(top_srcdir)/Makefile.tpl | sed -e '/^$$/{N;/^\\n$$/D;}' > $@.new 
|| (rm -f $@.new; exit 1)
+$(top_srcdir)/Makefile.util.am: $(top_srcdir)/gentpl.py 
$(top_srcdir)/Makefile.util.def $(top_srcdir)/Makefile.utilgcry.def
+   python $^ > $@.new || (rm -f $@.new; exit 1)
mv $@.new $@
 
 .PRECIOUS: $(top_srcdir)/grub-core/Makefile.core.am
-$(top_srcdir)/grub-core/Makefile.core.am: 
$(top_srcdir)/grub-core/Makefile.core.def 
$(top_srcdir)/grub-core/Makefile.gcry.def $(top_srcdir)/Makefile.tpl
+$(top_srcdir)/grub-core/Makefile.core.am: $(top_srcdir)/gentpl.py 
$(top_srcdir)/grub-core/Makefile.core.def 
$(top_srcdir)/grub-core/Makefile.gcry.def
if [ "x$$GRUB_CONTRIB" != x ]; then echo "You need to run ./autogen.sh 
manually." >&2; exit 1; fi
-   cat $(top_srcdir)/grub-core/Makefile.core.def 
$(top_srcdir)/grub-core/Makefile.gcry.def | autogen -T 
$(top_srcdir)/Makefile.tpl | sed -e '/^$$/{N;/^\\n$$/D;}' > $@.new || (rm -f 
$@.new; exit 1)
+   python $^ > $@.new || (rm -f $@.new; exit 1)
mv $@.new $@
diff --git a/conf/Makefile.extra-dist b/conf/Makefile.extra-dist
index 72b00c0..e0c915c 100644
--- a/conf/Makefile.extra-dist
+++ b/conf/Makefile.extra-dist
@@ -2,7 +2,6 @@ EXTRA_DIST += autogen.sh
 EXTRA_DIST += geninit.sh
 
 EXTRA_DIST += gentpl.py
-EXTRA_DIST += Makefile.tpl
 EXTRA_DIST += Makefile.util.def
 EXTR

[PATCH 3/3] Handle #if/#endif and C-style comments in AutoGen definitions files.

2013-11-26 Thread Colin Watson
---
 gentpl.py | 46 ++
 1 file changed, 42 insertions(+), 4 deletions(-)

diff --git a/gentpl.py b/gentpl.py
index 5bee43a..0c5eabb 100644
--- a/gentpl.py
+++ b/gentpl.py
@@ -106,9 +106,11 @@ for platform in GRUB_PLATFORMS:
 #
 
 # We support a subset of the AutoGen definitions file syntax.  Specifically,
-# compound names are disallowed; C-style comments and preprocessing
-# directives are disallowed; and shell-generated strings, Scheme-generated
-# strings, and here strings are disallowed.
+# compound names are disallowed; some preprocessing directives are
+# disallowed (though #if/#endif are allowed; note that, like AutoGen, #if
+# skips everything to the next #endif regardless of the value of the
+# conditional); and shell-generated strings, Scheme-generated strings, and
+# here strings are disallowed.
 
 class AutogenToken:
 (autogen, definitions, eof, var_name, other_name, string, number,
@@ -189,7 +191,27 @@ class AutogenParser:
 if offset >= end:
 break
 c = data[offset]
-if c == "{":
+if c == "#":
+offset += 1
+try:
+end_directive = data.index("\n", offset)
+directive = data[offset:end_directive]
+offset = end_directive
+except ValueError:
+directive = data[offset:]
+offset = end
+name, value = directive.split(None, 1)
+if name == "if":
+try:
+end_if = data.index("\n#endif", offset)
+new_offset = end_if + len("\n#endif")
+self.cur_line += data[offset:new_offset].count("\n")
+offset = new_offset
+except ValueError:
+self.error("#if without matching #endif")
+else:
+self.error("Unhandled directive '#%s'" % name)
+elif c == "{":
 yield AutogenToken.lbrace, c
 offset += 1
 elif c == "=":
@@ -234,6 +256,22 @@ class AutogenParser:
 else:
 s.append(data[offset])
 yield AutogenToken.string, "".join(s)
+elif c == "/":
+offset += 1
+if data[offset] == "*":
+offset += 1
+try:
+end_comment = data.index("*/", offset)
+new_offset = end_comment + len("*/")
+self.cur_line += data[offset:new_offset].count("\n")
+offset = new_offset
+except ValueError:
+self.error("/* without matching */")
+elif data[offset] == "/":
+try:
+offset = data.index("\n", offset)
+except ValueError:
+pass
 elif (c.isdigit() or
   (c == "-" and offset < end - 1 and
data[offset + 1].isdigit())):
-- 
1.8.4.3

___
Grub-devel mailing list
Grub-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/grub-devel