date:20110905

Re: coreutils-8.12.178-df9cd on Solaris 10

2011-09-05 Thread Bruno Haible

> > So, in summary, there are two problems:
> >   - 'cp' creates a nontrivial ACL for a directory when the original
> > directory had none.
> > ...
> This patch fixes it. I'm committing it in gnulib.

We shouldn't need coreutils tests in order to detect that a gnulib module
is not working well. I'm extending the ACL tests in gnulib so that they
detect the failure before the fix:

  mode = 4700: tmpfile1 got an ACL
  FAIL: test-set-mode-acl-2.sh
  files tmpfile0 and tmpfile1: different ACL entry #2: different ids -1 and 0
  FAIL: test-copy-acl-2.sh

Basically, it's enough to try different values of TMPDIR, instead of always
trying only TMPDIR=/tmp.


2011-09-05  Bruno Haible  

copy-file: Try unit tests on more file systems.
* tests/test-copy-file-1.sh: New file.
* tests/test-copy-file-2.sh: New file.
* modules/copy-file-tests (Files): Add them.
(Makefile.am): Add them to TESTS.

acl: Try unit tests on more file systems.
* tests/test-file-has-acl-1.sh: New file.
* tests/test-file-has-acl-2.sh: New file.
* tests/test-set-mode-acl-1.sh: New file.
* tests/test-set-mode-acl-2.sh: New file.
* tests/test-copy-acl-1.sh: New file.
* tests/test-copy-acl-2.sh: New file.
* modules/acl-tests (Files): Add them.
(Makefile.am): Add them to TESTS.

-- 
In memoriam Moshe Weinberg

maint.mk syntax check problems

2011-09-05 Thread Martin von Gagern

Hi!

I'm currently updating GNU wdiff to use latest gnulib, 2c53fc42. In the
process, I've encountered a number of problems with maint.mk syntax checks.


1. main.mk fails its own checks:

The checks sc_makefile_at_at_check and sc_prohibit_undesirable_word_seq
both fail for me, as the maint.mk file itself violates these checks.

I know, this will probably only affect projects which have gnulib
included in their own repository, but according to the manual
 that
approach is officially supported, so the checks should deal with it.


2. sc_prohibit_undesirable_word_seq and gettext:

Makefile.in.in as generated by gettextize will contain the undesirable
phrase "can not", due to
.
Will someone who is a native English speaker take this up to the gettext
guys, and kindlky ask them to address this? Or are people like Bruno
Haible sufficiently involved in both projects that someone will likely
read this mail and fix that phrase, without further mails required?


3. sc_prohibit_doubled_word and non-ASCII text:

In my po/pt_BR.po file

I have the text "no conteúdo do arquivo", encoded in ISO-8859-1. Perl
wasn't told to use any specific unicode-aware mode, and apparently
didn't consider the 'ú' a word character. So the change from 'ú' to 'd'
constituted a word boundary, causing the line to be forbidden.

I tried to address this using ignore_doubled_word_match_RE_, but
unfortunately the perl script only prints the matching part of the line,
not the whole line. So I cannot mask the error using a regexp like
'conte..?do.do', as the line only contains the "do do" all by itself. It
would generally be better to print the whole offending line.

And in the long run, it would be nice if you could persuade Perl to
consider any non-ascii characters as word characters for this test. In
dubio pro reo.


4. Obscure sc_tight_scope:

The syntax check printed a message indicating that it skipped the
tight_scope check. I still don't know what this check is about, but
reading the code I found that it sometimes sets a variable called
"fail", but that variable doesn't appear to affect the result in any
way. Other syntax checks have "test $$fail = 1" as a condition
somewhere, but tight_scope does not. So no matter what the test is
supposed do, I'd gues it currently doesn't do it.


5. sc_prohibit_always-defined_macros reports missing files:

The sc_prohibit_always-defined_macros check will cause error messages
about missing files to be emitted if elements from the gl_other_headers_
list are not present (i.e. not imported). This can be confusing. I've
added a "test -e $$f" check to the def_sym_regex code:
https://github.com/gagern/gnulib/commit/74524a2c993a809bbc20dcc3
Feel free to merge this.


6. sc_po_check and generated files:

If gnulib has its own po-base, then $(generated_files) should not be
included in the list of files to check, as the main project POTFILES.in
isn't responsible for these. Checking whether the po-base specific file
$(srcdir)/lib/po/POTFILES.in exists might be a good way to discern these
scenarios.


On the whole, I wonder whether it would be better to factor all of these
syntax checks from the makefile into a separate shell script file. After
all, most of them are largely composed of shell syntax snippets already.
And a shell script file would get rid of a HUGE number of line
continuation backslashes. It would probably also make things like output
coloring easier, I believe, as you could nest functions more easily.
What do you think?

By the way, big thanks to Jim Meyering for commit 0baae9ca, finally
making those source code exclusions use variables instead of files!
Finally obsoletes my own modification to that effect:
http://bzr.savannah.gnu.org/lh/wdiff/trunk/revision/66 as discussed in
http://thread.gmane.org/gmane.comp.lib.gnulib.bugs/20443/focus=20454


Greetings,
 Martin von Gagern



signature.asc
Description: OpenPGP digital signature

Re: Merge autoconf-archive into gnulib?

2011-09-05 Thread Reuben Thomas

On 5 September 2011 00:43, Bruno Haible  wrote:
> I think it
> would be useful to allow more projects of this kind to use the same
> tool for propagating the source code, namely gnulib-tool, extended
>  1. to allow multiple --local-dir options,
>  2. to allow fetching source code from git, hg, bzr, or svn servers,
>     or even from tarballs.

This is an attractive suggestion.

> If autoconf-archives macros have dependencies that so far have not
> been formally stated, then I would encourage them to adopt gnulib's
> module description format and hack gnulib-tool to collect the
> dependencies automatically.

I guess that would consist of parsing AC_REQUIRE lines.

Thanks for the examples of poor documentation and overlapping
functionality, which I'll chase up individually.

I see two obvious implementation strategies here:

1. Have a modified gnulib-tool for autoconf-archive, which parses
AC_REQUIRE lines and the listing of m4/*.m4 files to generate virtual
ax_foo modules and their inter-dependencies.

2. Use the actual gnulib-tool modified to support multiple --local-dir
options, and add an automatically-generated modules/ directory to
autoconf-archive to provide the data for it to work with.

(Option 2 is better for my money as it avoids forking gnulib-tool.)

How does this sound?

-- 
http://rrt.sc3d.org

RE: ACLs on HP NonStop

2011-09-05 Thread Schmitz, Joachim

Hi Bruno

I know, it's been a while... but I think I found a problem in ACL support for 
NonStop. There are file systems and OS version that don't support them, and 
gnu-lib code thows an error for these. The code I'm referring to is in 
file-has-acl.c and goes like this:

…
# elif HAVE_ACLSORT /* NonStop Kernel */

  int count;
  struct acl entries[NACLENTRIES];

  for (;;)
{
  count = acl ((char *) name, ACL_CNT, NACLENTRIES, NULL);

  if (count < 0)
return -1;

  if (count == 0)
return 0;
…

I think this could/should get changed to the following (which is very similar 
to how HPUX does it):

…
# elif HAVE_ACLSORT /* NonStop Kernel */

  int count;
  struct acl entries[NACLENTRIES];

  for (;;)
{
  count = acl ((char *) name, ACL_CNT, NACLENTRIES, NULL);

  if (count < 0)
return (errno == ENOSYS || errno == ENOTSUP ? 0 : -1);

  if (count == 0)
return 0;
…

What do you think? Not too sure whether this is the only change we need though.

Bye, Jojo
-Original Message-
From: Bruno Haible [mailto:br...@clisp.org] 
Sent: Sunday, October 03, 2010 6:05 PM
To: Schmitz, Joachim
Cc: 'Paolo Bonzini'; 'bug-gnulib'
Subject: Re: ACLs on HP NonStop

Joachim Schmitz wrote:
> Ah, now I see, 'class' is mandatory, so we'd need
>  
> setacl -s user::6,group::0,class:0,other:0 tmpfile0

Good. I'll use class:7 instead of class:0, since this is more realistic.

> and that indeed works:

Great! I'm thus committing the ACL support.

2010-10-03  Bruno Haible  
Joachim Schmitz(tiny change)

acl: Add support for ACLs on NonStop Kernel.
* m4/acl.m4 (gl_FUNC_ACL): For Solaris, test for facl(), not for acl().
Check whether the function aclsort() exists.
* lib/acl-internal.h: For Solaris, test HAVE_FACL, not HAVE_ACL.
(acl_nontrivial) [HAVE_ACLSORT]: New declaration.
* lib/file-has-acl.c: For Solaris, test HAVE_FACL, not HAVE_ACL.
(acl_nontrivial [HAVE_ACLSORT]: New function.
(file_has_acl): Implement for NonStop Kernel.
* lib/set-mode-acl.c: For Solaris, test HAVE_FACL, not HAVE_ACL.
(qset_acl): Implement for NonStop Kernel.
* lib/copy-acl.c (qcopy_acl): Implement for NonStop Kernel.
* tests/test-sameacls.c: For Solaris, test HAVE_FACL, not HAVE_ACL.
(main): Implement for NonStop Kernel.
* tests/test-file-has-acl.sh (acl_flavor): Set to 'nsk' on NonStop
Kernel. Handle this flavor.
* tests/test-set-mode-acl.sh: Likewise.
* tests/test-copy-acl.sh: Likewise.
* tests/test-copy-file.sh: Likewise.

--- lib/acl-internal.h.orig Sun Oct  3 17:46:20 2010
+++ lib/acl-internal.h  Sun Oct  3 04:21:58 2010
@@ -26,7 +26,7 @@
 #if HAVE_SYS_ACL_H
 # include 
 #endif
-#if defined HAVE_ACL && ! defined GETACLCNT && defined ACL_CNT
+#if defined HAVE_FACL && ! defined GETACLCNT && defined ACL_CNT
 # define GETACLCNT ACL_CNT
 #endif

@@ -158,7 +158,7 @@
 extern int acl_access_nontrivial (acl_t);
 #  endif

-# elif HAVE_ACL && defined GETACL /* Solaris, Cygwin, not HP-UX */
+# elif HAVE_FACL && defined GETACL /* Solaris, Cygwin, not HP-UX */

 /* Set to 1 if a file's mode is implicit by the ACL.
Set to 0 if a file's mode is stored independently from the ACL.  */
@@ -216,6 +216,12 @@
Return 0 if it is trivial, i.e. equivalent to a simple stat() mode.  */
 extern int acl_nontrivial (struct acl *a);

+# elif HAVE_ACLSORT /* NonStop Kernel */
+
+/* Return 1 if the given ACL is non-trivial.
+   Return 0 if it is trivial, i.e. equivalent to a simple stat() mode.  */
+extern int acl_nontrivial (int count, struct acl *entries);
+
 # endif

 #endif
--- lib/copy-acl.c.orig Sun Oct  3 17:46:20 2010
+++ lib/copy-acl.c  Sun Oct  3 14:40:31 2010
@@ -516,6 +516,68 @@

   return 0;

+#elif USE_ACL && HAVE_ACLSORT /* NonStop Kernel */
+
+  int count;
+  struct acl entries[NACLENTRIES];
+  int ret;
+
+  for (;;)
+{
+  count = acl ((char *) src_name, ACL_CNT, NACLENTRIES, NULL);
+
+  if (count < 0)
+{
+  if (0)
+{
+  count = 0;
+  break;
+}
+  else
+return -2;
+}
+
+  if (count == 0)
+break;
+
+  if (count > NACLENTRIES)
+/* If NACLENTRIES cannot be trusted, use dynamic memory allocation.  */
+abort ();
+
+  if (acl ((char *) src_name, ACL_GET, count, entries) == count)
+break;
+  /* Huh? The number of ACL entries changed since the last call.
+ Repeat.  */
+}
+
+  if (count == 0)
+return qset_acl (dst_name, dest_desc, mode);
+
+  ret = acl ((char *) dst_name, ACL_SET, count, entries);
+  if (ret < 0)
+{
+  int saved_errno = errno;
+
+  if (0)
+{
+  if (!acl_nontrivial (count, entries))
+return chmod_or_fchmod (dst_name, dest_desc, mode);
+}
+

Re: test-float fails on regular PowerPC, too.

2011-09-05 Thread Andreas Metzler

On 2011-09-04 Bruno Haible  wrote:
> Andreas Metzler wrote:
> > test-float fails on PowerPC (not PowerPC64), too. This causes
[...]
> Does the test failure go away if you change in lib/float.in.h the
> condition

> (defined _ARCH_PPC || defined _POWER) && defined _AIX

> to

> (defined _ARCH_PPC || defined _POWER || defined __ppc__ || defined 
> __powerpc__)

> and adjust m4/float_h.m4 to set FLOAT_H=float.h and REPLACE_FLOAT_LDBL=1
> (like it is done for AIX)?

Like I tried in the attached patched? - No did not help.

(I am running this from grep-2.9, as the whole thing is much smaller
and less interdependent than gnutls)

cu andreas

-- 
`What a good friend you are to him, Dr. Maturin. His other friends are
so grateful to you.'
`I sew his ears on from time to time, sure'
Description: Upstream changes introduced in version 2.9-2.1
 This patch has been created by dpkg-source during the package build.
 Here's the last changelog entry, hopefully it gives details on why
 those changes were made:
 .
 grep (2.9-2.1) unstable; urgency=low
 .
   * Non-maintainer upload.
   * testsuite error
 .
 The person named in the Author field signed this changelog entry.
Author: Andreas Metzler 

---
The information above should follow the Patch Tagging Guidelines, please
checkout http://dep.debian.net/deps/dep3/ to learn about the format. Here
are templates for supplementary fields that you might want to add:

Origin: , 
Bug: 
Bug-Debian: http://bugs.debian.org/
Bug-Ubuntu: https://launchpad.net/bugs/
Forwarded: 
Reviewed-By: 
Last-Update: 

--- grep-2.9.orig/m4/float_h.m4
+++ grep-2.9/m4/float_h.m4
@@ -40,6 +40,12 @@ changequote([,])dnl
 REPLACE_FLOAT_LDBL=1
   fi
   ;;
+linux*)
+  if test "$host_cpu" = "powerpc" && test -n "$FLOAT_H"; then
+FLOAT_H=float.h
+REPLACE_FLOAT_LDBL=1
+  fi
+  ;;
   esac
   if test -n "$FLOAT_H"; then
 gl_NEXT_HEADERS([float.h])
--- grep-2.9.orig/gnulib-tests/float.in.h
+++ grep-2.9/gnulib-tests/float.in.h
@@ -112,7 +112,7 @@ extern const union gl_long_double_union
 
 /* On AIX 7.1 with gcc 4.2, the values of LDBL_MIN_EXP, LDBL_MIN, LDBL_MAX are
wrong.  */
-#if (defined _ARCH_PPC || defined _POWER) && defined _AIX && (LDBL_MANT_DIG == 
106) && defined __GNUC__
+#if (defined _ARCH_PPC || defined _POWER || defined __ppc__ || defined 
__powerpc__) && (LDBL_MANT_DIG == 106) && defined __GNUC_
 # undef LDBL_MIN_EXP
 # define LDBL_MIN_EXP DBL_MIN_EXP
 # undef LDBL_MIN_10_EXP

Re: btowc(0) configure check hangs clang? [was: Possible configure bug]

2011-09-05 Thread Bruno Haible

On 2011-09-03 I wrote:
> I guess that clang hangs or produces an endless loop because of
> this inline definition in :
> 
> extern wint_t __btowc_alias (int __c) __asm ("btowc");
> __extern_inline wint_t
> __NTH (btowc (int __c))
> { return (__builtin_constant_p (__c) && __c >= '\0' && __c <= '\x7f'
>   ? (wint_t) __c : __btowc_alias (__c)); }
> 
> Evidently, clang gives a different semantics to __asm than GCC.
> But GCC's semantics of __asm is fixed for 20 years.
> 
> Conclusion: Needs to be fixed in clang.

And it's confirmed by this bug report in clang's bug tracker:


Bruno
-- 
In memoriam Moshe Weinberg

Re: maint.mk syntax check problems

2011-09-05 Thread Bruno Haible

Hi,

Martin von Gagern wrote:
> 2. sc_prohibit_undesirable_word_seq and gettext:
> 
> Makefile.in.in as generated by gettextize will contain the undesirable
> phrase "can not" ...
> 
> 3. sc_prohibit_doubled_word and non-ASCII text:
> 
> In my po/pt_BR.po file
> 
> I have the text "no conteúdo do arquivo"

I don't think it makes sense to run such stylistic checks on files that
are not under your control. po/Makefile.in.in is owned by the gettext
maintainer, and the *.po files are in the hands of the translators.
In other words: I would disable these checks for the po/ directory.

Bruno
-- 
In memoriam Moshe Weinberg

Re: coreutils-8.12.178-df9cd on Solaris 10

2011-09-05 Thread Bruno Haible

> I'm extending the ACL tests in gnulib so that they
> detect the failure before the fix:
> 
>   mode = 4700: tmpfile1 got an ACL
>   FAIL: test-set-mode-acl-2.sh
>   files tmpfile0 and tmpfile1: different ACL entry #2: different ids -1 and 0
>   FAIL: test-copy-acl-2.sh

The change to lib/set-mode-acl.c fixed the first test failure. For the
second test failure, I'm applying this fix:


2011-09-05  Bruno Haible  

acl: Improve support of NFSv4 ACLs on Solaris 10 (newer version).
* lib/acl-internal.h (acl_nontrivial): Declare also on newer Solaris
10.
* lib/file-has-acl.c (acl_nontrivial): Define also on newer Solaris 10.
(acl_ace_nontrivial): Likewise. Recognize the trivial ACLs with 6 ACEs.
* lib/copy-acl.c (qcopy_acl): On newer Solaris 10, use acl or facl
instead of acl_get, facl_get, acl_set, facl_set.

--- lib/acl-internal.h.orig Mon Sep  5 23:41:05 2011
+++ lib/acl-internal.h  Mon Sep  5 21:59:52 2011
@@ -183,7 +183,7 @@
 #   define MODE_INSIDE_ACL 1
 #  endif
 
-#  if !defined ACL_NO_TRIVIAL /* Solaris <= 10, Cygwin */
+#  if !(defined ACL_NO_TRIVIAL && 0) /* Solaris <= 10, Cygwin */
 
 /* Return 1 if the given ACL is non-trivial.
Return 0 if it is trivial, i.e. equivalent to a simple stat() mode.  */
--- lib/copy-acl.c.orig Mon Sep  5 23:41:05 2011
+++ lib/copy-acl.c  Mon Sep  5 22:08:19 2011
@@ -181,7 +181,7 @@
 
 #elif USE_ACL && defined GETACL /* Solaris, Cygwin, not HP-UX */
 
-# if defined ACL_NO_TRIVIAL
+# if defined ACL_NO_TRIVIAL && 0
   /* Solaris 10 (newer version), which has additional API declared in
   (acl_t) and implemented in libsec (acl_set, acl_trivial,
  acl_fromtext, ...).  */
--- lib/file-has-acl.c.orig Mon Sep  5 23:41:05 2011
+++ lib/file-has-acl.c  Mon Sep  5 23:40:51 2011
@@ -120,7 +120,7 @@
 
 #elif USE_ACL && HAVE_FACL && defined GETACL /* Solaris, Cygwin, not HP-UX */
 
-# if !defined ACL_NO_TRIVIAL /* Solaris <= 10, Cygwin */
+# if !(defined ACL_NO_TRIVIAL && 0) /* Solaris <= 10, Cygwin */
 
 /* Test an ACL retrieved with GETACL.
Return 1 if the given ACL, consisting of COUNT entries, is non-trivial.
@@ -167,7 +167,7 @@
   int old_convention = 0;
 
   for (i = 0; i < count; i++)
-if (entries[i].a_flags & (ACE_OWNER | ACE_GROUP | ACE_OTHER))
+if (entries[i].a_flags & (OLD_ACE_OWNER | OLD_ACE_GROUP | OLD_ACE_OTHER))
   {
 old_convention = 1;
 break;
@@ -183,28 +183,119 @@
If ace->a_flags = ACE_OWNER, ace->a_who is the st_uid from stat().
If ace->a_flags = ACE_GROUP, ace->a_who is the st_gid from stat().
We don't need to check ace->a_who in these cases.  */
-if (!(ace->a_type == ALLOW
-  && (ace->a_flags == ACE_OWNER
-  || ace->a_flags == ACE_GROUP
-  || ace->a_flags == ACE_OTHER)))
+if (!(ace->a_type == OLD_ALLOW
+  && (ace->a_flags == OLD_ACE_OWNER
+  || ace->a_flags == OLD_ACE_GROUP
+  || ace->a_flags == OLD_ACE_OTHER)))
   return 1;
   }
   else
-/* Running on Solaris 10 (newer version) or Solaris 11.  */
-for (i = 0; i < count; i++)
-  {
-ace_t *ace = &entries[i];
+{
+  /* Running on Solaris 10 (newer version) or Solaris 11.  */
+  unsigned int access_masks[6] =
+{
+  0, /* owner@ deny */
+  0, /* owner@ allow */
+  0, /* group@ deny */
+  0, /* group@ allow */
+  0, /* everyone@ deny */
+  0  /* everyone@ allow */
+};
+
+  for (i = 0; i < count; i++)
+{
+  ace_t *ace = &entries[i];
+  unsigned int index1;
+  unsigned int index2;
+
+  if (ace->a_type == NEW_ACE_ACCESS_ALLOWED_ACE_TYPE)
+index1 = 1;
+  else if (ace->a_type == NEW_ACE_ACCESS_DENIED_ACE_TYPE)
+index1 = 0;
+  else
+return 1;
+
+  if (ace->a_flags == NEW_ACE_OWNER)
+index2 = 0;
+  else if (ace->a_flags == (NEW_ACE_GROUP | NEW_ACE_IDENTIFIER_GROUP))
+index2 = 2;
+  else if (ace->a_flags == ACE_EVERYONE)
+index2 = 4;
+  else
+return 1;
 
-if (!(ace->a_type == ACE_ACCESS_ALLOWED_ACE_TYPE
-  && (ace->a_flags == NEW_ACE_OWNER
-  || ace->a_flags
- == (NEW_ACE_GROUP | NEW_ACE_IDENTIFIER_GROUP)
-  || ace->a_flags == ACE_EVERYONE)
-  && (ace->a_access_mask
-  & ~(NEW_ACE_READ_DATA | NEW_ACE_WRITE_DATA | 
NEW_ACE_EXECUTE))
- == 0))
-  return 1;
-  }
+  access_masks[index1 + index2] |= ace->a_access_mask;
+}
+
+  /* The same bit shouldn't be both allowed and denied.  */
+  if (access_masks[0] & access_masks[1])
+return 1;
+  if (access_masks[2] & access_masks[3])
+return 1;
+  if (access_masks[4] & access_masks[5])
+

Re: coreutils-8.12.178-df9cd on Solaris 10

2011-09-05 Thread Bruno Haible

> The change to lib/set-mode-acl.c fixed the first test failure. For the
> second test failure, I'm applying this fix:

And finally, clean up the pieces of unused Solaris code:


2011-09-05  Bruno Haible  

acl: Clean up Solaris code.
* lib/acl-internal.h: Remove no-op #if.
* lib/file-has-acl.c: Likewise.
* lib/set-mode-acl.c (qset_acl): Remove unused Solaris code.
* lib/copy-acl.c (qcopy_acl): Likewise.

--- lib/acl-internal.h.orig Tue Sep  6 00:23:04 2011
+++ lib/acl-internal.h  Tue Sep  6 00:03:11 2011
@@ -183,14 +183,10 @@
 #   define MODE_INSIDE_ACL 1
 #  endif
 
-#  if !(defined ACL_NO_TRIVIAL && 0) /* Solaris <= 10, Cygwin */
-
 /* Return 1 if the given ACL is non-trivial.
Return 0 if it is trivial, i.e. equivalent to a simple stat() mode.  */
 extern int acl_nontrivial (int count, aclent_t *entries);
 
-#  endif
-
 #  ifdef ACE_GETACL /* Solaris 10 */
 
 /* Test an ACL retrieved with ACE_GETACL.
--- lib/copy-acl.c.orig Tue Sep  6 00:23:04 2011
+++ lib/copy-acl.c  Tue Sep  6 00:03:11 2011
@@ -181,57 +181,20 @@
 
 #elif USE_ACL && defined GETACL /* Solaris, Cygwin, not HP-UX */
 
-# if defined ACL_NO_TRIVIAL && 0
-  /* Solaris 10 (newer version), which has additional API declared in
-  (acl_t) and implemented in libsec (acl_set, acl_trivial,
- acl_fromtext, ...).  */
-
-  int ret;
-  acl_t *aclp = NULL;
-  ret = (source_desc < 0
- ? acl_get (src_name, ACL_NO_TRIVIAL, &aclp)
- : facl_get (source_desc, ACL_NO_TRIVIAL, &aclp));
-  if (ret != 0 && errno != ENOSYS)
-return -2;
-
-  ret = qset_acl (dst_name, dest_desc, mode);
-  if (ret != 0)
-return -1;
-
-  if (aclp)
-{
-  ret = (dest_desc < 0
- ? acl_set (dst_name, aclp)
- : facl_set (dest_desc, aclp));
-  if (ret != 0)
-{
-  int saved_errno = errno;
-
-  acl_free (aclp);
-  errno = saved_errno;
-  return -1;
-}
-  acl_free (aclp);
-}
-
-  return 0;
-
-# else /* Solaris, Cygwin, general case */
-
   /* Solaris 2.5 through Solaris 10, Cygwin, and contemporaneous versions
  of Unixware.  The acl() call returns the access and default ACL both
  at once.  */
-#  ifdef ACE_GETACL
+# ifdef ACE_GETACL
   int ace_count;
   ace_t *ace_entries;
-#  endif
+# endif
   int count;
   aclent_t *entries;
   int did_chmod;
   int saved_errno;
   int ret;
 
-#  ifdef ACE_GETACL
+# ifdef ACE_GETACL
   /* Solaris also has a different variant of ACLs, used in ZFS and NFSv4
  file systems (whereas the other ones are used in UFS file systems).
  There is an API
@@ -284,7 +247,7 @@
   /* Huh? The number of ACL entries changed since the last call.
  Repeat.  */
 }
-#  endif
+# endif
 
   for (;;)
 {
@@ -327,9 +290,9 @@
 }
 
   /* Is there an ACL of either kind?  */
-#  ifdef ACE_GETACL
+# ifdef ACE_GETACL
   if (ace_count == 0)
-#  endif
+# endif
 if (count == 0)
   return qset_acl (dst_name, dest_desc, mode);
 
@@ -367,7 +330,7 @@
 }
   free (entries);
 
-#  ifdef ACE_GETACL
+# ifdef ACE_GETACL
   if (ace_count > 0)
 {
   ret = (dest_desc != -1
@@ -382,7 +345,7 @@
 }
 }
   free (ace_entries);
-#  endif
+# endif
 
   if (MODE_INSIDE_ACL
   && did_chmod <= ((mode & (S_ISUID | S_ISGID | S_ISVTX)) ? 1 : 0))
@@ -404,8 +367,6 @@
 }
   return 0;
 
-# endif
-
 #elif USE_ACL && HAVE_GETACL /* HP-UX */
 
   int count;
--- lib/file-has-acl.c.orig Tue Sep  6 00:23:04 2011
+++ lib/file-has-acl.c  Tue Sep  6 00:03:11 2011
@@ -120,8 +120,6 @@
 
 #elif USE_ACL && HAVE_FACL && defined GETACL /* Solaris, Cygwin, not HP-UX */
 
-# if !(defined ACL_NO_TRIVIAL && 0) /* Solaris <= 10, Cygwin */
-
 /* Test an ACL retrieved with GETACL.
Return 1 if the given ACL, consisting of COUNT entries, is non-trivial.
Return 0 if it is trivial, i.e. equivalent to a simple stat() mode.  */
@@ -148,7 +146,7 @@
   return 0;
 }
 
-#  ifdef ACE_GETACL
+# ifdef ACE_GETACL
 
 /* Test an ACL retrieved with ACE_GETACL.
Return 1 if the given ACL, consisting of COUNT entries, is non-trivial.
@@ -300,8 +298,6 @@
   return 0;
 }
 
-#  endif
-
 # endif
 
 #elif USE_ACL && HAVE_GETACL /* HP-UX */
--- lib/set-mode-acl.c.orig Tue Sep  6 00:23:04 2011
+++ lib/set-mode-acl.c  Tue Sep  6 00:03:11 2011
@@ -203,56 +203,9 @@
 
 # elif HAVE_FACL && defined GETACLCNT /* Solaris, Cygwin, not HP-UX */
 
-#  if defined ACL_NO_TRIVIAL && 0
-  /* Solaris 10 (newer version), which has additional API declared in
-  (acl_t) and implemented in libsec (acl_set, acl_trivial,
- acl_fromtext, ...).  */
-
-  acl_t *aclp;
-  char acl_text[] = "user::---,group::---,mask:---,other:---";
-  int ret;
-  int saved_errno;
-
-  if (mode & S_IRUSR) acl_text[ 6] = 'r';
-  if (mode & S_IWUSR) acl_text[ 7] = 'w';
-  if (mode & S_IXUSR) acl_text[ 8] = 'x';
-  if (mode & S_IRGRP) acl_text[17] = acl_text[26] = 'r';
-  if (mode & S_IWGRP) acl_text[18] = acl_text[27] = 'w';
-  if (mode &

Improve sha1sum speed

2011-09-05 Thread Loïc Le Loarer

Hi,
I saw in the todo list for coreutils that improving sha1sum speed was a
target. So I worked a bit on that.

By using the ideas from the sha1.c file from git sources (
http://git.kernel.org/?p=git/git.git;a=tree;f=block-sha1;hb=pu) which is
clearly faster than the one in gnulib, I have been able to aligned the speed
of Linus' git.

Here is what I did:
First create a test file:
$ dd if=/dev/zero of=big_zero count=1 seek=$(( 1000 * 100 - 1 )) bs=1
Test sha1sum on it:
$ \time sha1sum < big_zero
1dd775261d7abab0b66910acc1d827a2c3799eaf  -
3.49user 0.12system 0:03.61elapsed 99%CPU (0avgtext+0avgdata
2512maxresident)k
0inputs+0outputs (0major+202minor)pagefaults 0swaps
And Linus' sha1:
$ \time ../git-1.7.1/test-sha1 < big_zero
1dd775261d7abab0b66910acc1d827a2c3799eaf
2.54user 0.13system 0:02.70elapsed 98%CPU (0avgtext+0avgdata
2352maxresident)k
976inputs+0outputs (2major+191minor)pagefaults 0swaps

Here is my system:
$ gcc -v
Using built-in specs.
Target: x86_64-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Ubuntu/Linaro
4.4.4-14ubuntu5' --with-bugurl=file:///usr/share/doc/gcc-4.4/README.Bugs
--enable-languages=c,c++,fortran,objc,obj-c++ --prefix=/usr
--program-suffix=-4.4 --enable-shared --enable-multiarch
--enable-linker-build-id --with-system-zlib --libexecdir=/usr/lib
--without-included-gettext --enable-threads=posix
--with-gxx-include-dir=/usr/include/c++/4.4 --libdir=/usr/lib --enable-nls
--with-sysroot=/ --enable-clocale=gnu --enable-libstdcxx-debug
--enable-objc-gc --disable-werror --with-arch-32=i686 --with-tune=generic
--enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu
--target=x86_64-linux-gnu
Thread model: posix
gcc version 4.4.5 (Ubuntu/Linaro 4.4.4-14ubuntu5)

$ cat /proc/cpuinfo
processor: 0
vendor_id: GenuineIntel
cpu family: 6
model: 42
model name: Genuine Intel(R) CPU 0 @ 2.50GHz
stepping: 5
cpu MHz: 800.000
cache size: 3072 KB
physical id: 0
siblings: 4
core id: 0
cpu cores: 2
apicid: 0
initial apicid: 0
fpu: yes
fpu_exception: yes
cpuid level: 13
wp: yes
flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov
pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp
lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc
aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16
xtpr pdcm sse4_1 sse4_2 x2apic popcnt aes xsave avx lahf_lm ida arat pln pts
dts tpr_shadow vnmi flexpriority ept vpid
bogomips: 4988.10
clflush size: 64
cache_alignment: 64
address sizes: 36 bits physical, 48 bits virtual
power management:

You can see that it is working in 64 mode.

Checkout coreutils git lastest version and compile, here is original speed:
$ \time ./src/sha1sum < big_zero
1dd775261d7abab0b66910acc1d827a2c3799eaf  -
3.71user 0.20system 0:03.92elapsed 99%CPU (0avgtext+0avgdata
2528maxresident)k
0inputs+0outputs (0major+195minor)pagefaults 0swaps

Using -O3 to compile: (make CFLAGS="-g -O3") instead of default -O2:
$ \time ./src/sha1sum < big_zero
1dd775261d7abab0b66910acc1d827a2c3799eaf  -
3.47user 0.14system 0:03.62elapsed 99%CPU (0avgtext+0avgdata
2512maxresident)k
0inputs+0outputs (0major+195minor)pagefaults 0swaps

Just changing SWAP to ntohl does have a significant effect on my platform
(the bswap instruction is used). According to Linus comment, this should be
done only in platform having fast unalinged load:
$ diff -uN lib/sha1.c.orig lib/sha1.c
--- lib/sha1.c.orig2011-09-05 23:31:23.128552012 +0200
+++ lib/sha1.c2011-09-05 23:29:58.988552016 +0200
@@ -29,6 +29,7 @@
 #include 
 #include 
 #include 
+#include 

 #if USE_UNLOCKED_IO
 # include "unlocked-io.h"
@@ -333,7 +334,7 @@
   int t;
   for (t = 0; t < 16; t++)
 {
-  x[t] = SWAP (*words);
+  x[t] = ntohl(*words);
   words++;
 }

$ \time ./src/sha1sum.v1 < big_zero
1dd775261d7abab0b66910acc1d827a2c3799eaf  -
3.28user 0.10system 0:03.39elapsed 99%CPU (0avgtext+0avgdata
2512maxresident)k
0inputs+0outputs (0major+194minor)pagefaults 0swaps

Then add a second patch to avoid the (,) construction and reorder the
assignement to x[] array in the main define:

--- lib/sha1.c.orig2011-09-05 23:45:19.988552016 +0200
+++ lib/sha1.c2011-09-05 23:47:57.188552016 +0200
@@ -317,112 +317,107 @@

 #define rol(x, n) (((x) << (n)) | ((uint32_t) (x) >> (32 - (n

-#define M(I) ( tm =   x[I&0x0f] ^ x[(I-14)&0x0f] \
-^ x[(I-8)&0x0f] ^ x[(I-3)&0x0f] \
-   , (x[I&0x0f] = rol(tm, 1)) )
+#define X(I) ntohl(words[I])
+#define M(I) rol(  x[I&0x0f] ^ x[(I-14)&0x0f] \
+ ^ x[(I-8)&0x0f] ^ x[(I-3)&0x0f], 1) \

-#define R(A,B,C,D,E,F,K,M)  do { E += rol( A, 5 ) \
+#define R(A,B,C,D,E,F,K,M,I) do { unsigned int TEMP = M(I); x[I&0x0f] =
TEMP;\
+ E += rol( A, 5 ) \
   + F( B, C, D

Re: ACLs on HP NonStop

2011-09-05 Thread Bruno Haible

Hi Joachim,

> I know, it's been a while... but I think I found a problem in ACL support for 
> NonStop. There are file systems and OS version that don't support them, and 
> gnu-lib code thows an error for these. The code I'm referring to is in 
> file-has-acl.c and goes like this:
> 
> …
> # elif HAVE_ACLSORT /* NonStop Kernel */
> 
>   int count;
>   struct acl entries[NACLENTRIES];
> 
>   for (;;)
> {
>   count = acl ((char *) name, ACL_CNT, NACLENTRIES, NULL);
> 
>   if (count < 0)
> return -1;
> 
>   if (count == 0)
> return 0;
> …
> 
> I think this could/should get changed to the following (which is very similar 
> to how HPUX does it):
> 
> …
> # elif HAVE_ACLSORT /* NonStop Kernel */
> 
>   int count;
>   struct acl entries[NACLENTRIES];
> 
>   for (;;)
> {
>   count = acl ((char *) name, ACL_CNT, NACLENTRIES, NULL);
> 
>   if (count < 0)
> return (errno == ENOSYS || errno == ENOTSUP ? 0 : -1);
> 
>   if (count == 0)
> return 0;

Makes sense. I've applied it as follows:


2011-09-05  Joachim Schmitz(tiny change)
Bruno Haible  

acl: Avoid errors on NonStop Kernel.
* lib/file-has-acl.c (file_has_acl) [NonStop Kernel]: Ignore ENOSYS and
ENOTSUP errors.

--- lib/file-has-acl.c.orig Tue Sep  6 00:31:09 2011
+++ lib/file-has-acl.c  Tue Sep  6 00:31:01 2011
@@ -810,7 +810,12 @@
   count = acl ((char *) name, ACL_CNT, NACLENTRIES, NULL);
 
   if (count < 0)
-return -1;
+{
+  if (errno == ENOSYS || errno == ENOTSUP)
+break;
+  else
+return -1;
+}
 
   if (count == 0)
 return 0;

-- 
In memoriam Moshe Weinberg

Re: Improve sha1sum speed

2011-09-05 Thread Pádraig Brady

Thanks for working on this.

Looks like my TODO list is reaping rewards :)

A few general points.
You essentially used Linus' code (albeit by
very helpfully isolating the significant differences).
It might be easier/required to just include it in gnulib?
There are a few files in gnulib that are not copyright of the FSF,
so would Nicolas and Linus need to assign copyright?

For performance testing I've found gcc generates
much more deterministic results with a -march
as close to native as possible or otherwise
the code is very susceptible to alignment issues etc.
Your compiler supports -march=native.
Note also gcc 4.6 has much better support for your sandy bridge CPU,
either with -march=native or -march=corei7-avx

As for the SSE version, I would also like to see that included,
given the proportion of hardware supporting that these days.
I previously noticed a coreutils SSE2 patch here:
http://www.arctic.org/~dean/crypto/sha1.html
Though we'd probably need some runtime SSE detection to include that.

cheers,
Pádraig.

Re: Improve sha1sum speed

2011-09-05 Thread Karl Berry

There are a few files in gnulib that are not copyright of the FSF,

There are?

so would Nicolas and Linus need to assign copyright?

Evidently.

Re: Improve sha1sum speed

2011-09-05 Thread Pádraig Brady

On 09/06/2011 12:41 AM, Karl Berry wrote:
> There are a few files in gnulib that are not copyright of the FSF,
> 
> There are?

only a few, like sinl.c

> 
> so would Nicolas and Linus need to assign copyright?
> 
> Evidently.

Re: Improve sha1sum speed

2011-09-05 Thread Karl Berry

only a few, like sinl.c

GNU policy is all or nothing on this.  There shouldn't be any.

Re: files in gnulib copyright by someone other than the FSF

2011-09-05 Thread Paul Eggert

On 09/05/11 16:41, Karl Berry wrote:
> There are a few files in gnulib that are not copyright of the FSF,
> 
> There are?

Yes, there are.  I expect there are several trivial files without
any copyright notices (and which are public domain) but I just now
audited gnulib for files that are copyright by someone other than the
FSF and found the following.  Most seem benign, but the last entry
may be problematic.

build-aux/install-sh is copyrighted by X Consortium with an MIT license.

build-aux/po/Makefile.in.in is copyrighted by Ulrich Drepper and
"can be copied and used freely without restrictions" (effectively
public domain)

lib/fts_.h, lib/fts.c, lib/random_r.c are copyright by the Regents of the
University of California with a modified BSD license.

lib/cosl.h, lib/asinl.c, lib/sinl.c, lib/acosl.c, lib/trigl.c,
lib/tanl.c, are copyright Sun Microsystems with an MIT-like license.

lib/atanl.c and lib/logl.c are copyright Stephen L. Moshier under GPLv3
or later.

lib/inet_ntop.c and lib/inet_pton.c are copyright Internet Software Consortium
with an MIT-like license.

lib/filevercmp.[ch] are copyright Ian Jackson and Anthony Towns under GPLv3
or later.

lib/gc.h is copyright Simon Josefsson under the GPLv2 or later.

lib/git-merge-changelog.c is copyright Bruno Haible under GPLv2 or
later.

tests/unigbrk/GraphemeBreakTest.txt and tests/uninorm/NormalizationTest.txt
are copyright Unicode, Inc. under the Unicode Terms of Use
.  Hmmm, those
terms don't seem to allow changes, which would not be free software.

[PATCH] openat: port to AIX 7.1 with large files

2011-09-05 Thread Paul Eggert

Further investigation into problems GNU tar has on AIX 7.1
revealed a problem with openat that is similar to the recent
fstatat problem.  I installed the following to fix it.
Given the email cited below, this should be the last such
issue with AIX 7.1, as gnulib now addresses the issue for openat, stat,
lstat, and fstatat, and it doesn't wrap creat or fstat.


openat: port to AIX 7.1 with large files
AIX 7.1 does a "#define openat open64at" if large files are in use,
so we can't simply #undef openat.  Use the orig_openat trick (similar
to orig_open in lib/open.c) to work around the problem.  Problem
reported by Kevin Brott for GNU tar, in the thread containing
.
* lib/openat.c (__need_system_fcntl_h): Define first.
Include  and  before undefining.
(orig_openat) [HAVE_OPENAT]: New inline function.
(openat) [HAVE_OPENAT]: Do not undef.
(rpl_openat): Use orig_openat, not openat.
diff --git a/lib/openat.c b/lib/openat.c
index 18491a6..cc4b9d0 100644
--- a/lib/openat.c
+++ b/lib/openat.c
@@ -16,8 +16,25 @@

 /* written by Jim Meyering */

+/* If the user's config.h happens to include , let it include only
+   the system's  here, so that orig_openat doesn't recurse to
+   rpl_openat.  */
+#define __need_system_fcntl_h
 #include 

+/* Get the original definition of open.  It might be defined as a macro.  */
+#include 
+#include 
+#undef __need_system_fcntl_h
+
+#if HAVE_OPENAT
+static inline int
+orig_openat (int fd, char const *filename, int flags, mode_t mode)
+{
+  return openat (fd, filename, flags, mode);
+}
+#endif
+
 #include "openat.h"

 #include 
@@ -31,8 +48,6 @@

 #if HAVE_OPENAT

-# undef openat
-
 /* Like openat, but work around Solaris 9 bugs with trailing slash.  */
 int
 rpl_openat (int dfd, char const *filename, int flags, ...)
@@ -86,7 +101,7 @@ rpl_openat (int dfd, char const *filename, int flags, ...)
 }
 # endif

-  fd = openat (dfd, filename, flags, mode);
+  fd = orig_openat (dfd, filename, flags, mode);

 # if OPEN_TRAILING_SLASH_BUG
   /* If the filename ends in a slash and fd does not refer to a directory,

Re: files in gnulib copyright by someone other than the FSF

2011-09-05 Thread Jim Meyering

Paul Eggert wrote:

> On 09/05/11 16:41, Karl Berry wrote:
>> There are a few files in gnulib that are not copyright of the FSF,
>>
>> There are?
>
> Yes, there are.  I expect there are several trivial files without
> any copyright notices (and which are public domain) but I just now
> audited gnulib for files that are copyright by someone other than the
> FSF and found the following.  Most seem benign, but the last entry
> may be problematic.
>
> build-aux/install-sh is copyrighted by X Consortium with an MIT license.

>From what I recall, for a build tool like this it's not as important.

> build-aux/po/Makefile.in.in is copyrighted by Ulrich Drepper and
> "can be copied and used freely without restrictions" (effectively
> public domain)

This is a Makefile snippet.  Isn't that ok?

> lib/fts_.h, lib/fts.c, lib/random_r.c are copyright by the Regents of the
> University of California with a modified BSD license.

These initially came from glibc.

> lib/cosl.h, lib/asinl.c, lib/sinl.c, lib/acosl.c, lib/trigl.c,
> lib/tanl.c, are copyright Sun Microsystems with an MIT-like license.

And these.

> lib/atanl.c and lib/logl.c are copyright Stephen L. Moshier under GPLv3
> or later.

And these.

> lib/inet_ntop.c and lib/inet_pton.c are copyright Internet Software Consortium
> with an MIT-like license.

Also glibc-derived.

> lib/filevercmp.[ch] are copyright Ian Jackson and Anthony Towns under GPLv3
> or later.

If it's really so important, we can ask them to assign copyright to the FSF.

> lib/gc.h is copyright Simon Josefsson under the GPLv2 or later.

I suspect that Simon will be happy to adjust that.

> lib/git-merge-changelog.c is copyright Bruno Haible under GPLv2 or
> later.

This is a separate application.

> tests/unigbrk/GraphemeBreakTest.txt and tests/uninorm/NormalizationTest.txt
> are copyright Unicode, Inc. under the Unicode Terms of Use
> .  Hmmm, those
> terms don't seem to allow changes, which would not be free software.

These seem to be mere test inputs.
Is the license unacceptable even for them?

Re: coreutils-8.12.178-df9cd on Solaris 10

maint.mk syntax check problems

Re: Merge autoconf-archive into gnulib?

RE: ACLs on HP NonStop

Re: test-float fails on regular PowerPC, too.

Re: btowc(0) configure check hangs clang? [was: Possible configure bug]

Re: maint.mk syntax check problems

Re: coreutils-8.12.178-df9cd on Solaris 10

Re: coreutils-8.12.178-df9cd on Solaris 10

Improve sha1sum speed

Re: ACLs on HP NonStop

Re: Improve sha1sum speed

Re: Improve sha1sum speed

Re: Improve sha1sum speed

Re: Improve sha1sum speed

Re: files in gnulib copyright by someone other than the FSF

[PATCH] openat: port to AIX 7.1 with large files

Re: files in gnulib copyright by someone other than the FSF

18 matches

Site Navigation

Mail list logo

Footer information