Re: Circular dependency with glibc libpthread libihash
Manolis Ragkousis, le Wed 28 May 2014 18:47:57 +, a écrit :
> I sent the mail by mistake incomplete, I am sorry.
>
> > Could you try this patch for the Hurd’s configure.in?
>
> I had to put one more line after AC_NO_EXECUTABLES, otherwise it would fail
> with
> >> syntax error near unexpected token `fi'.
> Other than that it works as expected for me.
>
> diff --git a/configure.ac b/configure.ac
> index ecabfdf..7ede6db 100644
> --- a/configure.ac
> +++ b/configure.ac
> @@ -83,6 +83,13 @@ AC_PROG_INSTALL
> AC_PROG_AWK
> AC_PROG_SED
>
> +if test "x$cross_compiling" = "xyes"; then
> + # It may be that we don't have a working libc yet, for instance
> + # because we're bootstrapping the cross-compilation tool chain.
> + # Thus, use this undocumented Autoconf macro designed for this.
> + AC_NO_EXECUTABLES
> + echo done
Could you try
AC_MSG_WARN("cross-compiling, disabling linking")
?
Samuel
[PATCH 1/6] libdiskfs: add diskfs_make_node_alloc to allocate fat nodes
libdiskfs has two kind of nodes, struct node and struct netnode.
struct node is used to store libdiskfs specific data, while struct
netnode contains user supplied data. Previously, both objects were
allocated separatly, and a pointer from the node to the netnode
provided a mapping from the former to the latter.
Provide a function diskfs_make_node_alloc that allocates both nodes in
a contiguous region.
This reduces the memory allocation overhead when creating nodes. It
also makes the relation between node and netnode a simple offset
calculation. Provide two functions to compute the netnode address
from the node address and vice-versa.
Most notably, this makes implementing a cache on top of libdiskfs
easier. Auxiliary data for the cache can be stored in the
user-defined netnode, and the fat node can be used as the value.
* libdiskfs/node-make.c (init_node): Move initialization here.
(diskfs_make_node): Use init_node.
(diskfs_make_node_alloc): New function to allocate fat nodes.
* libdiskfs/diskfs.h (diskfs_make_node_alloc): New declaration.
(_diskfs_sizeof_struct_node): Likewise.
(diskfs_node_disknode): Compute disknode address from node address.
(diskfs_disknode_node): And vice-versa.
* libdiskfs/init-init.c (_diskfs_sizeof_struct_node): New variable.
---
libdiskfs/diskfs.h| 27 +++
libdiskfs/init-init.c | 4
libdiskfs/node-make.c | 39 ++-
3 files changed, 61 insertions(+), 9 deletions(-)
diff --git a/libdiskfs/diskfs.h b/libdiskfs/diskfs.h
index ae1a150..2c68aa3 100644
--- a/libdiskfs/diskfs.h
+++ b/libdiskfs/diskfs.h
@@ -686,6 +686,33 @@ diskfs_notice_filechange (struct node *np, enum
file_changed_type type,
The new node will have one hard reference and no light references. */
struct node *diskfs_make_node (struct disknode *dn);
+/* Create a new node structure. Also allocate SIZE bytes for the
+ disknode. The address of the disknode can be obtained using
+ diskfs_node_disknode. The new node will have one hard reference
+ and no light references. */
+struct node *diskfs_make_node_alloc (size_t size);
+
+/* To avoid breaking the ABI whenever sizeof (struct node) changes, we
+ explicitly provide the size. The following two functions will use
+ this value for offset calculations. */
+extern const size_t _diskfs_sizeof_struct_node;
+
+/* Return the address of the disknode for NODE. NODE must have been
+ allocated using diskfs_make_node_alloc. */
+static inline struct disknode *
+diskfs_node_disknode (struct node *node)
+{
+ return (struct disknode *) ((char *) node + _diskfs_sizeof_struct_node);
+}
+
+/* Return the address of the node for DISKNODE. DISKNODE must have
+ been allocated using diskfs_make_node_alloc. */
+static inline struct node *
+diskfs_disknode_node (struct disknode *disknode)
+{
+ return (struct node *) ((char *) disknode - _diskfs_sizeof_struct_node);
+}
+
/* The library also exports the following functions; they are not generally
useful unless you are redefining other functions the library provides. */
diff --git a/libdiskfs/init-init.c b/libdiskfs/init-init.c
index 35be7ed..7a7f248 100644
--- a/libdiskfs/init-init.c
+++ b/libdiskfs/init-init.c
@@ -25,6 +25,10 @@ the Free Software Foundation, 675 Mass Ave, Cambridge, MA
02139, USA. */
#include
#include
+/* For safe inlining of diskfs_node_disknode and
+ diskfs_disknode_node. */
+size_t const _diskfs_sizeof_struct_node = sizeof (struct node);
+
mach_port_t diskfs_default_pager;
mach_port_t diskfs_auth_server_port;
volatile struct mapped_time_value *diskfs_mtime;
diff --git a/libdiskfs/node-make.c b/libdiskfs/node-make.c
index 2b6ef2a..ff0cc0d 100644
--- a/libdiskfs/node-make.c
+++ b/libdiskfs/node-make.c
@@ -19,16 +19,9 @@
#include
-/* Create a and return new node structure with DN as its physical disknode.
- The node will have one hard reference and no light references. */
-struct node *
-diskfs_make_node (struct disknode *dn)
+static struct node *
+init_node (struct node *np, struct disknode *dn)
{
- struct node *np = malloc (sizeof (struct node));
-
- if (np == 0)
-return 0;
-
np->dn = dn;
np->dn_set_ctime = 0;
np->dn_set_atime = 0;
@@ -52,3 +45,31 @@ diskfs_make_node (struct disknode *dn)
return np;
}
+
+/* Create a and return new node structure with DN as its physical disknode.
+ The node will have one hard reference and no light references. */
+struct node *
+diskfs_make_node (struct disknode *dn)
+{
+ struct node *np = malloc (sizeof (struct node));
+
+ if (np == 0)
+return 0;
+
+ return init_node (np, dn);
+}
+
+/* Create a new node structure. Also allocate SIZE bytes for the
+ disknode. The address of the disknode can be obtained using
+ diskfs_node_disknode. The new node will have one hard reference
+ and no light references. */
+struct node *
+diskfs_make_node_alloc (size_t size)
+{
+ struct node *np = malloc (sizeof (struct node) + size);
+
+
[PATCH 3/6] trans/fakeroot: use fat nodes to simplify the node cache
Previously, fakeroot stored netnodes in the hash table. But we are
not interested in a cache for netnodes, we need a node cache. So
fakeroot kept pointers to the associated node object in each netnode
object.
Use fat netfs nodes, which combine node and netnode objects.
* trans/fakeroot.c (struct netnode): Remove np.
(idport_ihash): Fix ihash location pointer offset.
(new_node): Allocate fat nodes, store the node pointer in the hash
table.
(netfs_node_norefs): Adjust accordingly.
(netfs_S_dir_lookup): Likewise.
---
trans/fakeroot.c | 36
1 file changed, 12 insertions(+), 24 deletions(-)
diff --git a/trans/fakeroot.c b/trans/fakeroot.c
index 4175b55..59f8a86 100644
--- a/trans/fakeroot.c
+++ b/trans/fakeroot.c
@@ -47,7 +47,6 @@ static auth_t fakeroot_auth_port;
struct netnode
{
- struct node *np; /* our node */
hurd_ihash_locp_t idport_locp;/* easy removal pointer in idport ihash */
mach_port_t idport; /* port from io_identity */
int openmodes; /* O_READ | O_WRITE | O_EXEC */
@@ -64,7 +63,8 @@ struct netnode
pthread_mutex_t idport_ihash_lock = PTHREAD_MUTEX_INITIALIZER;
struct hurd_ihash idport_ihash
- = HURD_IHASH_INITIALIZER (offsetof (struct netnode, idport_locp));
+= HURD_IHASH_INITIALIZER (sizeof (struct node)
+ + offsetof (struct netnode, idport_locp));
/* Make a new virtual node. Always consumes the ports. If
@@ -74,8 +74,9 @@ new_node (file_t file, mach_port_t idport, int locked, int
openmodes,
struct node **np)
{
error_t err;
- struct netnode *nn = calloc (1, sizeof *nn);
- if (nn == 0)
+ struct netnode *nn;
+ *np = netfs_make_node_alloc (sizeof *nn);
+ if (*np == 0)
{
mach_port_deallocate (mach_task_self (), file);
if (idport != MACH_PORT_NULL)
@@ -84,6 +85,7 @@ new_node (file_t file, mach_port_t idport, int locked, int
openmodes,
pthread_mutex_unlock (&idport_ihash_lock);
return ENOMEM;
}
+ nn = netfs_node_netnode (*np);
nn->file = file;
nn->openmodes = openmodes;
if (idport != MACH_PORT_NULL)
@@ -97,35 +99,26 @@ new_node (file_t file, mach_port_t idport, int locked, int
openmodes,
if (err)
{
mach_port_deallocate (mach_task_self (), file);
- free (nn);
+ free (*np);
return err;
}
}
if (!locked)
pthread_mutex_lock (&idport_ihash_lock);
- err = hurd_ihash_add (&idport_ihash, nn->idport, nn);
+ err = hurd_ihash_add (&idport_ihash, nn->idport, *np);
if (err)
goto lose;
- *np = nn->np = netfs_make_node (nn);
- if (*np == 0)
-{
- err = ENOMEM;
- goto lose_hash;
-}
-
pthread_mutex_lock (&(*np)->lock);
pthread_mutex_unlock (&idport_ihash_lock);
return 0;
- lose_hash:
- hurd_ihash_locp_remove (&idport_ihash, nn->idport_locp);
lose:
pthread_mutex_unlock (&idport_ihash_lock);
mach_port_deallocate (mach_task_self (), nn->idport);
mach_port_deallocate (mach_task_self (), file);
- free (nn);
+ free (*np);
return err;
}
@@ -161,8 +154,6 @@ set_faked_attribute (struct node *np, unsigned int faked)
void
netfs_node_norefs (struct node *np)
{
- assert (np->nn->np == np);
-
pthread_mutex_unlock (&np->lock);
pthread_spin_unlock (&netfs_node_refcnt_lock);
@@ -172,7 +163,6 @@ netfs_node_norefs (struct node *np)
mach_port_deallocate (mach_task_self (), np->nn->file);
mach_port_deallocate (mach_task_self (), np->nn->idport);
- free (np->nn);
free (np);
pthread_spin_lock (&netfs_node_refcnt_lock);
@@ -358,13 +348,12 @@ netfs_S_dir_lookup (struct protid *diruser,
refcount lock so that, if a node is found, its reference counter cannot
drop to 0 before we get our own reference. */
pthread_spin_lock (&netfs_node_refcnt_lock);
- struct netnode *nn = hurd_ihash_find (&idport_ihash, idport);
- if (nn != NULL)
+ np = hurd_ihash_find (&idport_ihash, idport);
+ if (np != NULL)
{
- assert (nn->np->nn == nn);
/* We already know about this node. */
- if (nn->np->references == 0)
+ if (np->references == 0)
{
/* But it might be in the process of being released. If so,
unlock the hash table to give the node a chance to actually
@@ -376,7 +365,6 @@ netfs_S_dir_lookup (struct protid *diruser,
}
/* Otherwise, reference it right away. */
- np = nn->np;
np->references++;
pthread_spin_unlock (&netfs_node_refcnt_lock);
--
2.0.0.rc2
[PATCH 2/6] libnetfs: add netfs_make_node_alloc to allocate fat nodes
libnetfs has two kind of nodes, struct node and struct netnode.
struct node is used to store libnetfs specific data, while struct
netnode contains user supplied data. Previously, both objects were
allocated separatly, and a pointer from the node to the netnode
provided a mapping from the former to the latter.
Provide a function netfs_make_node_alloc that allocates both nodes in
a contiguous region.
This reduces the memory allocation overhead when creating nodes. It
also makes the relation between node and netnode a simple offset
calculation. Provide two functions to compute the netnode address
from the node address and vice-versa.
Most notably, this makes implementing a cache on top of libnetfs
easier. Auxiliary data for the cache can be stored in the
user-defined netnode, and the fat node can be used as the value.
* libnetfs/make-node.c (init_node): Move initialization here.
(netfs_make_node): Use init_node.
(netfs_make_node_alloc): New function to allocate fat nodes.
* libnetfs/netfs.h (netfs_make_node_alloc): New declaration.
(_netfs_sizeof_struct_node): Likewise.
(netfs_node_netnode): Compute netnode address from node address.
(netfs_netnode_node): And vice-versa.
* libnetfs/init-init.c (_netfs_sizeof_struct_node): New variable.
---
libnetfs/init-init.c | 3 +++
libnetfs/make-node.c | 29 +++--
libnetfs/netfs.h | 27 +++
3 files changed, 53 insertions(+), 6 deletions(-)
diff --git a/libnetfs/init-init.c b/libnetfs/init-init.c
index e98b656..a088ad5 100644
--- a/libnetfs/init-init.c
+++ b/libnetfs/init-init.c
@@ -21,6 +21,9 @@
#include "netfs.h"
+/* For safe inlining of netfs_node_netnode and netfs_netnode_node. */
+size_t const _netfs_sizeof_struct_node = sizeof (struct node);
+
pthread_spinlock_t netfs_node_refcnt_lock = PTHREAD_SPINLOCK_INITIALIZER;
struct node *netfs_root_node = 0;
diff --git a/libnetfs/make-node.c b/libnetfs/make-node.c
index f20ada1..6bd8109 100644
--- a/libnetfs/make-node.c
+++ b/libnetfs/make-node.c
@@ -21,13 +21,9 @@
#include "netfs.h"
#include
-struct node *
-netfs_make_node (struct netnode *nn)
+static struct node *
+init_node (struct node *np, struct netnode *nn)
{
- struct node *np = malloc (sizeof (struct node));
- if (! np)
-return NULL;
-
np->nn = nn;
pthread_mutex_init (&np->lock, NULL);
@@ -40,3 +36,24 @@ netfs_make_node (struct netnode *nn)
return np;
}
+
+struct node *
+netfs_make_node (struct netnode *nn)
+{
+ struct node *np = malloc (sizeof (struct node));
+ if (! np)
+return NULL;
+
+ return init_node (np, nn);
+}
+
+struct node *
+netfs_make_node_alloc (size_t size)
+{
+ struct node *np = malloc (sizeof (struct node) + size);
+
+ if (np == NULL)
+return NULL;
+
+ return init_node (np, netfs_node_netnode (np));
+}
diff --git a/libnetfs/netfs.h b/libnetfs/netfs.h
index aef4a3d..fbe2c60 100644
--- a/libnetfs/netfs.h
+++ b/libnetfs/netfs.h
@@ -372,6 +372,33 @@ extern int netfs_maxsymlinks;
If an error occurs, NULL is returned. */
struct node *netfs_make_node (struct netnode *);
+/* Create a new node structure. Also allocate SIZE bytes for the
+ netnode. The address of the netnode can be obtained using
+ netfs_node_netnode. The new node will have one hard reference and
+ no light references. If an error occurs, NULL is returned. */
+struct node *netfs_make_node_alloc (size_t size);
+
+/* To avoid breaking the ABI whenever sizeof (struct node) changes, we
+ explicitly provide the size. The following two functions will use
+ this value for offset calculations. */
+extern const size_t _netfs_sizeof_struct_node;
+
+/* Return the address of the netnode for NODE. NODE must have been
+ allocated using netfs_make_node_alloc. */
+static inline struct netnode *
+netfs_node_netnode (struct node *node)
+{
+ return (struct netnode *) ((char *) node + _netfs_sizeof_struct_node);
+}
+
+/* Return the address of the node for NETNODE. NETNODE must have been
+ allocated using netfs_make_node_alloc. */
+static inline struct node *
+netfs_netnode_node (struct netnode *netnode)
+{
+ return (struct node *) ((char *) netnode - _netfs_sizeof_struct_node);
+}
+
/* Whenever node->references is to be touched, this lock must be
held. Cf. netfs_nrele, netfs_nput, netfs_nref and netfs_drop_node. */
extern pthread_spinlock_t netfs_node_refcnt_lock;
--
2.0.0.rc2
[PATCH 5/6] libdiskfs: remove the statistics code from the name cache
The current name cache lookup operation completes in O(n) time. This
means that making the cache too large would decrease the performance.
Therefore it was required to tune the size, hence the need for
statistics.
We will use a data structure with worst case constant lookup times in
the future, removing the need to fine tune the cache size.
* libdiskfs/name-cache.c: Remove the statistics code from the name
cache.
---
libdiskfs/name-cache.c | 66 +-
1 file changed, 1 insertion(+), 65 deletions(-)
diff --git a/libdiskfs/name-cache.c b/libdiskfs/name-cache.c
index a212a6d..25b5d0d 100644
--- a/libdiskfs/name-cache.c
+++ b/libdiskfs/name-cache.c
@@ -45,29 +45,12 @@ struct lookup_cache
/* Strlen of NAME. If this is zero, it's an unused entry. */
size_t name_len;
-
- /* XXX */
- int stati;
};
/* The contents of the cache in no particular order */
static struct cacheq lookup_cache = { sizeof (struct lookup_cache) };
static pthread_spinlock_t cache_lock = PTHREAD_SPINLOCK_INITIALIZER;
-
-/* Buffer to hold statistics */
-static struct stats
-{
- long pos_hits;
- long neg_hits;
- long miss;
- long fetch_errors;
-} statistics;
-
-#define PARTIAL_THRESH 100
-#define NPARTIALS MAXCACHE / PARTIAL_THRESH
-struct stats partial_stats [NPARTIALS];
-
/* If there's an entry for NAME, of length NAME_LEN, in directory DIR in the
cache, return its entry, otherwise 0. CACHE_LOCK must be held. */
@@ -85,10 +68,7 @@ find_cache (struct node *dir, const char *name, size_t
name_len)
if (c->name_len == name_len
&& c->dir_cache_id == dir->cache_id
&& c->name[0] == name[0] && strcmp (c->name, name) == 0)
- {
- c->stati = i / 100;
- return c;
- }
+ return c;
return 0;
}
@@ -152,46 +132,6 @@ diskfs_purge_lookup_cache (struct node *dp, struct node
*np)
pthread_spin_unlock (&cache_lock);
}
-/* Register a negative hit for an entry in the Nth stat class */
-void
-register_neg_hit (int n)
-{
- int i;
-
- statistics.neg_hits++;
-
- for (i = 0; i < n; i++)
-partial_stats[i].miss++;
- for (; i < NPARTIALS; i++)
-partial_stats[i].neg_hits++;
-}
-
-/* Register a positive hit for an entry in the Nth stat class */
-void
-register_pos_hit (int n)
-{
- int i;
-
- statistics.pos_hits++;
-
- for (i = 0; i < n; i++)
-partial_stats[i].miss++;
- for (; i < NPARTIALS; i++)
-partial_stats[i].pos_hits++;
-}
-
-/* Register a miss */
-void
-register_miss ()
-{
- int i;
-
- statistics.miss++;
- for (i = 0; i < NPARTIALS; i++)
-partial_stats[i].miss++;
-}
-
-
/* Scan the cache looking for NAME inside DIR. If we don't know
anything entry at all, then return 0. If the entry is confirmed to
@@ -214,14 +154,12 @@ diskfs_check_lookup_cache (struct node *dir, const char
*name)
if (id == 0)
/* A negative cache entry. */
{
- register_neg_hit (c->stati);
pthread_spin_unlock (&cache_lock);
return (struct node *)-1;
}
else if (id == dir->cache_id)
/* The cached node is the same as DIR. */
{
- register_pos_hit (c->stati);
pthread_spin_unlock (&cache_lock);
diskfs_nref (dir);
return dir;
@@ -232,7 +170,6 @@ diskfs_check_lookup_cache (struct node *dir, const char
*name)
struct node *np;
error_t err;
- register_pos_hit (c->stati);
pthread_spin_unlock (&cache_lock);
if (name[0] == '.' && name[1] == '.' && name[2] == '\0')
@@ -259,7 +196,6 @@ diskfs_check_lookup_cache (struct node *dir, const char
*name)
}
}
- register_miss ();
pthread_spin_unlock (&cache_lock);
return 0;
--
2.0.0.rc2
[PATCH 4/6] trans/fakeroot: use netfs_node_netnode instead of np->nn
When using fat nodes, expressions of the form E->nn can be rewritten
as netfs_node_netnode (E). This is much faster as it only involves a
offset calculation. For reference, I used the following semantic
patch to create the patch:
@@
expression E;
@@
- E->nn
+ netfs_node_netnode (E)
* trans/fakeroot.c: Use netfs_node_netnode instead of np->nn.
---
trans/fakeroot.c | 99 ++--
1 file changed, 53 insertions(+), 46 deletions(-)
diff --git a/trans/fakeroot.c b/trans/fakeroot.c
index 59f8a86..32a34ec 100644
--- a/trans/fakeroot.c
+++ b/trans/fakeroot.c
@@ -125,7 +125,7 @@ new_node (file_t file, mach_port_t idport, int locked, int
openmodes,
static void
set_default_attributes (struct node *np)
{
- np->nn->faked = FAKE_UID | FAKE_GID | FAKE_DEFAULT;
+ netfs_node_netnode (np)->faked = FAKE_UID | FAKE_GID | FAKE_DEFAULT;
np->nn_stat.st_uid = 0;
np->nn_stat.st_gid = 0;
}
@@ -133,9 +133,9 @@ set_default_attributes (struct node *np)
static void
set_faked_attribute (struct node *np, unsigned int faked)
{
- np->nn->faked |= faked;
+ netfs_node_netnode (np)->faked |= faked;
- if (np->nn->faked & FAKE_DEFAULT)
+ if (netfs_node_netnode (np)->faked & FAKE_DEFAULT)
{
/* Now that the node has non-default faked attributes, they have to be
retained for future accesses. Account for the hash table reference.
@@ -146,7 +146,7 @@ set_faked_attribute (struct node *np, unsigned int faked)
easy enough if it's ever needed, although scalability could be
improved. */
netfs_nref (np);
- np->nn->faked &= ~FAKE_DEFAULT;
+ netfs_node_netnode (np)->faked &= ~FAKE_DEFAULT;
}
}
@@ -158,11 +158,11 @@ netfs_node_norefs (struct node *np)
pthread_spin_unlock (&netfs_node_refcnt_lock);
pthread_mutex_lock (&idport_ihash_lock);
- hurd_ihash_locp_remove (&idport_ihash, np->nn->idport_locp);
+ hurd_ihash_locp_remove (&idport_ihash, netfs_node_netnode (np)->idport_locp);
pthread_mutex_unlock (&idport_ihash_lock);
- mach_port_deallocate (mach_task_self (), np->nn->file);
- mach_port_deallocate (mach_task_self (), np->nn->idport);
+ mach_port_deallocate (mach_task_self (), netfs_node_netnode (np)->file);
+ mach_port_deallocate (mach_task_self (), netfs_node_netnode (np)->idport);
free (np);
pthread_spin_lock (&netfs_node_refcnt_lock);
@@ -245,7 +245,8 @@ error_t
netfs_check_open_permissions (struct iouser *user, struct node *np,
int flags, int newnode)
{
- return check_openmodes (np->nn, flags & (O_RDWR|O_EXEC), MACH_PORT_NULL);
+ return check_openmodes (netfs_node_netnode (np),
+ flags & (O_RDWR|O_EXEC), MACH_PORT_NULL);
}
error_t
@@ -271,12 +272,12 @@ netfs_S_dir_lookup (struct protid *diruser,
dnp = diruser->po->np;
- mach_port_t dir = dnp->nn->file;
+ mach_port_t dir = netfs_node_netnode (dnp)->file;
redo_lookup:
err = dir_lookup (dir, filename,
flags & (O_NOLINK|O_RDWR|O_EXEC|O_CREAT|O_EXCL|O_NONBLOCK),
mode, do_retry, retry_name, &file);
- if (dir != dnp->nn->file)
+ if (dir != netfs_node_netnode (dnp)->file)
mach_port_deallocate (mach_task_self (), dir);
if (err)
return err;
@@ -380,7 +381,8 @@ netfs_S_dir_lookup (struct protid *diruser,
pthread_mutex_unlock (&dnp->lock);
}
- err = check_openmodes (np->nn, (flags & (O_RDWR|O_EXEC)), file);
+ err = check_openmodes (netfs_node_netnode (np),
+(flags & (O_RDWR|O_EXEC)), file);
pthread_mutex_unlock (&idport_ihash_lock);
}
else
@@ -448,17 +450,17 @@ error_t
netfs_validate_stat (struct node *np, struct iouser *cred)
{
struct stat st;
- error_t err = io_stat (np->nn->file, &st);
+ error_t err = io_stat (netfs_node_netnode (np)->file, &st);
if (err)
return err;
- if (np->nn->faked & FAKE_UID)
+ if (netfs_node_netnode (np)->faked & FAKE_UID)
st.st_uid = np->nn_stat.st_uid;
- if (np->nn->faked & FAKE_GID)
+ if (netfs_node_netnode (np)->faked & FAKE_GID)
st.st_gid = np->nn_stat.st_gid;
- if (np->nn->faked & FAKE_AUTHOR)
+ if (netfs_node_netnode (np)->faked & FAKE_AUTHOR)
st.st_author = np->nn_stat.st_author;
- if (np->nn->faked & FAKE_MODE)
+ if (netfs_node_netnode (np)->faked & FAKE_MODE)
st.st_mode = np->nn_stat.st_mode;
np->nn_stat = st;
@@ -528,7 +530,7 @@ netfs_attempt_chmod (struct iouser *cred, struct node *np,
mode_t mode)
/* We don't bother with error checking since the fake mode change should
always succeed--worst case a later open will get EACCES. */
- (void) file_chmod (np->nn->file, mode);
+ (void) file_chmod (netfs_node_netnode (np)->file, mode);
set_faked_attribute (np, FAKE_MODE);
np->nn_stat.st_mode = mode;
return 0;
@@ -543,7 +545,7 @@ netfs_attempt_mksymlink (struct iouser *cred, struct node
*np, char *name)
char trans[sizeof _HURD_SYMLINK + namel
[PATCH 6/6] libdiskfs: use a hash table for the name cache
Previously, name cache lookup operation completed in O(n) time. This
means that making the cache too large would decrease the performance.
Therefore it was required to tune the size.
Implement the name cache using a hash table.
We use buckets of a fixed size. We approximate the least-frequently
used cache algorithm by counting the number of lookups using
saturating arithmetic in the two lowest bits of the pointer to the
name. Using this strategy we achieve a constant worst-case lookup and
insertion time.
Since we are not bound by the size of the cache anymore, increase its
size from 200 to 1024.
* libdiskfs/name-cache.c: Implement the name cache using a hash table.
(diskfs_enter_lookup_cache): Change accordingly.
(diskfs_purge_lookup_cache): Likewise.
(diskfs_check_lookup_cache): Likewise. Also, hard code a
cache miss for the parent of the root directory and merge unlocking
and releasing of node references.
---
libdiskfs/name-cache.c | 374 -
1 file changed, 274 insertions(+), 100 deletions(-)
diff --git a/libdiskfs/name-cache.c b/libdiskfs/name-cache.c
index 25b5d0d..d8f86b1 100644
--- a/libdiskfs/name-cache.c
+++ b/libdiskfs/name-cache.c
@@ -1,6 +1,6 @@
/* Directory name lookup caching
- Copyright (C) 1996, 1997, 1998 Free Software Foundation, Inc.
+ Copyright (C) 1996, 1997, 1998, 2014 Free Software Foundation, Inc.
Written by Michael I. Bushnell, p/BSG, & Miles Bader.
This file is part of the GNU Hurd.
@@ -20,118 +20,290 @@
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111, USA. */
#include "priv.h"
+#include
#include
-#include
-/* Maximum number of names to cache at once */
-#define MAXCACHE 200
+/* The name cache is implemented using a hash table.
-/* Maximum length of file name we bother caching */
-#define CACHE_NAME_LEN 100
+ We use buckets of a fixed size. We approximate the
+ least-frequently used cache algorithm by counting the number of
+ lookups using saturating arithmetic in the two lowest bits of the
+ pointer to the name. Using this strategy we achieve a constant
+ worst-case lookup and insertion time. */
-/* Cache entry */
-struct lookup_cache
+/* Number of buckets. Must be a power of two. */
+#define CACHE_SIZE 256
+
+/* Entries per bucket. */
+#define BUCKET_SIZE4
+
+/* A mask for fast binary modulo. */
+#define CACHE_MASK (CACHE_SIZE - 1)
+
+/* Cache bucket with BUCKET_SIZE entries.
+
+ The layout of the bucket is chosen so that it will be straight
+ forward to use vector operations in the future. */
+struct cache_bucket
{
- struct cacheq_hdr hdr;
+ /* Name of the node NODE_CACHE_ID in the directory DIR_CACHE_ID. If
+ NULL, the entry is unused. */
+ unsigned long name[BUCKET_SIZE];
- /* Used to indentify nodes to the fs dependent code. 0 for NODE_CACHE_ID
- means a `negative' entry -- recording that there's definitely no node with
- this name. */
- ino64_t dir_cache_id, node_cache_id;
+ /* The key. */
+ unsigned long key[BUCKET_SIZE];
- /* Name of the node NODE_CACHE_ID in the directory DIR_CACHE_ID. Entries
- with names too long to fit in this buffer aren't cached at all. */
- char name[CACHE_NAME_LEN];
+ /* Used to indentify nodes to the fs dependent code. */
+ ino64_t dir_cache_id[BUCKET_SIZE];
- /* Strlen of NAME. If this is zero, it's an unused entry. */
- size_t name_len;
+ /* 0 for NODE_CACHE_ID means a `negative' entry -- recording that
+ there's definitely no node with this name. */
+ ino64_t node_cache_id[BUCKET_SIZE];
};
-/* The contents of the cache in no particular order */
-static struct cacheq lookup_cache = { sizeof (struct lookup_cache) };
+/* The cache. */
+static struct cache_bucket name_cache[CACHE_SIZE];
-static pthread_spinlock_t cache_lock = PTHREAD_SPINLOCK_INITIALIZER;
+/* Protected by this lock. */
+static pthread_mutex_t cache_lock = PTHREAD_MUTEX_INITIALIZER;
-/* If there's an entry for NAME, of length NAME_LEN, in directory DIR in the
- cache, return its entry, otherwise 0. CACHE_LOCK must be held. */
-static struct lookup_cache *
-find_cache (struct node *dir, const char *name, size_t name_len)
+/* Given VALUE, return the char pointer. */
+static inline char *
+charp (unsigned long value)
{
- struct lookup_cache *c;
- int i;
-
- /* Search the list. All unused entries are contiguous at the end of the
- list, so we can stop searching when we see the first one. */
- for (i = 0, c = lookup_cache.mru;
- c && c->name_len;
- c = c->hdr.next, i++)
-if (c->name_len == name_len
- && c->dir_cache_id == dir->cache_id
- && c->name[0] == name[0] && strcmp (c->name, name) == 0)
- return c;
+ return (char *) (value & ~3L);
+}
- return 0;
+/* Given VALUE, return the approximation of use frequency. */
+static inline unsigned long
+frequ (unsigned long value)
+{
+ return value & 3;
}
-/* Node NP has just been found in
Re: [PATCH 1/6] libdiskfs: add diskfs_make_node_alloc to allocate fat nodes
Justus Winter, le Thu 29 May 2014 18:40:59 +0200, a écrit :
> libdiskfs has two kind of nodes, struct node and struct netnode.
> struct node is used to store libdiskfs specific data, while struct
> netnode contains user supplied data. Previously, both objects were
> allocated separatly, and a pointer from the node to the netnode
> provided a mapping from the former to the latter.
>
> Provide a function diskfs_make_node_alloc that allocates both nodes in
> a contiguous region.
>
> This reduces the memory allocation overhead when creating nodes. It
> also makes the relation between node and netnode a simple offset
> calculation. Provide two functions to compute the netnode address
> from the node address and vice-versa.
>
> Most notably, this makes implementing a cache on top of libdiskfs
> easier. Auxiliary data for the cache can be stored in the
> user-defined netnode, and the fat node can be used as the value.
Ack.
> * libdiskfs/node-make.c (init_node): Move initialization here.
> (diskfs_make_node): Use init_node.
> (diskfs_make_node_alloc): New function to allocate fat nodes.
> * libdiskfs/diskfs.h (diskfs_make_node_alloc): New declaration.
> (_diskfs_sizeof_struct_node): Likewise.
> (diskfs_node_disknode): Compute disknode address from node address.
> (diskfs_disknode_node): And vice-versa.
> * libdiskfs/init-init.c (_diskfs_sizeof_struct_node): New variable.
> ---
> libdiskfs/diskfs.h| 27 +++
> libdiskfs/init-init.c | 4
> libdiskfs/node-make.c | 39 ++-
> 3 files changed, 61 insertions(+), 9 deletions(-)
>
> diff --git a/libdiskfs/diskfs.h b/libdiskfs/diskfs.h
> index ae1a150..2c68aa3 100644
> --- a/libdiskfs/diskfs.h
> +++ b/libdiskfs/diskfs.h
> @@ -686,6 +686,33 @@ diskfs_notice_filechange (struct node *np, enum
> file_changed_type type,
> The new node will have one hard reference and no light references. */
> struct node *diskfs_make_node (struct disknode *dn);
>
> +/* Create a new node structure. Also allocate SIZE bytes for the
> + disknode. The address of the disknode can be obtained using
> + diskfs_node_disknode. The new node will have one hard reference
> + and no light references. */
> +struct node *diskfs_make_node_alloc (size_t size);
> +
> +/* To avoid breaking the ABI whenever sizeof (struct node) changes, we
> + explicitly provide the size. The following two functions will use
> + this value for offset calculations. */
> +extern const size_t _diskfs_sizeof_struct_node;
> +
> +/* Return the address of the disknode for NODE. NODE must have been
> + allocated using diskfs_make_node_alloc. */
> +static inline struct disknode *
> +diskfs_node_disknode (struct node *node)
> +{
> + return (struct disknode *) ((char *) node + _diskfs_sizeof_struct_node);
> +}
> +
> +/* Return the address of the node for DISKNODE. DISKNODE must have
> + been allocated using diskfs_make_node_alloc. */
> +static inline struct node *
> +diskfs_disknode_node (struct disknode *disknode)
> +{
> + return (struct node *) ((char *) disknode - _diskfs_sizeof_struct_node);
> +}
> +
>
> /* The library also exports the following functions; they are not generally
> useful unless you are redefining other functions the library provides. */
> diff --git a/libdiskfs/init-init.c b/libdiskfs/init-init.c
> index 35be7ed..7a7f248 100644
> --- a/libdiskfs/init-init.c
> +++ b/libdiskfs/init-init.c
> @@ -25,6 +25,10 @@ the Free Software Foundation, 675 Mass Ave, Cambridge, MA
> 02139, USA. */
> #include
> #include
>
> +/* For safe inlining of diskfs_node_disknode and
> + diskfs_disknode_node. */
> +size_t const _diskfs_sizeof_struct_node = sizeof (struct node);
> +
> mach_port_t diskfs_default_pager;
> mach_port_t diskfs_auth_server_port;
> volatile struct mapped_time_value *diskfs_mtime;
> diff --git a/libdiskfs/node-make.c b/libdiskfs/node-make.c
> index 2b6ef2a..ff0cc0d 100644
> --- a/libdiskfs/node-make.c
> +++ b/libdiskfs/node-make.c
> @@ -19,16 +19,9 @@
> #include
>
>
> -/* Create a and return new node structure with DN as its physical disknode.
> - The node will have one hard reference and no light references. */
> -struct node *
> -diskfs_make_node (struct disknode *dn)
> +static struct node *
> +init_node (struct node *np, struct disknode *dn)
> {
> - struct node *np = malloc (sizeof (struct node));
> -
> - if (np == 0)
> -return 0;
> -
>np->dn = dn;
>np->dn_set_ctime = 0;
>np->dn_set_atime = 0;
> @@ -52,3 +45,31 @@ diskfs_make_node (struct disknode *dn)
>
>return np;
> }
> +
> +/* Create a and return new node structure with DN as its physical disknode.
> + The node will have one hard reference and no light references. */
> +struct node *
> +diskfs_make_node (struct disknode *dn)
> +{
> + struct node *np = malloc (sizeof (struct node));
> +
> + if (np == 0)
> +return 0;
> +
> + return init_node (np, dn);
> +}
> +
> +/* Create a new node structure
Re: [PATCH 2/6] libnetfs: add netfs_make_node_alloc to allocate fat nodes
Justus Winter, le Thu 29 May 2014 18:41:00 +0200, a écrit :
> libnetfs has two kind of nodes, struct node and struct netnode.
> struct node is used to store libnetfs specific data, while struct
> netnode contains user supplied data. Previously, both objects were
> allocated separatly, and a pointer from the node to the netnode
> provided a mapping from the former to the latter.
>
> Provide a function netfs_make_node_alloc that allocates both nodes in
> a contiguous region.
>
> This reduces the memory allocation overhead when creating nodes. It
> also makes the relation between node and netnode a simple offset
> calculation. Provide two functions to compute the netnode address
> from the node address and vice-versa.
>
> Most notably, this makes implementing a cache on top of libnetfs
> easier. Auxiliary data for the cache can be stored in the
> user-defined netnode, and the fat node can be used as the value.
Ack.
> * libnetfs/make-node.c (init_node): Move initialization here.
> (netfs_make_node): Use init_node.
> (netfs_make_node_alloc): New function to allocate fat nodes.
> * libnetfs/netfs.h (netfs_make_node_alloc): New declaration.
> (_netfs_sizeof_struct_node): Likewise.
> (netfs_node_netnode): Compute netnode address from node address.
> (netfs_netnode_node): And vice-versa.
> * libnetfs/init-init.c (_netfs_sizeof_struct_node): New variable.
> ---
> libnetfs/init-init.c | 3 +++
> libnetfs/make-node.c | 29 +++--
> libnetfs/netfs.h | 27 +++
> 3 files changed, 53 insertions(+), 6 deletions(-)
>
> diff --git a/libnetfs/init-init.c b/libnetfs/init-init.c
> index e98b656..a088ad5 100644
> --- a/libnetfs/init-init.c
> +++ b/libnetfs/init-init.c
> @@ -21,6 +21,9 @@
>
> #include "netfs.h"
>
> +/* For safe inlining of netfs_node_netnode and netfs_netnode_node. */
> +size_t const _netfs_sizeof_struct_node = sizeof (struct node);
> +
> pthread_spinlock_t netfs_node_refcnt_lock = PTHREAD_SPINLOCK_INITIALIZER;
>
> struct node *netfs_root_node = 0;
> diff --git a/libnetfs/make-node.c b/libnetfs/make-node.c
> index f20ada1..6bd8109 100644
> --- a/libnetfs/make-node.c
> +++ b/libnetfs/make-node.c
> @@ -21,13 +21,9 @@
> #include "netfs.h"
> #include
>
> -struct node *
> -netfs_make_node (struct netnode *nn)
> +static struct node *
> +init_node (struct node *np, struct netnode *nn)
> {
> - struct node *np = malloc (sizeof (struct node));
> - if (! np)
> -return NULL;
> -
>np->nn = nn;
>
>pthread_mutex_init (&np->lock, NULL);
> @@ -40,3 +36,24 @@ netfs_make_node (struct netnode *nn)
>
>return np;
> }
> +
> +struct node *
> +netfs_make_node (struct netnode *nn)
> +{
> + struct node *np = malloc (sizeof (struct node));
> + if (! np)
> +return NULL;
> +
> + return init_node (np, nn);
> +}
> +
> +struct node *
> +netfs_make_node_alloc (size_t size)
> +{
> + struct node *np = malloc (sizeof (struct node) + size);
> +
> + if (np == NULL)
> +return NULL;
> +
> + return init_node (np, netfs_node_netnode (np));
> +}
> diff --git a/libnetfs/netfs.h b/libnetfs/netfs.h
> index aef4a3d..fbe2c60 100644
> --- a/libnetfs/netfs.h
> +++ b/libnetfs/netfs.h
> @@ -372,6 +372,33 @@ extern int netfs_maxsymlinks;
> If an error occurs, NULL is returned. */
> struct node *netfs_make_node (struct netnode *);
>
> +/* Create a new node structure. Also allocate SIZE bytes for the
> + netnode. The address of the netnode can be obtained using
> + netfs_node_netnode. The new node will have one hard reference and
> + no light references. If an error occurs, NULL is returned. */
> +struct node *netfs_make_node_alloc (size_t size);
> +
> +/* To avoid breaking the ABI whenever sizeof (struct node) changes, we
> + explicitly provide the size. The following two functions will use
> + this value for offset calculations. */
> +extern const size_t _netfs_sizeof_struct_node;
> +
> +/* Return the address of the netnode for NODE. NODE must have been
> + allocated using netfs_make_node_alloc. */
> +static inline struct netnode *
> +netfs_node_netnode (struct node *node)
> +{
> + return (struct netnode *) ((char *) node + _netfs_sizeof_struct_node);
> +}
> +
> +/* Return the address of the node for NETNODE. NETNODE must have been
> + allocated using netfs_make_node_alloc. */
> +static inline struct node *
> +netfs_netnode_node (struct netnode *netnode)
> +{
> + return (struct node *) ((char *) netnode - _netfs_sizeof_struct_node);
> +}
> +
> /* Whenever node->references is to be touched, this lock must be
> held. Cf. netfs_nrele, netfs_nput, netfs_nref and netfs_drop_node. */
> extern pthread_spinlock_t netfs_node_refcnt_lock;
> --
> 2.0.0.rc2
>
--
Samuel
Yep. Moi j'ai un clavier à une touche.
Par contre, ma souris a 102 boutons, c'est pas toujours pratique.
-+- OG in: Guide du Cabaliste Usenet - Le mulot contre attaque -+-
Re: [PATCH 3/6] trans/fakeroot: use fat nodes to simplify the node cache
Justus Winter, le Thu 29 May 2014 18:41:01 +0200, a écrit :
> Previously, fakeroot stored netnodes in the hash table. But we are
> not interested in a cache for netnodes, we need a node cache. So
> fakeroot kept pointers to the associated node object in each netnode
> object.
>
> Use fat netfs nodes, which combine node and netnode objects.
Ack.
> * trans/fakeroot.c (struct netnode): Remove np.
> (idport_ihash): Fix ihash location pointer offset.
> (new_node): Allocate fat nodes, store the node pointer in the hash
> table.
> (netfs_node_norefs): Adjust accordingly.
> (netfs_S_dir_lookup): Likewise.
> ---
> trans/fakeroot.c | 36
> 1 file changed, 12 insertions(+), 24 deletions(-)
>
> diff --git a/trans/fakeroot.c b/trans/fakeroot.c
> index 4175b55..59f8a86 100644
> --- a/trans/fakeroot.c
> +++ b/trans/fakeroot.c
> @@ -47,7 +47,6 @@ static auth_t fakeroot_auth_port;
>
> struct netnode
> {
> - struct node *np; /* our node */
>hurd_ihash_locp_t idport_locp;/* easy removal pointer in idport ihash */
>mach_port_t idport;/* port from io_identity */
>int openmodes; /* O_READ | O_WRITE | O_EXEC */
> @@ -64,7 +63,8 @@ struct netnode
>
> pthread_mutex_t idport_ihash_lock = PTHREAD_MUTEX_INITIALIZER;
> struct hurd_ihash idport_ihash
> - = HURD_IHASH_INITIALIZER (offsetof (struct netnode, idport_locp));
> += HURD_IHASH_INITIALIZER (sizeof (struct node)
> + + offsetof (struct netnode, idport_locp));
>
>
> /* Make a new virtual node. Always consumes the ports. If
> @@ -74,8 +74,9 @@ new_node (file_t file, mach_port_t idport, int locked, int
> openmodes,
> struct node **np)
> {
>error_t err;
> - struct netnode *nn = calloc (1, sizeof *nn);
> - if (nn == 0)
> + struct netnode *nn;
> + *np = netfs_make_node_alloc (sizeof *nn);
> + if (*np == 0)
> {
>mach_port_deallocate (mach_task_self (), file);
>if (idport != MACH_PORT_NULL)
> @@ -84,6 +85,7 @@ new_node (file_t file, mach_port_t idport, int locked, int
> openmodes,
> pthread_mutex_unlock (&idport_ihash_lock);
>return ENOMEM;
> }
> + nn = netfs_node_netnode (*np);
>nn->file = file;
>nn->openmodes = openmodes;
>if (idport != MACH_PORT_NULL)
> @@ -97,35 +99,26 @@ new_node (file_t file, mach_port_t idport, int locked,
> int openmodes,
>if (err)
> {
> mach_port_deallocate (mach_task_self (), file);
> - free (nn);
> + free (*np);
> return err;
> }
> }
>
>if (!locked)
> pthread_mutex_lock (&idport_ihash_lock);
> - err = hurd_ihash_add (&idport_ihash, nn->idport, nn);
> + err = hurd_ihash_add (&idport_ihash, nn->idport, *np);
>if (err)
> goto lose;
>
> - *np = nn->np = netfs_make_node (nn);
> - if (*np == 0)
> -{
> - err = ENOMEM;
> - goto lose_hash;
> -}
> -
>pthread_mutex_lock (&(*np)->lock);
>pthread_mutex_unlock (&idport_ihash_lock);
>return 0;
>
> - lose_hash:
> - hurd_ihash_locp_remove (&idport_ihash, nn->idport_locp);
> lose:
>pthread_mutex_unlock (&idport_ihash_lock);
>mach_port_deallocate (mach_task_self (), nn->idport);
>mach_port_deallocate (mach_task_self (), file);
> - free (nn);
> + free (*np);
>return err;
> }
>
> @@ -161,8 +154,6 @@ set_faked_attribute (struct node *np, unsigned int faked)
> void
> netfs_node_norefs (struct node *np)
> {
> - assert (np->nn->np == np);
> -
>pthread_mutex_unlock (&np->lock);
>pthread_spin_unlock (&netfs_node_refcnt_lock);
>
> @@ -172,7 +163,6 @@ netfs_node_norefs (struct node *np)
>
>mach_port_deallocate (mach_task_self (), np->nn->file);
>mach_port_deallocate (mach_task_self (), np->nn->idport);
> - free (np->nn);
>free (np);
>
>pthread_spin_lock (&netfs_node_refcnt_lock);
> @@ -358,13 +348,12 @@ netfs_S_dir_lookup (struct protid *diruser,
> refcount lock so that, if a node is found, its reference counter cannot
> drop to 0 before we get our own reference. */
>pthread_spin_lock (&netfs_node_refcnt_lock);
> - struct netnode *nn = hurd_ihash_find (&idport_ihash, idport);
> - if (nn != NULL)
> + np = hurd_ihash_find (&idport_ihash, idport);
> + if (np != NULL)
> {
> - assert (nn->np->nn == nn);
>/* We already know about this node. */
>
> - if (nn->np->references == 0)
> + if (np->references == 0)
> {
> /* But it might be in the process of being released. If so,
>unlock the hash table to give the node a chance to actually
> @@ -376,7 +365,6 @@ netfs_S_dir_lookup (struct protid *diruser,
> }
>
>/* Otherwise, reference it right away. */
> - np = nn->np;
>np->references++;
>pthread_spin_unlock (&netfs_node_refcnt_lock);
>
> --
> 2.0.0.rc2
>
--
Samuel
T'as pas de portable ?
J'ai un nokia, dans le bassin d'arcachon
Re: [PATCH 4/6] trans/fakeroot: use netfs_node_netnode instead of np->nn
Justus Winter, le Thu 29 May 2014 18:41:02 +0200, a écrit :
> When using fat nodes, expressions of the form E->nn can be rewritten
> as netfs_node_netnode (E). This is much faster as it only involves a
> offset calculation. For reference, I used the following semantic
> patch to create the patch:
>
> @@
> expression E;
> @@
>
> - E->nn
> + netfs_node_netnode (E)
Ack.
> * trans/fakeroot.c: Use netfs_node_netnode instead of np->nn.
> ---
> trans/fakeroot.c | 99
> ++--
> 1 file changed, 53 insertions(+), 46 deletions(-)
>
> diff --git a/trans/fakeroot.c b/trans/fakeroot.c
> index 59f8a86..32a34ec 100644
> --- a/trans/fakeroot.c
> +++ b/trans/fakeroot.c
> @@ -125,7 +125,7 @@ new_node (file_t file, mach_port_t idport, int locked,
> int openmodes,
> static void
> set_default_attributes (struct node *np)
> {
> - np->nn->faked = FAKE_UID | FAKE_GID | FAKE_DEFAULT;
> + netfs_node_netnode (np)->faked = FAKE_UID | FAKE_GID | FAKE_DEFAULT;
>np->nn_stat.st_uid = 0;
>np->nn_stat.st_gid = 0;
> }
> @@ -133,9 +133,9 @@ set_default_attributes (struct node *np)
> static void
> set_faked_attribute (struct node *np, unsigned int faked)
> {
> - np->nn->faked |= faked;
> + netfs_node_netnode (np)->faked |= faked;
>
> - if (np->nn->faked & FAKE_DEFAULT)
> + if (netfs_node_netnode (np)->faked & FAKE_DEFAULT)
> {
>/* Now that the node has non-default faked attributes, they have to be
>retained for future accesses. Account for the hash table reference.
> @@ -146,7 +146,7 @@ set_faked_attribute (struct node *np, unsigned int faked)
>easy enough if it's ever needed, although scalability could be
>improved. */
>netfs_nref (np);
> - np->nn->faked &= ~FAKE_DEFAULT;
> + netfs_node_netnode (np)->faked &= ~FAKE_DEFAULT;
> }
> }
>
> @@ -158,11 +158,11 @@ netfs_node_norefs (struct node *np)
>pthread_spin_unlock (&netfs_node_refcnt_lock);
>
>pthread_mutex_lock (&idport_ihash_lock);
> - hurd_ihash_locp_remove (&idport_ihash, np->nn->idport_locp);
> + hurd_ihash_locp_remove (&idport_ihash, netfs_node_netnode
> (np)->idport_locp);
>pthread_mutex_unlock (&idport_ihash_lock);
>
> - mach_port_deallocate (mach_task_self (), np->nn->file);
> - mach_port_deallocate (mach_task_self (), np->nn->idport);
> + mach_port_deallocate (mach_task_self (), netfs_node_netnode (np)->file);
> + mach_port_deallocate (mach_task_self (), netfs_node_netnode (np)->idport);
>free (np);
>
>pthread_spin_lock (&netfs_node_refcnt_lock);
> @@ -245,7 +245,8 @@ error_t
> netfs_check_open_permissions (struct iouser *user, struct node *np,
> int flags, int newnode)
> {
> - return check_openmodes (np->nn, flags & (O_RDWR|O_EXEC), MACH_PORT_NULL);
> + return check_openmodes (netfs_node_netnode (np),
> + flags & (O_RDWR|O_EXEC), MACH_PORT_NULL);
> }
>
> error_t
> @@ -271,12 +272,12 @@ netfs_S_dir_lookup (struct protid *diruser,
>
>dnp = diruser->po->np;
>
> - mach_port_t dir = dnp->nn->file;
> + mach_port_t dir = netfs_node_netnode (dnp)->file;
> redo_lookup:
>err = dir_lookup (dir, filename,
> flags & (O_NOLINK|O_RDWR|O_EXEC|O_CREAT|O_EXCL|O_NONBLOCK),
> mode, do_retry, retry_name, &file);
> - if (dir != dnp->nn->file)
> + if (dir != netfs_node_netnode (dnp)->file)
> mach_port_deallocate (mach_task_self (), dir);
>if (err)
> return err;
> @@ -380,7 +381,8 @@ netfs_S_dir_lookup (struct protid *diruser,
> pthread_mutex_unlock (&dnp->lock);
> }
>
> - err = check_openmodes (np->nn, (flags & (O_RDWR|O_EXEC)), file);
> + err = check_openmodes (netfs_node_netnode (np),
> + (flags & (O_RDWR|O_EXEC)), file);
>pthread_mutex_unlock (&idport_ihash_lock);
> }
>else
> @@ -448,17 +450,17 @@ error_t
> netfs_validate_stat (struct node *np, struct iouser *cred)
> {
>struct stat st;
> - error_t err = io_stat (np->nn->file, &st);
> + error_t err = io_stat (netfs_node_netnode (np)->file, &st);
>if (err)
> return err;
>
> - if (np->nn->faked & FAKE_UID)
> + if (netfs_node_netnode (np)->faked & FAKE_UID)
> st.st_uid = np->nn_stat.st_uid;
> - if (np->nn->faked & FAKE_GID)
> + if (netfs_node_netnode (np)->faked & FAKE_GID)
> st.st_gid = np->nn_stat.st_gid;
> - if (np->nn->faked & FAKE_AUTHOR)
> + if (netfs_node_netnode (np)->faked & FAKE_AUTHOR)
> st.st_author = np->nn_stat.st_author;
> - if (np->nn->faked & FAKE_MODE)
> + if (netfs_node_netnode (np)->faked & FAKE_MODE)
> st.st_mode = np->nn_stat.st_mode;
>
>np->nn_stat = st;
> @@ -528,7 +530,7 @@ netfs_attempt_chmod (struct iouser *cred, struct node
> *np, mode_t mode)
>
>/* We don't bother with error checking since the fake mode change should
> always succeed--worst case a later open will get EACCES. */
> - (void) fil
Re: [PATCH 6/6] libdiskfs: use a hash table for the name cache
Justus Winter, le Thu 29 May 2014 18:41:04 +0200, a écrit :
> Previously, name cache lookup operation completed in O(n) time. This
> means that making the cache too large would decrease the performance.
> Therefore it was required to tune the size.
>
> Implement the name cache using a hash table.
>
> We use buckets of a fixed size. We approximate the least-frequently
> used cache algorithm by counting the number of lookups using
> saturating arithmetic in the two lowest bits of the pointer to the
> name. Using this strategy we achieve a constant worst-case lookup and
> insertion time.
>
> Since we are not bound by the size of the cache anymore, increase its
> size from 200 to 1024.
Ack.
> * libdiskfs/name-cache.c: Implement the name cache using a hash table.
> (diskfs_enter_lookup_cache): Change accordingly.
> (diskfs_purge_lookup_cache): Likewise.
> (diskfs_check_lookup_cache): Likewise. Also, hard code a
> cache miss for the parent of the root directory and merge unlocking
> and releasing of node references.
> ---
> libdiskfs/name-cache.c | 374
> -
> 1 file changed, 274 insertions(+), 100 deletions(-)
>
> diff --git a/libdiskfs/name-cache.c b/libdiskfs/name-cache.c
> index 25b5d0d..d8f86b1 100644
> --- a/libdiskfs/name-cache.c
> +++ b/libdiskfs/name-cache.c
> @@ -1,6 +1,6 @@
> /* Directory name lookup caching
>
> - Copyright (C) 1996, 1997, 1998 Free Software Foundation, Inc.
> + Copyright (C) 1996, 1997, 1998, 2014 Free Software Foundation, Inc.
> Written by Michael I. Bushnell, p/BSG, & Miles Bader.
>
> This file is part of the GNU Hurd.
> @@ -20,118 +20,290 @@
> Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111, USA. */
>
> #include "priv.h"
> +#include
> #include
> -#include
>
> -/* Maximum number of names to cache at once */
> -#define MAXCACHE 200
> +/* The name cache is implemented using a hash table.
>
> -/* Maximum length of file name we bother caching */
> -#define CACHE_NAME_LEN 100
> + We use buckets of a fixed size. We approximate the
> + least-frequently used cache algorithm by counting the number of
> + lookups using saturating arithmetic in the two lowest bits of the
> + pointer to the name. Using this strategy we achieve a constant
> + worst-case lookup and insertion time. */
>
> -/* Cache entry */
> -struct lookup_cache
> +/* Number of buckets. Must be a power of two. */
> +#define CACHE_SIZE 256
> +
> +/* Entries per bucket. */
> +#define BUCKET_SIZE 4
> +
> +/* A mask for fast binary modulo. */
> +#define CACHE_MASK (CACHE_SIZE - 1)
> +
> +/* Cache bucket with BUCKET_SIZE entries.
> +
> + The layout of the bucket is chosen so that it will be straight
> + forward to use vector operations in the future. */
> +struct cache_bucket
> {
> - struct cacheq_hdr hdr;
> + /* Name of the node NODE_CACHE_ID in the directory DIR_CACHE_ID. If
> + NULL, the entry is unused. */
> + unsigned long name[BUCKET_SIZE];
>
> - /* Used to indentify nodes to the fs dependent code. 0 for NODE_CACHE_ID
> - means a `negative' entry -- recording that there's definitely no node
> with
> - this name. */
> - ino64_t dir_cache_id, node_cache_id;
> + /* The key. */
> + unsigned long key[BUCKET_SIZE];
>
> - /* Name of the node NODE_CACHE_ID in the directory DIR_CACHE_ID. Entries
> - with names too long to fit in this buffer aren't cached at all. */
> - char name[CACHE_NAME_LEN];
> + /* Used to indentify nodes to the fs dependent code. */
> + ino64_t dir_cache_id[BUCKET_SIZE];
>
> - /* Strlen of NAME. If this is zero, it's an unused entry. */
> - size_t name_len;
> + /* 0 for NODE_CACHE_ID means a `negative' entry -- recording that
> + there's definitely no node with this name. */
> + ino64_t node_cache_id[BUCKET_SIZE];
> };
>
> -/* The contents of the cache in no particular order */
> -static struct cacheq lookup_cache = { sizeof (struct lookup_cache) };
> +/* The cache. */
> +static struct cache_bucket name_cache[CACHE_SIZE];
>
> -static pthread_spinlock_t cache_lock = PTHREAD_SPINLOCK_INITIALIZER;
> +/* Protected by this lock. */
> +static pthread_mutex_t cache_lock = PTHREAD_MUTEX_INITIALIZER;
>
> -/* If there's an entry for NAME, of length NAME_LEN, in directory DIR in the
> - cache, return its entry, otherwise 0. CACHE_LOCK must be held. */
> -static struct lookup_cache *
> -find_cache (struct node *dir, const char *name, size_t name_len)
> +/* Given VALUE, return the char pointer. */
> +static inline char *
> +charp (unsigned long value)
> {
> - struct lookup_cache *c;
> - int i;
> -
> - /* Search the list. All unused entries are contiguous at the end of the
> - list, so we can stop searching when we see the first one. */
> - for (i = 0, c = lookup_cache.mru;
> - c && c->name_len;
> - c = c->hdr.next, i++)
> -if (c->name_len == name_len
> - && c->dir_cache_id == dir->cache_id
> -
Re: [PATCH 5/6] libdiskfs: remove the statistics code from the name cache
Justus Winter, le Thu 29 May 2014 18:41:03 +0200, a écrit :
> The current name cache lookup operation completes in O(n) time. This
> means that making the cache too large would decrease the performance.
> Therefore it was required to tune the size, hence the need for
> statistics.
>
> We will use a data structure with worst case constant lookup times in
> the future, removing the need to fine tune the cache size.
Ack.
> * libdiskfs/name-cache.c: Remove the statistics code from the name
> cache.
> ---
> libdiskfs/name-cache.c | 66
> +-
> 1 file changed, 1 insertion(+), 65 deletions(-)
>
> diff --git a/libdiskfs/name-cache.c b/libdiskfs/name-cache.c
> index a212a6d..25b5d0d 100644
> --- a/libdiskfs/name-cache.c
> +++ b/libdiskfs/name-cache.c
> @@ -45,29 +45,12 @@ struct lookup_cache
>
>/* Strlen of NAME. If this is zero, it's an unused entry. */
>size_t name_len;
> -
> - /* XXX */
> - int stati;
> };
>
> /* The contents of the cache in no particular order */
> static struct cacheq lookup_cache = { sizeof (struct lookup_cache) };
>
> static pthread_spinlock_t cache_lock = PTHREAD_SPINLOCK_INITIALIZER;
> -
> -/* Buffer to hold statistics */
> -static struct stats
> -{
> - long pos_hits;
> - long neg_hits;
> - long miss;
> - long fetch_errors;
> -} statistics;
> -
> -#define PARTIAL_THRESH 100
> -#define NPARTIALS MAXCACHE / PARTIAL_THRESH
> -struct stats partial_stats [NPARTIALS];
> -
>
> /* If there's an entry for NAME, of length NAME_LEN, in directory DIR in the
> cache, return its entry, otherwise 0. CACHE_LOCK must be held. */
> @@ -85,10 +68,7 @@ find_cache (struct node *dir, const char *name, size_t
> name_len)
> if (c->name_len == name_len
> && c->dir_cache_id == dir->cache_id
> && c->name[0] == name[0] && strcmp (c->name, name) == 0)
> - {
> - c->stati = i / 100;
> - return c;
> - }
> + return c;
>
>return 0;
> }
> @@ -152,46 +132,6 @@ diskfs_purge_lookup_cache (struct node *dp, struct node
> *np)
>pthread_spin_unlock (&cache_lock);
> }
>
> -/* Register a negative hit for an entry in the Nth stat class */
> -void
> -register_neg_hit (int n)
> -{
> - int i;
> -
> - statistics.neg_hits++;
> -
> - for (i = 0; i < n; i++)
> -partial_stats[i].miss++;
> - for (; i < NPARTIALS; i++)
> -partial_stats[i].neg_hits++;
> -}
> -
> -/* Register a positive hit for an entry in the Nth stat class */
> -void
> -register_pos_hit (int n)
> -{
> - int i;
> -
> - statistics.pos_hits++;
> -
> - for (i = 0; i < n; i++)
> -partial_stats[i].miss++;
> - for (; i < NPARTIALS; i++)
> -partial_stats[i].pos_hits++;
> -}
> -
> -/* Register a miss */
> -void
> -register_miss ()
> -{
> - int i;
> -
> - statistics.miss++;
> - for (i = 0; i < NPARTIALS; i++)
> -partial_stats[i].miss++;
> -}
> -
> -
>
> /* Scan the cache looking for NAME inside DIR. If we don't know
> anything entry at all, then return 0. If the entry is confirmed to
> @@ -214,14 +154,12 @@ diskfs_check_lookup_cache (struct node *dir, const char
> *name)
>if (id == 0)
> /* A negative cache entry. */
> {
> - register_neg_hit (c->stati);
> pthread_spin_unlock (&cache_lock);
> return (struct node *)-1;
> }
>else if (id == dir->cache_id)
> /* The cached node is the same as DIR. */
> {
> - register_pos_hit (c->stati);
> pthread_spin_unlock (&cache_lock);
> diskfs_nref (dir);
> return dir;
> @@ -232,7 +170,6 @@ diskfs_check_lookup_cache (struct node *dir, const char
> *name)
> struct node *np;
> error_t err;
>
> - register_pos_hit (c->stati);
> pthread_spin_unlock (&cache_lock);
>
> if (name[0] == '.' && name[1] == '.' && name[2] == '\0')
> @@ -259,7 +196,6 @@ diskfs_check_lookup_cache (struct node *dir, const char
> *name)
> }
> }
>
> - register_miss ();
>pthread_spin_unlock (&cache_lock);
>
>return 0;
> --
> 2.0.0.rc2
>
--
Samuel
Profitant de cette occasion, vous serait-il possible de rebooter
aussi Modérator et son petit copain qui gère les ressources de
download ?
-+- OB in NPC : Apprendre à flasher son personnel -+-
Re: let's make libpager single-threaded
Richard Braun, le Tue 27 May 2014 14:12:07 +0200, a écrit : > > But the paging part also comes from client events, doesn't it? E.g. > > on-demand loads of parts of mapped libraries, etc. > > If on-demand here means on page fault, then no. In Mach, client threads > don't process their page faults, the kernel merely sends async requests > to external pagers. There is normally no I/O when the underlying > physical pages are already resident. External pagers are used only when > I/O must be done, Sure, sure. > in which case a single thread per backing device is > fine since processing incoming requests is normally orders of magnitude > faster than the underlying device latency. But precisely: once the only thread gets a data_request, it'll call pager_read_page, which will typically call store_read, which will wait on the eventual I/O, and no other request will be processed during that time, and thus not pipelining the I/O requests. Samuel
Re: let's make libpager single-threaded
On Thu, May 29, 2014 at 07:04:48PM +0200, Samuel Thibault wrote: > But precisely: once the only thread gets a data_request, it'll call > pager_read_page, which will typically call store_read, which will wait > on the eventual I/O, and no other request will be processed during that > time, and thus not pipelining the I/O requests. Right, I assumed the store interface wasn't synchronous... -- Richard Braun
Re: Circular dependency with glibc libpthread libihash
> Could you try
>
> AC_MSG_WARN("cross-compiling, disabling linking")
>
> ?
diff --git a/configure.ac b/configure.ac
index ecabfdf..7ede6db 100644
--- a/configure.ac
+++ b/configure.ac
@@ -83,6 +83,13 @@ AC_PROG_INSTALL
AC_PROG_AWK
AC_PROG_SED
+if test "x$cross_compiling" = "xyes"; then
+ # It may be that we don't have a working libc yet, for instance
+ # because we're bootstrapping the cross-compilation tool chain.
+ # Thus, use this undocumented Autoconf macro designed for this.
+ AC_NO_EXECUTABLES
+ AC_MSG_WARN("cross-compiling, disabling linking")
+fi
AC_PROG_CC
# Require GCC.
if test x$GCC != xyes; then
Works like a charm.
Manolis
Re: Circular dependency with glibc libpthread libihash
Manolis Ragkousis, le Thu 29 May 2014 21:01:02 +, a écrit : > Works like a charm. Good, applied, thanks! Samuel
Re: [PATCH 06/11] libstore: provide function declaration until available upstream
Justus Winter, le Fri 23 May 2014 08:39:56 +0200, a écrit : > Quoting Samuel Thibault (2014-05-23 00:22:14) > > Justus Winter, le Thu 22 May 2014 22:13:28 +0200, a écrit : > > > Until the Hurd specific header is available, provide a local > > > declaration of ped_device_new_from_store. > > > > Mmm, does it really fix something? I'd prefer to keep the warning > > showing up, so somebody eventually takes the time to provide a header. > > Otherwise we'll just hide a warning, and possibly nobody will eventually > > take the time to actually fix it. > > With this change, Hurd compiles with -Werror=implicit. Then I'd say put a #warning next to the added declaration :) Samuel
State of patch in debian/hurd glibc
What is the state of the patch unsubmitted-pthread_posix-option.diff
from the debian/hurd glibc package?
It is critical in building glibc, as without it fails with
> In file included from gai_sigqueue.c:22:0:
> ./gai_misc.h:32:5: error: unknown type name 'pthread_cond_t'
> pthread_cond_t *cond;
> ^
> ./gai_misc.h:74:1: error: unknown type name 'pthread_mutex_t'
> extern pthread_mutex_t __gai_requests_mutex;
Also if I don't patch Versions.def file with
--- a/Versions.def
+++ b/Versions.def
@@ -102,6 +102,7 @@ libpthread {
GLIBC_2.6
GLIBC_2.11
GLIBC_2.12
+ GLIBC_2.17
GLIBC_PRIVATE
}
libresolv {
will fail with
> version GLIBC_2.17 not defined for libpthread
Maybe some of these changes should be pushed to upstream.What do you think?
Even if glibc doesn't have an integrated pthread library, it depends on the hurd
packages which have a pthread library, so we should provide all the defines etc.
Avoid letting glibc try to install its own headers for libpthread.
No topgit branch, TODO?
---
sysdeps/mach/hurd/Makefile |4 +++
sysdeps/mach/hurd/bits/posix_opt.h | 40 ++---
sysdeps/mach/hurd/gai_misc.h | 44 +
3 files changed, 75 insertions(+), 13 deletions(-)
--- a/sysdeps/mach/hurd/bits/posix_opt.h
+++ b/sysdeps/mach/hurd/bits/posix_opt.h
@@ -71,24 +71,38 @@
/* XPG4.2 shared memory is supported. */
#define _XOPEN_SHM 1
-/* We do not have the POSIX threads interface. */
-#define _POSIX_THREADS -1
+/* Tell we have POSIX threads. */
+#define _POSIX_THREADS 200112L
/* We have the reentrant functions described in POSIX. */
#define _POSIX_REENTRANT_FUNCTIONS 1
#define _POSIX_THREAD_SAFE_FUNCTIONS 200809L
-/* These are all things that won't be supported when _POSIX_THREADS is not. */
+/* We do not provide priority scheduling for threads. */
#define _POSIX_THREAD_PRIORITY_SCHEDULING -1
-#define _POSIX_THREAD_ATTR_STACKSIZE -1
-#define _POSIX_THREAD_ATTR_STACKADDR -1
+
+/* We support user-defined stack sizes. */
+#define _POSIX_THREAD_ATTR_STACKSIZE 200112L
+
+/* We support user-defined stacks. */
+#define _POSIX_THREAD_ATTR_STACKADDR 200112L
+
+/* We do not support priority inheritence. */
#define _POSIX_THREAD_PRIO_INHERIT -1
+
+/* We do not support priority protection. */
#define _POSIX_THREAD_PRIO_PROTECT -1
+
#ifdef __USE_XOPEN2K8
+/* We do not support priority inheritence for robust mutexes. */
# define _POSIX_THREAD_ROBUST_PRIO_INHERIT -1
+
+/* We do not support priority protection for robust mutexes. */
# define _POSIX_THREAD_ROBUST_PRIO_PROTECT -1
#endif
-#define _POSIX_SEMAPHORES -1
+
+/* We support POSIX.1b semaphores. */
+#define _POSIX_SEMAPHORES 200112L
/* Real-time signals are not yet supported. */
#define _POSIX_REALTIME_SIGNALS-1
@@ -121,17 +135,17 @@
/* GNU libc provides regular expression handling. */
#define _POSIX_REGEXP 1
-/* Reader/Writer locks are not available. */
-#define _POSIX_READER_WRITER_LOCKS -1
+/* Reader/Writer locks are available. */
+#define _POSIX_READER_WRITER_LOCKS 200112L
/* We have a POSIX shell. */
#define _POSIX_SHELL 1
-/* We cannot support the Timeouts option without _POSIX_THREADS. */
-#define _POSIX_TIMEOUTS-1
+/* We support the Timeouts option. */
+#define _POSIX_TIMEOUTS200112L
-/* We do not support spinlocks. */
-#define _POSIX_SPIN_LOCKS -1
+/* We support spinlocks. */
+#define _POSIX_SPIN_LOCKS 200112L
/* The `spawn' function family is supported. */
#define _POSIX_SPAWN 200809L
@@ -140,7 +154,7 @@
#define _POSIX_TIMERS 0
/* The barrier functions are not available. */
-#define _POSIX_BARRIERS-1
+#define _POSIX_BARRIERS200112L
/* POSIX message queues could be available in future. */
#define_POSIX_MESSAGE_PASSING 0
--- a/sysdeps/mach/hurd/Makefile
+++ b/sysdeps/mach/hurd/Makefile
@@ -167,6 +167,10 @@
CFLAGS-dl-load.c = -DEXTERNAL_MAP_FROM_FD
endif
+ifeq ($(subdir),posix)
+CFLAGS-confstr.c += -DLIBPTHREAD_VERSION='"libpthread 0.3"'
+endif
+
# Override the generic Makeconfig values so we link against the RPC libs.
link-libc-static := -Wl,--start-group \
$(patsubst %,$(common-objpfx)%.a,\
--- /dev/null
+++ b/sysdeps/mach/hurd/gai_misc.h
@@ -0,0 +1,44 @@
+#include
+#include
+
+#define gai_start_notify_thread __gai_start_notify_thread
+#define gai_create_helper_thread __gai_create_helper_thread
+
+extern inline void
+__gai_start_notify_thread (void)
+{
+ sigset_t ss;
+ sigemptyset (&ss);
+ sigprocmask(SIG_SETMASK, &ss, NULL);
+}
+
+extern inline int
+__gai_create_helper_thread (pthread_t *threadp, void *(*tf) (void *),
+ void *arg)
+{
+ pthread_attr_t attr;
+
+ /* Make sure the thread is created detached. */
+ pthread_attr_init (&attr);
+ p
Re: State of patch in debian/hurd glibc
Manolis Ragkousis, le Thu 29 May 2014 22:34:35 +, a écrit : > Maybe some of these changes should be pushed to upstream.What do you think? Well, all our patches should be pushed to upstream, whenever anybody gets the time to submit them :) I don't really have much more to say than the comment, except that in the upstream repository there isn't our libpthread yet, so it doesn't really make sense to claim in headers that we have libpthread etc. I don't really know why we don't have a topgit branch for this yet. I'll just create it, then. Samuel
Re: State of patch in debian/hurd glibc
That's great thank you :-) Manolis
Re: pbuilder problems
Justus Winter, le Sat 08 Jun 2013 14:15:14 +0200, a écrit : > Quoting Samuel Thibault (2013-06-08 13:34:18) > > Justus Winter, le Sat 08 Jun 2013 13:27:07 +0200, a écrit : > > > I'm trying to setup pbuilder on one of my virtual Hurd > > > systems. Problem is that debootstrap eventually dies: > > > > I don't think pbuilder has ever been made to work on hurd-i386. There > > are quite a few things to do for a chroot, see > > http://www.gnu.org/software/hurd/hurd/running/chroot > > I see. This separate translator the page is talking about, does it > have to be a filesystem or would any translator do the trick? Just to confirm: there is no need for a separate translator for chroot, I've removed the mention of it from the page. Samuel
