ABI compatibility checker for shared libraries
Hi, I'd like to introduce shlib-compat -- an ABI compatibility checker for shared libraries with symbol versioning. The idea of such tool was discussed on mail lists before. shlib-compat uses debugging info (dwarf) from compiled library to compare definitions of exported symbols, i.e. no sources is necessary. Code quality is pre-alpha, loops in struct/union definitions are not handled properly, values for enums and const/volatile are ignored. Reporting is not yet implemented, it has knob to dump complete symbol definitions in json which is rather useless. It just prints if symbol definitions match. shlib-compat parses libc.so.7 on my system (amd64), but there is a lot warnings about symbols it can't find in dwarf dump (looks like these are implemented in asm). Sources contain several trivial test cases. devel/dwarfdump port is required (as well as objdump which is in base). Source code available on github: http://github.com/glk/shlib-compat Or as tarball: http://github.com/downloads/glk/shlib-compat/shlib-compat-0.1.tar.gz I don't have much free time to continue developing it right now, but feel free to add a ticket on github, submit a patch or fork it. Thanks, Gleb. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Userland debug symbols directory
On (05/11/2010 15:14), Mark Johnston wrote:
> Hi all,
>
> I have some tentative patches which add support for creating a separate
> directory containing all of the userland debugging symbols.
>
> I posted about this a week or so ago:
> http://lists.freebsd.org/pipermail/freebsd-hackers/2010-October/033437.html
>
> Some future work will involve finding out if any changes are necessary to
> support Clang/LLVM, and seeing whether there's any interest in adding
> support to the FreeBSD installer to install the debug symbols to a
> user-defined directory.
>
> Note that DEBUG_FLAGS must be set somewhere when building world -
> adding WITH_DEBUG_SYMBOLS_DIR=yes to src.conf beforehand is sufficient,
> as it causes DEBUG_FLAGS+=-g to be set; building world with DEBUG_FLAGS=-g
> alone is also sufficient. If the binaries are built without debug symbols
> and one tries to use my new option, nothing happens, i.e., no new
> files/directories are created, unless specific programs explicitly
> build with -g somehow (see below).
Hi,
I like the idea a lot, but why not to leave symbol files in /usr/obj,
otherwise it creates a problem of mismatch between installed binaries
and stale symbol files. Kernel case is very different, it preserves
previous version during 'make installkernel' besides one would hardly
need symbols for all installed binaries. It's for developers only after
all. If there is a problem to be debugged one can always find binary
and corresponding symbols file under /usr/obj.
I find this feature to be particularly useful for shlib-compat ABI
compatibility checker:
http://marc.info/?l=freebsd-hackers&m=128861933016176&w=2
> My changes are below. There's a new file (stripbin.sh) which invokes
> objcopy(1) and strip(1). At the moment it's in usr/src, but it should
> probably go elsewhere... perhaps usr/src/tools? The other changes are to
> bsd.prog.mk, bsd.own.mk and the src.conf man page.
>
> I've also noticed a few Makefiles that explicitly set DEBUG_FLAGS=-g
> or CFLAGS+=-g for some reason. They are in
>
> usr.bin/tar/
> cddl/usr.bin/ctfconvert/
> cddl/usr.sbin/usr.sbin/lockstat/
> usr.sbin/wpa/wpa_supplicant/
> usr.sbin/wpa/hostapd/
> usr.sbin/bluetooth/bthidd/
>
> I'm not sure if this is intentional.
>
> Feedback and suggestions are extremely welcome. =)
>
> Thanks,
> -Mark
>
> diff --git a/stripbin.sh b/stripbin.sh
> new file mode 100755
> index 000..be2e9ad
> --- /dev/null
> +++ b/stripbin.sh
> @@ -0,0 +1,29 @@
> +#!/bin/sh
> +
> +# This script is invoked by install(1) on all binaries when installing world.
> +# It determines whether any debug info is present in the binary; if so, it
> +# will extract the debug symbols to $SYMBOLS_FULLPATH, strip the binary and
> +# add a link to the binary which points to the file containing the debugging
> +# symbols.
> +
> +: ${READELFEXEC:=/usr/obj/usr/src/gnu/usr.bin/binutils/readelf/readelf}
> +: ${STRIPEXEC:=/usr/obj/usr/src/gnu/usr.bin/binutils/strip/strip}
> +: ${OBJCEXEC:=/usr/obj/usr/src/gnu/usr.bin/binutils/objcopy/objcopy}
> +: ${DIRNAMEEXEC:=/usr/obj/usr/src/usr.bin/dirname/dirname}
Did it survive make universe? Looks incorrect for me. At least you
should depend on binaries which are part of the build toolchain, and not
hard code paths.
I'd propose not to add any scripts and handle it within makefiles.
> +
> +SYMBOLS_FULLPATH="${SYMBOLS_DIR}`${DIRNAMEEXEC} ${1}`"
> +
> +case $1 in
> +/*...@*)
> + exit 0
> + ;;
> +esac
> +
> +# Make sure that some debug info is actually present.
> +[ -z `$READELFEXEC -wi $1` ] && exit 0
> +
> +mkdir -p $SYMBOLS_FULLPATH
> +
> +$STRIPEXEC --only-keep-debug -o ${SYMBOLS_DIR}${1}.symbols $1
> +$STRIPEXEC --strip-debug $1
> +$OBJCEXEC --add-gnu-debuglink=${SYMBOLS_DIR}${1}.symbols $1
> diff --git a/share/mk/bsd.prog.mk b/share/mk/bsd.prog.mk
> index 47b43ab..9809db0 100644
> --- a/share/mk/bsd.prog.mk
> +++ b/share/mk/bsd.prog.mk
> @@ -29,6 +29,12 @@ CFLAGS+=${CRUNCH_CFLAGS}
>
> .if !defined(DEBUG_FLAGS)
> STRIP?= -s
> +.else
> +
> +.if defined(STRIPSCRIPT)
> +STRIP?= -s
> +INSTALL:= /usr/bin/env SYMBOLS_DIR=${SYMBOLS_DIR} STRIPBIN=${STRIPSCRIPT}
> ${INSTALL}
> +.endif
> .endif
>
> .if defined(NO_SHARED) && (${NO_SHARED} != "no" && ${NO_SHARED} != "NO")
> diff --git a/share/mk/bsd.own.mk b/share/mk/bsd.own.mk
> index 3aa832c..80b42ad 100644
> --- a/share/mk/bsd.own.mk
> +++ b/share/mk/bsd.own.mk
> @@ -541,6 +541,12 @@ MK_${vv:H}:= ${MK_${vv:T}}
> .endif
> .endfor
>
> +.if defined(WITH_DEBUG_SYMBOLS_DIR)
> +DEBUG_FLAGS+=-g
> +STRIPSCRIPT= /usr/src/stripbin.sh
> +SYMBOLS_DIR?=/usr/local/lib/debug
> +.endif
> +
> .endif # !_WITHOUT_SRCCONF
>
> .endif # !target()
> diff --git a/share/man/man5/src.conf.5 b/share/man/man5/src.conf.5
> index 4f8f9a1..d787746 100644
> --- a/share/man/man5/src.conf.5
> +++ b/share/man/man5/src.conf.5
> @@ -283,6 +283,15 @@ Set to not build CVS.
> Set to not build
> .Xr g++ 1
> and related libraries.
> +.It Va WITH_DEBUG_SYM
[rfc] Replacing FNV and hash32 with Paul Hsieh's SuperFastHash
Hi,
I've recently noticed that hash table use in nullfs was inefficient, 1/3
to half of buckets remained unused. I've started investigating it
further and came across SuperFastHash hashing function, SFH
(SuperFastHash) has BSD license, used in WebKit and other open source
projects. Detailed description and Comparision with FNV and Bob Jenkin's
hash can be found here:
http://www.azillionmonkeys.com/qed/hash.html
In my tests SFH (SuperFastHash) was 1.3 to 4 times faster than FNV (it
depends on size of data being hashed) e.g. performance of hashing 56
bytes struct (Core i5 CPU, 2 cores + 2 hyperthreads):
SFH -- 1339.79 MB/s
FNV -- 330.27 MB/s
I've prepared a patch to change FNV and hash32 with SFH in the tree.
For testing I've used dbench with 16 processes on 1 Gb swap back md
device, UFS + SoftUpdates:
Old hash (Mb/s): 599.94 600.096 599.536
SFH hash (Mb/s): 612.439 612.341 609.673
It's just ~1% improvement, but dbench is not a VFS metadata intensive
benchmark. Subjectively it feels faster accessing maildir mailboxes
with ~10.000 messages : )
I'd also wish hash32 (often called Bernstein hash) to be replaced with
better function (it might be FNV if not SFH). Hash32 is inappropriate
for binary data that results in poor distribution.
Thanks,
Gleb.
commit 6e19401826b6769fa95b1e4f9e561dec7101082b
Author: Gleb Kurtsou
Date: Thu Dec 16 08:05:14 2010 +0200
Import Paul Hsieh's SuperFastHash.
Replace FNV and Bernstein hash uses with SFH.
In most cases Bernstein is simply inappropriate choice.
diff --git a/lib/libkvm/kvm_minidump_amd64.c b/lib/libkvm/kvm_minidump_amd64.c
index 15630b1..7c9c4dd 100644
--- a/lib/libkvm/kvm_minidump_amd64.c
+++ b/lib/libkvm/kvm_minidump_amd64.c
@@ -35,7 +35,7 @@ __FBSDID("$FreeBSD$");
#include
#include
#include
-#include
+#include
#include
#include
#include
@@ -74,26 +74,26 @@ static void
hpt_insert(kvm_t *kd, vm_paddr_t pa, int64_t off)
{
struct hpte *hpte;
- uint32_t fnv = FNV1_32_INIT;
+ uint32_t hash;
- fnv = fnv_32_buf(&pa, sizeof(pa), fnv);
- fnv &= (HPT_SIZE - 1);
+ hash = hash_sfh_buf(&pa, sizeof(pa), sizeof(pa));
+ hash &= (HPT_SIZE - 1);
hpte = malloc(sizeof(*hpte));
hpte->pa = pa;
hpte->off = off;
- hpte->next = kd->vmst->hpt_head[fnv];
- kd->vmst->hpt_head[fnv] = hpte;
+ hpte->next = kd->vmst->hpt_head[hash];
+ kd->vmst->hpt_head[hash] = hpte;
}
static int64_t
hpt_find(kvm_t *kd, vm_paddr_t pa)
{
struct hpte *hpte;
- uint32_t fnv = FNV1_32_INIT;
+ uint32_t hash;
- fnv = fnv_32_buf(&pa, sizeof(pa), fnv);
- fnv &= (HPT_SIZE - 1);
- for (hpte = kd->vmst->hpt_head[fnv]; hpte != NULL; hpte = hpte->next) {
+ hash = hash_sfh_buf(&pa, sizeof(pa), sizeof(pa));
+ hash &= (HPT_SIZE - 1);
+ for (hpte = kd->vmst->hpt_head[hash]; hpte != NULL; hpte = hpte->next) {
if (pa == hpte->pa)
return (hpte->off);
}
diff --git a/lib/libkvm/kvm_minidump_arm.c b/lib/libkvm/kvm_minidump_arm.c
index d48c1bc..f42f4ca 100644
--- a/lib/libkvm/kvm_minidump_arm.c
+++ b/lib/libkvm/kvm_minidump_arm.c
@@ -38,7 +38,7 @@ __FBSDID("$FreeBSD$");
#include
#include
#include
-#include
+#include
#include
#include
#include
@@ -77,26 +77,26 @@ static void
hpt_insert(kvm_t *kd, uint64_t pa, int64_t off)
{
struct hpte *hpte;
- uint32_t fnv = FNV1_32_INIT;
+ uint32_t hash;
- fnv = fnv_32_buf(&pa, sizeof(pa), fnv);
- fnv &= (HPT_SIZE - 1);
+ hash = hash_sfh_buf(&pa, sizeof(pa), sizeof(pa));
+ hash &= (HPT_SIZE - 1);
hpte = malloc(sizeof(*hpte));
hpte->pa = pa;
hpte->off = off;
- hpte->next = kd->vmst->hpt_head[fnv];
- kd->vmst->hpt_head[fnv] = hpte;
+ hpte->next = kd->vmst->hpt_head[hash];
+ kd->vmst->hpt_head[hash] = hpte;
}
static int64_t
hpt_find(kvm_t *kd, uint64_t pa)
{
struct hpte *hpte;
- uint32_t fnv = FNV1_32_INIT;
+ uint32_t hash;
- fnv = fnv_32_buf(&pa, sizeof(pa), fnv);
- fnv &= (HPT_SIZE - 1);
- for (hpte = kd->vmst->hpt_head[fnv]; hpte != NULL; hpte = hpte->next)
+ hash = hash_sfh_buf(&pa, sizeof(pa), sizeof(pa));
+ hash &= (HPT_SIZE - 1);
+ for (hpte = kd->vmst->hpt_head[hash]; hpte != NULL; hpte = hpte->next)
if (pa == hpte->pa)
return (hpte->off);
diff --git a/lib/libkvm/kvm_minidump_i386.c b/lib/libkvm/kvm_minidump_i386.c
index 0d31705..343d70e 100644
--- a/lib/libkvm/kvm_minidump_i386.c
+++ b/lib/libkvm/kvm_minidump_i386.c
@@ -35,7 +35,7 @@ __FBSDID("$FreeBSD$");
#include
#inclu
Re: [rfc] Replacing FNV and hash32 with Paul Hsieh's SuperFastHash
On (24/12/2010 00:46), Gleb Kurtsou wrote:
> Hi,
>
> I've recently noticed that hash table use in nullfs was inefficient, 1/3
> to half of buckets remained unused. I've started investigating it
Nullfs patch I've forgotten to attach before.
It adds vfs.nullfs.buckets tunable to change number of hash table
buckets, vfs.nullfs.nodes sysctl to get number of active vnodes, and
increases default number of buckets from 16 to 32. Note that nullfs
nodes - are currently opened vnodes, so number is so low, one may want
to further increase it only if nullfs is heavily used (lots of null
mounts and active clients).
The patch requires SFH, but might be useful without it if
null_hash_mixptr() changed accordingly.
> further and came across SuperFastHash hashing function, SFH
> (SuperFastHash) has BSD license, used in WebKit and other open source
> projects. Detailed description and Comparision with FNV and Bob Jenkin's
> hash can be found here:
> http://www.azillionmonkeys.com/qed/hash.html
>
> In my tests SFH (SuperFastHash) was 1.3 to 4 times faster than FNV (it
> depends on size of data being hashed) e.g. performance of hashing 56
> bytes struct (Core i5 CPU, 2 cores + 2 hyperthreads):
> SFH -- 1339.79 MB/s
> FNV -- 330.27 MB/s
>
> I've prepared a patch to change FNV and hash32 with SFH in the tree.
>
> For testing I've used dbench with 16 processes on 1 Gb swap back md
> device, UFS + SoftUpdates:
> Old hash (Mb/s): 599.94 600.096 599.536
> SFH hash (Mb/s): 612.439 612.341 609.673
>
> It's just ~1% improvement, but dbench is not a VFS metadata intensive
> benchmark. Subjectively it feels faster accessing maildir mailboxes
> with ~10.000 messages : )
>
> I'd also wish hash32 (often called Bernstein hash) to be replaced with
> better function (it might be FNV if not SFH). Hash32 is inappropriate
> for binary data that results in poor distribution.
>
> Thanks,
> Gleb.
commit dd778e797cbfd821fc43ff74a8d5681ed2b93da1
Author: Gleb Kurtsou
Date: Thu Dec 16 08:13:58 2010 +0200
nullfs: Use SFH. Add sysctls to control hash table size
Add vfs.nullfs sysctls and tunable to change number hash table buckets
Increase default number of buckets to 32 (was 16)
diff --git a/sys/fs/nullfs/null_subr.c b/sys/fs/nullfs/null_subr.c
index 5717a01..2acd85b 100644
--- a/sys/fs/nullfs/null_subr.c
+++ b/sys/fs/nullfs/null_subr.c
@@ -36,18 +36,21 @@
#include
#include
+#include
#include
#include
#include
#include
#include
#include
+#include
#include
#include
-#define LOG2_SIZEVNODE 8 /* log2(sizeof struct vnode) */
-#defineNNULLNODECACHE 16
+#defineNULL_BUCKETS_MIN16
+#defineNULL_BUCKETS_DEFAULT32
+#defineNULL_BUCKETS_TUNABLE"vfs.nullfs.buckets"
/*
* Null layer cache:
@@ -58,7 +61,7 @@
*/
#defineNULL_NHASH(vp) \
- (&null_node_hashtbl[(((uintptr_t)vp)>>LOG2_SIZEVNODE) & null_node_hash])
+ (&null_node_hashtbl[null_hash_mixptr(vp) & null_node_hash])
static LIST_HEAD(null_node_hashhead, null_node) *null_node_hashtbl;
static u_long null_node_hash;
@@ -70,6 +73,15 @@ MALLOC_DEFINE(M_NULLFSNODE, "nullfs_node", "NULLFS vnode
private part");
static struct vnode * null_hashget(struct mount *, struct vnode *);
static struct vnode * null_hashins(struct mount *, struct null_node *);
+static u_long null_nodes;
+static u_long null_buckets = NULL_BUCKETS_DEFAULT;
+
+SYSCTL_NODE(_vfs, OID_AUTO, nullfs, CTLFLAG_RW, 0, "null file system");
+SYSCTL_ULONG(_vfs_nullfs, OID_AUTO, nodes, CTLFLAG_RD, &null_nodes, 0,
+"Allocated nodes");
+SYSCTL_ULONG(_vfs_nullfs, OID_AUTO, buckets, CTLFLAG_RD, &null_buckets, 0,
+"Allocated node hash table buckets");
+
/*
* Initialise cache headers
*/
@@ -77,10 +89,15 @@ int
nullfs_init(vfsp)
struct vfsconf *vfsp;
{
-
NULLFSDEBUG("nullfs_init\n"); /* printed during system boot */
- null_node_hashtbl = hashinit(NNULLNODECACHE, M_NULLFSHASH,
&null_node_hash);
+ TUNABLE_ULONG_FETCH(NULL_BUCKETS_TUNABLE, &null_buckets);
+ if (null_buckets < NULL_BUCKETS_MIN)
+ null_buckets = NULL_BUCKETS_MIN;
+ else if (null_buckets > desiredvnodes)
+ null_buckets = NULL_BUCKETS_DEFAULT;
+ null_node_hashtbl = hashinit(null_buckets, M_NULLFSHASH,
&null_node_hash);
mtx_init(&null_hashmtx, "nullhs", NULL, MTX_DEF);
+ null_nodes = 0;
return (0);
}
@@ -94,6 +111,12 @@ nullfs_uninit(vfsp)
return (0);
}
+static __inline uint32_t
+null_hash_mixptr(void *ptr)
+{
+ return (hash_sfh_buf(&ptr, sizeof(ptr), 33554467UL));
+}
+
/*
* Return a VREF'ed alias for lo
Re: [rfc] Replacing FNV and hash32 with Paul Hsieh's SuperFastHash
On (25/12/2010 20:29), Ivan Voras wrote: > On 23.12.2010 23:46, Gleb Kurtsou wrote: > > > For testing I've used dbench with 16 processes on 1 Gb swap back md > > device, UFS + SoftUpdates: > > Old hash (Mb/s): 599.94 600.096 599.536 > > SFH hash (Mb/s): 612.439 612.341 609.673 > > > > It's just ~1% improvement, but dbench is not a VFS metadata intensive > > benchmark. Subjectively it feels faster accessing maildir mailboxes > > with ~10.000 messages : ) > > Try blogbench if you need metadata-intensive operations, or even fsx. blogbench should be good, but I've always had hard time interpreting its results. Besides results tend to very a lot, there is no way to set seed value like in fsx, so that I could run exactly the same test in different configurations. I prefer to use blogbench for stability testing. fsx is a different beast, it reads/writes/truncates at random offsets - great tool for debugging mmap/truncate issues. Patch doesn't improve it in any way. > > ___ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org" ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: [rfc] Replacing FNV and hash32 with Paul Hsieh's SuperFastHash
On (26/12/2010 15:20), Ivan Voras wrote: > On 26 December 2010 14:24, Gleb Kurtsou wrote: > > On (25/12/2010 20:29), Ivan Voras wrote: > >> On 23.12.2010 23:46, Gleb Kurtsou wrote: > >> > >> > For testing I've used dbench with 16 processes on 1 Gb swap back md > >> > device, UFS + SoftUpdates: > >> > Old hash (Mb/s): 599.94 600.096 599.536 > >> > SFH hash (Mb/s): 612.439 612.341 609.673 > >> > > >> > It's just ~1% improvement, but dbench is not a VFS metadata intensive > >> > benchmark. Subjectively it feels faster accessing maildir mailboxes > >> > with ~10.000 messages : ) > >> > >> Try blogbench if you need metadata-intensive operations, or even fsx. > > > blogbench should be good, but I've always had hard time interpreting its > > results. Besides results tend to very a lot, there is no way to set seed > > value like in fsx, so that I could run exactly the same test in different > > configurations. > > I think the exact sequence of blogbench operations depends on duration > of previous operations (it's multithreaded) so from that angle you are Why should it? Operation order in dbench or fsx doesn't depend on duration of previous operations. > right - you can't do a repeatable run except in the trivial cases. On > the other hand, it uses rand() without seeding it with > srand()/sranddev() so this part is actually very repeatable :) I've once tried to make its behaviour more predictable, I can't find the patch and can't recall any specifics, but there were architectural issues. You are right, setting seed and calling rand() should give stable results, that's what I was trying to achieve. The other way to work around such "limitation" is too run sufficiently large number of tests. Which requires patience :) > > fsx is a different beast, it reads/writes/truncates at random offsets - > > great tool for debugging mmap/truncate issues. Patch doesn't improve it > > in any way. > > It depends on what metadata operations you require - blogbench will > create, find and write files (if we ignore atime); fsx will create a > decent amount of traffic with file size and mtime changes. In your > case you'll probably need to run it on a memory file system or tmpfs > due to sensitivity to disk IO latencies (if your improvements is on > the order of few percent). I meant create/readdir/remove as metadata intensive operations -- blogbench is very good for it. fsx creates single file. Most people will only notice changes in vfs_cache.c and UFS' dirhash, that's 600 Mb/s vs 613 Mb/s improvement I've written about above. I'd appreciate if someone could benchmark if_lagg, it was using hash32 for binary data, which could result in poor hash table usage, which could possibly make most of the data go on single interface. But there would be hardly any performance improvement due to limited network bandwidth. Besides old hash32 is faster than new SFH. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Getting full binary path in MAC Framework
On (28/12/2010 14:03), Jakub Szafrański wrote:
> Hi,
> At first sory for my bad english and for my behaviour - english is not my
> native language, and I am new to mail lists.
>
> I'm trying to get the *FULL* path to a binary launched by the user, so
> that I could use it later. I've managed to get just the binary name, OR get
> the binary name from /proc, but I'd like it to be better (and don't require
> /proc).
Due to VFS design there is no reliable way of getting full path to
vnode. In some cases getting full path is impossible, e.g. file may be
deleted but still open.
It looks like you are working on a security policy to verify executable
before running it, I'd suggest you attach signature to executable itself
or use extended attributes. Among other issues path-based security
solutions are inherently race-prone and thus generally not as secure as
advertised.
>
> This is what I've already written:
>
> #include
> #include
> #include
> #include
> #include
> #include
> #include
> #include
> #include
> #include
> #include
>
> #include
>
>
> SYSCTL_DECL(_security_mac);
>
> SYSCTL_NODE(_security_mac, OID_AUTO, veriexec, CTLFLAG_RW, 0,
> "MAC veriexec implementation");
>
>
> static int veriexec_enabled = 0;
> SYSCTL_INT(_security_mac_veriexec, OID_AUTO, enabled, CTLFLAG_RW,
> &veriexec_enabled, 0, "Enforce mac_veriexec policy");
>
> static int veriexec_level = 0;
> SYSCTL_INT(_security_mac_veriexec, OID_AUTO, level, CTLFLAG_RW,
> &veriexec_level, 0, "Veriexec security level");
>
> static int veriexec_vnode_check_exec(struct ucred *cred, struct vnode *vp,
> struct label *vplabel, struct image_params *imgp,
> struct label *execlabel)
> {
> if (veriexec_enabled) {
> if (cred && imgp && imgp->execpath) {
> log(LOG_NOTICE, "UID %d launched PID %d, veriexec_level: %d %s\n",
> cred->cr_uid, imgp->proc->p_pid, veriexec_level, imgp->execpath);
> }
> }
> return 0;
> }
>
> static struct mac_policy_ops veriexec_ops =
> {
> .mpo_vnode_check_exec = veriexec_vnode_check_exec,
> };
>
> MAC_POLICY_SET(&veriexec_ops, mac_veriexec, "MAC veriexec implementation",
> MPC_LOADTIME_FLAG_UNLOADOK, NULL);
>
> I'll be glad for any help
>
> Jakub 'samu' Szafrański
> ___
> freebsd-hackers@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
> To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Building "third-party" modules for kernel with debug options?
On (07/01/2011 15:15), Lev Serebryakov wrote:
> Hello, Freebsd-hackers.
>
>
> I've found, that "struct bio" is depend on state of "DIAGNOSTIC"
> flag ("options DIAGNOSTIC" in kernel config). But when I build
> third-party GEOM (or any other) module with using of ,
> there is no access to these options. So, module, built from ports, can
> fail on user's kernel, even if it built with proper kernel sources in
> "/usr/src/sys". Is here any solution for this problem?
Set KERNBUILDDIR before building module for options to be picked up,
e.g.:
export KERNBUILDDIR=`make -C /sys -V .OBJDIR`/`uname -i`
cd /module/path; make
>
> P.S. NB: GEOM module is only example, question is about modules &
> kernel options in general, so I put this message on Hackers list.
>
> --
> // Black Lion AKA Lev Serebryakov
>
> ___
> freebsd-hackers@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
> To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Why not give git a try? (was "Re: [head tinderbox] failure on amd64/amd64")
On (24/01/2011 11:33), Alexander Best wrote: > On Mon Jan 24 11, Garrett Cooper wrote: > > On Sun, Jan 23, 2011 at 9:16 PM, Peter Jeremy wrote: > > > On 2011-Jan-21 20:01:32 +0100, "Simon L. B. Nielsen" > > > wrote: > > >>Perhaps we should just set the tinderbox up to sync directly of > > >>cvsup-master instead if that makes it more useful? > > > > > > Can cvsup-master still lose atomicity of commits? I suspect it can, > > > in which case syncing directly off the SVN master would seem a better > > > approach. > > > > I've seen a lot of `self-healing' failures lately w.r.t. cvsup, so I > > wonder if it's time to look at another solution to this problem as > > these annoying stability issues don't appear to be going away. What > > about git? > > > > Just some things I'm able to rattle off that come to mind with git.. > > it would also be nice to have github running on freebsd.org. that way it would > be much easier to discuss src changes without having to point people at a > file, > a function or even a specific line. also it would finally kill the > mailinglists, which have lots of issues: spam, broken mailman installation, > people going berserker when they see lines > 80 etc. there have been a few > attempts to introduce a code review system, but since that was all hosted on > foreign websites the idea never cought on and afaik those websites weren't > being supported/promoted by freebsd.org. Having github would be nice, but it's not open source. Another option could be gitorious, there are merge requests with review option[1], patch review, already hosted freebsd repository[2]. All we need as a first step is developers starting accepting merge requests from each other, people use it already[3]. 1. http://blog.gitorious.org/2009/11/06/awesome-code-review/ 2. http://gitorious.org/freebsd 3. http://gitorious.org/freebsd/repositories > but personally i don't expect a change like this to happen in the near future. > basically most of the freebsd administrative people are quite conservative. i > wouldn't be surprised if some them are still trying to run freebsd on their > typewriters. ;) > > cheers. > alex > > > > > Some arguments `for git'... > > > > 1. One tool to rule them all: > >- cvsup/csup can be replaced with git archive [1]. > >- cvs git scales a bit better. > >- less support cost for p4 and lower likelihood of downtime in the > > event of critical failure (perforce's proprietary DB is a pain to > > recover I've recently discovered from other dealings). > >- svn <-> cvs exporter is no longer required as it's all one SCM. > >- As a side-effect, the bits present in CVS and SVN would now be > > 100% in sync, unlike cvs which can lead svn in terms of commits (at > > least that was the case when I last talked to someone about version > > numbering in pkg_install done by re@). > > 2. More evolved tool: > >- branches are cheap and can be local or remote. > >- distributed SCM seem to work well with large groups of developers. > >- works better with branching and merging from what I've seen. > > > > Some arguments against git... > > - The one caveat to cvsup/csup that's awesome is its componentization > > capability, i.e. being able to selectively download components in src > > / ports; I'm not 100% sure but there doesn't appear to be a clear > > analog in git. It might be achievable through gits remote. in > > git-config, git-remote, etc, but I would need to prototype whether or > > not this is true. > > - Higher learning curve. > > - Some slightly annoying nits with stashing local changes when working > > on separate branches (need to talk to git maintainers). > > - > > > > Some more git experienced folks could comment here, but it would > > be nice to unify all of the systems under `one flag' for the sake of > > simplicity and hopefully the sanity of the tool maintainers (Simon, et > > all). > > Thanks! > > -Garrett > > -- > a13x > ___ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org" ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Namecache lock contention?
On (28/01/2011 18:20), Ivan Voras wrote:
> On 28 January 2011 16:25, Dan Nelson wrote:
>
> > My guess would be:
> >
> > kern/vfs_cache.c:151 static struct rwlock cache_lock;
> > kern/vfs_cache.c:152 RW_SYSINIT(vfscache, &cache_lock, "Name Cache");
> >
> > The CACHE_*LOCK() macros.c in vfs_cache use cache_lock, so you've got lots
> > of possible contention points. procstat -ka and/or dtrace might help you
> > determine exactly where.
>
> I'm new with dtrace so I tried this:
>
> lockstat:::rw-block
> {
> @traces[stack()] = count();
> }
>
> with these results:
>
> http://ivoras.net/stuff/rw-block.txt
>
> It's informative because most of the traces are namecache-related. As
> suspected, the most blocking occurs in stat().
>
> As this is a rwlock I'd interpret it as waiting for a write lock to be
> lifted so the readers can acquire it, but I need to confirm this as
> there's a lot of things that can in theory be stat()ed here.
>
> I'm going to continue investigating with dtrace but I'd appreciate
> pointers on how to make the output more useful (like including
> filenames from stat()).
You could try replacing rwlock with plain mutex to check if there are
priority propagation issues among readers/writers. SX locks should also
work but would likely to be a considerable performance regression.
Finding out home much activity is there outside of storage/a/b/file
(sessions storage) could also be helpful.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Namecache lock contention?
On (28/01/2011 22:59), Ivan Voras wrote: > On 28 January 2011 22:18, Gleb Kurtsou wrote: > > > You could try replacing rwlock with plain mutex to check if there are > > priority propagation issues among readers/writers. > > How would that manifest? (i.e. how would it be detectable) Benchmark. If there are prio propagation issues mutex version should be faster than original locking. Let me know if you need help with patch. > > SX locks should also > > work but would likely to be a considerable performance regression. > > With mutexes I'd lose all shared (read) acquisitions so I doubt sx > locks would do much more harm :) Yeah, but sxlocks are more heavy weight. The question is what actual readers/writers ratio in your test is, e.g. all namecache entries for a dir may be purged on rename. > > Finding out home much activity is there outside of storage/a/b/file > > (sessions storage) could also be helpful. > > Here's more information: > > * The session storage (i.e. mostly file creates / writes in this > particular workload) is on a separate file system than the core of the > application (which only does reads) Changing namecache lock to become per-file system would hardly improve situation in general. > * The dtrace output I've send is from around thirty seconds of > operation, so around 2000 PHP runs. (PHP in this case is FastCGI, so > the processes are persistent instead of constantly respawning). In > these 2000 runs there have been around 20,000 rw-block events in > cache_lookup - which is strange. Are there rename, rmdir calls? - these purge namecache. If cache is empty, VOP_LOOKUP acquires write lock to populate the cache. > * Here's another dtrace output without rwlock mutex inlining, showing > a different picture than what I've earlier thought: most rw-blocks > events are in wlock! http://ivoras.net/stuff/rw-block-noinline.txt -- > there are also some blocks without a rwlock function in the trace; I > don't understand how rwlock inlining is implemented, maybe the readers > are always inlined? Add options RWLOCK_NOINLINE, recompiling with -O0 might also be good idea. > > Next step - find out how to make dtrace print files for which this happens. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: OpenGrok for FreeBSD
On (12/05/2011 22:45), Zafer Aydoğan wrote: > Hello List, > > I've set up an OpenGrok service for the FreeBSD sources (current). > OpenGrok is a source code search engine and cross reference. > I'm updating the sources once a day. I'm a NetBSD developer and I'm > also maintaining http://nxr.netbsd.org > > Feel free to use it: > http://fxr.aydogan.net Thanks! What do you think about having index for different kernels in one place like now defunct http://grok.x12.su/source/ or http://fxr.watson.org/fxr/source/?v=HEAD > > Cheers, Zafer. > ___ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org" ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: kexec or similar for FreeBSD
On (16/06/2011 13:32), Russell Cattelan wrote: > I have been contacted about possibly implementing a fast reboot > mechanism for FreeBSD similar to kexec on Linux. > > I have just started looking into how this accomplished so I figured > a note to freebsd hackers would also be a good place to ask > for comments. > > Has anybody looked at doing something like kexec? I was working on similar project some time ago. First of all you have to leave hardware in known good state for a new kernel. Reseting devices can be generally accomplished by unloading corresponding kernel modules (even if they are compiled in kernel). The biggest problem for me was timers and programmable interrupt controller. I didn't make it work properly, but my goals where much wider than replacing with another FreeBSD kernel. Aim was to restore initial BIOS state as much as possible. > Is it the right thing to do for FreeBSD. I'm concerned that the way > FreeBSD handles early kernel modules (loaded via the boot loader) > vs linux which does everything via initrd is going to be a problem. I find loader code easy work with. You could write dummy filesystem implementation for libstand. So that customized loader will load both kernel and modules yet while running FreeBSD. Your "reboot" procedure wouldn't even use any BIOS io interrupts. Linux boot is a real mess imho. > > Thanks for any help on this. > > -Russell Cattelan ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: kexec or similar for FreeBSD
On (16/06/2011 22:35), Russell Cattelan wrote: > On 6/16/11 3:06 PM, Gleb Kurtsou wrote: > > On (16/06/2011 13:32), Russell Cattelan wrote: > >> I have been contacted about possibly implementing a fast reboot > >> mechanism for FreeBSD similar to kexec on Linux. > >> > >> I have just started looking into how this accomplished so I figured > >> a note to freebsd hackers would also be a good place to ask > >> for comments. > >> > >> Has anybody looked at doing something like kexec? > > I was working on similar project some time ago. First of all you have to > > leave hardware in known good state for a new kernel. Reseting devices > > can be generally accomplished by unloading corresponding kernel modules > > (even if they are compiled in kernel). The biggest problem for me was > > timers and programmable interrupt controller. I didn't make it work > > properly, but my goals where much wider than replacing with another > > FreeBSD kernel. Aim was to restore initial BIOS state as much as > > possible. > What were your goals beyond booting a new kernel? I think getting back > to a known BIOS state is kinda required to get a kernel through the boot > process. From what I can tell the main goal of kexec is to avoid the process > of re-initializing the hardware via BIOS. Yes it's required to be able load device drivers once again. I had to get back BIOS USB support, both USB HID devices and USB mass storage access with int 13h. Which is not documented by BIOS vendors. So decision was made not to boot full OS, but to extend loader and boot FreeBSD kernel only if necessary rebooting afterwards. > >> Is it the right thing to do for FreeBSD. I'm concerned that the way > >> FreeBSD handles early kernel modules (loaded via the boot loader) > >> vs linux which does everything via initrd is going to be a problem. > > I find loader code easy work with. You could write dummy filesystem > > implementation for libstand. So that customized loader will load both > > kernel and modules yet while running FreeBSD. Your "reboot" procedure > > wouldn't even use any BIOS io interrupts. Linux boot is a real mess > > imho. > > > So it is possible to get back to the loader once the kernel is booted? > So the running kernel could load the new kernel / modules and then jump > back to the loader to start the boot process. I meant that you can allocate memory and load new kernel and modules still while running original FreeBSD kernel, i.e. reuse loader code to load elf objects, pass boot args to kernel, etc. That would require very specific memory layout and proper BIOS memory map (SMAP). Unload all drivers, disable timers and interrupt controller. Then you can start new kernel directly without going to loader, thus avoiding real mode crap, finding original boot device BIOS number and boot0+boot altogether. I'm not even sure int 13h will always be functional after device was claimed by FreeBSD driver. It's not the case for USB, ATA should work, booting from anything else is likely to be a problem. > > >> Thanks for any help on this. > >> > >> -Russell Cattelan > ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: kexec or similar for FreeBSD
On (21/06/2011 16:05), Warner Losh wrote: > On Jun 21, 2011, at 2:40 PM, Gleb Kurtsou wrote: > > On (16/06/2011 22:35), Russell Cattelan wrote: > >> On 6/16/11 3:06 PM, Gleb Kurtsou wrote: > >>> On (16/06/2011 13:32), Russell Cattelan wrote: > >>>> I have been contacted about possibly implementing a fast reboot > >>>> mechanism for FreeBSD similar to kexec on Linux. > >>>> > >>>> I have just started looking into how this accomplished so I figured > >>>> a note to freebsd hackers would also be a good place to ask > >>>> for comments. > >>>> > >>>> Has anybody looked at doing something like kexec? > >>> I was working on similar project some time ago. First of all you have to > >>> leave hardware in known good state for a new kernel. Reseting devices > >>> can be generally accomplished by unloading corresponding kernel modules > >>> (even if they are compiled in kernel). The biggest problem for me was > >>> timers and programmable interrupt controller. I didn't make it work > >>> properly, but my goals where much wider than replacing with another > >>> FreeBSD kernel. Aim was to restore initial BIOS state as much as > >>> possible. > >> What were your goals beyond booting a new kernel? I think getting back > >> to a known BIOS state is kinda required to get a kernel through the boot > >> process. From what I can tell the main goal of kexec is to avoid the > >> process > >> of re-initializing the hardware via BIOS. > > Yes it's required to be able load device drivers once again. I had to > > get back BIOS USB support, both USB HID devices and USB mass storage > > access with int 13h. Which is not documented by BIOS vendors. So > > decision was made not to boot full OS, but to extend loader and boot > > FreeBSD kernel only if necessary rebooting afterwards. > > Well, if it is really kexec, wouldn't the kernel be able to use its > drivers to load the stuff, rendering the BIOS state irrelevant? Or > were you kexecing /boot/loader, which would be a lot harder... The project I've mentioned was about implementing a solution similar to kexec-loader, it boots Linux kernel and uses kexec to boot another loader or Linux kernel. Linux kernel execution starts in real mode(!) unlike FreeBSD. The project was very different from FreeBSD-kexec I describe. I'd like to avoid running /boot/loader booting FreeBSD kernel directly and to use full-fledged OS to load new kernel and modules. http://www.solemnwarning.net/kexec-loader/ > > >>>> Is it the right thing to do for FreeBSD. I'm concerned that the way > >>>> FreeBSD handles early kernel modules (loaded via the boot loader) > >>>> vs linux which does everything via initrd is going to be a problem. > >>> I find loader code easy work with. You could write dummy filesystem > >>> implementation for libstand. So that customized loader will load both > >>> kernel and modules yet while running FreeBSD. Your "reboot" procedure > >>> wouldn't even use any BIOS io interrupts. Linux boot is a real mess > >>> imho. > >>> > >> So it is possible to get back to the loader once the kernel is booted? > >> So the running kernel could load the new kernel / modules and then jump > >> back to the loader to start the boot process. > > I meant that you can allocate memory and load new kernel and modules > > still while running original FreeBSD kernel, i.e. reuse loader code to > > load elf objects, pass boot args to kernel, etc. That would require very > > specific memory layout and proper BIOS memory map (SMAP). Unload all > > drivers, disable timers and interrupt controller. Then you can start new > > kernel directly without going to loader, thus avoiding real mode crap, > > finding original boot device BIOS number and boot0+boot altogether. > > Yea, what he said :) > > > I'm not even sure int 13h will always be functional after device was > > claimed by FreeBSD driver. It's not the case for USB, ATA should work, > > booting from anything else is likely to be a problem. > > Yup. Once you're in the kernel, all bets are off on BIOS functions on amd64 > (you can't go back to a state where you can call the BIOS without resetting > the CPU aka rebooting). Many bets are off on i386. > > Warner ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Capsicum project: Ideas needed
On (09/07/2011 15:54), Gabor Kovesdan wrote: > Em 08-07-2011 13:23, Ivan Voras escreveu: > > On 08/07/2011 05:42, Ilya Bakulin wrote: > >> Hi hackers, > >> As a part of ongoing effort to enhance usage of Capsicum in FreeBSD base > >> system, I want to ask you, which applications in the base system should > >> receive sandboxing support. > > > > How about a small description what sandboxing can bring to applications? > > > > I'm browsing the documents at > > http://www.cl.cam.ac.uk/research/security/capsicum/documentation.html > > but it looks like it still mostly describes the generic framework > > rather than what you can do with it. From it, it looks like you can > > set limits on file handle operations (e.g. (lc_limitfd(STDOUT_FILENO, > > CAP_FSTAT | CAP_SEEK | CAP_WRITE)), but what else? > Yes, I've been reading the thread and I don't know either what are the > deliverables of a Capsicum sandbox. > > Anyway, consider sendmail and BIND. I think these are important enough > to get some more protection. Both sendmail and bind are very complicated peaces of software. I thinks it would be necessary to split them up into several independent daemons first and than place each into capsicum sandbox. Privilege separation makes sshd a better condidate here (sshd is already sandboxed). I'd really like to see lwresd sandboxed and enabled by default, ntpdate may also be a good candidate but it's not that important. > > Gabor > ___ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org" ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Fwd: my git development snapshot(s)
On (18/09/2011 12:14), Andriy Gapon wrote: > > Just decided to follow the global trends and trying to throw all of my > local/private changes at you in hope that the "crowd-sourcing magic" might > somehow happen :-) This seems definitely easier than carefully producing the > patch files and keeping them up-to-date. > > So, my newly cloned gitorious repository: > https://gitorious.org/~avg/freebsd/avgbsd > > And the first branch of interest: > https://gitorious.org/~avg/freebsd/avgbsd/commits/devel-20110915 > > This is a snapshot of almost all of my local changes to the FreeBSD src tree. > Please note that the branch is not intended to be public in the git sense. > That > is, there will be no linear history - I periodically rebase my changes on top > of > the svn head and also frequently reshuffle/merge/split commits. > The snapshot is not tidied up, there are quite a few commits that should be > merged into other commits, some commit messages are not accurate/pretty, etc. > The older the commits, the more mature they are supposed to be. > > Based on the above, no new commits are expected to this snapshot branch. > I will produce new snapshot branches from time to time. > > I am posting this information to this list initially, later I plan to share > the > code with the wider audience e.g. via hackers@. > > P.S. This code sharing is made easier for me by git, Gitorious and "git rebase > --onto" in particular. Thanks to Fabien Thomas for the initial FreeBSD clone > repository at Gitorious! Let me share my experience as well. My repo: https://github.com/glk/freebsd-head/ I used rebase to keep local branches as well, but no longer do so. Such setup worked for me at least for two years, I had local changes and worked on a larger patchset for 7,8-STABLE and CURRENT branches simultaneously (no longer do it either). But (imho) this approach his serious drawbacks. The biggest one is that it's hard to check if regression comes from your code or recently merged upstream code. The second one: once you screw rebase (merge) you are in big trouble. Both issues can be worked around by keeping previous master branches, but it hardly helps: once there are conflicts with upstream or your local changes get commited upstream it becomes only harder and harder. (I used to have master-prev-DATE similar to devel-DATE Andriy uses.) I like and use rebase a lot while working on the code: reorganize commits, small fixups, etc. But try not to touch code that is more than several days old unless really necessary. The reason is to maintain code base in well tested state: imagine you've changed or small bit in HEAD~20, a week later you find a bug, bisect to find that it showed up in orig-HEAD~10 because of your small fix in orig-HEAD~20. Fixing it would have taken less time without that rebase, or bug wouldn't appear in the first place. Once history stabilizes (i.e. remains untouched for 3-4 days) I push it public or backup (for local projects) repository. There is no history rewriting after this point. Use separate branches for independent patchsets. E.g. tmpfs branch - for tmpfs changes, misc - for smaller changes, etc. It lets me bisect and test code easier. For example if I get strange panic I could create temporal branch with all patchsets except pefs branch. Test it, find faulty branch, bisect individual changes. Besides it's also possible to bisect master branch, merge appropriate patchset revisions while bisecting upstream, etc. Upgrade procedure. Patchset branches are independently merged with upstream (vendor/master in my case). And all patchset branches are merged into master branch. In other words resolve conflicts in patchset branches, merge everything back to master. I usually merge individual branches only if there are conflicts or after important milestone. Octopus merge in git may not be as good as you I'd like it to if there are conflicts in several branches, but it's manageable. I also find identifying my changes vs upstream easier than in rebase scenario: - Use 'git log --first-parent'. Limiting number of commits to show in gitk is also good option for large repositories. - Use 'git merge --no-log --no-ff' to merge with upstream, it strips excessive logs from commit message, --no-ff is necessary only for the first time because fast-forward merges won't be possible. To keep history extremely clean: create empty git repo, make non-empty commit and merge with upstream. And there is hope that keeping independent branches would make "crowd-sourcing magic" more feasible :) I'd prefer to be able to merge code I'm interested in without touching unrelated bits. Thanks, Gleb. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
gcc 4.2 miscompilation with -O2 -fno-omit-frame-pointer on amd64
Hi, I was lucky to write a bit of code which gcc 4.2 fails to compile correctly with -O2. Too keep long story short the code fails for gcc from base system and last gcc 4.2 snapshot from ports. It works with gcc 4.3, gcc 4.4 on FreeBSD and Linux. Clang from base is also good. -O and -Os optimization levels are fine (I've tried with all -f* flags mentioned in documentation) -O2 -fno-omit-frame-pointer combination is troublesome on amd64. I presume i386 should be fine. These options are also used for compilation of kernel (with debugging enabled) and modules. I'm not able to share the code, but have a test case reproducing the bug. I've encountered the issue over a week ago and tried narrowing it down to a simple test I could share but without much success. The code itself is very common: initialize two structs on stack, call a function with pointers to those stucts as arguments. A number of inlined assertion functions. gcc fails to correctly optimize struct assignments with -fno-omit-frame-pointer, I have a number of small structs assigned, gcc decides not to use data coping but to assign fields directly. I've tried disabling sra, tweaking sra parameters -- no luck in forcing it to copy data. Replacing one particular assignment with memcpy produces correct code, but that's not a solution. -O2 -fno-omit-frame-pointer -fno-inline is buggy -O2 -fno-omit-frame-pointer -frename-registers is buggy I found similar issue with gcc 4.6, but I'm not able to reproduce it with gcc test case: https://bugzilla.redhat.com/show_bug.cgi?id=679924 http://gcc.gnu.org/bugzilla/show_bug.cgi?id=47893 I'll be glad to help debugging it and will be hanging on #bsddev during weekend as glk. Thanks, Gleb. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: gcc 4.2 miscompilation with -O2 -fno-omit-frame-pointer on amd64
On (19/11/2011 12:25), Alexander Best wrote: > On Sat Nov 19 11, Gleb Kurtsou wrote: > > Hi, > > > > I was lucky to write a bit of code which gcc 4.2 fails to compile > > correctly with -O2. Too keep long story short the code fails for gcc > > from base system and last gcc 4.2 snapshot from ports. It works with gcc > > 4.3, gcc 4.4 on FreeBSD and Linux. Clang from base is also good. -O and > > -Os optimization levels are fine (I've tried with all -f* flags > > mentioned in documentation) > > > > -O2 -fno-omit-frame-pointer combination is troublesome on amd64. I > > presume i386 should be fine. These options are also used for > > compilation of kernel (with debugging enabled) and modules. > > > > I'm not able to share the code, but have a test case reproducing the > > bug. I've encountered the issue over a week ago and tried narrowing it down > > to a simple test I could share but without much success. > > > > The code itself is very common: initialize two structs on stack, call a > > function with pointers to those stucts as arguments. A number of inlined > > assertion functions. gcc fails to correctly optimize struct assignments > > with -fno-omit-frame-pointer, I have a number of small structs assigned, > > gcc decides not to use data coping but to assign fields directly. I've > > tried disabling sra, tweaking sra parameters -- no luck in forcing it > > to copy data. Replacing one particular assignment with memcpy produces > > correct code, but that's not a solution. > > > > -O2 -fno-omit-frame-pointer -fno-inline is buggy > > -O2 -fno-omit-frame-pointer -frename-registers is buggy > > does the issue also come up with '-fno-builtin' in addition to those flags? > is '-O2 -fno-omit-frame-pointer' alone also buggy? does the issue also exist > when using -O1 and -O3? -O0 -g -std=gnu99 -pipe -fno-omit-frame-pointer -fno-strict-aliasing -- OK -Os -g -std=gnu99 -pipe -fno-omit-frame-pointer -fno-strict-aliasing -- OK -O -g -std=gnu99 -pipe -fno-omit-frame-pointer -fno-strict-aliasing -- BAD -O1 -g -std=gnu99 -pipe -fno-omit-frame-pointer -fno-strict-aliasing -- BAD -O2 -g -std=gnu99 -pipe -fno-omit-frame-pointer -- BAD -O2 -g -std=gnu99 -pipe -fno-omit-frame-pointer -fno-builtin -fno-strict-aliasing -- BAD -O3 -g -std=gnu99 -pipe -fno-omit-frame-pointer -fno-strict-aliasing -- OK!! -O3 -fno-inline-functions -fno-unswitch-loops -fno-gcse-after-reload -g -std=gnu99 -pipe -fno-omit-frame-pointer -fno-strict-aliasing -- BAD -O3 -fno-inline-functions -g -std=gnu99 -pipe -fno-omit-frame-pointer -fno-strict-aliasing -- BAD -O2 -finline-functions -g -std=gnu99 -pipe -fno-omit-frame-pointer -fno-strict-aliasing -- OK So, it's -finline-functions that makes it work. But I'm afraid it just masks the real problem. The rest of CFLAGS is warnings and includes: -D_GNU_SOURCE -Wsystem-headers -Werror -Wall -Wno-format-y2k -W -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wreturn-type -Wcast-qual -Wno-write-strings -Wswitch -Wshadow -Wunused-parameter -Wcast-align -Wno-pointer-sign -Wstrict-aliasing=2 -Wno-error=strict-aliasing > > cheers. > alex > > > > > I found similar issue with gcc 4.6, but I'm not able to reproduce it > > with gcc test case: > > https://bugzilla.redhat.com/show_bug.cgi?id=679924 > > http://gcc.gnu.org/bugzilla/show_bug.cgi?id=47893 > > > > I'll be glad to help debugging it and will be hanging on #bsddev during > > weekend as glk. > > > > > > Thanks, > > Gleb. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: gcc 4.2 miscompilation with -O2 -fno-omit-frame-pointer on amd64
On (19/11/2011 07:26), m...@freebsd.org wrote: > On Sat, Nov 19, 2011 at 2:01 AM, Gleb Kurtsou wrote: > > Hi, > > > > I was lucky to write a bit of code which gcc 4.2 fails to compile > > correctly with -O2. Too keep long story short the code fails for gcc > > from base system and last gcc 4.2 snapshot from ports. It works with gcc > > 4.3, gcc 4.4 on FreeBSD and Linux. Clang from base is also good. -O and > > -Os optimization levels are fine (I've tried with all -f* flags > > mentioned in documentation) > > > > -O2 -fno-omit-frame-pointer combination is troublesome on amd64. I > > presume i386 should be fine. These options are also used for > > compilation of kernel (with debugging enabled) and modules. > > > > I'm not able to share the code, but have a test case reproducing the > > bug. I've encountered the issue over a week ago and tried narrowing it down > > to a simple test I could share but without much success. > > > > The code itself is very common: initialize two structs on stack, call a > > function with pointers to those stucts as arguments. A number of inlined > > assertion functions. gcc fails to correctly optimize struct assignments > > with -fno-omit-frame-pointer, I have a number of small structs assigned, > > gcc decides not to use data coping but to assign fields directly. I've > > tried disabling sra, tweaking sra parameters -- no luck in forcing it > > to copy data. Replacing one particular assignment with memcpy produces > > correct code, but that's not a solution. > > How small are the structs? gcc has an optimization for structs that > are no larger than a register, but it's buggy in 4.2 and we disabled > it at $WORK. I can dig up the patch if this is the problem. struct sockaddr_in in this particular test. 16 bytes. Register size structs are rather common, e.g. struct in_addr. I could test the patch. Adding -finline-functions seems to fix the issue for me. Thanks, Gleb. > > Thanks, > matthew ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: gcc 4.2 miscompilation with -O2 -fno-omit-frame-pointer on amd64
On (19/11/2011 09:11), m...@freebsd.org wrote: > On Sat, Nov 19, 2011 at 8:19 AM, Gleb Kurtsou wrote: > > On (19/11/2011 07:26), m...@freebsd.org wrote: > >> On Sat, Nov 19, 2011 at 2:01 AM, Gleb Kurtsou > >> wrote: > >> > Hi, > >> > > >> > I was lucky to write a bit of code which gcc 4.2 fails to compile > >> > correctly with -O2. Too keep long story short the code fails for gcc > >> > from base system and last gcc 4.2 snapshot from ports. It works with gcc > >> > 4.3, gcc 4.4 on FreeBSD and Linux. Clang from base is also good. -O and > >> > -Os optimization levels are fine (I've tried with all -f* flags > >> > mentioned in documentation) > >> > > >> > -O2 -fno-omit-frame-pointer combination is troublesome on amd64. I > >> > presume i386 should be fine. These options are also used for > >> > compilation of kernel (with debugging enabled) and modules. > >> > > >> > I'm not able to share the code, but have a test case reproducing the > >> > bug. I've encountered the issue over a week ago and tried narrowing it > >> > down > >> > to a simple test I could share but without much success. > >> > > >> > The code itself is very common: initialize two structs on stack, call a > >> > function with pointers to those stucts as arguments. A number of inlined > >> > assertion functions. gcc fails to correctly optimize struct assignments > >> > with -fno-omit-frame-pointer, I have a number of small structs assigned, > >> > gcc decides not to use data coping but to assign fields directly. I've > >> > tried disabling sra, tweaking sra parameters -- no luck in forcing it > >> > to copy data. Replacing one particular assignment with memcpy produces > >> > correct code, but that's not a solution. > >> > >> How small are the structs? gcc has an optimization for structs that > >> are no larger than a register, but it's buggy in 4.2 and we disabled > >> it at $WORK. I can dig up the patch if this is the problem. > > struct sockaddr_in in this particular test. 16 bytes. > > > > Register size structs are rather common, e.g. struct in_addr. > > > > I could test the patch. Adding -finline-functions seems to fix the issue > > for me. > > I can't find the thing I'm thinking of. The only potentially relevant > patch I see in our gcc sources is this: It could be related but doesn't fix bug I observe. I've installed fresh 9.0-RC2 virtual machine and reran tests in clean environment. Do you plan committing it? > > > Index: opts.c > === > --- opts.c(.../vendor.branches/freebsd/stable/7/src/contrib/gcc/opts.c) > (revision > 211574) > +++ opts.c(.../head/src/contrib/gcc/opts.c) (revision 211574) > @@ -457,11 +457,7 @@ >flag_tree_dse = 1; >flag_tree_ter = 1; >flag_tree_live_range_split = 1; > + /** > + * 7dot1MERGE: tree-sra in gcc 4.2.x is buggy and > + * breaks bitfield structs. > + */ > + flag_tree_sra = 0; > - flag_tree_sra = 1; >flag_tree_copyrename = 1; >flag_tree_fre = 1; >flag_tree_copy_prop = 1; > > Thanks, > matthew ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: gcc 4.2 miscompilation with -O2 -fno-omit-frame-pointer on amd64
On (20/11/2011 01:57), Alexander Best wrote: > On Sat Nov 19 11, Gleb Kurtsou wrote: > > On (19/11/2011 12:25), Alexander Best wrote: > > > On Sat Nov 19 11, Gleb Kurtsou wrote: > > > > Hi, > > > > > > > > I was lucky to write a bit of code which gcc 4.2 fails to compile > > > > correctly with -O2. Too keep long story short the code fails for gcc > > > > from base system and last gcc 4.2 snapshot from ports. It works with gcc > > > > 4.3, gcc 4.4 on FreeBSD and Linux. Clang from base is also good. -O and > > > > -Os optimization levels are fine (I've tried with all -f* flags > > > > mentioned in documentation) > > > > > > > > -O2 -fno-omit-frame-pointer combination is troublesome on amd64. I > > > > presume i386 should be fine. These options are also used for > > > > compilation of kernel (with debugging enabled) and modules. > > > > > > > > I'm not able to share the code, but have a test case reproducing the > > > > bug. I've encountered the issue over a week ago and tried narrowing it > > > > down > > > > to a simple test I could share but without much success. > > > > > > > > The code itself is very common: initialize two structs on stack, call a > > > > function with pointers to those stucts as arguments. A number of inlined > > > > assertion functions. gcc fails to correctly optimize struct assignments > > > > with -fno-omit-frame-pointer, I have a number of small structs assigned, > > > > gcc decides not to use data coping but to assign fields directly. I've > > > > tried disabling sra, tweaking sra parameters -- no luck in forcing it > > > > to copy data. Replacing one particular assignment with memcpy produces > > > > correct code, but that's not a solution. > > > > > > > > -O2 -fno-omit-frame-pointer -fno-inline is buggy > > > > -O2 -fno-omit-frame-pointer -frename-registers is buggy > > > > > > does the issue also come up with '-fno-builtin' in addition to those > > > flags? > > > is '-O2 -fno-omit-frame-pointer' alone also buggy? does the issue also > > > exist > > > when using -O1 and -O3? > > > > -O0 -g -std=gnu99 -pipe -fno-omit-frame-pointer -fno-strict-aliasing -- OK > > > > -Os -g -std=gnu99 -pipe -fno-omit-frame-pointer -fno-strict-aliasing -- OK > > > > -O -g -std=gnu99 -pipe -fno-omit-frame-pointer -fno-strict-aliasing -- BAD > > > > -O1 -g -std=gnu99 -pipe -fno-omit-frame-pointer -fno-strict-aliasing -- BAD > > > > -O2 -g -std=gnu99 -pipe -fno-omit-frame-pointer -- BAD > > > > -O2 -g -std=gnu99 -pipe -fno-omit-frame-pointer -fno-builtin > > -fno-strict-aliasing -- BAD > > > > -O3 -g -std=gnu99 -pipe -fno-omit-frame-pointer -fno-strict-aliasing -- OK!! > > > > -O3 -fno-inline-functions -fno-unswitch-loops -fno-gcse-after-reload -g > > -std=gnu99 -pipe -fno-omit-frame-pointer -fno-strict-aliasing -- BAD > > > > -O3 -fno-inline-functions -g -std=gnu99 -pipe -fno-omit-frame-pointer > > -fno-strict-aliasing -- BAD > > > > -O2 -finline-functions -g -std=gnu99 -pipe -fno-omit-frame-pointer > > -fno-strict-aliasing -- OK > > btw: for any optimisation > -O1, -fno-strict-aliasing isn't needed, since it > gets added automatically (see sys/conf/kern.pre.mk). For kernel builds. I'm building 3rd party software with base system compiler. On the other hand kernel and modules are built with -O2 -fno-omit-frame-pointer.. > > cheers. > alex > > > > > So, it's -finline-functions that makes it work. But I'm afraid it just > > masks the real problem. > > > > The rest of CFLAGS is warnings and includes: > > -D_GNU_SOURCE -Wsystem-headers -Werror -Wall -Wno-format-y2k -W > > -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wreturn-type > > -Wcast-qual -Wno-write-strings -Wswitch -Wshadow -Wunused-parameter > > -Wcast-align -Wno-pointer-sign -Wstrict-aliasing=2 > > -Wno-error=strict-aliasing > > > > > > > > cheers. > > > alex > > > > > > > > > > > I found similar issue with gcc 4.6, but I'm not able to reproduce it > > > > with gcc test case: > > > > https://bugzilla.redhat.com/show_bug.cgi?id=679924 > > > > http://gcc.gnu.org/bugzilla/show_bug.cgi?id=47893 > > > > > > > > I'll be glad to help debugging it and will be hanging on #bsddev during > > > > weekend as glk. > > > > > > > > > > > > Thanks, > > > > Gleb. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: gcc 4.2 miscompilation with -O2 -fno-omit-frame-pointer on amd64
On (02/12/2011 01:56), Stanislav Sedov wrote: > On Sat, 19 Nov 2011 12:01:50 +0200 > Gleb Kurtsou mentioned: > > > Hi, > > > > I was lucky to write a bit of code which gcc 4.2 fails to compile > > correctly with -O2. Too keep long story short the code fails for gcc > > from base system and last gcc 4.2 snapshot from ports. It works with gcc > > 4.3, gcc 4.4 on FreeBSD and Linux. Clang from base is also good. -O and > > -Os optimization levels are fine (I've tried with all -f* flags > > mentioned in documentation) > > > > -O2 -fno-omit-frame-pointer combination is troublesome on amd64. I > > presume i386 should be fine. These options are also used for > > compilation of kernel (with debugging enabled) and modules. > > > > I'm not able to share the code, but have a test case reproducing the > > bug. I've encountered the issue over a week ago and tried narrowing it down > > to a simple test I could share but without much success. > > > > The code itself is very common: initialize two structs on stack, call a > > function with pointers to those stucts as arguments. A number of inlined > > assertion functions. gcc fails to correctly optimize struct assignments > > with -fno-omit-frame-pointer, I have a number of small structs assigned, > > gcc decides not to use data coping but to assign fields directly. I've > > tried disabling sra, tweaking sra parameters -- no luck in forcing it > > to copy data. Replacing one particular assignment with memcpy produces > > correct code, but that's not a solution. > > > > -O2 -fno-omit-frame-pointer -fno-inline is buggy > > -O2 -fno-omit-frame-pointer -frename-registers is buggy > > > > I found similar issue with gcc 4.6, but I'm not able to reproduce it > > with gcc test case: > > https://bugzilla.redhat.com/show_bug.cgi?id=679924 > > http://gcc.gnu.org/bugzilla/show_bug.cgi?id=47893 > > > > I'll be glad to help debugging it and will be hanging on #bsddev during > > weekend as glk. > > > > Hi! > > I'm not sure this is relevant to your case, but our base gcc used to have > a bug with strict aliasing, which was fixed only in a GPLv3 version of > it. That's why we have -fno-strict-aliasing in default CFALGS. So you > might try to build using -fno-strict-aliasing. I always have -fno-strict-aliasing, the whole idea of misusing undefined behaviour to perform optimization is crazy. I guess it seemed evident to me so I've skipped the flag above. Besides gcc was barking with aliasing warnings on 3rd party party code in my case, I had to change warnings flags to run tests without -fno-strict-aliasing. I've dropped -fno-omit-frame-pointer, async unwind tables are ok for userland. Another work around was adding -finline-functions. Kernel and modules won't build with -finline-functions. So we are just lucky not to catch it. Thanks, Gleb. > -- > Stanislav Sedov > ST4096-RIPE > > () ascii ribbon campaign - against html e-mail > /\ www.asciiribbon.org - against proprietary attachments ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: gcc 4.2 miscompilation with -O2 -fno-omit-frame-pointer on amd64
On (08/12/2011 10:01), Piotr Nowak wrote: > We're working on PowerPC target using GCC 4.2.1 > and FreeBSD 6.1. It seems like we have similar > problem. In our case GCC sometimes very unfortunately > optimize code with -fno-omit-frame-pointer. > > Example shown below covers file sys/powerc/booke/pmap.c > and function pmap_kenter. If we disassemble kernel binary > we have: > > c019998c: 4b ec 6a ed bl c0060478 <_mtx_unlock_spin_flags> > c010: 81 61 00 00 lwz r11,0(r1) > c014: 80 0b 00 04 lwz r0,4(r11) > c018: 7d 61 5b 78 mr r1,r11 > c01c: 82 ab ff d4 lwz r21,-44(r11) > c01999a0: 7c 08 03 a6 mtlrr0 > c01999a4: 82 cb ff d8 lwz r22,-40(r11) > c01999a8: 82 eb ff dc lwz r23,-36(r11) > c01999ac: 83 0b ff e0 lwz r24,-32(r11) > c01999b0: 83 2b ff e4 lwz r25,-28(r11) > c01999b4: 83 4b ff e8 lwz r26,-24(r11) > c01999b8: 83 6b ff ec lwz r27,-20(r11) > > As you can see stack pointer on R1 is being updated > before stashed data were pulled off stack. (mr r1,r11) > As a result of this we have chance to get crash when > any interrupt hit shortly after stack pointer update. > The interrupt prologue will override not yet pulled off > pmap_kenter function data. > > The problem occures only with -fno-omit-frame-pointer > and not every branch returns are beeing corrupted. > > Do you think this issue may be somehow related to yours? > Are there any patches/solutions to fix it? Adding -finline-functions fixed/masked issue for me. Unfortunately building kernel with -finline-functions is not supported. You can try tweaking conf/files to change build options for this file only. Issue not sra-related, but sra is also known to be buggy in gcc 4.2. > > Regards, > Piotr ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: gcc 4.2 miscompilation with -O2 -fno-omit-frame-pointer on amd64
On (09/12/2011 16:15), Rafal Jaworowski wrote: > > On 2011-12-08, at 17:53, Nathan Whitehorn wrote: > > > On 12/08/11 03:01, Piotr Nowak wrote: > >> We're working on PowerPC target using GCC 4.2.1 > >> and FreeBSD 6.1. It seems like we have similar > >> problem. In our case GCC sometimes very unfortunately > >> optimize code with -fno-omit-frame-pointer. > >> > >> Example shown below covers file sys/powerc/booke/pmap.c > >> and function pmap_kenter. If we disassemble kernel binary > >> we have: > >> > >> c019998c: 4b ec 6a ed bl c0060478<_mtx_unlock_spin_flags> > >> c010: 81 61 00 00 lwz r11,0(r1) > >> c014: 80 0b 00 04 lwz r0,4(r11) > >> c018: 7d 61 5b 78 mr r1,r11 > >> c01c: 82 ab ff d4 lwz r21,-44(r11) > >> c01999a0: 7c 08 03 a6 mtlrr0 > >> c01999a4: 82 cb ff d8 lwz r22,-40(r11) > >> c01999a8: 82 eb ff dc lwz r23,-36(r11) > >> c01999ac: 83 0b ff e0 lwz r24,-32(r11) > >> c01999b0: 83 2b ff e4 lwz r25,-28(r11) > >> c01999b4: 83 4b ff e8 lwz r26,-24(r11) > >> c01999b8: 83 6b ff ec lwz r27,-20(r11) > >> > >> As you can see stack pointer on R1 is being updated > >> before stashed data were pulled off stack. (mr r1,r11) > >> As a result of this we have chance to get crash when > >> any interrupt hit shortly after stack pointer update. > >> The interrupt prologue will override not yet pulled off > >> pmap_kenter function data. > >> > >> The problem occures only with -fno-omit-frame-pointer > >> and not every branch returns are beeing corrupted. > >> > >> Do you think this issue may be somehow related to yours? > >> Are there any patches/solutions to fix it? > > > > Should we turn off -fno-omit-frame-frame-pointer on PPC then? It's enabled > > in default kernel builds. > > I think that's a good idea. Even though we have managed to trigger > this only in rare cases, the problem is real and the code generated is > broken i.e. leads to corruption and panics. -fno-omit-frame-pointer is there for kernel debugger to be able to generate backtraces. Hacking long abandoned gcc version won't get us far either. IMO we'd better concentrate on external toolchain support and clang. I wasn't able to come up with a small test case for the problem month ago, I'll try again once I have free time. > > Rafal > ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: gcc 4.2 miscompilation with -O2 -fno-omit-frame-pointer on amd64
Please find test case and test results attached. (gcc-test1.shar.txt)
The long story short: only gcc-4.2 is affected, gcc 3.4, 4.4 and 4.6 are
ok. clang is ok. (test-cc.txt)
Nearly all of the workarounds I used in original test doesn't work in
this test case (see #ifdef BAD_FIX in sources).
gcc 4.2 fails even with -O1, it has nothing to do with
-fno-omit-frame-pointer, finline-functions, etc. (test-cflags.txt)
Compile with -DFIX1 to work around problem (replace struct assignment
with memcpy):
% make test XFLAGS="-O2 -DFIX1"
rm -f gcc-test1 src1.o src2.o src3.o
cc -O2 -DFIX1 -g -std=gnu99 -c src1.c
cc -O2 -DFIX1 -g -std=gnu99 -c src2.c
cc -O2 -DFIX1 -g -std=gnu99 -c src3.c
cc -O2 -DFIX1 -g -std=gnu99-o gcc-test1 src1.o src2.o src3.o
/home/gleb/projects/gcc-test1/gcc-test1
ok
Happy hacking!
# This is a shell archive. Save it in a file, remove anything before
# this line, and then unpack it by entering "sh file". Note, it may
# create directories; files and directories will be owned by you and
# have default permissions.
#
# This archive contains:
#
# Makefile
# array.h
# debug.h
# hdr1.h
# src1.c
# src2.c
# src3.c
#
echo x - Makefile
sed 's/^X//' >Makefile << 'b67911656ef5d18c4ae36cb6741b7965'
XPROG= gcc-test1
XSRCS= src1.c src2.c src3.c hdr1.h
XNO_MAN=
X
XXFLAGS?=-O2 -fno-omit-frame-pointer
X
X#WARNS=6
X#NO_WERROR=
X
XSSP_CFLAGS=
XDEBUG_FLAGS= -g
XCSTD= gnu99
X
XCFLAGS= ${XFLAGS}
X
X.PHONY: test
Xtest: clean ${PROG}
X ${.OBJDIR}/${PROG}
X
X.include
b67911656ef5d18c4ae36cb6741b7965
echo x - array.h
sed 's/^X//' >array.h << '3ff8e27b821109f0551768c5e1b6bc0d'
X#include "debug.h"
X
X#define ARRAY_HEAD(name, type, capacity) \
X struct name { \
X size_t arr_count; \
X struct type arr_items[capacity];\
X }
X
X#define ARRAY_INIT(head) do { \
X (head)->arr_count = 0; \
X} while (0)
X
X#define ARRAY_CAPACITY(head) \
X (sizeof((head)->arr_items) / sizeof((head)->arr_items[0]))
X
X#define ARRAY_COUNT(head) (head)->arr_count
X
X#ifdef FIX1
X
X#define ARRAY_INSERT_TAIL(head, elm) do { \
X ASSERT(ARRAY_COUNT((head)) < ARRAY_CAPACITY((head))); \
X memcpy(&(head)->arr_items[(head)->arr_count++], &(elm), sizeof(elm)); \
X} while (0)
X
X#else
X
X#define ARRAY_INSERT_TAIL(head, elm) do { \
X ASSERT(ARRAY_COUNT((head)) < ARRAY_CAPACITY((head))); \
X (head)->arr_items[(head)->arr_count++] = (elm); \
X} while (0)
X
X#endif
X
X#define ARRAY_FOREACH(var, head) \
X for ((var) = &(head)->arr_items[0]; \
X (var) < &(head)->arr_items[0] + (head)->arr_count; \
X (var)++)
3ff8e27b821109f0551768c5e1b6bc0d
echo x - debug.h
sed 's/^X//' >debug.h << '0483bb2079adc3936e07b79b224930aa'
X#ifndef XXX_DEBUG_H
X#define XXX_DEBUG_H
X
X#include
X
X#define ASSERT(cond) \
X (__predict_false(cond) ? (void)0 : \
X debug_assert(#cond, __FILE__, __LINE__, __func__))
X
Xvoid debug_assert(const char *msg, const char *file, int line,
Xconst char *func);
X
X#endif
0483bb2079adc3936e07b79b224930aa
echo x - hdr1.h
sed 's/^X//' >hdr1.h << '023696bd679b9970bb745000978348c8'
X#include
X
X#include "array.h"
X#include "debug.h"
X
Xtypedef union {
X struct {
X boolflag1:1;
X boolflag2:1;
X boolflag3:1;
X boolflag4:1;
X boolflag5:1;
X boolflag6:1;
X boolflag7:1;
X boolflag8:1;
X boolflag9:1;
X boolflag10:1;
X };
X uint32_tbits;
X} xflags_t;
X
Xstruct session {
X xflags_t flags;
X struct sockaddr_in addr;
X};
X
Xstruct dummy1 {
X int a, b, c;
X void *ptr;
X};
X
XARRAY_HEAD(addr_list, sockaddr_in, 2);
X
Xstruct addr_list;
X
Xvoid test_fail(void *arg1, void *arg2,
Xvoid *arg3, const void *arg4,
Xconst struct addr_list *addr_list,
Xconst void *arg5);
X
Xbool test_run(struct session *sess);
023696bd679b9970bb745000978348c8
echo x - src1.c
sed 's/^X//' >src1.c << 'd76eceed15fcbc6c2cf5da795270acb3'
X#include
X#include
X#include
X
X#include
X#include
X#include
X
X#include "hdr1.h"
X
Xstatic void test_set_dummy(struct dummy1 *arg1, void *arg2);
X
Xbool
Xtest_run(struct session *sess)
X{
X struct dummy1 d1 __unused;
X struct addr_list tmp_addr_list;
X struct sockaddr_in tmp_addr;
X bool rv = tru
Re: devd based AUTOMOUNTER
On (18/02/2012 10:48), vermaden wrote:
> Added a check if ntfs-3g is available, if not then mount_ntfs is used instead.
> Added deleting of empty directories at ${MNTPREFIX}.
> Added ${MNTPREFIX} to be set to /mnt or /media according to preference
>
> #! /bin/sh
>
> PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin
> MNTPREFIX="/media"
> LOG="/var/log/automount.log"
> STATE="/var/run/automount.state"
> DATEFMT="%Y-%m-%d %H:%M:%S"
>
> __create_mount_point() { # /* 1=DEV */
> MNT="${MNTPREFIX}/$( basename ${1} )"
> mkdir -p ${MNT}
> }
>
> __state_lock() {
> while [ -f ${STATE}.lock ]; do sleep 0.5; done
> :> ${STATE}.lock
> }
Why not keep it stateless, unmounting by hand would kill integrity.
Usage of `file` is a big security concern.
I think geom config and list of mounted file systems should be
sufficient for collecting information at runtime. Although it may not be
that easy for disks without partitioning.
I'm using a python script for mounting/unmounting removable disks,
but having similar tool written in sh would be great.
https://github.com/glk/mmount
Thanks,
Gleb.
>
> __state_unlock() {
> rm ${STATE}.lock
> }
>
> __state_add() { # /* 1=DEV 2=PROVIDER 3=MNT */
> __state_lock
> grep -E "${3}" ${STATE} 1> /dev/null 2> /dev/null && {
> __log "${1}:duplicated '${STATE}'"
> return 1
> }
> echo "${1} ${2} ${3}" >> ${STATE}
> __state_unlock
> }
>
> __state_remove() { # /* 1=MNT 2=STATE 3=LINE */
> BSMNT=$( echo ${1} | sed 's/\//\\\//g' )
> sed -i '' "/${BSMNT}\$/d" ${2}
> }
>
> __log() { # /* @=MESSAGE */
> echo $( date +"${DATEFMT}" ) ${@} >> ${LOG}
> }
>
> case ${2} in
> (attach)
> for I in /dev/${1}*
> do
> case $( file -L -s ${I} | sed -E 's/label:\ \".*\"//g' ) in
> (*NTFS*)
> dd < ${I} count=1 2> /dev/null \
> | strings \
> | head -1 \
> | grep -q "NTFS" && {
> __create_mount_point ${I}
> which ntfs-3g 1> /dev/null 2> /dev/null && {
> ntfs-3g ${I} ${MNT} # /* sysutils/fusefs-ntfs */
> } || {
> mount_ntfs ${I} ${MNT}
> }
> __log "${I}:mount (ntfs)"
> }
> ;;
> (*FAT*)
> dd < ${I} count=1 2> /dev/null \
> | strings \
> | grep -q "FAT32" && {
> __create_mount_point ${I}
> fsck_msdosfs -y ${I}
> mount_msdosfs -o large -l -L pl_PL.ISO8859-2 -D cp852 ${I}
> ${MNT}
> __log "${I}:mount (fat)"
> }
> ;;
> (*ext2*)
> __create_mount_point ${I}
> fsck.ext2 -y ${I}
> mount -t ext2fs ${I} ${MNT}
> __log "${I}:mount (ext2)"
> ;;
> (*ext3*)
> __create_mount_point ${I}
> fsck.ext3 -y ${I}
> mount -t ext2fs ${I} ${MNT}
> __log "${I}:mount (ext3)"
> ;;
> (*ext4*)
> __create_mount_point ${I}
> fsck.ext4 -y ${I}
> ext4fuse ${I} ${MNT} # /* sysutils/fusefs-ext4fuse */
> __log "${I}:mount (ext4)"
> ;;
> (*Unix\ Fast\ File*)
> __create_mount_point ${I}
> fsck_ufs -y ${I}
> mount ${I} ${MNT}
> __log "${I}:mount (ufs)"
> ;;
> (*)
> case $( dd < ${I} count=1 2> /dev/null | strings | head -1 ) in
> (EXFAT)
> __create_mount_point ${I}
> mount.exfat ${I} ${MNT} # /* sysutils/fusefs-exfat */
> __log "${I}:mount (ufs)"
> ;;
> (*) continue ;;
> esac
> ;;
> esac
> __state_add ${I} $( mount | grep -m 1 " ${MNT} " | awk '{printf $1}' ) \
> ${MNT} || continue
> done
> ;;
>
> (detach)
> MOUNT=$( mount )
> __state_lock
> grep ${1} ${STATE} \
> | while read DEV PROVIDER MNT
> do
> TARGET=$( echo "${MOUNT}" | grep -E "^${PROVIDER} " | awk '{print
> $3}' )
> [ -z ${TARGET} ] && {
> __state_remove ${MNT} ${STATE} ${LINE}
> continue
> }
> umount -f ${TARGET} &
> unset TARGET
> __state_remove ${MNT} ${STATE} ${LINE}
> __log "${DEV}:umount"
> done
> __state_unlock
> __log "/dev/${1}:detach"
> find ${MNTPREFIX} -type d -empty -delete
> ;;
>
> esac
>
>
>
> > Not sure if you've looked at disktype in sysutils
> > but it may be useful to you.
>
> I will get look into that, thanks ;)
>
>
>
> > Neat scripts!
> > Matt
>
> Thanks mate.
>
>
>
> Regards,
> vermaden
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: small change to du, so it will accepts unit suffixes for negative thresholds
On (01/03/2012 23:38), Alexander Best wrote:
> hi there,
>
> i just noticed that du will not accepts something like the following:
>
> du -t-500M
>
> whereas
>
> du -t500M
>
> will work. i've attached a patch, which makes unit suffixes in connection with
> negative thresholds possible.
Good catch, thanks. A few minor comments below.
>
> cheers.
> alex
> diff --git a/usr.bin/du/du.1 b/usr.bin/du/du.1
> index 3db1367..01d2ec1 100644
> --- a/usr.bin/du/du.1
> +++ b/usr.bin/du/du.1
> @@ -137,6 +137,10 @@ If
> is negative, display only entries for which size is less than the absolute
> value of
> .Ar threshold .
> +For both positive and negative values,
> +.Ar threshold
> +accepts unit suffixes
> +.Po see Fl h Li option Pc .
> .It Fl x
> File system mount points are not traversed.
> .El
> diff --git a/usr.bin/du/du.c b/usr.bin/du/du.c
> index 7b47b71..51bfd07 100644
> --- a/usr.bin/du/du.c
> +++ b/usr.bin/du/du.c
> @@ -175,13 +175,18 @@ main(int argc, char *argv[])
> break;
> case 'r':/* Compatibility. */
> break;
> - case 't' :
> + case 't':
> + if (strncmp(optarg, "-", 1) == 0) {
Why not optarg[0] == '-', it makes intent more clear.
I think we should support "+500M" as well.
Perhaps we'd better use temporal variable for string value instead of
changing optarg.
Or initialize threshold_sign with 0, set it either to 1 or -1 if optarg
starts with '+' or '-' accordingly, and do
expand_number(optarg + (threshold_sign != 0 ? 1 : 0), &threshold)
> + optarg++;
> + threshold_sign = -1;
> + }
> if (expand_number(optarg, &threshold) != 0 ||
> threshold == 0) {
> warnx("invalid threshold: %s", optarg);
optarg can differ from original value because of optarg++ above.
> usage();
> - } else if (threshold < 0)
> - threshold_sign = -1;
> + }
> + if (threshold_sign == -1)
> + threshold = -threshold;
> break;
> case 'x':
> ftsoptions |= FTS_XDEV;
> ___
> freebsd-hackers@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
> To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: NTFS GSoC Project Idea
On (26/03/2012 21:13), Efstratios Karatzas wrote: > Greetings, > > I am a FreeBSD GSoC 2010 student, looking to participate in this years > GSoC. The project that I wish to work on is the FreeBSD NTFS driver. > > I 've already discussed my project idea with Attilio@ who suggested that I > forward my proposal to the hackers mailing list in order to get more > feedback and see if there's anyone interested in mentoring a NTFS project. > > The current draft of the technical part of my proposal(pdf & plain text) > can be found here: > > http://cgi.di.uoa.gr/~std06101/ntfs/ntfs_proposal.tar > > The project idea focuses on mp-safing the NTFS driver as well as adding > extra write support. I've tried to merge the two conflicting NTFS project > ideas in the ideas wiki page, into one. One of them suggesting that work > should be done on the current driver (mp-safe it etc) and the other one > suggesting that we port Apple's NTFS driver from scratch. The concept is > that we keep most of our vnode/vfs code (i.e. ntfs_vfsops.c, ntfs_vnops.c) > and rework existing infrastructure as needed as well as implement new > features using Apple's driver as a guide. I'm not sure I follow your idea, but I'd suggest sticking to a single project: either improve FreeBSD NTFS or do the port. FreeBSD and Darwin ntfs implementations are completely unrelated thus porting features from Darwin won't be easy. > This way, we avoid the major > changes in Apple's VFS (is there any documentation to be found?) and port > code that won't break current functionality. I bet "major changes in Apple's VFS" are easier to deal with than "merging" two unrelated file systems. XNU VFS is based on FreeBSD 5 VFS and they still share a lot in common, e.g. vnode operations themselves are nearly the same, e.g. extended attributes, locking, buffer cache are different. Take a look at HFS+ port. It's unmaintained and outdated but page contains link to CVS repository snapshot. http://people.freebsd.org/~yar/hfs/ > I tried to keep this e-mail brief so If you wish to know more, please refer > to the proposal. > > Thank you > > -- > > Efstratios "GPF" Karatzas ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: NTFS GSoC Project Idea
On (28/03/2012 18:43), Pedro Giffuni wrote: > Hello Efstratios ; > > In general, I agree with Gleb that you should start from > the Apple Darwin port instead of spending time on the > current FreeBSD driver. I'd suggest submitting two proposals -- too much depends on a mentor availability for a particular project. > Please note that last year someone attempted to bring > in smbfs from Darwin with your same strategy and > failed: > http://lists.freebsd.org/pipermail/soc-status/2011-June/000340.html > > Making it MPsafe may look relatively easy but it will > take valuable time. In the case of ext2fs, for example, > I am convinced it was only done in time because the > ext2fs code is based on UFS1. > > Also the lack of write support has made the current > NTFS driver undesirable and for a while, even before > the MP-unsafe axing was defined, the driver was > being considered for deprecation. > > Quite honestly I think we want the Darwin driver. If it > serves as further encouragement, when I asked Yar > about his HFS port he said everything he took from > Darwin "basically" compiled. My experience with Darwin NTFS was similar. I had most of it compiling in several evenings. At least extended attributes and utf-8 routines had to be disabled. Sorry, I can't recall the details, had no time to get back to it afterwards. Thanks, Gleb. > > cheers, > > Pedro. > ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: [Hash function Ipv4]
On (02/06/2012 20:14), enrico d'urso wrote: > > Hi, > I'm looking for an Hash function for Ipv4 addresses. > > What are good ones? Have you tried good general purpose hash functions like murmur3 or cityhash? Another option is to use "hash" function that is bijection on integers and exploit this fact in data structure, e.g. by using hash array mapped trie or another prefix tree. The easiest way to build such function is Feistel network on top of general purpose hash function as round function. Li and Ri will be most and less significant 16 bits of ipv4 address accordingly. At least 3 Fiestel rounds required. Play with function to achieve better performance/distribution. https://en.wikipedia.org/wiki/Feistel_cipher Reduced round and block size RC5 also looks very attractive, but it's patented :( Thanks, Gleb. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: boot menu option to disable graphics mode
On (07/06/2012 11:56), Andriy Gapon wrote: > on 07/06/2012 11:47 Konstantin Belousov said the following: > > On Thu, Jun 07, 2012 at 10:45:26AM +0300, Andriy Gapon wrote: > >> > >> It's long been a wish of mine to have an ability to decide at boot time > >> that a > >> system should boot in "console-only" mode. That is, that no graphics/X > >> applications like e.g. xdm/kdm/gdm are automatically started even when > >> they are > >> configured to do so. > >> > >> Here is my attempt at implementing that: > >> https://gitorious.org/~avg/freebsd/avgbsd/commit/96f7051d63d4286ef6f0196d241e7855338a6ed7?format=patch > >> > >> All the option does at boot time is setting of 'inhibit_gui' variable for > >> kernel > >> environment. I envision that this variable could be properly and > >> gracefully > >> handled in various startup scripts and/or application startup logic. > >> But to ensure that the option is always honored I've also added "ultimate > >> protection" to syscons that prohibits KDSETMODE/KD_GRAPHICS ioctl. > > This is too much, IMO. I understand why you may want to disable > > auto-start of login manager, but preventing a user from running X at all > > until she learns about kenv -u _and_ obscure code somewhere in the kernel, > > is unreasonable. > > A user doesn't have to select the option unless he needs to. > A "simple user" can just reboot without selecting the option to get back his > X. > A user doesn't have to learn anything about the code, just about kenv and > "magic" inhibit_gui variable. What do you think about adding generic support for overriding *_enable options in rc.conf? I'd like to be able to disable services at boot prompt, e.g. # set rc.slim_enable="no" -- overrides slim_enable="yes" in rc.conf Similarly rc.pf_enable="no" Then introduce x_enable knob (=yes by default) to disable login managers. User will be able to override this setting with # service xdm forcestart Thanks, Gleb. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
[patch] savecore can't create dump from encrypted swap
Issue was mentioned recently on stable@ http://docs.freebsd.org/cgi/mid.cgi?47F548D9.8060905 The problem is in order of rc.d scripts. After encswap finishes it's impossible to open original (not encrypted) swap device for write so savecore fails. The following patch adds 'encswap' target to be run after 'disks' but before 'swap1'. With the patch order of scripts becomes: dumpon geli savecore encswap swap1 instead of: dumpon geli encswap swap1 savecore The problem is that savecore looses ability to use syslog. You still get messages on console. The only solution I see is to use some hackery magic to inject log messages into kern.msgbuf with the help of /dev/console. I'm not sure it's really needed. diff -ur rc.d.orig/encswap rc.d/encswap --- rc.d.orig/encswap 2008-04-27 14:53:38.0 +0300 +++ etc/rc.d/encswap2008-04-27 14:53:27.0 +0300 @@ -4,7 +4,7 @@ # -# PROVIDE: disks -# REQUIRE: initrandom +# PROVIDE: encswap +# REQUIRE: initrandom disks # KEYWORD: nojail . /etc/rc.subr diff -ur rc.d.orig/savecore rc.d/savecore --- rc.d.orig/savecore 2008-04-27 14:53:38.0 +0300 +++ etc/rc.d/savecore 2008-04-27 14:53:27.0 +0300 @@ -4,8 +4,7 @@ # # PROVIDE: savecore -# REQUIRE: syslogd -# BEFORE: SERVERS +# BEFORE: encswap # KEYWORD: nojail . /etc/rc.subr diff -ur rc.d.orig/swap1 rc.d/swap1 --- rc.d.orig/swap1 2008-04-27 14:53:38.0 +0300 +++ etc/rc.d/swap1 2008-04-27 14:53:27.0 +0300 @@ -4,7 +4,7 @@ # # PROVIDE: localswap -# REQUIRE: disks +# REQUIRE: encswap disks # KEYWORD: nojail shutdown . /etc/rc.subr ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
GSoC: namecache improvements
Hello, My project is about reimplementing namecache. In a few words it's about generalizing UFS-like dirhash cache and exposing it to upper layers so that it can be used for reliable full path lookup. The idea is quite different from existing implementations in DragonflyBSD and Linux, instead of making namecache first class interface for vnode lookup and thus making VFS name centric, I propose that filesystem itself is to keep cache in sync, thus eliminating the need of changing existing filesystems, avoiding problems with network filesystems (especially NFS) and keeping existing VFS design intact. More detailed proposal: http://lists.freebsd.org/pipermail/freebsd-arch/2010-April/010083.html I'm going to update my blog on the progress during the summer: http://glebkurtsou.blogspot.com/ Thanks, Gleb. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: USB key && kernel: da0: Attempt to query device size failed: UNIT ?ATTENTION, Medium not present
On (06/08/2008 19:29), Oliver Fromme wrote:
> Matthias Apitz wrote:
> > I've updated usb/80361, see
> > http://www.freebsd.org/cgi/query-pr.cgi?pr=80361
> > because I have the same problem as well that an USB key attaches fine
> > when plugged in at boot time, but not later:
Situation here is somewhat opposite. Device doesn't attach at boot or
attaches some times. And needs special patch to attach later.
% usbdevs -v
port 1 addr 2: high speed, power 300 mA, config 1, USB DRIVE(0x0111),
0(0x04e8), rev 2.00
>
> I'm just wondering what happens if you enforce a rescan
> on the (virtual) SCSI bus. That is, after you have
> plugged in the USB stick and the problem occured, type
> "camcontrol rescan 0".
>
> If that doesn't help, please try this patch:
>
> ===
> --- src/sys/dev/usb/umass.c.orig 2008-05-21 16:22:03.0 +0200
> +++ src/sys/dev/usb/umass.c 2008-08-06 19:23:01.0 +0200
> @@ -2690,7 +2690,7 @@
>* completed, when interrupts have been enabled.
>*/
>
> - callout_reset(&sc->cam_scsi_rescan_ch, MS_TO_TICKS(200),
> + callout_reset(&sc->cam_scsi_rescan_ch, MS_TO_TICKS(2000),
> umass_cam_rescan, sc);
> }
With this patch it gives following error:
Aug 6 23:33:44 h1 kernel: umass0: <0 USB DRIVE, class 0/0, rev 2.00/2.00, addr
2> on uhub2
Aug 6 23:33:46 h1 kernel: da0 at umass-sim0 bus 0 target 0 lun 0
Aug 6 23:33:46 h1 kernel: da0: < USB DRIVE 2.00> Removable Direct Access
SCSI-2 device
Aug 6 23:33:46 h1 kernel: da0: 40.000MB/s transfers
Aug 6 23:33:46 h1 kernel: da0: Attempt to query device size failed: UNIT
ATTENTION, Not ready to ready change,
I've tried to increase delay up to 4000, nothing changed.
Without patch error message is different. Can reproduce it if somebody
is interested in it.
But what surprises is that with this patch device attaches during boot.
> Note that this patch is not a solution. It's purpose is
> to find out if the cause of your problem is the same as
> the one in PR usb/80361. If it is, the patch from the PR
> should be committed (it introduces a quirk for cases like
> this), and your USB stick should be added to the quirks
> list.
I've been using another homemade patch for >2 years. Hope it can help to
find a real solution. I have no idea what it does, I'm not sure a had
one during writing it back then. Anyway key attaches and just works.
diff -r 24788dc12d11 -r 519a067f2475 sys/dev/usb/umass.c
--- a/sys/dev/usb/umass.c Sat Jun 03 13:05:16 2006 +0300
+++ b/sys/dev/usb/umass.c Sat Jun 03 13:08:35 2006 +0300
@@ -2514,7 +2514,7 @@ umass_cam_action(struct cam_sim *sim, un
sense->extra_len = 10;
ccb->csio.scsi_status = SCSI_STATUS_CHECK_COND;
ccb->ccb_h.status = CAM_SCSI_STATUS_ERROR |
- CAM_AUTOSNS_VALID;
+ /* CAM_AUTOSNS_VALID */ 0;
xpt_done(ccb);
return;
}
@@ -2805,7 +2805,7 @@ umass_cam_sense_cb(struct umass_softc *s
*/
ccb->ccb_h.status = CAM_SCSI_STATUS_ERROR
- | CAM_AUTOSNS_VALID;
+ | /* CAM_AUTOSNS_VALID */ 0;
csio->scsi_status = SCSI_STATUS_CHECK_COND;
#if 0
@@ -2836,7 +2836,7 @@ umass_cam_sense_cb(struct umass_softc *s
break;
} else {
ccb->ccb_h.status = CAM_SCSI_STATUS_ERROR
- | CAM_AUTOSNS_VALID;
+ | /* CAM_AUTOSNS_VALID */ 0;
csio->scsi_status = SCSI_STATUS_CHECK_COND;
}
xpt_done(ccb);
@@ -2872,7 +2872,7 @@ umass_cam_quirk_cb(struct umass_softc *s
ccb->ccb_h.status = CAM_REQ_CMP;
#endif
ccb->ccb_h.status = CAM_SCSI_STATUS_ERROR
- | CAM_AUTOSNS_VALID;
+ | /* CAM_AUTOSNS_VALID */ 0;
ccb->csio.scsi_status = SCSI_STATUS_CHECK_COND;
xpt_done(ccb);
}
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: FreeBSD 7.2 + NFS + nullfs + unlink + fstat = Stale NFS File Handle
On (28/10/2009 09:48), Linda Messerschmidt wrote:
> On Wed, Oct 28, 2009 at 8:48 AM, Andrey Simonenko
> wrote:
> > As I understand when a file is opened in NULLFS its vnode gets new
> > reference on 'count of users', but this new reference is not propagated
> > to the lower vnode (vnode that is under NULLFS). When a file is removed
> > NULLFS passes this op to the lower FS (NFS in this example) and that
> > FS sees that its vnode has only a single reference on 'count of users'.
> >
> > In case of NFS when there is a request to remove a vnode it checks that
> > value of 'count of users' for this vnode. If this count is equal to 1,
> > then NFS client code does 'RPC remove'. If this count is greater than 1
> > (for example when a file is opened), then NFS client code renames pathname
> > to .nfs-file, but does not send 'RPC remove' to the NFS server.
>
> That sounds like a pretty reasonable explanation of what's going on.
>
> Unfortunately it does sound like this would be tough to fix. Since
> NFS deletes are a special case, short of making an NFS-aware nullfs,
> which seems silly, it sounds like the "solution" would be rewriting
> nullfs to propagate the reference count. I don't know enough about
> nullfs to know exactly how hard that would be, but I'm guessing it's
> not the work of a lazy afternoon. :-)
I think that's not about nullfs. Nullfs behaves correctly. It grabs
reference for a vnode and even tries too release it as soon as possible.
NFS logic is wrong here, imho. vrefcnt(vp) == 1 supposed to mean that
vnode is going to be reclaimed on next reference release and nothing
more. And it doesn't mean that reference is going to be released any
time soon. Although network filesystems in the tree abuse it the same
way NFS does.
Probably what NFS tries to do is to check if file descriptor is opened
for a vnode. This assumption holds only for a single code path out of
many: syscall by user. But there is plenty of code invoking vnode
operations without even allocating file descriptor.
Propagating per file descriptor reference counting into filesystem
itself is a layer violation and should be avoided in my opinion. There
should be a way to fix NFS by replacing vrefcnt check.
> >> if (!fd) {
> > ^ should be (fd < 0)
>
> Oops, you are right!
>
> Thanks very much!
> ___
> freebsd-hackers@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
> To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: dumpon to an encrypted swap partition?
On (31/10/2009 19:59), remodeler wrote: > I am running 8.0 RC1 on a multi-user server with a few dozen vnet-enabled > jails and netgraph. The swap partition is encrypted by its /etc/fstab entry, > like: > > /dev/ad2s1b.eli noneswapsw 0 0 > > I am getting sporadic kernel panics on reboot, during the GEOM_JOURNAL > shutdown sequence. However, they occur after geli detaches the swap partition, > so I get an error like: > > Cannot dump. Device not defined or unavailable. As far as I remember you should configure dump device to be raw swap partition. Like /dev/ad2s1b in your case, and you can continue using it for encrypted swap. I suppose you are using one time passwords for swap partitions, so dump can't be restored after reboot anyway. But there are issues with saving dump from encrypted swap after reboot. See http://www.freebsd.org/cgi/query-pr.cgi?pr=conf/124747 It's about dependencies during startup and the patch from PR is not entirely correct/complete. > I know I can set dumpdev in /etc/rc.conf to a file rather than a swap > partition, but is there a way to (1) have an encrypted swap partition, and (2) > dump a core to a swap partition without failure? If I set up a second > unencrypted swap, I can't let the system write potentially confidential > information into that space. No, using file as dumpdev is impossible, not all device drivers support crash dumps (because after kernel panic all interrupts are masked, acquiring mutex always succeeds, driver should be able to operate in poling mode, etc). I've never tried, but it seems dumping to umass devices should be supported now, if you are concerned with security. Otherwise solution would be to create special unencrypted partition for dumps. > Also, at the end of the panic, I get the message: > > Automatic reboot in 15 seconds - press a key on the console to abort > > but then the server hangs and requires manual power-down and reboot. I thought > a reboot was inevitable after a kernel panic - that nothing could prevent it > in terms of misbehaving processes, etc. Any idea what could cause such a > freeze? > > Thank you. > ___ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org" ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: PUFFS SoC project?
On (21/11/2009 21:40), Ivan Voras wrote: > What is the status of this year's SoC projects? Specifically, does > anyone know what happened to PUFFS? > (http://wiki.freebsd.org/SOC2009TatsianaSeveryna) ? As far as I know it is in a pretty good shape. Tatsiana is busy with real job, and has "passed" maintainership to me. ntfs-3g, puffs-ssh and some other filesystems work, but are largely untested, some fuse filesystems are broken as their support of inode numbers is to weak (take a look at fuse-sshfs). There are some issues regarding cache. Original NetBSD puffs code tends to mmap data to cache it in vm, puffs port doesn't. Besides mmaped pages can go out of sync. If you are interested in working on it, do not hesitate contacting me, I'm interested in finishing the port. Thanks, Gleb. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: [PATCH] Support for thresholds in du(1)
On (25/02/2009 17:24), Mel wrote:
> Hi,
>
> attached is a small patch to add threshold support to du(1). I've been using
> it on 7-STABLE machines for a while, cause I got tired of the noise I get
> when sorting and then reformatting to human-readable. Especially since
> sorting isn't part of the equasion "I'd like to see all dirs exceeding a
> given size".
> I've not updated the manpage on -STABLE yet, should be the same as HEAD.
>
> Example usage:
> # du -xht 20m .
> 29M./contrib/binutils
> 52M./contrib/gcc
> 237M./contrib
> 35M./crypto
> 28M./lib
> 20M./share
> 55M./sys/dev
> 139M./sys
> 545M.
>
> I'll file a PR for it, if there's no objections to this feature /
> implementation, the style(9) or the usage of -t.
> --
> Mel
Hi,
I've cleaned up original patch:
* fixed style and some bugs
* as suggested changed it to use expand_number
* implemented support for negative threshold values
I find the patch very useful. Does it look ok to be commited, or should
I file a PR so it won't get lost once again.
Example usage:
src/sys/crypto % du -ht 100k
137K./des
482K.
src/sys/crypto % du -A -ht +100k
129K./des
446K.
src/sys/crypto % du -ht -100k
56K./camellia
11K./salsa20
38K./sha2
9.0K./rc4
68K./des/arch/i386
70K./des/arch
88K./rijndael
37K./via
8.0K./hmac
36K./blowfish/arch/i386
37K./blowfish/arch
85K./blowfish
diff --git a/usr.bin/du/du.1 b/usr.bin/du/du.1
index af2ff84..b0e1748 100644
--- a/usr.bin/du/du.1
+++ b/usr.bin/du/du.1
@@ -42,7 +42,7 @@
.Nm
.Op Fl A
.Op Fl H | L | P
-.Op Fl a | s | d Ar depth
+.Op Fl a | s | d Ar depth | t Ar threshold
.Op Fl c
.Op Fl l
.Op Fl h | k | m | B Ar blocksize
@@ -107,6 +107,14 @@ This option exists solely for conformance with
Display an entry for each specified file.
(Equivalent to
.Fl d Li 0 )
+.It Fl t Ar threshold
+Display only entries for which size exceeds
+.Ar threshold .
+If
+.Ar threshold
+is negative, display only entries for which size is less then absolute
+value of
+.Ar threshold .
.It Fl d Ar depth
Display an entry for all files and directories
.Ar depth
diff --git a/usr.bin/du/du.c b/usr.bin/du/du.c
index 63677f5..ff62c5b 100644
--- a/usr.bin/du/du.c
+++ b/usr.bin/du/du.c
@@ -90,6 +90,7 @@ main(int argc, char *argv[])
FTS *fts;
FTSENT *p;
off_t savednumber, curblocks;
+ int64_t threshold, threshold_sign;
int ftsoptions;
int listall;
int depth;
@@ -106,12 +107,14 @@ main(int argc, char *argv[])
save = argv;
ftsoptions = 0;
savednumber = 0;
+ threshold = 0;
+ threshold_sign = 1;
cblocksize = DEV_BSIZE;
blocksize = 0;
depth = INT_MAX;
SLIST_INIT(&ignores);
- while ((ch = getopt(argc, argv, "AB:HI:LPasd:chklmnrx")) != -1)
+ while ((ch = getopt(argc, argv, "AB:HI:LPasd:chklmnrt:x")) != -1)
switch (ch) {
case 'A':
Aflag = 1;
@@ -179,6 +182,14 @@ main(int argc, char *argv[])
break;
case 'r':/* Compatibility. */
break;
+ case 't' :
+ if (expand_number(optarg, &threshold) != 0 ||
+ threshold == 0) {
+ warnx("invalid threshold: %s", optarg);
+ usage();
+ } else if (threshold < 0)
+ threshold_sign = -1;
+ break;
case 'x':
ftsoptions |= FTS_XDEV;
break;
@@ -248,6 +259,10 @@ main(int argc, char *argv[])
blocksize /= DEV_BSIZE;
}
+ if (threshold != 0)
+ threshold = howmany(threshold / DEV_BSIZE * cblocksize,
+ blocksize);
+
rval = 0;
(void)signal(SIGINFO, siginfo);
@@ -271,7 +286,9 @@ main(int argc, char *argv[])
p->fts_parent->fts_bignum += p->fts_bignum +=
curblocks;
- if (p->fts_level <= depth) {
+ if (p->fts_level <= depth && threshold <=
+ threshold_sign * howmany(p->fts_bignum *
+ cblocksize, blocksize)) {
if (hflag) {
prthumanval(p->fts_bignum);
(void)printf("\t%s\n", p->fts_path);
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: [PATCH] Support for thresholds in du(1)
On (04/02/2010 23:24), jhell wrote:
>
> On Thu, 4 Feb 2010 06:55, gleb.kurtsou@ wrote:
> > On (25/02/2009 17:24), Mel wrote:
> >> Hi,
> >>
> >> attached is a small patch to add threshold support to du(1). I've been
> >> using
> >> it on 7-STABLE machines for a while, cause I got tired of the noise I get
> >> when sorting and then reformatting to human-readable. Especially since
> >> sorting isn't part of the equasion "I'd like to see all dirs exceeding a
> >> given size".
> >> I've not updated the manpage on -STABLE yet, should be the same as HEAD.
> >>
> >> Example usage:
> >> # du -xht 20m .
> >> 29M./contrib/binutils
> >> 52M./contrib/gcc
> >> 237M./contrib
> >> 35M./crypto
> >> 28M./lib
> >> 20M./share
> >> 55M./sys/dev
> >> 139M./sys
> >> 545M.
> >>
> >> I'll file a PR for it, if there's no objections to this feature /
> >> implementation, the style(9) or the usage of -t.
> >> --
> >> Mel
> >
> > Hi,
> >
> > I've cleaned up original patch:
> > * fixed style and some bugs
> > * as suggested changed it to use expand_number
> > * implemented support for negative threshold values
> >
> > I find the patch very useful. Does it look ok to be commited, or should
> > I file a PR so it won't get lost once again.
> >
> > Example usage:
> > src/sys/crypto % du -ht 100k
> > 137K./des
> > 482K.
> > src/sys/crypto % du -A -ht +100k
> > 129K./des
> > 446K.
> > src/sys/crypto % du -ht -100k
> > 56K./camellia
> > 11K./salsa20
> > 38K./sha2
> > 9.0K./rc4
> > 68K./des/arch/i386
> > 70K./des/arch
> > 88K./rijndael
> > 37K./via
> > 8.0K./hmac
> > 36K./blowfish/arch/i386
> > 37K./blowfish/arch
> > 85K./blowfish
> >
>
>
> Also not to be picky but I can see this being raised in another email at a
> later time.
>
> For Each
> du -ht +1.5{M,G,T,P}
>
> Would be something nice to compliment this so you do not have to revert
> back to say "1440k" when wanting to say 1.5m and so on for larger sizes.
>
> Also I think to be fully supportive it should have the capability to do at
> the least -N bits & +-N bytes.
Let's keep it simple, these features are are not hard to implement, but
overcomplicate such simple utility as du(1). If one needs more
sophisticated disk usage reporting he's likely to have 3rd party utils
installed already.
> Thanks for the work that you have done on this.
>
> --
>
> jhell
>
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: How to change vnode operations ?
On (22/04/2010 16:02), Lukáš Czerner wrote: > Hi all, > > this may sound a little odd, since I have noticed that there is much > work done to not allow such a thing ($SUBJ). But may be you can help > me and point me to the right direction. > > I am writing a kernel module with somewhat similar functionality > like nullfs has, BUT it has to have some features which nullfs > itself does not provide : > > 1. I need the new layer to completely hide underlaying layer so no > one can bypass it. Is hypothetic 'mount -t mynullfs /a /a' good enough for you? I'm not sure what your goals are but completely finding underlaying filesystem won't be easy because of VFS_GET, getfh and other stuff operating with inode numbers. > 2. Nullfs allows me to to overlay just one directory, but i want to > include another directories and/or exclude subdirectories/files. > 3. Nullfs just redirects vnode operations to lower layer, I need to > catch that operation, do something (for example alter the arguments > somehow etc..), pass the operation (with possibly altered arguments) > to the lower layer, get the result and then return the result. I'd suggest to take a look at pefs: http://github.com/glk/pefs It's cryptographic stacked filesystem for FreeBSD. It changes file names, hides directory entries, modifies data from lower layer (encrypts or decrypts), supports mounting on same directory, etc. > The best way to do that (I think) is to change vnode operations of > particular vnodes to point to functions defined in that module. At > this point, I can catch any operations with the vnode and this is > the base of what i want. > > So my question is. I there any "clean" way to chande vnode > operations ? If not, is there any "not so clean" way ? Anyway I will > appreciate any good idea how to do what I have described. Imho, stacked filesystem is the only right way to do it (see null, unionfs, pefs). > Thanks! > -Lukas. > ___ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org" ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: How to change vnode operations ?
On (23/04/2010 08:10), Lukáš Czerner wrote: > On Thu, 22 Apr 2010, Gleb Kurtsou wrote: > > > Date: Thu, 22 Apr 2010 22:18:49 +0300 > > From: Gleb Kurtsou > > To: Lukáš Czerner > > Cc: freebsd-hackers@freebsd.org > > Subject: Re: How to change vnode operations ? > > > > On (22/04/2010 16:02), Lukáš Czerner wrote: > > > Hi all, > > > > > > this may sound a little odd, since I have noticed that there is much > > > work done to not allow such a thing ($SUBJ). But may be you can help > > > me and point me to the right direction. > > > > > > I am writing a kernel module with somewhat similar functionality > > > like nullfs has, BUT it has to have some features which nullfs > > > itself does not provide : > > > > > > 1. I need the new layer to completely hide underlaying layer so no > > > one can bypass it. > > Is hypothetic 'mount -t mynullfs /a /a' good enough for you? I'm not sure > > what your goals are but completely finding underlaying filesystem won't > > be easy because of VFS_GET, getfh and other stuff operating with inode > > numbers. > > Well, it may be good enough, or not. Thats what I am trying to find > out. Obviously there are problems, as you mentioned, which will not > exist when I change the vop_vector of the vnode, but as I thought > and you mentioned it as well, this is not very clean way. Why don't you like stacked filesystem approach? It's designed to solve the problem you are describing if I get it right. Although creating pefs-like filesystem altering data and names is not so easy within existing framework. > > > 2. Nullfs allows me to to overlay just one directory, but i want to > > > include another directories and/or exclude subdirectories/files. > > > 3. Nullfs just redirects vnode operations to lower layer, I need to > > > catch that operation, do something (for example alter the arguments > > > somehow etc..), pass the operation (with possibly altered arguments) > > > to the lower layer, get the result and then return the result. > > I'd suggest to take a look at pefs: http://github.com/glk/pefs > > It's cryptographic stacked filesystem for FreeBSD. It changes file > > names, hides directory entries, modifies data from lower layer > > (encrypts or decrypts), supports mounting on same directory, etc. > > Thats great, thanks! I will look at it. > > > > > > The best way to do that (I think) is to change vnode operations of > > > particular vnodes to point to functions defined in that module. At > > > this point, I can catch any operations with the vnode and this is > > > the base of what i want. > > > > > > So my question is. I there any "clean" way to chande vnode > > > operations ? If not, is there any "not so clean" way ? Anyway I will > > > appreciate any good idea how to do what I have described. > > Imho, stacked filesystem is the only right way to do it (see null, > > unionfs, pefs). > > OK. Thanks for pointing me to the pefs, it is interesting and looks > like a good start. But I would appreciate more comment on the side > of the whole idea about changing vnode operations from the kernel > module. It is a little hacky, but aside this I do not see any bigger > problems, do you ? Changing vop_vector is too hackish for me. Basically, changing vnode operations is what stacked filesystems are about. Vnode operations are the top of the problem, you would also have to deal with parent lookup, namecache consistency and buffering, which is going to be complicated. I.e. you'd have to partially reimplement part of VFS layer. nullfs and unionfs pass vnode vobject (buffering layer) from lower layer, adding your own vobject to vnode would complicate filesystem significantly. Besides you won't be able to assign 2 vobjects (original and your own) to a single vnode if you decide to change operations vector. Thanks, Gleb. > Thanks. > -Lukas ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: How to change vnode operations ?
On (24/04/2010 12:26), Lukáš Czerner wrote: > On Fri, 23 Apr 2010, Gleb Kurtsou wrote: > > > Date: Fri, 23 Apr 2010 12:22:57 +0300 > > From: Gleb Kurtsou > > To: Lukáš Czerner > > Cc: freebsd-hackers@freebsd.org > > Subject: Re: How to change vnode operations ? > > > > On (23/04/2010 08:10), Lukáš Czerner wrote: > > > On Thu, 22 Apr 2010, Gleb Kurtsou wrote: > > > > > > > Date: Thu, 22 Apr 2010 22:18:49 +0300 > > > > From: Gleb Kurtsou > > > > To: Lukáš Czerner > > > > Cc: freebsd-hackers@freebsd.org > > > > Subject: Re: How to change vnode operations ? > > > > > > > > On (22/04/2010 16:02), Lukáš Czerner wrote: > > > > > Hi all, > > > > > > > > > > this may sound a little odd, since I have noticed that there is much > > > > > work done to not allow such a thing ($SUBJ). But may be you can help > > > > > me and point me to the right direction. > > > > > > > > > > I am writing a kernel module with somewhat similar functionality > > > > > like nullfs has, BUT it has to have some features which nullfs > > > > > itself does not provide : > > > > > > > > > > 1. I need the new layer to completely hide underlaying layer so no > > > > > one can bypass it. > > > > Is hypothetic 'mount -t mynullfs /a /a' good enough for you? I'm not > > > > sure > > > > what your goals are but completely finding underlaying filesystem won't > > > > be easy because of VFS_GET, getfh and other stuff operating with inode > > > > numbers. > > > > > > Well, it may be good enough, or not. Thats what I am trying to find > > > out. Obviously there are problems, as you mentioned, which will not > > > exist when I change the vop_vector of the vnode, but as I thought > > > and you mentioned it as well, this is not very clean way. > > Why don't you like stacked filesystem approach? It's designed to solve > > the problem you are describing if I get it right. Although creating > > pefs-like filesystem altering data and names is not so easy within > > existing framework. > > > > > > > 2. Nullfs allows me to to overlay just one directory, but i want to > > > > > include another directories and/or exclude subdirectories/files. > > > > > 3. Nullfs just redirects vnode operations to lower layer, I need to > > > > > catch that operation, do something (for example alter the > > > > > arguments > > > > > somehow etc..), pass the operation (with possibly altered > > > > > arguments) > > > > > to the lower layer, get the result and then return the result. > > > > I'd suggest to take a look at pefs: http://github.com/glk/pefs > > > > It's cryptographic stacked filesystem for FreeBSD. It changes file > > > > names, hides directory entries, modifies data from lower layer > > > > (encrypts or decrypts), supports mounting on same directory, etc. > > > > > > Thats great, thanks! I will look at it. > > > > > > > > > > > > The best way to do that (I think) is to change vnode operations of > > > > > particular vnodes to point to functions defined in that module. At > > > > > this point, I can catch any operations with the vnode and this is > > > > > the base of what i want. > > > > > > > > > > So my question is. I there any "clean" way to chande vnode > > > > > operations ? If not, is there any "not so clean" way ? Anyway I will > > > > > appreciate any good idea how to do what I have described. > > > > Imho, stacked filesystem is the only right way to do it (see null, > > > > unionfs, pefs). > > > > > > OK. Thanks for pointing me to the pefs, it is interesting and looks > > > like a good start. But I would appreciate more comment on the side > > > of the whole idea about changing vnode operations from the kernel > > > module. It is a little hacky, but aside this I do not see any bigger > > > problems, do you ? > > Changing vop_vector is too hackish for me. Basically, changing vnode > > operations is what stacked filesystems are about. Vnode operations are > > the top of the problem, you would also have to deal with parent lookup, > > namecache consistency and buffering, which
