Re: Communication between kernel and userspace via local socket
* Maxim Ignatenko , 2015 21:18: > I'm currently inventing the wheel^W^W^Wwriting a firewall from scratch and > looking for most convenient way to establish communication between userspace > processes and kernel part. Communication pattern best fits to listening > PF_LOCAL socket opened from kernel and userspace processes connecting to it. What's wrong with a character device? -- Ed Schouten WWW: http://80386.nl/ pgpdZgbXYqVSE.pgp Description: PGP signature
Re: easy way to determine if a stream or fd is seekable
On Tue Nov 15 11, Brandon Gooch wrote:
> On Nov 15, 2011 2:25 PM, "Alexander Best" wrote:
> >
> > hi there,
> >
> > one of the things i'm missing is an easy way to determine, whether a
> stream or
> > fd is seekable. i checked the dd(1) and hd(1) sources and those tools are
> > performing so much stuff just to find out if this is the case, and they
> still
> > are doing a very poor job.
> >
> > dd(1) e.g. identifies /dev/zero as non-seekable, even though it is. the
> result:
> >
> > `dd if=/dev/zero bs=1m count=1 skip=10`:
> >
> > on freebsd:
> > 57,41 real 0,05 user43,21 sys
> >
> > on openbsd:
> > 0,88 real 0,00 user 0,07 sys
> >
> > on freebsd dd(1) is very easy fixable (1 line diff). the point however is:
> >
> > there doesn't seem to exist a unified way of checking seekable == TRUE. so
> > every userspace application seems to do it differently and most of them
> (dd(1)
> > and hd(1) e.g) aren't doing it right. hd(1) e.g. believes that only
> regular
> > files are seekable. this means that hd(1) fired at /dev/ada* with a high
> skip
> > value takes ages! dd(1) is a bit smarter in this case, but still not
> correct.
> >
> > idealy there would be something like stat.st_mode (see stat(2)) where one
> > could simply do a S_ISSEEK(m). so far the best and easiest solution i've
> seen
> > is:
> >
> > fd = open(argv[1], O_RDONLY);
> > if (fd == -1)
> > exit(1);
> > if (lseek(fd, 0, SEEK_CUR) != -1)
> >printf ("%d is seekable.\n", fd);
> > else
> >printf ("%d is not seekable.\n", fd);
> >
> > seeing an application iterate through a stream or fd via getchar(),
> although
> > a simple seek() would work is very frustrating. i think what would be
> needed
> > is a simple function or macro that userspace applications can make use of.
> > maybe such a thing already exists in linux, openbsd, netbsd, dragonflybsd,
> > solaris or plan9?
> >
> > cheers.
> > alex
> >
> > references:
> > [1]
> http://www.mavetju.org/mail/view_thread.php?list=freebsd-hackers&id=3290708&thread=yes
> > [2] http://www.freebsd.org/cgi/query-pr.cgi?pr=152485
> > [3] http://www.freebsd.org/cgi/query-pr.cgi?pr=86485
>
> So, according to APUE 2nd Edition, seek should be supported on anything
> that's not a pipe, FIFO, or socket. In fact, the "test for seekability"
> you've provided above closely resembles the example given in that text.
if this really is the case, things could even be easier in a freebsd specific
manor than the code above:
!S_ISFIFO(m) && !S_ISSOCK(m)
this means it would be trivial to write a new macro S_ISSEEK which would test
stat.st_mode against the according bits.
cheers.
alex
>
> Need to think about this more before commenting further...
>
> -Brandon
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: easy way to determine if a stream or fd is seekable
On Tue, Nov 15, 2011 at 08:24:50PM +, Alexander Best wrote: > one of the things i'm missing is an easy way to determine, whether a stream or > fd is seekable. i checked the dd(1) and hd(1) sources and those tools are > performing so much stuff just to find out if this is the case, and they still > are doing a very poor job. Isn't the primary issue that FreeBSD doesn't properly report errors for lseek(2)? I think you should start from that and not hack around the fallout... Joerg ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: easy way to determine if a stream or fd is seekable
On Wed Nov 16 11, Alexander Best wrote:
> On Tue Nov 15 11, Brandon Gooch wrote:
> > On Nov 15, 2011 2:25 PM, "Alexander Best" wrote:
> > >
> > > hi there,
> > >
> > > one of the things i'm missing is an easy way to determine, whether a
> > stream or
> > > fd is seekable. i checked the dd(1) and hd(1) sources and those tools are
> > > performing so much stuff just to find out if this is the case, and they
> > still
> > > are doing a very poor job.
> > >
> > > dd(1) e.g. identifies /dev/zero as non-seekable, even though it is. the
> > result:
> > >
> > > `dd if=/dev/zero bs=1m count=1 skip=10`:
> > >
> > > on freebsd:
> > > 57,41 real 0,05 user43,21 sys
> > >
> > > on openbsd:
> > > 0,88 real 0,00 user 0,07 sys
> > >
> > > on freebsd dd(1) is very easy fixable (1 line diff). the point however is:
> > >
> > > there doesn't seem to exist a unified way of checking seekable == TRUE. so
> > > every userspace application seems to do it differently and most of them
> > (dd(1)
> > > and hd(1) e.g) aren't doing it right. hd(1) e.g. believes that only
> > regular
> > > files are seekable. this means that hd(1) fired at /dev/ada* with a high
> > skip
> > > value takes ages! dd(1) is a bit smarter in this case, but still not
> > correct.
> > >
> > > idealy there would be something like stat.st_mode (see stat(2)) where one
> > > could simply do a S_ISSEEK(m). so far the best and easiest solution i've
> > seen
> > > is:
> > >
> > > fd = open(argv[1], O_RDONLY);
> > > if (fd == -1)
> > > exit(1);
> > > if (lseek(fd, 0, SEEK_CUR) != -1)
> > >printf ("%d is seekable.\n", fd);
> > > else
> > >printf ("%d is not seekable.\n", fd);
> > >
> > > seeing an application iterate through a stream or fd via getchar(),
> > although
> > > a simple seek() would work is very frustrating. i think what would be
> > needed
> > > is a simple function or macro that userspace applications can make use of.
> > > maybe such a thing already exists in linux, openbsd, netbsd, dragonflybsd,
> > > solaris or plan9?
> > >
> > > cheers.
> > > alex
> > >
> > > references:
> > > [1]
> > http://www.mavetju.org/mail/view_thread.php?list=freebsd-hackers&id=3290708&thread=yes
> > > [2] http://www.freebsd.org/cgi/query-pr.cgi?pr=152485
> > > [3] http://www.freebsd.org/cgi/query-pr.cgi?pr=86485
> >
> > So, according to APUE 2nd Edition, seek should be supported on anything
> > that's not a pipe, FIFO, or socket. In fact, the "test for seekability"
> > you've provided above closely resembles the example given in that text.
>
> if this really is the case, things could even be easier in a freebsd specific
> manor than the code above:
>
> !S_ISFIFO(m) && !S_ISSOCK(m)
>
> this means it would be trivial to write a new macro S_ISSEEK which would test
> stat.st_mode against the according bits.
here's a patch, which also fixes a comment.
cheers.
alex
>
> cheers.
> alex
>
> >
> > Need to think about this more before commenting further...
> >
> > -Brandon
diff --git a/sys/sys/stat.h b/sys/sys/stat.h
index 1b03bd2..ab5ae44 100644
--- a/sys/sys/stat.h
+++ b/sys/sys/stat.h
@@ -238,10 +238,11 @@ struct nstat {
#defineS_ISCHR(m) (((m) & 017) == 002)/* char special
*/
#defineS_ISBLK(m) (((m) & 017) == 006)/* block
special */
#defineS_ISREG(m) (((m) & 017) == 010)/* regular file
*/
-#defineS_ISFIFO(m) (((m) & 017) == 001)/* fifo or
socket */
+#defineS_ISFIFO(m) (((m) & 017) == 001)/* pipe or fifo
*/
#if __POSIX_VISIBLE >= 200112
#defineS_ISLNK(m) (((m) & 017) == 012)/* symbolic
link */
#defineS_ISSOCK(m) (((m) & 017) == 014)/* socket */
+#defineS_ISSEEK(m) (((m) & 015) == 0) /* seekable
file */
#endif
#if __BSD_VISIBLE
#defineS_ISWHT(m) (((m) & 017) == 016)/* whiteout */
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: The zombie has involved into /dev/null
So, if I understand you correctly, you are reporting a bug in which a jailed process is holding (the jailed instance of) /dev/null open and "umount -f" doesn't work on the jailed /dev ? On 14/11/2011 23:52, Slono Slono wrote: > On one of servers where installed cacti in jail there is strange enough > situation. Sometimes processes poller.php haven't time to successful complete > until to beginning of the following session (absence of lock is other problem > - its ok) therefore processes breed yet won't begin them kill. During such > moments appear zombie processes. However, these zombie show that keep devfs > the device. Possibly because are started as > > php /poller.php 2>/dev/null 2>&1 > > Sending of any signals (SIGCHILD too) changes nothing. Strange that with -f > (force) optons through a umount command is impossible to unmount devfs with > which worked as the zombie. > > ps axf shows: > .. > > 99551 ?? DsJ 0:00.12 /usr/local/bin/php > /usr/local/share/cacti/poller.php > 99554 ?? ZJ 0:00.02 > . > > lsof -p 99551 > COMMAND PID USER FD TYPE DEVICE SIZE/OFFNODE NAME > php 99551 root cwd VBAD (revoked) > php 99551 root rtd VDIR 225,10355344423 678909 > /usr/jails/jails/mon > php 99551 root jld VDIR 225,10355344423 678909 > /usr/jails/jails/mon > php 99551 root txt VREG 225,1035534442 3261754 1620922 > /usr/jails/jails-data/mon-data/usr/local/bin/php > php 99551 root txt VREG 225,1035534442 246776 626780 > /usr/jails/jails-data/mon-data/libexec/ld-elf.so.1 > php 99551 root txt VREG 225,103553444233600 626862 > /usr/jails/jails-data/mon-data/lib/libcrypt.so.5 > php 99551 root txt VREG 225,1035534442 377814 1267501 > /usr/jails/jails-data/mon-data/usr/local/lib/libpcre.so.0 > php 99551 root txt VREG 225,1035534442 150656 626861 > /usr/jails/jails-data/mon-data/lib/libm.so.5 > php 99551 root txt VREG 225,1035534442 1495740 649173 > /usr/jails/jails-data/mon-data/usr/local/lib/libxml2.so.5 > php 99551 root txt VREG 225,103553444284848 626828 > /usr/jails/jails-data/mon-data/lib/libz.so.5 > php 99551 root txt VREG 225,1035534442 1074175 649584 > /usr/jails/jails-data/mon-data/usr/local/lib/libiconv.so.3 > php 99551 root txt VREG 225,1035534442 1270640 626857 > /usr/jails/jails-data/mon-data/lib/libc.so.7 > php 99551 root txt VREG 225,103553444274189 636259 > /usr/jails/jails-data/mon-data/usr/local/lib/php/20090626/session.so > php 99551 root txt VREG 225,103553444263195 637380 > /usr/jails/jails-data/mon-data/usr/local/lib/php/20090626/xml.so > php 99551 root txt VREG 225,103553444240650 638507 > /usr/jails/jails-data/mon-data/usr/local/lib/php/20090626/snmp.so > php 99551 root txt VREG 225,1035534442 337128 665903 > /usr/jails/jails-data/mon-data/usr/lib/libssl.so.6 > php 99551 root txt VREG 225,1035534442 730269 8050234 > /usr/jails/jails-data/mon-data/usr/local/lib/libnetsnmp.so.30 > php 99551 root txt VREG 225,103553444235264 626850 > /usr/jails/jails-data/mon-data/lib/libkvm.so.5 > php 99551 root txt VREG 225,103553444219720 626858 > /usr/jails/jails-data/mon-data/lib/libdevstat.so.7 > php 99551 root txt VREG 225,1035534442 1693344 626824 > /usr/jails/jails-data/mon-data/lib/libcrypto.so.6 > php 99551 root txt VREG 225,1035534442 105904 666224 > /usr/jails/jails-data/mon-data/usr/lib/libelf.so.1 > php 99551 root txt VREG 225,103553444261034 635955 > /usr/jails/jails-data/mon-data/usr/local/lib/php/20090626/mysql.so > php 99551 root txt VREG 225,103553444254114 637132 > /usr/jails/jails-data/mon-data/usr/local/lib/php/20090626/sockets.so > php 99551 root0u PIPE 0xfe07514ab5b016384 > ->0xfe07514ab708 > php 99551 root1w VCHR 0,27 0t0 27 > /usr/jails/jails/mon/dev (devfs) (like character special /dev/null) > php 99551 root2w VCHR 0,27 0t0 27 > /usr/jails/jails/mon/dev (devfs) (like character special /dev/null) > php 99551 root3u unix 0xfe074ad832a8 0t0 ->(none) > php 99551 root5u PIPE 0xfe043c62fcb80 > ->0xfe043c62fb60 > > mount -t devfs |grep mon > devfs on /usr/jails/jails/mon/dev (devfs, local, multilabel) > > umount -f /usr/jails/jails/mon/dev > umount: unmount of /usr/jails/jails/mon/dev failed: Device busy > > However apparently devfs is unmount when executed jail stop: > > ls -la /usr/jails/jails/mon/dev > total 5 > drwxr-xr-x 2 root wheel 2 Nov 14 22:36 . > drwxr-xr-x 3 root wheel 3 Nov 14 22:36 .. > > As can be that zombie blocks devfs or that in system there is an information > on active mou
Base compiler and amdfam10 - anybody/anything?
Hi,
Are there any attempts to bring to -CURRENT newer AMD chips support?
Personally, I've just tried to apply the patches from openSUSE's gcc-4.2.1
SRPM. With slight adaptation they've applied and gave rather significant
boost in resulting code speed. At least, testfcpy by Alexander Konovalenko
(http://daemon.safety.sci.kth.se/~kono/testfcpu) gave me ~20% (!) speedup
with -march=amdfam10 compared to our -march=athlon64-sse3 on Phenom II
970.
Unfortunately, the patched compiler with -march=amdfam10 fails in
buildworld ("internal compiler error"'s while compiling clang). The
buildworld was successful with patched compiler and -march=athlon64-sse3
but since this is my main working system... Well, I had to come back to
our unpatched compiler :-(
If anyone is interested, the patches were taken from
gcc42-4.2.1_20070724-17.src.rpm (actually, I applied all the patches
marked as AMD stuff), the resulting patches towards our src/contrib/gcc
and share/mk/bsd.cpu.mk are attached (or I can send them by email), and I
am quite ready to test what comes out of it.
WBR,
Vladimir
gcc-amdfam10.diff.gz
Description: Binary data
bsd.cpu.mk.amdfam10.gz
Description: Binary data
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Communication between kernel and userspace via local socket
On вт, 15 лис 2011 23:17:41 m...@freebsd.org wrote: > On Tue, Nov 15, 2011 at 12:18 PM, Maxim Ignatenko wrote: > > frHi, > > > > I'm currently inventing the wheel^W^W^Wwriting a firewall from scratch > > and looking for most convenient way to establish communication between > > userspace processes and kernel part. Communication pattern best fits to > > listening PF_LOCAL socket opened from kernel and userspace processes > > connecting to it. Clients should be able to send requests and receive > > responses from kernel (to retrieve list of loaded modules, active > > ruleset, add or remove rules, ...) and vice versa: kernel should be able > > to send request to userspace process and receive response (I'm planning > > to add interactive features like in most firewalls for windows(r)). > > > > First part can be implemented via ioctl, but it should be called not only > > by processes with euid == 0, so supplied pointer to receive buffer > > cannot be trusted (is there any mechanism to check memory allocation?) > > and any unprivileged user can instruct kernel to write some trash at > > arbitrary address (for example, VM just rebooted ungracefully when I > > supplied (void*)123 as pointer to destination buffer). > > Were you using copyout(9)? I think FreeBSD's memory isolation between > processes is pretty decent. I would be very surprised if copyout to an > invalid address did something other than return EFAULT. At least the > amd64 implementation of copyout(9) will also explicitly check that the > address is a user address, so that you can't corrupt kernel memory > with a rogue pointer from user-space. > Yep. I've used this https://gitorious.org/acpi_call-freebsd/acpi_call- freebsd/blobs/master/acpi_call.c#line49 for tests. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Communication between kernel and userspace via local socket
On 16 November 2011 10:55, Ed Schouten wrote: > * Maxim Ignatenko , 2015 21:18: >> I'm currently inventing the wheel^W^W^Wwriting a firewall from scratch and >> looking for most convenient way to establish communication between userspace >> processes and kernel part. Communication pattern best fits to listening >> PF_LOCAL socket opened from kernel and userspace processes connecting to it. > > What's wrong with a character device? > With character device I'll need to manually maintain "per-connection" buffers, this will bloat the code. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: The zombie has involved into /dev/null
On Wednesday 16 November 2011 14:35:04 Ivan Voras wrote: > So, if I understand you correctly, you are reporting a bug in which a > jailed process is holding (the jailed instance of) /dev/null open and > "umount -f" doesn't work on the jailed /dev ? Hello, Yes, correct. > > On 14/11/2011 23:52, Slono Slono wrote: > > On one of servers where installed cacti in jail there is strange enough > > situation. Sometimes processes poller.php haven't time to successful > > complete until to beginning of the following session (absence of lock is > > other problem - its ok) therefore processes breed yet won't begin them > > kill. During such moments appear zombie processes. However, these zombie > > show that keep devfs the device. Possibly because are started as > > > > php /poller.php 2>/dev/null 2>&1 > > > > Sending of any signals (SIGCHILD too) changes nothing. Strange that with > > -f (force) optons through a umount command is impossible to unmount > > devfs with which worked as the zombie. > > > > ps axf shows: > > .. > > > > 99551 ?? DsJ 0:00.12 /usr/local/bin/php > > /usr/local/share/cacti/poller.php 99554 ?? ZJ 0:00.02 > > . > > > > lsof -p 99551 > > COMMAND PID USER FD TYPE DEVICE SIZE/OFFNODE NAME > > php 99551 root cwd VBAD > > (revoked) php 99551 root rtd VDIR 225,10355344423 > > 678909 /usr/jails/jails/mon php 99551 root jld VDIR > > 225,10355344423 678909 /usr/jails/jails/mon php 99551 root > > txt VREG 225,1035534442 3261754 1620922 > > /usr/jails/jails-data/mon-data/usr/local/bin/php php 99551 root txt > > VREG 225,1035534442 246776 626780 > > /usr/jails/jails-data/mon-data/libexec/ld-elf.so.1 php 99551 root > > txt VREG 225,103553444233600 626862 > > /usr/jails/jails-data/mon-data/lib/libcrypt.so.5 php 99551 root txt > > VREG 225,1035534442 377814 1267501 > > /usr/jails/jails-data/mon-data/usr/local/lib/libpcre.so.0 php 99551 > > root txt VREG 225,1035534442 150656 626861 > > /usr/jails/jails-data/mon-data/lib/libm.so.5 php 99551 root txt > > VREG 225,1035534442 1495740 649173 > > /usr/jails/jails-data/mon-data/usr/local/lib/libxml2.so.5 php 99551 > > root txt VREG 225,103553444284848 626828 > > /usr/jails/jails-data/mon-data/lib/libz.so.5 php 99551 root txt > > VREG 225,1035534442 1074175 649584 > > /usr/jails/jails-data/mon-data/usr/local/lib/libiconv.so.3 php 99551 > > root txt VREG 225,1035534442 1270640 626857 > > /usr/jails/jails-data/mon-data/lib/libc.so.7 php 99551 root txt > > VREG 225,103553444274189 636259 > > /usr/jails/jails-data/mon-data/usr/local/lib/php/20090626/session.so php > > 99551 root txt VREG 225,103553444263195 637380 > > /usr/jails/jails-data/mon-data/usr/local/lib/php/20090626/xml.so php > > 99551 root txt VREG 225,103553444240650 638507 > > /usr/jails/jails-data/mon-data/usr/local/lib/php/20090626/snmp.so php > > 99551 root txt VREG 225,1035534442 337128 665903 > > /usr/jails/jails-data/mon-data/usr/lib/libssl.so.6 php 99551 root > > txt VREG 225,1035534442 730269 8050234 > > /usr/jails/jails-data/mon-data/usr/local/lib/libnetsnmp.so.30 php > > 99551 root txt VREG 225,103553444235264 626850 > > /usr/jails/jails-data/mon-data/lib/libkvm.so.5 php 99551 root txt > > VREG 225,103553444219720 626858 > > /usr/jails/jails-data/mon-data/lib/libdevstat.so.7 php 99551 root > > txt VREG 225,1035534442 1693344 626824 > > /usr/jails/jails-data/mon-data/lib/libcrypto.so.6 php 99551 root > > txt VREG 225,1035534442 105904 666224 > > /usr/jails/jails-data/mon-data/usr/lib/libelf.so.1 php 99551 root > > txt VREG 225,103553444261034 635955 > > /usr/jails/jails-data/mon-data/usr/local/lib/php/20090626/mysql.so php > > 99551 root txt VREG 225,103553444254114 637132 > > /usr/jails/jails-data/mon-data/usr/local/lib/php/20090626/sockets.so php > > 99551 root0u PIPE 0xfe07514ab5b016384 > > ->0xfe07514ab708 php 99551 root1w VCHR 0,27 > >0t0 27 /usr/jails/jails/mon/dev (devfs) (like character special > > /dev/null) php 99551 root2w VCHR 0,27 0t0 > > 27 /usr/jails/jails/mon/dev (devfs) (like character special /dev/null) > > php 99551 root3u unix 0xfe074ad832a8 0t0 > > ->(none) php 99551 root5u PIPE 0xfe043c62fcb80 > >->0xfe043c62fb60 > > > > mount -t devfs |grep mon > > devfs on /usr/jails/jails/mon/dev (devfs, local, multilabel) > > > > umount -f /usr/jails/jails/mon/dev > > umount: unmount of /usr/jails/jails/mon/dev failed: Device busy > > > > However apparently devfs is unmount when executed jail stop: > > > > ls -la /usr/jail
Re: easy way to determine if a stream or fd is seekable
On Wed Nov 16 11, Alexander Best wrote:
> On Wed Nov 16 11, Alexander Best wrote:
> > On Tue Nov 15 11, Brandon Gooch wrote:
> > > On Nov 15, 2011 2:25 PM, "Alexander Best" wrote:
> > > >
> > > > hi there,
> > > >
> > > > one of the things i'm missing is an easy way to determine, whether a
> > > stream or
> > > > fd is seekable. i checked the dd(1) and hd(1) sources and those tools
> > > > are
> > > > performing so much stuff just to find out if this is the case, and they
> > > still
> > > > are doing a very poor job.
> > > >
> > > > dd(1) e.g. identifies /dev/zero as non-seekable, even though it is. the
> > > result:
> > > >
> > > > `dd if=/dev/zero bs=1m count=1 skip=10`:
> > > >
> > > > on freebsd:
> > > > 57,41 real 0,05 user43,21 sys
> > > >
> > > > on openbsd:
> > > > 0,88 real 0,00 user 0,07 sys
> > > >
> > > > on freebsd dd(1) is very easy fixable (1 line diff). the point however
> > > > is:
> > > >
> > > > there doesn't seem to exist a unified way of checking seekable == TRUE.
> > > > so
> > > > every userspace application seems to do it differently and most of them
> > > (dd(1)
> > > > and hd(1) e.g) aren't doing it right. hd(1) e.g. believes that only
> > > regular
> > > > files are seekable. this means that hd(1) fired at /dev/ada* with a high
> > > skip
> > > > value takes ages! dd(1) is a bit smarter in this case, but still not
> > > correct.
> > > >
> > > > idealy there would be something like stat.st_mode (see stat(2)) where
> > > > one
> > > > could simply do a S_ISSEEK(m). so far the best and easiest solution i've
> > > seen
> > > > is:
> > > >
> > > > fd = open(argv[1], O_RDONLY);
> > > > if (fd == -1)
> > > > exit(1);
> > > > if (lseek(fd, 0, SEEK_CUR) != -1)
> > > >printf ("%d is seekable.\n", fd);
> > > > else
> > > >printf ("%d is not seekable.\n", fd);
> > > >
> > > > seeing an application iterate through a stream or fd via getchar(),
> > > although
> > > > a simple seek() would work is very frustrating. i think what would be
> > > needed
> > > > is a simple function or macro that userspace applications can make use
> > > > of.
> > > > maybe such a thing already exists in linux, openbsd, netbsd,
> > > > dragonflybsd,
> > > > solaris or plan9?
> > > >
> > > > cheers.
> > > > alex
> > > >
> > > > references:
> > > > [1]
> > > http://www.mavetju.org/mail/view_thread.php?list=freebsd-hackers&id=3290708&thread=yes
> > > > [2] http://www.freebsd.org/cgi/query-pr.cgi?pr=152485
> > > > [3] http://www.freebsd.org/cgi/query-pr.cgi?pr=86485
> > >
> > > So, according to APUE 2nd Edition, seek should be supported on anything
> > > that's not a pipe, FIFO, or socket. In fact, the "test for seekability"
> > > you've provided above closely resembles the example given in that text.
> >
> > if this really is the case, things could even be easier in a freebsd
> > specific
> > manor than the code above:
> >
> > !S_ISFIFO(m) && !S_ISSOCK(m)
> >
> > this means it would be trivial to write a new macro S_ISSEEK which would
> > test
> > stat.st_mode against the according bits.
>
> here's a patch, which also fixes a comment.
hmmm...the patch doesn't seem to work for directories and symbolic links
unfortunately. does anybody spot the problem?
i've attached two testing applications. mode.c will do the tests via the S_IS*
macros and requires the patch i've posted beforehand to stat.h. the other
application is seekable.c, which will use lseek(2) to test, whether a file
argument is seekable or not. basically both applications should report the
same!
cheers.
>
> cheers.
> alex
>
> >
> > cheers.
> > alex
> >
> > >
> > > Need to think about this more before commenting further...
> > >
> > > -Brandon
> diff --git a/sys/sys/stat.h b/sys/sys/stat.h
> index 1b03bd2..ab5ae44 100644
> --- a/sys/sys/stat.h
> +++ b/sys/sys/stat.h
> @@ -238,10 +238,11 @@ struct nstat {
> #define S_ISCHR(m) (((m) & 017) == 002)/* char special
> */
> #define S_ISBLK(m) (((m) & 017) == 006)/* block
> special */
> #define S_ISREG(m) (((m) & 017) == 010)/* regular file
> */
> -#define S_ISFIFO(m) (((m) & 017) == 001)/* fifo or
> socket */
> +#define S_ISFIFO(m) (((m) & 017) == 001)/* pipe or fifo
> */
> #if __POSIX_VISIBLE >= 200112
> #define S_ISLNK(m) (((m) & 017) == 012)/* symbolic
> link */
> #define S_ISSOCK(m) (((m) & 017) == 014)/* socket */
> +#define S_ISSEEK(m) (((m) & 015) == 0) /* seekable
> file */
> #endif
> #if __BSD_VISIBLE
> #define S_ISWHT(m) (((m) & 017) == 016)/* whiteout */
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: easy way to determine if a stream or fd is seekable
On Wed Nov 16 11, Joerg Sonnenberger wrote: > On Tue, Nov 15, 2011 at 08:24:50PM +, Alexander Best wrote: > > one of the things i'm missing is an easy way to determine, whether a stream > > or > > fd is seekable. i checked the dd(1) and hd(1) sources and those tools are > > performing so much stuff just to find out if this is the case, and they > > still > > are doing a very poor job. > > Isn't the primary issue that FreeBSD doesn't properly report errors for > lseek(2)? I think you should start from that and not hack around the > fallout... what do you mean? lseek(2) returns -1, when the file descriptor is not seekable. i fired lseek(2) at all sorts of file types (dir, sockets, ...) and it always returned the correct result. cheers. alex > > Joerg ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: easy way to determine if a stream or fd is seekable
On Wed Nov 16 11, Alexander Best wrote:
> On Wed Nov 16 11, Alexander Best wrote:
> > On Wed Nov 16 11, Alexander Best wrote:
> > > On Tue Nov 15 11, Brandon Gooch wrote:
> > > > On Nov 15, 2011 2:25 PM, "Alexander Best" wrote:
> > > > >
> > > > > hi there,
> > > > >
> > > > > one of the things i'm missing is an easy way to determine, whether a
> > > > stream or
> > > > > fd is seekable. i checked the dd(1) and hd(1) sources and those tools
> > > > > are
> > > > > performing so much stuff just to find out if this is the case, and
> > > > > they
> > > > still
> > > > > are doing a very poor job.
> > > > >
> > > > > dd(1) e.g. identifies /dev/zero as non-seekable, even though it is.
> > > > > the
> > > > result:
> > > > >
> > > > > `dd if=/dev/zero bs=1m count=1 skip=10`:
> > > > >
> > > > > on freebsd:
> > > > > 57,41 real 0,05 user43,21 sys
> > > > >
> > > > > on openbsd:
> > > > > 0,88 real 0,00 user 0,07 sys
> > > > >
> > > > > on freebsd dd(1) is very easy fixable (1 line diff). the point
> > > > > however is:
> > > > >
> > > > > there doesn't seem to exist a unified way of checking seekable ==
> > > > > TRUE. so
> > > > > every userspace application seems to do it differently and most of
> > > > > them
> > > > (dd(1)
> > > > > and hd(1) e.g) aren't doing it right. hd(1) e.g. believes that only
> > > > regular
> > > > > files are seekable. this means that hd(1) fired at /dev/ada* with a
> > > > > high
> > > > skip
> > > > > value takes ages! dd(1) is a bit smarter in this case, but still not
> > > > correct.
> > > > >
> > > > > idealy there would be something like stat.st_mode (see stat(2)) where
> > > > > one
> > > > > could simply do a S_ISSEEK(m). so far the best and easiest solution
> > > > > i've
> > > > seen
> > > > > is:
> > > > >
> > > > > fd = open(argv[1], O_RDONLY);
> > > > > if (fd == -1)
> > > > > exit(1);
> > > > > if (lseek(fd, 0, SEEK_CUR) != -1)
> > > > >printf ("%d is seekable.\n", fd);
> > > > > else
> > > > >printf ("%d is not seekable.\n", fd);
> > > > >
> > > > > seeing an application iterate through a stream or fd via getchar(),
> > > > although
> > > > > a simple seek() would work is very frustrating. i think what would be
> > > > needed
> > > > > is a simple function or macro that userspace applications can make
> > > > > use of.
> > > > > maybe such a thing already exists in linux, openbsd, netbsd,
> > > > > dragonflybsd,
> > > > > solaris or plan9?
> > > > >
> > > > > cheers.
> > > > > alex
> > > > >
> > > > > references:
> > > > > [1]
> > > > http://www.mavetju.org/mail/view_thread.php?list=freebsd-hackers&id=3290708&thread=yes
> > > > > [2] http://www.freebsd.org/cgi/query-pr.cgi?pr=152485
> > > > > [3] http://www.freebsd.org/cgi/query-pr.cgi?pr=86485
> > > >
> > > > So, according to APUE 2nd Edition, seek should be supported on anything
> > > > that's not a pipe, FIFO, or socket. In fact, the "test for seekability"
> > > > you've provided above closely resembles the example given in that text.
> > >
> > > if this really is the case, things could even be easier in a freebsd
> > > specific
> > > manor than the code above:
> > >
> > > !S_ISFIFO(m) && !S_ISSOCK(m)
> > >
> > > this means it would be trivial to write a new macro S_ISSEEK which would
> > > test
> > > stat.st_mode against the according bits.
> >
> > here's a patch, which also fixes a comment.
>
> hmmm...the patch doesn't seem to work for directories and symbolic links
> unfortunately. does anybody spot the problem?
>
> i've attached two testing applications. mode.c will do the tests via the S_IS*
> macros and requires the patch i've posted beforehand to stat.h. the other
> application is seekable.c, which will use lseek(2) to test, whether a file
> argument is seekable or not. basically both applications should report the
> same!
g...the attachments got scrubbed!
>
> cheers.
>
> >
> > cheers.
> > alex
> >
> > >
> > > cheers.
> > > alex
> > >
> > > >
> > > > Need to think about this more before commenting further...
> > > >
> > > > -Brandon
>
> > diff --git a/sys/sys/stat.h b/sys/sys/stat.h
> > index 1b03bd2..ab5ae44 100644
> > --- a/sys/sys/stat.h
> > +++ b/sys/sys/stat.h
> > @@ -238,10 +238,11 @@ struct nstat {
> > #defineS_ISCHR(m) (((m) & 017) == 002)/* char special
> > */
> > #defineS_ISBLK(m) (((m) & 017) == 006)/* block
> > special */
> > #defineS_ISREG(m) (((m) & 017) == 010)/* regular file
> > */
> > -#defineS_ISFIFO(m) (((m) & 017) == 001)/* fifo or
> > socket */
> > +#defineS_ISFIFO(m) (((m) & 017) == 001)/* pipe or fifo
> > */
> > #if __POSIX_VISIBLE >= 200112
> > #defineS_ISLNK(m) (((m) & 017) == 012)/* symbolic
> > link */
> > #defineS_ISSOCK(m) (((m) & 017) == 014)/* socket */
> > +#defineS_ISSEEK(m) (((m) & 015) == 0) /* seekable
> > file */
>
Re: Include file search path
On Wed, Sep 14, 2011 at 8:23 PM, Arnaud Lacombe wrote: > Hi, ... > ping, Warner ? > > FWIW, I guess that in 4 months, either you had time to finish the > patch, or I could have found to time to complete it myself based on > your incomplete version. I'd really be interested to benchmark recent, > decent, compiler. I've pinged Warner multiple times with no response on this issue apart from the first reply. I'll try to compile all of the details in this thread and do my best to engineer the work from scratch because it appears that the work was never made available for public consumption. So rather than wait for 2038 to roll around, I'll talk to folks (brooks, etc) and just get it done. Thanks, -Garrett ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: easy way to determine if a stream or fd is seekable
On Wed Nov 16 11, Alexander Best wrote:
> On Wed Nov 16 11, Alexander Best wrote:
> > On Wed Nov 16 11, Alexander Best wrote:
> > > On Tue Nov 15 11, Brandon Gooch wrote:
> > > > On Nov 15, 2011 2:25 PM, "Alexander Best" wrote:
> > > > >
> > > > > hi there,
> > > > >
> > > > > one of the things i'm missing is an easy way to determine, whether a
> > > > stream or
> > > > > fd is seekable. i checked the dd(1) and hd(1) sources and those tools
> > > > > are
> > > > > performing so much stuff just to find out if this is the case, and
> > > > > they
> > > > still
> > > > > are doing a very poor job.
> > > > >
> > > > > dd(1) e.g. identifies /dev/zero as non-seekable, even though it is.
> > > > > the
> > > > result:
> > > > >
> > > > > `dd if=/dev/zero bs=1m count=1 skip=10`:
> > > > >
> > > > > on freebsd:
> > > > > 57,41 real 0,05 user43,21 sys
> > > > >
> > > > > on openbsd:
> > > > > 0,88 real 0,00 user 0,07 sys
> > > > >
> > > > > on freebsd dd(1) is very easy fixable (1 line diff). the point
> > > > > however is:
> > > > >
> > > > > there doesn't seem to exist a unified way of checking seekable ==
> > > > > TRUE. so
> > > > > every userspace application seems to do it differently and most of
> > > > > them
> > > > (dd(1)
> > > > > and hd(1) e.g) aren't doing it right. hd(1) e.g. believes that only
> > > > regular
> > > > > files are seekable. this means that hd(1) fired at /dev/ada* with a
> > > > > high
> > > > skip
> > > > > value takes ages! dd(1) is a bit smarter in this case, but still not
> > > > correct.
> > > > >
> > > > > idealy there would be something like stat.st_mode (see stat(2)) where
> > > > > one
> > > > > could simply do a S_ISSEEK(m). so far the best and easiest solution
> > > > > i've
> > > > seen
> > > > > is:
> > > > >
> > > > > fd = open(argv[1], O_RDONLY);
> > > > > if (fd == -1)
> > > > > exit(1);
> > > > > if (lseek(fd, 0, SEEK_CUR) != -1)
> > > > >printf ("%d is seekable.\n", fd);
> > > > > else
> > > > >printf ("%d is not seekable.\n", fd);
> > > > >
> > > > > seeing an application iterate through a stream or fd via getchar(),
> > > > although
> > > > > a simple seek() would work is very frustrating. i think what would be
> > > > needed
> > > > > is a simple function or macro that userspace applications can make
> > > > > use of.
> > > > > maybe such a thing already exists in linux, openbsd, netbsd,
> > > > > dragonflybsd,
> > > > > solaris or plan9?
> > > > >
> > > > > cheers.
> > > > > alex
> > > > >
> > > > > references:
> > > > > [1]
> > > > http://www.mavetju.org/mail/view_thread.php?list=freebsd-hackers&id=3290708&thread=yes
> > > > > [2] http://www.freebsd.org/cgi/query-pr.cgi?pr=152485
> > > > > [3] http://www.freebsd.org/cgi/query-pr.cgi?pr=86485
> > > >
> > > > So, according to APUE 2nd Edition, seek should be supported on anything
> > > > that's not a pipe, FIFO, or socket. In fact, the "test for seekability"
> > > > you've provided above closely resembles the example given in that text.
> > >
> > > if this really is the case, things could even be easier in a freebsd
> > > specific
> > > manor than the code above:
> > >
> > > !S_ISFIFO(m) && !S_ISSOCK(m)
> > >
> > > this means it would be trivial to write a new macro S_ISSEEK which would
> > > test
> > > stat.st_mode against the according bits.
> >
> > here's a patch, which also fixes a comment.
>
> hmmm...the patch doesn't seem to work for directories and symbolic links
> unfortunately. does anybody spot the problem?
here's an updated patch, which should work.
cheers.
alex
>
> i've attached two testing applications. mode.c will do the tests via the S_IS*
> macros and requires the patch i've posted beforehand to stat.h. the other
> application is seekable.c, which will use lseek(2) to test, whether a file
> argument is seekable or not. basically both applications should report the
> same!
>
> cheers.
>
> >
> > cheers.
> > alex
> >
> > >
> > > cheers.
> > > alex
> > >
> > > >
> > > > Need to think about this more before commenting further...
> > > >
> > > > -Brandon
>
> > diff --git a/sys/sys/stat.h b/sys/sys/stat.h
> > index 1b03bd2..ab5ae44 100644
> > --- a/sys/sys/stat.h
> > +++ b/sys/sys/stat.h
> > @@ -238,10 +238,11 @@ struct nstat {
> > #defineS_ISCHR(m) (((m) & 017) == 002)/* char special
> > */
> > #defineS_ISBLK(m) (((m) & 017) == 006)/* block
> > special */
> > #defineS_ISREG(m) (((m) & 017) == 010)/* regular file
> > */
> > -#defineS_ISFIFO(m) (((m) & 017) == 001)/* fifo or
> > socket */
> > +#defineS_ISFIFO(m) (((m) & 017) == 001)/* pipe or fifo
> > */
> > #if __POSIX_VISIBLE >= 200112
> > #defineS_ISLNK(m) (((m) & 017) == 012)/* symbolic
> > link */
> > #defineS_ISSOCK(m) (((m) & 017) == 014)/* socket */
> > +#defineS_ISSEEK(m) (((m) & 015) == 0) /* seeka
Re: easy way to determine if a stream or fd is seekable
On Wed, Nov 16, 2011 at 01:14:28PM +, Alexander Best wrote: > On Wed Nov 16 11, Joerg Sonnenberger wrote: > > On Tue, Nov 15, 2011 at 08:24:50PM +, Alexander Best wrote: > > > one of the things i'm missing is an easy way to determine, whether a > > > stream or > > > fd is seekable. i checked the dd(1) and hd(1) sources and those tools are > > > performing so much stuff just to find out if this is the case, and they > > > still > > > are doing a very poor job. > > > > Isn't the primary issue that FreeBSD doesn't properly report errors for > > lseek(2)? I think you should start from that and not hack around the > > fallout... > > what do you mean? lseek(2) returns -1, when the file descriptor is not > seekable. i fired lseek(2) at all sorts of file types (dir, sockets, ...) > and it always returned the correct result. If that were the case, you wouldn't need your flag to detect seek support. But e.g. some devices silently ignore seek requests without reporting errors. At least that is what I remember from the last time this has brought up. Joerg ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: easy way to determine if a stream or fd is seekable
On Thu Nov 17 11, Joerg Sonnenberger wrote: > On Wed, Nov 16, 2011 at 01:14:28PM +, Alexander Best wrote: > > On Wed Nov 16 11, Joerg Sonnenberger wrote: > > > On Tue, Nov 15, 2011 at 08:24:50PM +, Alexander Best wrote: > > > > one of the things i'm missing is an easy way to determine, whether a > > > > stream or > > > > fd is seekable. i checked the dd(1) and hd(1) sources and those tools > > > > are > > > > performing so much stuff just to find out if this is the case, and they > > > > still > > > > are doing a very poor job. > > > > > > Isn't the primary issue that FreeBSD doesn't properly report errors for > > > lseek(2)? I think you should start from that and not hack around the > > > fallout... > > > > what do you mean? lseek(2) returns -1, when the file descriptor is not > > seekable. i fired lseek(2) at all sorts of file types (dir, sockets, ...) > > and it always returned the correct result. > > If that were the case, you wouldn't need your flag to detect seek > support. But e.g. some devices silently ignore seek requests without > reporting errors. At least that is what I remember from the last time > this has brought up. this is the first time i hear about problems with seek requests. would be nice to see some examples cases. was this discussed on the mailinglists? or submitted as a problem report? even if lseek(2) would always work, it would be overhead. there's no need to test a file operand against seeking, because we can derive that from its type. a file operand that isn't a pipe, fifo or socket IS seekable. i consider running S_ISSEEK(m) much more intuitive than first opening a file, saving its filedescriptor, dealing with errors, running lseek(2) with a zero offset argument (which appears to be very hackish), dealing with lseek(2) errors and finally evaluating lseek(2)'s return value. also the way of checking, whether a file operand is seekable via lseek(2) as always existed and people don't seem to understood that. hexdump(1) and dd(1) weren't written by amateurs -- still they aren't using lseek(2) on their file operand or aren't doing it correctly. so i believe developers should be given a more intuitive way to check for the ability to seek on a given file operand. S_ISSEEK(m) seems to be a very good solution from my point of view. cheers. alex > > Joerg ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: easy way to determine if a stream or fd is seekable
On Nov 16, 2011, at 4:24 PM, Alexander Best wrote: > On Thu Nov 17 11, Joerg Sonnenberger wrote: >> On Wed, Nov 16, 2011 at 01:14:28PM +, Alexander Best wrote: >>> On Wed Nov 16 11, Joerg Sonnenberger wrote: On Tue, Nov 15, 2011 at 08:24:50PM +, Alexander Best wrote: > one of the things i'm missing is an easy way to determine, whether a > stream or > fd is seekable. i checked the dd(1) and hd(1) sources and those tools are > performing so much stuff just to find out if this is the case, and they > still > are doing a very poor job. Isn't the primary issue that FreeBSD doesn't properly report errors for lseek(2)? I think you should start from that and not hack around the fallout... >>> >>> what do you mean? lseek(2) returns -1, when the file descriptor is not >>> seekable. i fired lseek(2) at all sorts of file types (dir, sockets, ...) >>> and it always returned the correct result. >> >> If that were the case, you wouldn't need your flag to detect seek >> support. But e.g. some devices silently ignore seek requests without >> reporting errors. At least that is what I remember from the last time >> this has brought up. > > this is the first time i hear about problems with seek requests. would be nice > to see some examples cases. was this discussed on the mailinglists? or > submitted as a problem report? There was a version of bsdtar that made the mistake of assuming lseek() would return an error. lseek() on a tape drive does not return an error, nor does it actually do anything. After a few experiments, bsdtar stopped using lseek() on FreeBSD for anything other than regular files and block devices. I believe there are other things that do support seeking, but I don't believe there is an accurate mechanism for determining whether lseek() is correctly supported. Tim ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
