Fscking a partition mounted Read only...
Hi all. I didn't really know where to post this question, so I try "hackers" When rc starts, root filesystem is already mounted readonly, and fsck runs ok, then root is remounted read/write. Later, if root filesystem is remounted readonly, then fsck is called, it will says "NO WRITE ACCESS". If "later" is "in multi user mode with lots of running process", I guess it is a really bad idea to run fsck, but if "later" is "still very early during rc process", it should not generate more problems than the usual fsck. My exact situation is: - run a custom init * remounts root read/write * do some write operations on root filesystem * remount root read-only (mnt_flags = MNT_RDONLY|MNT_UPDATE, export.ex_flags = MNT_EXRDONLY in mount syscall). * execv real /sbin/init - init starts - rc starts - fsck says "NO WRITE ACCESS" Is there a way to remount root read only in the exact same state as it is when init starts ? Thanks, Yvan. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Fscking a partition mounted Read only...
On Fri, Oct 13, 2006 at 09:41:37AM +0200, VANHULLEBUS Yvan wrote: > > Later, if root filesystem is remounted readonly, then fsck is called, > it will says "NO WRITE ACCESS". mount -ur / or if your fstab is not matching the system configuration mount -ur /dev/$ROOT / Joerg ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Fscking a partition mounted Read only...
On Fri, Oct 13, 2006 at 10:04:07AM +0200, Joerg Sonnenberger wrote: > On Fri, Oct 13, 2006 at 09:41:37AM +0200, VANHULLEBUS Yvan wrote: > > > > Later, if root filesystem is remounted readonly, then fsck is called, > > it will says "NO WRITE ACCESS". > > mount -ur / > > or if your fstab is not matching the system configuration > > mount -ur /dev/$ROOT / That's what I tried (using the mount command or the mount syscall). Fsck runs, but starts by "NO WRITE ACCESS", and I don't know what will happen if it detects and try to fix some problems on the filesystem. Yvan. -- NETASQ http://www.netasq.com ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: LSI1064 and LSI1064E and mpt
Sounds like f/w issues. Let me see if I can't get some traction out of LSI on this. On 10/12/06, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: The problems are: (1) It's not a RAID-0 Vol but a RAID-1 (The PCI-X card reports it correctly, but not the PCI-Express. This should not differ. (2) The RAID volume is very slow (both cards are slow with RAID volume - this is the problem I am trying to track down), approx 4.5 Mbyte/sec instead of 30 Mbyte/sec I get when no RAID volume is configured. With the RAID volume, even just 1 disk in the volume get slow performance. (3) With the PCI-X card the config page has info in it, so I can add code to print the resync complete percentage on intervals I choose. (4) I added some code to print out the volume config page 0 info on the PCI-Express card: The header looks ok, but not the config page in the same structure, it's all zeroes. Snippet start -- . . mpt: MPI Version=1.5.13.0 ROLE: MPT_ROLE_INITIATOR IOC now at RUNSTATE IOC Page 2 Header: ver 4, len 8c, num 2, type 1 Capabilities: ( RAID-0 RAID-1E RAID-1netlog:mpt .. ) 1 Active Volume (2 Max) 2 Hidden Drive Members (10 Max) . . mpt: mpt_read_cfg_page: Config Info Status 22 mpt0:vol0(mpt0:0:0): mpt_refresh_raid_vol: Failed to read RAID Vol Page(0) Volume Config Page values Header PageVersion 0x6 Header PageLength 0x12 Header PageNumber 0x0 Header PageType0x8 VolumeID0 VolumeBus 0 VolumeIOC 0 VolumeType 0 VolumeType Flags 0x0 VolumeType State 0x0 VolumeType Reserved 0x0 VolumeSettings settings 0x0 VolumeSettings HotSparePool 0x0 VolumeSettings Reserved 0x0 MaxLBA 0 Reserved1 0 StripeSize 0 Reserved2 0 Reserved3 0 NumPhysDisks0 DataScrubRate 0 ResyncRate 0 InactiveStatus 0 mpt0:vol0(mpt0:0:0): Settings ( ) mpt0:vol0(mpt0:0:0): 0 Members: mpt0:vol0(mpt0:0:0): RAID-0 - Optimal . . . (mpt0:0:6): Physical (mpt0:0:6), Pass-thru (mpt0:1:0) (mpt0:0:6): Online (mpt0:0:4): Physical (mpt0:0:4), Pass-thru (mpt0:1:1) (mpt0:0:4): Online . . . (probe5:mpt0:0:5:0): Uninitialized Transport 8:1e? da0 at mpt0 bus 0 target 5 lun 0 da0: Fixed Direct Access SCSI-2 device da0: 300.000MB/s transfers da0: 37193MB (76171264 512 byte sectors: 255H 63S/T 4741C) . . --- Snippet end --- Johan ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Tracing binaries statically linked against vulnerable libs
On 10/7/06, Kris Kennaway <[EMAIL PROTECTED]> wrote:
On Fri, Oct 06, 2006 at 09:35:31AM +0400, Andrew Pantyukhin wrote:
> I wonder if there is a way to deal with statically linked binaries,
> which use vulnerable libraries.
The best way is to track them down and force them all to link
dynamically; static linking is a PITA from a systems management point
of view :)
Do you think we could do that without a serious impact on
performance? I know Gentoo has this Prelink feature
(http://www.gentoo.org/doc/en/prelink-howto.xml) which
helps with performance, but looks like a hack.
Anyway, maybe portmgr could issue some kind of a policy
about this. I.e. (1) use {build,run}_depends instead of lib_
when you depend on a port providing both shared and
static libraries, but link statically; (2) make an effort to
encourage dynamic linking - try to provide only shared
libs in new ports, remove unused static ones from old
ones, and so on.
The only secure way to deal with it now is to mark all
ports that depend on a vulnerable one, also vulnerable -
and then try to figure out which of them are indeed safe.
Of course, this will result in half of the tree being marked
vulnerable most of the time :-(
Thanks!
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: "tar -c|gzip" faster than "tar -cz"?!?
Doug Barton wrote:
> Oliver Fromme wrote:
> > In this case, the "real" time is much larger than the
> > "user" time. I guess that's the overhead of 85677 files
> > and 23399 directories (according to find(1)). :-)
>
> Did you perform your tests once only with each method, and one right
> after the other? If so, the effect you saw might be due to file system
> caching.
I performed each test several times in succession. If the
first run was much different from the rest, I ignored it,
so the caches were filled the same on all runs.
But 100,000 files still cause some I/O overhead, even if
the data is cached and the I/O requests don't actually hit
the physical disks.
Anyway, my point is not about caching and I/O. The numbers
are pretty normal in that regard. My point is about the
difference in CPU ("user") time when using "tar -cz" vs.
"tar -c | gzip".
Meanwhile I had a quick look at the code: gzip uses some
optimized assembler code (for x86 and 680x0), while libz
doesn't have such a thing. Maybe that's the reason why
gzip is noticeably faster.
Best regards
Oliver
--
Oliver Fromme, secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing
Dienstleistungen mit Schwerpunkt FreeBSD: http://www.secnetix.de/bsd
Any opinions expressed in this message may be personal to the author
and may not necessarily reflect the opinions of secnetix in any way.
"The last good thing written in C was
Franz Schubert's Symphony number 9."
-- Erwin Dieterich
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: "tar -c|gzip" faster than "tar -cz"?!?
Oliver Fromme wrote: Meanwhile I had a quick look at the code: gzip uses some optimized assembler code ... Maybe that's the reason why gzip is noticeably faster. Anyone care to try this test on PPC, ARM, or Sparc? There's a move afoot to replace the GPL gzip with a more openly-licensed gzip implemented on top of libz. I wonder if the libz implementors have similar assembly optimizations that we should be using? Tim Kientzle ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: "tar -c|gzip" faster than "tar -cz"?!?
> From: Pieter de Goeje <[EMAIL PROTECTED]> > Subject: Re: "tar -c|gzip" faster than "tar -cz"?!? > > The tar|gzip command uses 18% less CPU and is 10% faster. It > is clear the HDD is the bottleneck. Now it's clear to me :) This makes sense if tar is single-threaded: there's only one thread of execution, and it can either be waiting on the disk, or compressing data. With two processes, gzip can compress while tar blocks on disk IO. Maybe the rest of you figured this out immediately, but it took my coffee a while to take effect ;) Kelly ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: "tar -c|gzip" faster than "tar -cz"?!?
In the last episode (Oct 13), Tim Kientzle said: > Oliver Fromme wrote: > >Meanwhile I had a quick look at the code: gzip uses some optimized > >assembler code ... Maybe that's the reason why gzip is noticeably > >faster. > > Anyone care to try this test on PPC, ARM, or Sparc? The only assembly in our match.S is for x86 and 68k. Newer gzips also include an ia64 version. > There's a move afoot to replace the GPL gzip with a more > openly-licensed gzip implemented on top of libz. I wonder if the > libz implementors have similar assembly optimizations that we should > be using? Odd. I actually disabled the assembly file in my tree because gcc generated 20%-faster code from deflate.c than the provided assembly code in match.S , at least on a pIII. -- Dan Nelson [EMAIL PROTECTED] ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: "tar -c|gzip" faster than "tar -cz"?!?
: :Just a silly one but are you guys using the same :version of gzip, would be worth just checking? It could also simply be a piplining issue. If the pipe inbetween the 'tar' and the 'gzip' is too small (whether gzip is internal to tar or not), then the 'tar' portion could wind up getting blocked by the 'gzip' portion and not do disk I/O in parallel with the cpu that the gzip portion uses. Here I am presuming that there is in fact a fork internal to tar when using the built-in gzip. There had better be, or performance would be horrible! In anycase, the pipe buffer needs to be at least 2x the block size gzip uses internally when compressing. I would even recommend making it very large, like several hundred kilobytes (at least). It is the same problem one faces when, say, streaming data to a slow device such as a tape drive. You want a large pipe buffer to avoid unsightly stalls of the code scanning the filesystem. -Matt ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Fscking a partition mounted Read only...
On Fri, Oct 13, 2006 at 10:04:07AM +0200, Joerg Sonnenberger wrote: > On Fri, Oct 13, 2006 at 09:41:37AM +0200, VANHULLEBUS Yvan wrote: > > > > Later, if root filesystem is remounted readonly, then fsck is called, > > it will says "NO WRITE ACCESS". > > mount -ur / I'm pretty sure that's what he tried (hence "remounted readonly"). I've noticed this behavior as well and it is quite frustrating. If you boot single-user, / will be mounted read-only and you can fsck it. If you do: mount -u / mount -u -r / You can no longer fsck it. I've been meaning to track this down and/or file a PR. I'm pretty sure this used to work just fine in 3.x and 5.x. -- Rick C. Petty ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: "tar -c|gzip" faster than "tar -cz"?!?
On Fri, Oct 13, 2006 at 03:45:34PM +0200, Oliver Fromme wrote: > Meanwhile I had a quick look at the code: gzip uses some > optimized assembler code (for x86 and 680x0), while libz > doesn't have such a thing. Maybe that's the reason why > gzip is noticeably faster. I'm not sure what happened in the mean time, but when I did the tests in 2004, libz was faster. *sigh* The assembly doesn't seem to help much, at least not the longest_match version. Might be a good idea to look at crc32, but I don't have the time now :-( Joerg ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: "tar -c|gzip" faster than "tar -cz"?!?
Kelly Hall <[EMAIL PROTECTED]> wrote: > > From: Pieter de Goeje <[EMAIL PROTECTED]> > > Subject: Re: "tar -c|gzip" faster than "tar -cz"?!? > > > > The tar|gzip command uses 18% less CPU and is 10% faster. It > > is clear the HDD is the bottleneck. > > Now it's clear to me :) > > This makes sense if tar is single-threaded: there's only one thread of > execution, and it can either be waiting on the disk, or compressing > data. With two processes, gzip can compress while tar blocks on disk > IO. No. During my tests there was no physical disk I/O (the disk LED was *OFF* all the time). So tar certainly wasn't blocking on disk I/O. The difference in CPU time (and wall clock time) seems simply to be caused by different compression code. gzip is noticeably more efficient than libz, at least on the OS/processor combination where I tested it (Athlon64 with FreeBSD/i386 6.2-PRERELEASE). Depending on the type of data, the difference is between 25% and 40%. If the data is not compressible (like stuff from /dev/random, or already compressed files), it's only 25%. For very good compressible data (best case: /dev/null), the difference is 40%. Best regards Oliver -- Oliver Fromme, secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing Dienstleistungen mit Schwerpunkt FreeBSD: http://www.secnetix.de/bsd Any opinions expressed in this message may be personal to the author and may not necessarily reflect the opinions of secnetix in any way. cat man du : where Unix geeks go when they die ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Fscking a partition mounted Read only...
On Friday 13 October 2006 13:42, Rick C. Petty wrote: > On Fri, Oct 13, 2006 at 10:04:07AM +0200, Joerg Sonnenberger wrote: > > On Fri, Oct 13, 2006 at 09:41:37AM +0200, VANHULLEBUS Yvan wrote: > > > > > > Later, if root filesystem is remounted readonly, then fsck is called, > > > it will says "NO WRITE ACCESS". > > > > mount -ur / > > I'm pretty sure that's what he tried (hence "remounted readonly"). I've > noticed this behavior as well and it is quite frustrating. If you boot > single-user, / will be mounted read-only and you can fsck it. If you do: > > mount -u / > mount -u -r / > > You can no longer fsck it. I've been meaning to track this down and/or > file a PR. I'm pretty sure this used to work just fine in 3.x and 5.x. I think it's broken in 5.x as well. It's fallout from GEOM IIRC, and it is annoying. -- John Baldwin ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: "tar -c|gzip" faster than "tar -cz"?!?
:On Fri, Oct 13, 2006 at 03:45:34PM +0200, Oliver Fromme wrote: :> Meanwhile I had a quick look at the code: gzip uses some :> optimized assembler code (for x86 and 680x0), while libz :> doesn't have such a thing. Maybe that's the reason why :> gzip is noticeably faster. : :I'm not sure what happened in the mean time, but when I did the tests in :2004, libz was faster. *sigh* The assembly doesn't seem to help much, at :least not the longest_match version. Might be a good idea to look at :crc32, but I don't have the time now :-( : :Joerg I very vaguely remember someone mentioning assembly optimized code for our libz, or something... I don't remember precisely. I think my response was along the lines of "as long as a C version always works, knock yourself out". -Matt ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Fscking a partition mounted Read only...
On Fri, Oct 13, 2006 at 02:16:51PM -0400, John Baldwin wrote: > On Friday 13 October 2006 13:42, Rick C. Petty wrote: > > > > I'm pretty sure that's what he tried (hence "remounted readonly"). I've > > noticed this behavior as well and it is quite frustrating. If you boot > > single-user, / will be mounted read-only and you can fsck it. If you do: > > > > mount -u / > > mount -u -r / > > > > You can no longer fsck it. I've been meaning to track this down and/or > > file a PR. I'm pretty sure this used to work just fine in 3.x and 5.x. > > I think it's broken in 5.x as well. It's fallout from GEOM IIRC, and it is > annoying. Grr, I meant 4.x not 5.x, and I thought the problem started about the time bg fsck was introduced... -- Rick C. Petty ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
VBAD vnodes?
Hi there, This has been noticed on all 5.x, 6.x, and 7.0 systems around here: adjkerntz and xterm processes have VBAD type vnodes attached to some of their descriptors. What this is supposed to mean? $ fstat | grep -w bad root xterm9766 - - bad- root xterm8696 - - bad- root adjkerntz1470 - - bad- root adjkerntz1471 - - bad- root adjkerntz1472 - - bad- On RELENG_4, it's "s/bad/none/". Cheers, -- Ruslan Ermilov [EMAIL PROTECTED] FreeBSD committer pgplHds681edQ.pgp Description: PGP signature
Re: VBAD vnodes?
:Hi there, : :This has been noticed on all 5.x, 6.x, and 7.0 systems around :here: : :adjkerntz and xterm processes have VBAD type vnodes attached :to some of their descriptors. What this is supposed to mean? : :$ fstat | grep -w bad :root xterm9766 - - bad- :... : :On RELENG_4, it's "s/bad/none/". : :Cheers, : :Ruslan Ermilov These are almost certainly descriptors whos vnodes have been revoked with revoke(). This most commonly occurs on tty (or pty) descriptors. For example, if you logout of a tty session with background processes running, the tty is revoked so it can be reused and so the processes still running cannot mess around with someone else who reuses the tty. Here is an example: shell1# xterm (from xterm) shell2# sleep 300 & [1] 38290 shell2# exit shell1# fstat -p 38290 USER CMD PID FD PATH INUM MODE SZ|DV R/W dillon sleep 38290 root / 2 drwxr-xr-x 1024 r dillon sleep 38290 wd /home/dillon 224000 drwxr-xr-x 6144 r dillon sleep 38290 text /bin/sleep10652 -r-xr-xr-x101344 r dillon sleep 382900 - -none- dillon sleep 382901 - -none- dillon sleep 382902 - -none- -Matt ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: VBAD vnodes?
On Fri, Oct 13, 2006 at 01:43:38PM -0700, Matthew Dillon wrote: > These are almost certainly descriptors whos vnodes have been > revoked with revoke(). This most commonly occurs on tty (or pty) > descriptors. For example, if you logout of a tty session > with background processes running, the tty is revoked so > it can be reused and so the processes still running cannot mess > around with someone else who reuses the tty. > I've tracked it down to vgonerel() in the kernel, but didn't think it was so simple to reproduce. Thanks a lot! :-) Cheers, -- Ruslan Ermilov [EMAIL PROTECTED] FreeBSD committer pgpLEFVTfyMw8.pgp Description: PGP signature
Re: Tracing binaries statically linked against vulnerable libs
On Fri, Oct 13, 2006 at 05:18:57PM +0400, Andrew Pantyukhin wrote:
> On 10/7/06, Kris Kennaway <[EMAIL PROTECTED]> wrote:
> >On Fri, Oct 06, 2006 at 09:35:31AM +0400, Andrew Pantyukhin wrote:
> >> I wonder if there is a way to deal with statically linked binaries,
> >> which use vulnerable libraries.
> >
> >The best way is to track them down and force them all to link
> >dynamically; static linking is a PITA from a systems management point
> >of view :)
>
> Do you think we could do that without a serious impact on
> performance?
In most of the cases I've looked at the statically linked binary is
not performance critical or otherwise necessary (the only exception I
saw is for some tripwire-like port whose name I forget, which is
statically linked as a security enhancement, to make it lease easily
subverted). Static linking can be made an OPTION if someone thinks
it's really necessary for a given port.
> I know Gentoo has this Prelink feature
> (http://www.gentoo.org/doc/en/prelink-howto.xml) which
> helps with performance, but looks like a hack.
>
> Anyway, maybe portmgr could issue some kind of a policy
> about this. I.e. (1) use {build,run}_depends instead of lib_
> when you depend on a port providing both shared and
> static libraries, but link statically; (2) make an effort to
> encourage dynamic linking - try to provide only shared
> libs in new ports, remove unused static ones from old
> ones, and so on.
(1) is just a statement of correct behaviour, no need for a policy
about it (it could be clarified in the porters handbook if needed).
(2) could also be added to the porter's handbook as a recommendation-
I don't think we need a formal proclamation of policy about it.
Kris
P.S. I can provide a list of static binaries in ports if anyone wants
to work on fixing them.
pgpH9hc8m3z8n.pgp
Description: PGP signature
Re: Re: "tar -c|gzip" faster than "tar -cz"?!?
>From: Oliver Fromme <[EMAIL PROTECTED]> > > > The tar|gzip command uses 18% less CPU and is 10% faster. It > > > is clear the HDD is the bottleneck. > > > > Now it's clear to me :) > > > > This makes sense if tar is single-threaded: there's only one thread of > > execution, and it can either be waiting on the disk, or compressing > > data. With two processes, gzip can compress while tar blocks on disk > > IO. > >No. During my tests there was no physical disk I/O (the >disk LED was *OFF* all the time). So tar certainly wasn't >blocking on disk I/O. > >The difference in CPU time (and wall clock time) seems >simply to be caused by different compression code. gzip >is noticeably more efficient than libz, at least on the >OS/processor combination where I tested it (Athlon64 with >FreeBSD/i386 6.2-PRERELEASE). Any chance that gzip uses a different version of libz? Or maybe the buffer size is different? Yet another possibility could be if tar calls zlib with the SYNC (or is that FLUSH? something like that) flag on each chunk, this would kill both the performance and the compression rate. Then again, the default compression level may be different (but it should be making the speed higher if the ratio falls lower). -SB ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
