A few problems
Hello there. I've got a couple of things I don't get or can't handle. I'm running: FreeBSD diablo.miekoff.local 9.1-STABLE FreeBSD 9.1-STABLE #0 r248347: Sat Mar 16 03:20:58 MSK 2013 root@diablo.miekoff.local:/usr/obj/usr/src/sys/DIABLO64 amd64 1st of all, on dmesg there's something strange. It says: real memory = 6442450944 (6144 MB) avail memory = 4092743680 (3903 MB) So real memory is about 6Gb, but localy installed only 4Gb. I tried to disable swap and reboot but it's still the same output. And here goes dmidecode: SMBIOS 2.5 present. 72 structures occupying 2730 bytes. Table at 0x000FB710. Handle 0x, DMI type 0, 24 bytes BIOS Information Vendor: American Megatrends Inc. Version: V3.4 Release Date: 03/09/2009 Address: 0xF Runtime Size: 64 kB ROM Size: 4096 kB Characteristics: ISA is supported PCI is supported PNP is supported APM is supported BIOS is upgradeable BIOS shadowing is allowed ESCD support is available Boot from CD is supported Selectable boot is supported BIOS ROM is socketed EDD is supported 5.25"/1.2 MB floppy services are supported (int 13h) 3.5"/720 kB floppy services are supported (int 13h) 3.5"/2.88 MB floppy services are supported (int 13h) Print screen service is supported (int 5h) 8042 keyboard services are supported (int 9h) Serial services are supported (int 14h) Printer services are supported (int 17h) CGA/mono video services are supported (int 10h) ACPI is supported USB legacy is supported LS-120 boot is supported ATAPI Zip drive boot is supported BIOS boot specification is supported Targeted content distribution is supported BIOS Revision: 8.15 Handle 0x0001, DMI type 1, 27 bytes System Information Manufacturer: MICRO-STAR INTERNATIONAL CO.,LTD Product Name: MS-7512 Version: 1.0 Serial Number: To Be Filled By O.E.M. UUID: ----0021851C24FA Wake-up Type: Power Switch SKU Number: To Be Filled By O.E.M. Family: To Be Filled By O.E.M. Handle 0x0002, DMI type 2, 15 bytes Base Board Information Manufacturer: MICRO-STAR INTERNATIONAL CO.,LTD Product Name: P45 Neo2-FR (MS-7512) Version: 1.0 Serial Number: To be filled by O.E.M. Asset Tag: To Be Filled By O.E.M. Features: Board is a hosting board Board is replaceable Location In Chassis: To Be Filled By O.E.M. Chassis Handle: 0x0003 Type: Motherboard Contained Object Handles: 0 Handle 0x0003, DMI type 3, 21 bytes Chassis Information Manufacturer: MICRO-STAR INTERNATIONAL CO.,LTD Type: Desktop Lock: Not Present Version: 1.0 Serial Number: To Be Filled By O.E.M. Asset Tag: To Be Filled By O.E.M. Boot-up State: Safe Power Supply State: Safe Thermal State: Safe Security Status: None OEM Information: 0x Height: Unspecified Number Of Power Cords: 1 Contained Elements: 0 Handle 0x0004, DMI type 4, 40 bytes Processor Information Socket Designation: CPU 1 Type: Central Processor Family: Core 2 Duo Manufacturer: Intel ID: 7A 06 01 00 FF FB EB BF Signature: Type 0, Family 6, Model 23, Stepping 10 Flags: FPU (Floating-point unit on-chip) VME (Virtual mode extension) DE (Debugging extension) PSE (Page size extension) TSC (Time stamp counter) MSR (Model specific registers) PAE (Physical address extension) MCE (Machine check exception) CX8 (CMPXCHG8 instruction supported) APIC (On-chip APIC hardware supported) SEP (Fast system call) MTRR (Memory type range registers) PGE (Page global enable) MCA (Machine check architecture) CMOV (Conditional move instruction supported) PAT (Page attribute table) PSE-36 (36-bit page size extension) CLFSH (CLFLUSH instruction supported) DS (Debug store) ACPI (ACPI supported) MMX (MMX technology supported) FXSR (FXSAVE and FXSTOR instructions supported) SSE (Streaming SIMD extensions) SSE2 (Streaming SIMD extensions 2) SS (Self-snoop) HTT (Multi-threading) T
Re: svn - but smaller?
On Wed, 13 Mar 2013 21:11:28 -0500, John Mehr wrote:
> On Wed, 13 Mar 2013 16:29:45 +1100 (EST)
> Ian Smith wrote:
[..]
> > I have a small test system on which I'd installed (two instances of) 9.1 so
> > a couple of days ago I fetched ports with portsnap, installed svnup, and
> > ran it using the (just what I needed) example command in svnup(1).
Just to reiterate: this is vanilla releng/9.1, no ports but docs and
svnup installed, no X, 512MB RAM, nothing happening but occasional ARP.
For completeness, it's running powerd and acpi_ibm but not cam.ctl.
> > I get about 700KB/s here, and svnup took about 15 minutes to update 9.1
> > sources to 9-stable. This is fine. Last night I ran it again, but it took
> > 12:42 to make no changes. This seemed puzzling, as you'd said only a few
> > minutes for subsequent updates, but the reason appears to be that in both
> > cases, I ran it in script(1), and the default verbosity of 1 includes a
> > listing of every directory and file examined, followed by then > to eol> codes. Even in less -r (raw) mode it still has to display and skip
> > through all the (now invisible) lines; bit messy.
> >
> > Even the second do-nothing run made a 2MB script file, the original with
> > all 9.1 to -stable updates being 3.4MB. So I'd love the option to only
> > list the changes (- and +) and simply ignore unchanged dirs/files without
> > any display for use in script(1). Apart from that, I'm happy.
>
> Which mirror are you using? I ran several tests tonight repeatedly fetching
> 9/stable from svn0.us-west (so they would all be do-nothing runs) both inside
> and outside of a script(1) capture and on both an old SSD and on a ZFS
> mirrored array (to see if the target media made any difference) and they all
> completed in 2 minutes, 43 seconds +/- 2 seconds on my 350 KB/s connection.
svn0.us-west.freebsd.org as per example, but it's the closest to here
anyway; pings avg 217.5ms stddev 0.4ms, us-east avg 269.3ms sd 9.5ms.
fastest_cvsup used to find cvs mirrors in Australia at around 55-70ms,
so for another while I thought RTT might be the issue - but it's not.
> I'll definitely put in a verbosity level that does exactly what you suggest.
> Sorry about that.
Not at all, and thanks! But neither, I now find, is logging the extra
time issue; as mentioned elsewhere, running at -v0 makes next to no
difference, 12:48 west vs 12:54 east on 4 do-nothing runs, 700KB/s link,
not running in script(1) - though I think that matters not, or little.
So I began watching top more closely, also running iostat -w10, noticing
that svnup was pegging CPU at 100% with ~75% _system_? time for nearly 9
minutes of the run, doing little or no disk I/O and a max of 130KB/s in,
22.6KB/s out, no stress there, growing to ~20MB resident, then doing a
flurry of disk I/O at about 3-5MB/s for the last 3m40s odd. IOW, svnup
here is mostly heavily CPU-bound, which I don't understand, but then I
know nothing about what level of computation the protocol demands.
I've done some more controlled tests since, running both iostat -w10 and
netstat -w10 -dn before, during and until a bit after the run, attached,
hoping these might help pinpoint any bottlenecks? This was running -v1
default verbosity, hence the (not at all heavy) tty output numbers.
Just sometimes, running slower hardware can have advantages! On your
times, yours likely has more than 500% of my single-core P3-M's 1133MHz
performance, which would tend to mask this issue pretty well for you :)
John Baldwin has best described when c{,v}sup deletes (only in-tree)
files, covered in section 'delete' in csup(1). I think this should
probably be default behaviour, even if svn proper would do otherwise.
cheers, Ianinput(Total) output
packets errs idrops bytespackets errs bytes colls drops
0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
312 0 0 504213212 0 50511 0 0
587 0 0 758448384 0 108299 0 0
858 0 01307014601 0 139574 0 0
699 0 0 777451521 0 111510 0 0
612 0 0 821160429 0 99541 0 0
459 0 0 653828331 0 106364 0 0
453 0 0 510009336 0 94317 0 0
371 0 0 440457275 0 114857 0 0
341 0 0 409886273 0 80247 0 0
345 0 0 46772825
Re: amdtemp does not find my CPU.
On 3/16/13 2:20 AM, Jeremy Chadwick wrote: > On Fri, Mar 15, 2013 at 03:16:19PM -0400, Jim Ohlstein wrote: >> On 3/15/13 12:15 PM, Zoran Kolic wrote: >>> After I installed 9.1 amd64 on node with amd 8120, >>> I was not able to read temperatures out of the box. >>> I fetched source for head module and compiled. And >>> loaded module. Still nothing. I assume my cpu is >>> a bit different. >>> Best regards >> >> The module from head "works" for me with an 8120 on 9.1 stable (r247893) >> though the results are inconsistent. I am not certain of how useful they >> are. >> >> # sysctl hw.model >> hw.model: AMD FX(tm)-8120 Eight-Core Processor >> >> # kldstat | grep amd >> 51 0x8183e000 1043 amdtemp.ko >> >> # sysctl -a | grep dev.amdtemp >> dev.amdtemp.0.%desc: AMD CPU On-Die Thermal Sensors >> dev.amdtemp.0.%driver: amdtemp >> dev.amdtemp.0.%parent: hostb4 >> dev.amdtemp.0.sensor_offset: 0 >> dev.amdtemp.0.core0.sensor0: 47.7C >> >> Here are results taken at 0.1 second intervals using a shell script: >> >> dev.amdtemp.0.core0.sensor0: 42.1C >> dev.amdtemp.0.core0.sensor0: 42.2C >> dev.amdtemp.0.core0.sensor0: 42.0C >> dev.amdtemp.0.core0.sensor0: 42.1C >> dev.amdtemp.0.core0.sensor0: 41.8C >> dev.amdtemp.0.core0.sensor0: 41.7C >> dev.amdtemp.0.core0.sensor0: 51.1C >> dev.amdtemp.0.core0.sensor0: 51.0C >> dev.amdtemp.0.core0.sensor0: 50.7C >> dev.amdtemp.0.core0.sensor0: 50.5C >> dev.amdtemp.0.core0.sensor0: 50.1C >> dev.amdtemp.0.core0.sensor0: 49.8C >> dev.amdtemp.0.core0.sensor0: 49.5C >> dev.amdtemp.0.core0.sensor0: 49.2C >> dev.amdtemp.0.core0.sensor0: 49.2C >> >> >> and again: >> >> dev.amdtemp.0.core0.sensor0: 41.5C >> dev.amdtemp.0.core0.sensor0: 41.2C >> dev.amdtemp.0.core0.sensor0: 40.8C >> dev.amdtemp.0.core0.sensor0: 40.8C >> dev.amdtemp.0.core0.sensor0: 41.0C >> dev.amdtemp.0.core0.sensor0: 41.3C >> dev.amdtemp.0.core0.sensor0: 41.6C >> dev.amdtemp.0.core0.sensor0: 41.3C >> dev.amdtemp.0.core0.sensor0: 54.0C >> dev.amdtemp.0.core0.sensor0: 53.7C >> dev.amdtemp.0.core0.sensor0: 53.3C >> dev.amdtemp.0.core0.sensor0: 53.1C >> dev.amdtemp.0.core0.sensor0: 52.7C >> dev.amdtemp.0.core0.sensor0: 52.3C >> dev.amdtemp.0.core0.sensor0: 52.1C >> dev.amdtemp.0.core0.sensor0: 51.7C >> dev.amdtemp.0.core0.sensor0: 51.5C >> >> You can see during each series there are sudden increases of over 9C and >> almost 13C respectively. >> >> The same effect is seen if I track any of the individual cores with >> "dev.cpu.[0-7].temperature". Here's an example with a 9C jump in 0.1 second. >> >> dev.cpu.3.temperature: 41.5C >> dev.cpu.3.temperature: 41.5C >> dev.cpu.3.temperature: 41.7C >> dev.cpu.3.temperature: 41.7C >> dev.cpu.3.temperature: 41.3C >> dev.cpu.3.temperature: 41.0C >> dev.cpu.3.temperature: 40.7C >> dev.cpu.3.temperature: 49.8C >> dev.cpu.3.temperature: 49.5C >> dev.cpu.3.temperature: 49.2C >> dev.cpu.3.temperature: 48.8C >> dev.cpu.3.temperature: 48.6C >> dev.cpu.3.temperature: 48.2C >> dev.cpu.3.temperature: 48.0C >> >> I don't have hands on access to this box as it's in a datacenter 1000 >> miles from me, but the techs there had a look and all "seems to be OK". > > 1. While it's certainly possible the DTS reading routines and/or the > calculation formulas may be wrong in amdtemp(4), possibly for your model > of CPU, it is also certainly possible that what you're seeing is normal > and fully justified. This is especially the case for the > dev.cpu.X.temperature nodes on the K8 family. > > Respectfully, not combatively nor dismissively: you've not provided a > comparison base to prove there's an issue. You would need to provide > data from Linux (I forget what daemon/tool they have to get this) or > Windows (Core Temp). Respectfully, not combatively nor dismissively: I hadn't attempted to "prove" anything. I said: "I am not certain of how useful they [the readings] are.". I had merely provided some observational data as an aside to the fact that yes, indeed, the module provides readings for me on the 8120 This was in direct response to to Zoran's issue with this module and that processor model. This started, for me, when I looked at a graph of average core temperatures taken at 30 second intervals on two different machines using Zabbix. The fluctuations were visibly (I know that's not scientific "proof") more wild than on this server than on another using the amdtemp module from 9 stable. I don't have access to another server with this model CPU on any other OS, or even on this OS, so I cannot provide the data to "prove" this is an issue according to your criteria. However, I will provide comparative data from the other machine with the module from stable and with the the module from head. Full data taken now: # sysctl hw.model hw.model: AMD FX(tm)-8120 Eight-Core Processor Using the module from head: http://pastebin.com/wqQ0FLq3 Note the big change between lines 34 and 35. # sysctl hw.model hw.model: AMD Phenom(tm) II X6 1055T Processor Using the module from stable: http://pas
Re: amdtemp does not find my CPU.
On 16 Mar, Jim Ohlstein wrote: > On 3/16/13 2:20 AM, Jeremy Chadwick wrote: >> On Fri, Mar 15, 2013 at 03:16:19PM -0400, Jim Ohlstein wrote: >>> On 3/15/13 12:15 PM, Zoran Kolic wrote: After I installed 9.1 amd64 on node with amd 8120, I was not able to read temperatures out of the box. I fetched source for head module and compiled. And loaded module. Still nothing. I assume my cpu is a bit different. Best regards >>> >>> The module from head "works" for me with an 8120 on 9.1 stable (r247893) >>> though the results are inconsistent. I am not certain of how useful they >>> are. >>> >>> # sysctl hw.model >>> hw.model: AMD FX(tm)-8120 Eight-Core Processor >>> >>> # kldstat | grep amd >>> 51 0x8183e000 1043 amdtemp.ko >>> >>> # sysctl -a | grep dev.amdtemp >>> dev.amdtemp.0.%desc: AMD CPU On-Die Thermal Sensors >>> dev.amdtemp.0.%driver: amdtemp >>> dev.amdtemp.0.%parent: hostb4 >>> dev.amdtemp.0.sensor_offset: 0 >>> dev.amdtemp.0.core0.sensor0: 47.7C >>> >>> Here are results taken at 0.1 second intervals using a shell script: >>> >>> dev.amdtemp.0.core0.sensor0: 42.1C >>> dev.amdtemp.0.core0.sensor0: 42.2C >>> dev.amdtemp.0.core0.sensor0: 42.0C >>> dev.amdtemp.0.core0.sensor0: 42.1C >>> dev.amdtemp.0.core0.sensor0: 41.8C >>> dev.amdtemp.0.core0.sensor0: 41.7C >>> dev.amdtemp.0.core0.sensor0: 51.1C >>> dev.amdtemp.0.core0.sensor0: 51.0C >>> dev.amdtemp.0.core0.sensor0: 50.7C >>> dev.amdtemp.0.core0.sensor0: 50.5C >>> dev.amdtemp.0.core0.sensor0: 50.1C >>> dev.amdtemp.0.core0.sensor0: 49.8C >>> dev.amdtemp.0.core0.sensor0: 49.5C >>> dev.amdtemp.0.core0.sensor0: 49.2C >>> dev.amdtemp.0.core0.sensor0: 49.2C >>> >>> >>> and again: >>> >>> dev.amdtemp.0.core0.sensor0: 41.5C >>> dev.amdtemp.0.core0.sensor0: 41.2C >>> dev.amdtemp.0.core0.sensor0: 40.8C >>> dev.amdtemp.0.core0.sensor0: 40.8C >>> dev.amdtemp.0.core0.sensor0: 41.0C >>> dev.amdtemp.0.core0.sensor0: 41.3C >>> dev.amdtemp.0.core0.sensor0: 41.6C >>> dev.amdtemp.0.core0.sensor0: 41.3C >>> dev.amdtemp.0.core0.sensor0: 54.0C >>> dev.amdtemp.0.core0.sensor0: 53.7C >>> dev.amdtemp.0.core0.sensor0: 53.3C >>> dev.amdtemp.0.core0.sensor0: 53.1C >>> dev.amdtemp.0.core0.sensor0: 52.7C >>> dev.amdtemp.0.core0.sensor0: 52.3C >>> dev.amdtemp.0.core0.sensor0: 52.1C >>> dev.amdtemp.0.core0.sensor0: 51.7C >>> dev.amdtemp.0.core0.sensor0: 51.5C >>> >>> You can see during each series there are sudden increases of over 9C and >>> almost 13C respectively. >>> >>> The same effect is seen if I track any of the individual cores with >>> "dev.cpu.[0-7].temperature". Here's an example with a 9C jump in 0.1 second. >>> >>> dev.cpu.3.temperature: 41.5C >>> dev.cpu.3.temperature: 41.5C >>> dev.cpu.3.temperature: 41.7C >>> dev.cpu.3.temperature: 41.7C >>> dev.cpu.3.temperature: 41.3C >>> dev.cpu.3.temperature: 41.0C >>> dev.cpu.3.temperature: 40.7C >>> dev.cpu.3.temperature: 49.8C >>> dev.cpu.3.temperature: 49.5C >>> dev.cpu.3.temperature: 49.2C >>> dev.cpu.3.temperature: 48.8C >>> dev.cpu.3.temperature: 48.6C >>> dev.cpu.3.temperature: 48.2C >>> dev.cpu.3.temperature: 48.0C >>> >>> I don't have hands on access to this box as it's in a datacenter 1000 >>> miles from me, but the techs there had a look and all "seems to be OK". >> >> 1. While it's certainly possible the DTS reading routines and/or the >> calculation formulas may be wrong in amdtemp(4), possibly for your model >> of CPU, it is also certainly possible that what you're seeing is normal >> and fully justified. This is especially the case for the >> dev.cpu.X.temperature nodes on the K8 family. >> >> Respectfully, not combatively nor dismissively: you've not provided a >> comparison base to prove there's an issue. You would need to provide >> data from Linux (I forget what daemon/tool they have to get this) or >> Windows (Core Temp). > > Respectfully, not combatively nor dismissively: I hadn't attempted to > "prove" anything. I said: "I am not certain of how useful they [the > readings] are.". I had merely provided some observational data as an > aside to the fact that yes, indeed, the module provides readings for me > on the 8120 This was in direct response to to Zoran's issue with this > module and that processor model. > > This started, for me, when I looked at a graph of average core > temperatures taken at 30 second intervals on two different machines > using Zabbix. The fluctuations were visibly (I know that's not > scientific "proof") more wild than on this server than on another using > the amdtemp module from 9 stable. > > I don't have access to another server with this model CPU on any other > OS, or even on this OS, so I cannot provide the data to "prove" this is > an issue according to your criteria. However, I will provide comparative > data from the other machine with the module from stable and with the the > module from head. > > > Full data taken now: > > # sysctl hw.model > hw.model: AMD FX(tm)-8120 Eight-Core Processor > > Using the module from head:
Re: amdtemp does not find my CPU.
On 3/16/13 2:24 PM, Don Lewis wrote: > On 16 Mar, Jim Ohlstein wrote: [snip] >>> On Fri, Mar 15, 2013 at 03:16:19PM -0400, Jim Ohlstein wrote: [snip] >> >> Note the big change between lines 34 and 35. > > My FX-4100 behaves the same way. I noticed it because on an idle system > sysctl -a | grep amdtemp > would read consistently higher than > sysctl dev.amdtemp > > I think the thermal sensor in this AMD CPU family has a much faster > response time so that it is more sensitive to temperature changes caused > by CPU load over the short term. Going from idle to 100% CPU load for > even 0.1 seconds and then back to idle is likely to change the die > temperature a lot, but will probably only have a negligible effect on > the heat sink temperature. The faster response time may be needed to > support AMD Turbo CORE. > That's very possible. This is not an idle system so I cannot really test that out without taking it offline. My results on that comparison are therefore perhaps not valid. However, I don't see much difference. See http://pastebin.com/Rk0sbhhL. When I stress the CPU the temps simply stay high without any fluctuation, but do show those rapid rises both before and after. This supports your explanation. Of course this still does not explain the observed differences between the two modules on the other machine. -- Jim Ohlstein ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: svn - but smaller?
On Sat, Mar 16, 2013 at 8:14 AM, Ian Smith wrote:
> On Wed, 13 Mar 2013 21:11:28 -0500, John Mehr wrote:
> > On Wed, 13 Mar 2013 16:29:45 +1100 (EST)
> > Ian Smith wrote:
> [..]
> > > I have a small test system on which I'd installed (two instances of)
> 9.1 so
> > > a couple of days ago I fetched ports with portsnap, installed svnup,
> and
> > > ran it using the (just what I needed) example command in svnup(1).
>
> Just to reiterate: this is vanilla releng/9.1, no ports but docs and
> svnup installed, no X, 512MB RAM, nothing happening but occasional ARP.
> For completeness, it's running powerd and acpi_ibm but not cam.ctl.
>
> > > I get about 700KB/s here, and svnup took about 15 minutes to update
> 9.1
> > > sources to 9-stable. This is fine. Last night I ran it again, but
> it took
> > > 12:42 to make no changes. This seemed puzzling, as you'd said only a
> few
> > > minutes for subsequent updates, but the reason appears to be that in
> both
> > > cases, I ran it in script(1), and the default verbosity of 1 includes
> a
> > > listing of every directory and file examined, followed by then
> > > to eol> codes. Even in less -r (raw) mode it still has to display
> and skip
> > > through all the (now invisible) lines; bit messy.
> > >
> > > Even the second do-nothing run made a 2MB script file, the original
> with
> > > all 9.1 to -stable updates being 3.4MB. So I'd love the option to
> only
> > > list the changes (- and +) and simply ignore unchanged dirs/files
> without
> > > any display for use in script(1). Apart from that, I'm happy.
> >
> > Which mirror are you using? I ran several tests tonight repeatedly
> fetching
> > 9/stable from svn0.us-west (so they would all be do-nothing runs) both
> inside
> > and outside of a script(1) capture and on both an old SSD and on a ZFS
> > mirrored array (to see if the target media made any difference) and
> they all
> > completed in 2 minutes, 43 seconds +/- 2 seconds on my 350 KB/s
> connection.
>
> svn0.us-west.freebsd.org as per example, but it's the closest to here
> anyway; pings avg 217.5ms stddev 0.4ms, us-east avg 269.3ms sd 9.5ms.
> fastest_cvsup used to find cvs mirrors in Australia at around 55-70ms,
> so for another while I thought RTT might be the issue - but it's not.
>
> > I'll definitely put in a verbosity level that does exactly what you
> suggest.
> > Sorry about that.
>
> Not at all, and thanks! But neither, I now find, is logging the extra
> time issue; as mentioned elsewhere, running at -v0 makes next to no
> difference, 12:48 west vs 12:54 east on 4 do-nothing runs, 700KB/s link,
> not running in script(1) - though I think that matters not, or little.
>
> So I began watching top more closely, also running iostat -w10, noticing
> that svnup was pegging CPU at 100% with ~75% _system_? time for nearly 9
> minutes of the run, doing little or no disk I/O and a max of 130KB/s in,
> 22.6KB/s out, no stress there, growing to ~20MB resident, then doing a
> flurry of disk I/O at about 3-5MB/s for the last 3m40s odd. IOW, svnup
> here is mostly heavily CPU-bound, which I don't understand, but then I
> know nothing about what level of computation the protocol demands.
>
> I've done some more controlled tests since, running both iostat -w10 and
> netstat -w10 -dn before, during and until a bit after the run, attached,
> hoping these might help pinpoint any bottlenecks? This was running -v1
> default verbosity, hence the (not at all heavy) tty output numbers.
>
> Just sometimes, running slower hardware can have advantages! On your
> times, yours likely has more than 500% of my single-core P3-M's 1133MHz
> performance, which would tend to mask this issue pretty well for you :)
>
> John Baldwin has best described when c{,v}sup deletes (only in-tree)
> files, covered in section 'delete' in csup(1). I think this should
> probably be default behaviour, even if svn proper would do otherwise.
>
> cheers, Ian
>
Ian,
Have you looked with either truss or ktrace? This looks a lot like some
silliness like repeated calls to expensive system service (e.g.
gettimeofday(3) or stat(3)). If it's something like that, It can possibly
be improved a lot fairly easily. My best guess is some sort of graph of the
ports tree is going on, but that is just a wild guess as I have never
looked at or installed the code.
--
R. Kevin Oberman, Network Engineer
E-mail: rkober...@gmail.com
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: svn - but smaller?
On Sun, 17 Mar 2013 02:14:30 +1100 (EST)
Ian Smith wrote:
On Wed, 13 Mar 2013 21:11:28 -0500, John Mehr wrote:
> On Wed, 13 Mar 2013 16:29:45 +1100 (EST)
> Ian Smith wrote:
svn0.us-west.freebsd.org as per example, but it's the
closest to here
anyway; pings avg 217.5ms stddev 0.4ms, us-east avg
269.3ms sd 9.5ms.
fastest_cvsup used to find cvs mirrors in Australia at
around 55-70ms,
so for another while I thought RTT might be the issue -
but it's not.
I'm seeing ~110ms pings to svn0.us-west.freebsd.org.
> I'll definitely put in a verbosity level that does
exactly what you suggest.
> Sorry about that.
Not at all, and thanks! But neither, I now find, is
logging the extra
time issue; as mentioned elsewhere, running at -v0 makes
next to no
difference, 12:48 west vs 12:54 east on 4 do-nothing
runs, 700KB/s link,
not running in script(1) - though I think that matters
not, or little.
So I began watching top more closely, also running
iostat -w10, noticing
that svnup was pegging CPU at 100% with ~75% _system_?
time for nearly 9
minutes of the run, doing little or no disk I/O and a
max of 130KB/s in,
22.6KB/s out, no stress there, growing to ~20MB
resident, then doing a
flurry of disk I/O at about 3-5MB/s for the last 3m40s
odd. IOW, svnup
here is mostly heavily CPU-bound, which I don't
understand, but then I
know nothing about what level of computation the
protocol demands.
Except for the time duration of the run, the behavior
you're describing is normal. Using the svn protocol, the
program runs in three phases:
1) It issues a series of get-dir commands to the server to
get a list of all the files/directories in the repository
and stores the structure in memory. There is some disk
activity during this as it creates the directory tree (it
uses one lstat per directory and creates the directory if
it doesn't exist). On my development machine, this phase
takes about 25% of the total time of a do-nothing run.
2) Because the svn protocol doesn't include MD5 signatures
and the last-author/committed date/committed revision
fields in the results of the get-dir commands
although they *are* included in the http protocol
responses it has to issue a get-file request for
each file to get this information. This phase takes about
50% of the total time of a do-nothing run.
3) The downloaded MD5 signatures are for versions of the
files that do not have their $FreeBSD$ tags expanded, so
each file on the local machine must be loaded into memory
and have their $FreeBSD: ... $ tags collapsed and MD5
summed to see if there's a match. If there's no match,
the file on the server is downloaded. This phase takes up
the remaining 25% and is I/O intensive.
This three step process is further complicated because of
the huge time penalty that occurs when sending commands
one-at-a-time. To get around this, as many commands that
can fit in 32KB are grouped together and sent in bulk.
I've done some more controlled tests since, running both
iostat -w10 and
netstat -w10 -dn before, during and until a bit after
the run, attached,
hoping these might help pinpoint any bottlenecks? This
was running -v1
default verbosity, hence the (not at all heavy) tty
output numbers.
Just sometimes, running slower hardware can have
advantages! On your
times, yours likely has more than 500% of my single-core
P3-M's 1133MHz
performance, which would tend to mask this issue pretty
well for you :)
I do all of my development and test runs on a nettop with
an AMD E-350 (dual core, 1.6GHz) processor, and with the
program only utilizing one core I'd be willing to bet the
two chips are in roughly the same league. Memory and hard
drives probably aren't, and with 512MB of memory, how does
your swap utilization look?
As soon as I can get a replacement CMOS battery, I'll be
dusting off my old Athlon 2700+ system to run tests on it.
John Baldwin has best described when c{,v}sup deletes
(only in-tree)
files, covered in section 'delete' in csup(1). I think
this should
probably be default behaviour, even if svn proper would
do otherwise.
Will do. Setting up both a default of deleting only files
that appear in previous checkouts plus giving each user
the option of skipping specific directories from any
pruning is on the to-do list. Thanks!
cheers, Ian
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
