Am 26.10.2006 um 15:47 schrieb Stefan Bethke:

Am 26.10.2006 um 15:37 schrieb Robert Watson:

On Thu, 26 Oct 2006, Stefan Bethke wrote:

          acpica         3024          159K     20026966
...
db> show uma
            Zone       Allocs        Frees         Used    Cache
              64      9990754      9986054         4700  9980755

Looks like acpica has gone crazy performing allocation/freeing at a very high rate, and that for some reason, UMA is failing to properly reuse/release memory. So there are two bugs/problems here: whatever is causing ACPI to behave this way, and then the fact that UMA is failing to deal properly with its misbehavior.

We had the machines running with ACPI disabled for a week or so, and we were still getting these panics, but I'll disable it again in the BIOS to make sure.

Alternatively, that we have a bug in the way statistics are handled. If you can generate a coredump, it would be quite useful to be able to run umstat (src/tools/tools/umastat in HEAD) on it. The tool probably needs a bit of tweaking to run on the core dump -- in particular, the first and second arguments of kvm_open() need to be the name of the kernel and dumpfile, rather than NULL. This would help confirm what actual state UMA is in.

So far, the machines always just hang instead of dumping core; I'll see if I can get them to write a dump.

On two of the three boxes, the panics have not happened for a week.

On the one I disabled ACPI on, no panics were seen for a week as well, so I enabled it again. Within an hour, the panic occurred. call doadump just hangs, so I won't be able to run umastat. I'll try to kick the box via the IPMI card and wait for the next panic.


Fri Nov  3 23:22:58 CET 2006

FreeBSD/i386 (discovery.tallence.de) (ttyd0)

login:         -0255: *** Error: UtCallocate: Could not allocate size 30
        -0255: *** Error: UtCallocate: Could not allocate size 30
        -0255: *** Error: UtCallocate: Could not allocate size 30
ACPI-1304: *** Error: Method execution failed [\SWFS] (Node 0xc63ee220), AE_NO_MEMORY ACPI-1304: *** Error: Method execution failed [\RBYT] (Node 0xc63ee1a0), AE_NO_MEMORY ACPI-1304: *** Error: Method execution failed [\RTMP] (Node 0xc63ee380), AE_NO_MEMORY ACPI-1304: *** Error: Method execution failed [\_TZ_.THRM._TMP] (Node 0xc63ede00), AE_NO_MEMORY
panic: kmem_malloc(4096): kmem_map too small: 699756544 total allocated
KDB: enter: panic
[thread pid 1119 tid 100074 ]
Stopped at      kdb_enter+0x30: leave
db> show uma
              Zone       Allocs        Frees         Used    Cache
            pfosfp          188            0          188      193
          pfospfen          345            0          345       51
         pfiaddrpl            0            0            0        0
      pfstatescrub            0            0            0        0
          pffrcent            0            0            0        0
         pffrcache            0            0            0        0
            pffrag            0            0            0        0
           pffrent            0            0            0        0
        pfrkentry2            0            0            0        0
         pfrkentry            0            0            0        0
         pfrktable            0            0            0        0
      pfpooladdrpl            0            0            0        0
          pfaltqpl            0            0            0        0
         pfstatepl          873          844           29       76
          pfrulepl           62            0           62       10
         pfsrctrpl            1            0            1       77
       FFS2 dinode         2117          193         1924       11
       FFS1 dinode            0            0            0        0
         FFS inode         2117          193         1924       48
       Mountpoints           19            0           19        5
          SWAPMETA            0            0            0        0
           rtentry           10            0           10       48
             unpcb          664          613           51       33
             ripcb            0            0            0        0
          sackhole            0            0            0        0
          tcpreass            9            9            0      169
         hostcache           39            0           39       61
          syncache          150          150            0       78
             tcptw           55           55            0      156
             tcpcb          229          198           31       25
             inpcb          229          198           31       35
             udpcb          979          959           20       24
               ipq            0            0            0        0
            socket         1873         1771          102        8
             KNOTE         1242         1242            0      112
              PIPE         1033         1022           11       16
           NFSNODE            0            0            0        0
          NFSMOUNT            0            0            0        0
           DIRHASH           91            0           91        5
             NAMEI        42625        42625            0       20
       L VFS Cache            0            0            0        0
       S VFS Cache         6029         4101         1928       32
         VNODEPOLL            0            0            0        0
             VNODE         2211          199         2012        4
     ata_composite            0            0            0        0
       ata_request        12690        12690            0      380
             g_bio        47901        47901            0     1247
      ACL UMA zone            0            0            0        0
    mbuf_jumbo_16k            0            0            0        0
     mbuf_jumbo_9k            0            0            0        0
mbuf_jumbo_pagesize            0            0            0        0
      mbuf_cluster          128            0          128        6
              mbuf         8800         8772           28      114
       mbuf_packet        47880        47812           68       60
           VMSPACE         2384         2317           67       37
            UPCALL            6            0            6      150
            KSEGRP          129            0          129       71
            THREAD          216           81          135       15
              PROC         2424         2317          107       19
             Files        17415        17156          259       59
              4096         5808         5666          142       25
              2048          554          185          369        5
              1024         2266         2218           48      144
               512         3396         3339           57       15
               256         6678         6240          438       42
               128        24937        22580         2357       73
                64      9966167      9961335         4832  9947229
                32         5515         2447         3068       96
                16     10000705      9997868         2837      208
           mt_zone          163            0          163       73
         DP fakepg            0            0            0        0
          PV ENTRY      1242630      1213645        28985     1320
         MAP ENTRY        88676        86884         1792        0
        KMAP ENTRY       171204       171178           26      142
               MAP            7            0            7       33
         VM OBJECT        45712        43104         2608      321
        128 Bucket           58            2           56        0
         64 Bucket           49            1           48        8
         32 Bucket           31            0           31       25
         16 Bucket           30            0           30       20
          UMA Hash            6            1            5       25
     UMA RCntSlabs           67            0           67        7
         UMA Slabs          669          193          476       55
         UMA Zones           83            0           83        7
          UMA Kegs           83            0           83       13
db> show malloc
              Type        InUse        MemUse     Requests
      NULLFS mount            8            1K            8
       NULLFS hash            1            1K            1
       NULLFS node            8            1K            8
             pflog            1            1K            1
             pf_if           26            6K           26
        MADT Table            0            0K            0
           acpipwr            0            0K            0
         acpi_perf            0            0K            0
           acpidev           93            3K           93
           acpisem           17            2K           17
         acpicmbat            0            0K            0
          PCI Link           64            6K           64
          acpitask            0            0K            2
            acpica         3024          159K     19943808
            KTRACE          100           13K          100
            prison            4            8K            4
              $PIR            0            0K            0
            DEVFS3          562           71K          563
          nexusdev            3            1K            3
          MP Table            0            0K            0
           memdesc            1            4K            1
         legacydrv            0            0K            0
           ithread           66            6K           66
          I/O APIC            1            1K            1
            zombie            0            0K         2317
         proc-args           52            4K         5377
            kqueue            0            0K         1234
              kenv          113            8K          114
          atkbddev            2            1K            2
             sigio            1            1K            1
file desc to leader            0            0K            0
         VM pgdata            2           65K            2
         file desc          108           27K         2425
            DEVFS2          100            4K          400
             USBHC            0            0K            0
              cdev           20            3K           20
            USBdev            3            1K            9
           UMAHash            1            1K            2
         UFS mount           18           82K           18
         UFS quota            0            0K            0
       UFS dirhash           87           16K           87
          savedino            0            0K            0
         newdirblk            0            0K            0
            dirrem            0            0K            0
             mkdir            0            0K            0
            diradd            0            0K            0
          freefile            0            0K            0
          freeblks            0            0K            0
          freefrag            0            0K            0
        allocindir            0            0K            0
          indirdep            0            0K            0
       allocdirect            0            0K            0
         bmsafemap            0            0K            0
            newblk            1            1K            1
          inodedep            1          256K            1
           pagedep            1           64K            1
           rpcclnt            0            0K            0
          p1003.1b            1            1K            1
               agp            0            0K            0
        NFS daemon            5           10K            5
     NFSV3 srvdesc            0            0K            0
       NFS srvsock            1            1K            1
           nlminfo            0            0K            0
          NFS lock            0            0K            0
      NFS DirectIO            0            0K            0
          NFS hash            0            0K            0
      NFSV3 diroff            0            0K            0
       NFSV3 bigfh            0            0K            0
           NFS req            0            0K            0
       NFS srvsock            0            0K            0
             idmap            0            0K            0
          NFS4 dev            0            0K            0
          syncache            1            8K            1
               USB           31            3K           31
         hostcache            1           24K            1
       ip_moptions            0            0K            0
       Export Host            0            0K            0
          in_multi            3            1K            3
              igmp            0            0K            0
          routetbl           20            2K           93
           entropy         1024           64K         1024
         ad_driver            3            1K            3
       ata_generic            4            4K            4
              vlan            2            1K            2
               tun            0            0K            0
                lo            1            1K            1
            arpcom            3            1K            3
             clone            3           12K            3
       ether_multi           12            1K           14
            ifaddr           30            7K           30
             ifnet            6            6K            6
               BPF            9           65K           10
        acd_driver            1            2K            1
       Export Host            0            0K            0
       vnodemarker            0            0K         2420
             mount          223            6K          314
            vnodes            1            1K            1
          VFS hash            1          256K            1
       Export Host            0            0K            0
cluster_save buffer            0            0K            6
          vfscache            1          512K            1
        BIO buffer          224          448K          235
            isadev           19            2K           19
               pcb           21            5K          703
            soname           13            1K         3568
          mbuf_tag            0            0K            2
          mbextcnt            0            0K            0
              accf            0            0K            0
              ptys            0            0K            0
              ttys         1177          165K         2648
               shm            1           12K            1
               sem            4            7K            4
               msg            4           25K            4
               iov            0            0K          828
            select            0            0K            0
          ioctlops            0            0K         2054
            Unitno            6            1K          178
        turnstiles          151           10K          151
         taskqueue            9            1K            9
             stack            0            0K            0
        MD sectors            0            0K            0
           MD disk            0            0K            0
      sleep queues          151            5K          151
              sbuf            0            0K          906
              rman          176           11K          544
               LED            0            0K            0
            kbdmux            6            9K            6
           ata_pci            0            0K            0
              kobj          115          230K          134
      eventhandler           55            4K           55
           devstat           14           29K           14
       mirror_data            3            1K          625
            bus-sc           82           32K         1841
               bus          799           38K         4368
              SWAP            2          549K            2
              umtx          150           10K          150
         sysctltmp            0            0K          333
         sysctloid         3214           97K         3214
            sysctl            0            0K          634
           uidinfo            6            2K          141
            plimit           25            7K         1334
           ata_dma            6            1K            6
              cred           50            7K        12035
           subproc          235          464K         2552
              proc            2            8K            2
           session           46            6K          223
              pgrp           46            3K          223
              GEOM          156           17K          832
          mtx_pool            1            8K            1
            module          183           12K          183
       pfs_vncache            0            0K            0
              free            0            0K            0
         pfs_nodes            0            0K            0
            ip6ndp            0            0K            0
            ip6opt            0            0K            0
              temp           13          211K         6822
            devbuf         1562         3592K         1564
             cache            0            0K            0
             lockf           21            2K         6581
             DEVFS           62            2K           63
            linker           50          243K          105
        DEVFS_RULE            0            0K            0
            DEVFS1          100           25K          102
db> call doadump



--
Stefan Bethke <[EMAIL PROTECTED]>   Fon +49 170 346 0140


_______________________________________________
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Reply via email to