2017-08-02 19:21 GMT+02:00 Enrico Olivelli <eolive...@gmail.com>: > > > Il mer 2 ago 2017, 18:51 Sijie Guo <guosi...@gmail.com> ha scritto: > >> On Wed, Aug 2, 2017 at 8:21 AM, Enrico Olivelli <eolive...@gmail.com> >> wrote: >> >> > Hi, >> > I am trying to understand because after running my benchmark (see [1]) >> the >> > overall performances slow down incredibly. The bench starts at 80,8 MB/s >> > throughput and drops to 10,2 MB/s throughput. >> > >> > I think that the problem could be the I am creating a lot of ledgers on >> a >> > single bookie. >> > It seems that the more active ledgers I have the more time it takes the >> > bookie to process internal activities, like >> > IndexInMemPageMgr#flushOneOrMoreLedgers(true). >> >> >> What is your disk layout? journal/ledger/index directory settings? >> > > All in one disk, on a tmp dir under target. > Hdd vs ssd is the same > >> >> What is your ledger storage? are you using sorted or not? >> > > Sorted > > Can you run the test? Just clone and run > Mvn test -Dtest= > BookKeeperWriteSynchClientsTest > > I am curious to know if the fact happens on other laptops >
I am comparing two different machines: machine 1: Fedora24 + ssd disk + 32 GB RAM + ext4 with mount options "defaults,relatime" machine 2: Centos6 + ssd disk + 32 GB RAM + ext4 with mount options "defaults,relatime" on machine 1 after 320 created ledgers, there is the 'slow down' situation on machine 2 there is never a "slow down" situation machines are really alike to each other, the real differences are: - SO - disk speed, on machine 2 the disk is slower and there is a avg thoughtput of 61 MB/s, on machine 1 we start from 80 MB/s and drops to 20 MB/s -- Enrico > > Enrico > >> >> >> > >> > The slow down starts after the first line in logs as: >> > org.apache.bookkeeper.bookie.EntryLogger flushRotatedLogs >> >> >> > I suspect that there is a piece of the story that I am missing. >> > >> > This behavior is the same on BK 4.4 and BK 4.5 >> >> >> > Is it possible ? >> >> >> > [1] >> > https://github.com/eolivelli/bookkeepers-benchs/blob/ >> master/src/test/java/ >> > BookKeeperWriteSynchClientsTest.java >> > >> > Thanks >> > >> > -- Enrico >> > >> > -- > > > -- Enrico Olivelli >
