Hi,
I also concerns about this problem. And my problem is how many threads will the
qemu-system-x86 has.
From what i tested, it could between 100 to 800, yeah, maybe it has
relationship with the osd number. But it
seems affect the performance when it has many threads. From what i tested, 4k
ran
If we're talking about RBD clients (qemu) then the number also grows with
number of volumes attached to the client. With a single volume it was <1000. It
grows when there's heavy IO happening in the guest.
I had to bump up the file open limits to several thusands (8000 was it?) to
accomodate cli
We've run into issues with the number of open TCP connections from a single
client to the OSDs in our Ceph cluster.
We can (& have) increased the open file limit to work around this, but
we're looking to understand what determines the number of open connections
maintained between a client and a pa
Hi All
I'm observing some weird behavior in the amount of space ceph reports
while copying files from an rbd image in one pool to an rbd image in another.
The AVAIL number reported by 'ceph df' goes up as the copy proceeds rather
than goes down!
The output of 'ceph df' shows t
This Hammer point release fixes a critical regression in librbd that can
cause Qemu/KVM to crash when caching is enabled on images that have been
cloned.
All v0.94.4 Hammer users are strongly encouraged to upgrade.
Notable Changes
===
* librbd: potential assertion failure during ca
Another point,
As Christian mentioned, try to evaluate O_DIRECT|O_DSYNC performance of a SSD
before choosing that for Ceph..
Try to run with direct=1 and sync =1 with fio to a raw ssd drive..
Thanks & Regards
Somnath
-Original Message-
From: ceph-users [mailto:ceph-users-boun...@lists.ce
I'd imagine this is a side-effect of the fact that RBD keeps a watch/notify
handle open on the header. Watch/notify is what allows you to manipulate an
image from the rbd cli on a running image (i.e. take a snapshot, resize, etc).
Even if the OSDs permitted this action, librbd would immediatel
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256
I'm trying to remove a cache tier while I have a VM running, but a few
of the objects will not evict. Do you have to stop all RBD access to
remove a cache tier?
# ceph osd tier cache-mode ssd-pool forward
# rados -p ssd-pool cache-flush-evict-all
-
One thing, *don't* trust iostat disk util% in case of SSDs..100% doesn't mean
you are saturating SSDs there..I have seen a large performance delta even if
iostat is reporting 100% disk util in both the cases.
Also, the ceph.conf file you are using is not optimal..Try to add these..
debug_lockdep
>but because there were only two copies it had no way to tell which one was
>correct, and when I forced it to choose it often chose wrong.
Yeah. This is a BIG problem with only running with two copies. Good luck if
your pgs ever get inconsistent :)
--Lincoln
> On Oct 26, 2015, at 10:41 AM, Que
Hi Brian,
> -Original Message-
> From: ceph-users [mailto:ceph-users-boun...@lists.ceph.com] On Behalf Of
> Brian Kroth
> Sent: 23 October 2015 21:31
> To: ceph-users@lists.ceph.com
> Subject: [ceph-users] cache tier write-back upper bound?
>
> Hi, I'm wondering when using a cache pool tie
TL;DR - Running two copies in my cluster cost me a weekend, and many more
hours of productive time during normal working hours. Networking problems
can be just as destructive as disk problems. I only run 2 copies on
throwaway data.
So, I have personal experience in data loss when running only two
I am. For our workloads it works fine. The biggest trick I found is to make
sure that Nova leaves enough free RAM to not starve the OSDs. In my case,
each node is running three OSDs, so in my nova.cfg I added
"reserved_host_memory_mb = 3072" to help ensure that. Each node has 72GB of
RAM, so there'
Hello,
On Mon, 26 Oct 2015 14:35:19 +0100 Wido den Hollander wrote:
>
>
> On 26-10-15 14:29, Matteo Dacrema wrote:
> > Hi Nick,
> >
> >
> >
> > I also tried to increase iodepth but nothing has changed.
> >
> >
> >
> > With iostat I noticed that the disk is fully utilized and write per
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256
That seems to have resolved the problem. Thanks! I guess the repair
does not increase the epoc of the PG or something so it loses it's
brain about it being clean. I'm glad it was a simple resolution.
-
Robert LeBlanc
PGP Fingerprint
On 26-10-15 14:29, Matteo Dacrema wrote:
> Hi Nick,
>
>
>
> I also tried to increase iodepth but nothing has changed.
>
>
>
> With iostat I noticed that the disk is fully utilized and write per
> seconds from iostat match fio output.
>
Ceph isn't fully optimized to get the maximum poten
Hi Nick,
I also tried to increase iodepth but nothing has changed.
With iostat I noticed that the disk is fully utilized and write per seconds
from iostat match fio output.
Matteo
From: Nick Fisk [mailto:n...@fisk.me.uk]
Sent: lunedì 26 ottobre 2015 13:06
To: Matteo Dacrema ; ceph-us...@ceph.c
On Thu, Oct 22, 2015 at 2:55 PM, Burkhard Linke
wrote:
> Hi,
>
>
> On 10/22/2015 02:54 AM, Gregory Farnum wrote:
>>
>> On Sun, Oct 18, 2015 at 8:27 PM, Yan, Zheng wrote:
>>>
>>> On Sat, Oct 17, 2015 at 1:42 AM, Burkhard Linke
>>> wrote:
Hi,
I've noticed that CephFS (both ceph
Hi Matteo,
Ceph introduces latency into the write path and so what you are seeing is
typical. If you increase the iodepth of the fio test you should get higher
results though, until you start maxing out your CPU.
Nick
From: ceph-users [mailto:ceph-users-boun...@lists.ceph.com] On Behalf
Hi all,
I've recently buy two Samsung SM951 256GB nvme PCIe SSDs and built a 2 OSD ceph
cluster with min_size = 1.
I've tested them with fio ad I obtained two very different results with these
two situations with fio.
This is the command : fio --ioengine=libaio --direct=1 --name=test
--filena
Hi all,
is anyone running nova compute on ceph OSD Servers and could share his
experience?
Thanks and Regards,
Felix
Forschungszentrum Juelich GmbH
52425 Juelich
Sitz der Gesellschaft: Juelich
Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498
Vorsitzender des
On Mon, Oct 26, 2015 at 4:38 AM, Robert LeBlanc wrote:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA256
>
> I set debug_osd = 20/20 and restarted the primary osd. The logs are at
> http://162.144.87.113/files/ceph-osd.110.log.xz .
>
> The PG in question is 9.e3 and it is one of 15 that have thi
22 matches
Mail list logo
