Hello,
I have also had some good experience with Micron M510DC. The disk has
pretty solid performance scores and works good with Ceph.
P.S.: Do not forget: If you are going to use raid controller, make sure
your raid card in HBA (Non-Raid) mode.
On Thu, Feb 25, 2016 at 8:23 AM, Shinobu Kinjo
I confirm that the bug is fixed with the 0.94.6 release packages.
thank you
Saverio
2016-02-22 10:20 GMT+01:00 Saverio Proto :
> Hello Jason,
>
> from this email on ceph-dev
> http://article.gmane.org/gmane.comp.file-systems.ceph.devel/29692
>
> it looks like 0.94.6 is coming out very soon. We
Hi,
We test sync iops with fio sync=1 for database workloads in VM,
the backend is librbd and ceph (all SSD setup).
The result is sad to me. we only get ~400 IOPS sync randwrite with
iodepth=1
to iodepth=32.
But test in physical machine with fio ioengine=rbd sync=1, we can reache
~35K IO
The way LRC works is that is creates an additionnal parity chunk every l
OSD.
So with k=m=l=2, you will have 2 data chunks, 2 parity chunks and 2
locality parity chunks.
Your ruleset-failure-domain is set to "host", as well as your
ruleset-locality, so you will need 6 hosts in order to create the
Hi,
Am 24.02.2016 um 17:27 schrieb Alfredo Deza:
> On Wed, Feb 24, 2016 at 4:31 AM, Dan van der Ster wrote:
>> Thanks Sage, looking forward to some scrub randomization.
>>
>> Were binaries built for el6? http://download.ceph.com/rpm-hammer/el6/x86_64/
>
> We are no longer building binaries for e
This has happened me before but in virtual machine environment.
The VM was KVM and storage was RBD. My problem was a bad cable in network.
You should check following details:
1-) Do you use any kind of hardware raid configuration? (Raid 0, 5 or 10)
Ceph does not work well on hardware raid syste
> -Original Message-
> From: ceph-users [mailto:ceph-users-boun...@lists.ceph.com] On Behalf Of
> Huan Zhang
> Sent: 25 February 2016 11:11
> To: josh.dur...@inktank.com
> Cc: ceph-users
> Subject: [ceph-users] Guest sync write iops so poor.
>
> Hi,
>We test sync iops with fio sync=
> -Original Message-
> From: ceph-users [mailto:ceph-users-boun...@lists.ceph.com] On Behalf Of
> Jason Dillaman
> Sent: 25 February 2016 01:30
> To: Christian Balzer
> Cc: ceph-us...@ceph.com
> Subject: Re: [ceph-users] ceph hammer : rbd info/Status : operation not
> supported (95) (EC+
> On 25 Feb 2016, at 14:39, Nick Fisk wrote:
>
>
>
>> -Original Message-
>> From: ceph-users [mailto:ceph-users-boun...@lists.ceph.com] On Behalf Of
>> Huan Zhang
>> Sent: 25 February 2016 11:11
>> To: josh.dur...@inktank.com
>> Cc: ceph-users
>> Subject: [ceph-users] Guest sync write
There’s two factors really
1. Suitability for use in ceph
2. Number of people using them
For #1, there are a number of people using various different drives, so lots of
options. The blog articled linked is a good place to start.
For #2 and I think this is quite important.
On 25 Feb 2016 1:47 pm, Jan Schermer wrote:
> On 25 Feb 2016, at 14:39, Nick Fisk wrote: > > >
>> -Original Message- >> From: ceph-users
[mailto:ceph-users-boun...@lists.ceph.com] On Behalf Of >> Huan
Zhang >> Sent: 25 February 2016 11:11 >> To: josh.dur...@inktank.com
>> Cc
We are very happy with S3610s in our cluster.
We had to flash a new firmware because of latency spikes (NCQ-related), but had
zero problems after that...
Just beware of HBA compatibility, even in passthrough mode some crappy
firmwares can try and be smart about what you can do (LSI-Avago, I'm loo
Hi Dan,
If download.ceph.com doesn't support IPv6, then why is there a
record for it?
Thanks,
Andy
On 25 February 2016 at 02:21, Dan Mick wrote:
> Yes. download.ceph.com does not currently support IPv6 access.
>
> On 02/14/2016 11:53 PM, Artem Fokin wrote:
>> Hi
>>
>> It seems like downlo
> 35K IOPS with ioengine=rbd sounds like the "sync=1" option doesn't actually
> work. Or it's not touching the same object (but I wonder whether write
> ordering is preserved at that rate?).
The fio rbd engine does not support "sync=1"; however, it should support
"fsync=1" to accomplish roughly t
> > Let's start from the top. Where are you stuck with [1]? I have noticed
> > that after evicting all the objects with RBD that one object for each
> > active RBD is still left, I think this is the head object.
> Precisely.
> That came up in my extensive tests as well.
Is this in reference to the
Hi,
We have two CEPH clusters running on Dumpling 0.67.11 and some of our
"multipart objects" are incompleted. It seems that some slow requests could
cause corruption of related S3 objects. Moveover GETs for that objects are
working without any error messages. There are only HTTP 200 in logs as
Because we thought that he infrastructure did at the time. We'll get that
removed; I can see where it could cause hassles.
Sent from Nine
From: Andy Allan
Sent: Feb 25, 2016 6:11 AM
To: Dan Mick
Cc: Artem Fokin; ceph-users
Subject: Re: [ceph-users] [Ceph-maintainers] download.ceph.com has AA
After evacuated all the PGs from a node in hammer 0.94.5, I noticed that
each of the OSDs was still using ~8GB of storage. After investigating it
appears like all the data is coming from around 13,000 files in
/usr/lib/ceph/osd/ceph-*/current/meta/ with names like:
DIR_4/DIR_0/DIR_0/osdmap.303231
We've seen this as well as early as 0.94.3 and have a bug,
http://tracker.ceph.com/issues/13990 which we're working through
currently. Nothing fixed yet, still trying to nail down exactly why the
osd maps aren't being trimmed as they should.
On Thu, Feb 25, 2016 at 10:16 AM, Stillwell, Bryan <
b
I'm just trying to understand the steps each IO goes through and have been
looking at the output dump historic ops command from the admin socket.
There's a couple of steps I'm not quite sure what they mean and also
slightly puzzled by the delay and was wondering if anybody could share some
knowledg
On Thu, Feb 25, 2016 at 7:00 AM, Udo Lembke wrote:
> Hi,
>
> Am 24.02.2016 um 17:27 schrieb Alfredo Deza:
>> On Wed, Feb 24, 2016 at 4:31 AM, Dan van der Ster
>> wrote:
>>> Thanks Sage, looking forward to some scrub randomization.
>>>
>>> Were binaries built for el6? http://download.ceph.com/rpm
It's good to hear that I'm not the only one affected by this! After the
node was brought back into the cluster (I weighted it out for hardware
repairs) it appears to have removed some of the old maps, as I'm down to
8,000 now. Although I did find another OSD in the cluster which has
95,000 osd ma
Hi,
I'm trying to track shards of an EC4+2 ceph filesystem back to users and
buckets. Is there a procedure outlined somewhere for this? All I have
is a file name from an osd data pool, e.g:
default.724733.17\u\ushadow\uprostate\srnaseq\sd959d5dd-2454-4f07-b69e-9ead4a58b5f2\sUNCID\u2256596.bf4
Hi folks,
A few of us at RH are working on a project called "ceph-installer",
which is a Pecan web app that exposes endpoints for running
ceph-ansible under the hood.
The idea is that other applications will be able to consume this REST
API in order to orchestrate Ceph installations.
Another tea
Have you tried restarting each OSD one-by-one to see if that clears up the
problem?
Also, what does the output of this command look like:
ceph osd dump | grep 'replicated size'
As for whether or not 'ceph pg repair' will work, I doubt it. It uses
copy on the primary OSD to fix the other OSDs i
Hi,
am I right, that official 0.94.5 el6 was built here?
http://gitbuilder.sepia.ceph.com/gitbuilder-ceph-rpm-centos6-5-amd64-basic/log.cgi?log=9764da52395923e0b32908d83a9f7304401fee43
If yes, it seems like hammer autobuild was broken more than one month ago (11th
of Jan there is a first failed
> Just beware of HBA compatibility, even in passthrough mode some crappy
> firmwares can try and be smart about what you can do (LSI-Avago, I'm looking
> your way for crippling TRIM, seriously WTH).
This is very good to know.
Can anybody elaborate on this a bit more?
Rgds,
Shinobu
- Origin
> On 25 Feb 2016, at 22:41, Shinobu Kinjo wrote:
>
>> Just beware of HBA compatibility, even in passthrough mode some crappy
>> firmwares can try and be smart about what you can do (LSI-Avago, I'm looking
>> your way for crippling TRIM, seriously WTH).
>
> This is very good to know.
> Can any
Where should I go to get "ceph-installer" source code?
Rgds,
Shinobu
- Original Message -
From: "Ken Dreyer"
To: "ceph-devel" , "ceph-users"
Sent: Friday, February 26, 2016 6:07:54 AM
Subject: [ceph-users] "ceph-installer" in GitHub
Hi folks,
A few of us at RH are working on a projec
The code is here https://github.com/ceph/ceph-installer
Thx
YuriW
On Thu, Feb 25, 2016 at 2:57 PM, Shinobu Kinjo wrote:
> Where should I go to get "ceph-installer" source code?
>
> Rgds,
> Shinobu
>
> - Original Message -
> From: "Ken Dreyer"
> To: "ceph-devel" , "ceph-users"
>
> Sent
Thank you for the pointer.
Rgds,
Shinobu
- Original Message -
From: "Yuri Weinstein"
To: "Shinobu Kinjo"
Cc: "Ken Dreyer" , "ceph-devel"
, "ceph-users"
Sent: Friday, February 26, 2016 8:01:36 AM
Subject: Re: [ceph-users] "ceph-installer" in GitHub
The code is here https://github.com/
Thanks for your suggestion,
I have re-test my cluster and result is much better.
Regards,
Yang
-- Original --
From: "Christian Balzer";;
Date: Tue, Feb 23, 2016 09:49 PM
To: "ceph-users";
Cc: "yang";
Subject: Re: [ceph-users] Why my cluster performance is so
Hello Robert,
Thanks for the speedy reply.
On Wed, 24 Feb 2016 22:44:47 -0700 Robert LeBlanc wrote:
> We have not seen this issue, but we don't run EC pools yet (we are
> waiting for multiple layers to be available).
Yeah, that seems to be the consensus here, only EC is affected.
> We are n
Hello,
I know somebody will ask me to open a tracker issue, etc, but I feel
sufficiently frustrated to rant a bit here.
Case in point:
http://docs.ceph.com/docs/master/rados/operations/cache-tiering/
Let me start on a positive note, though.
Somebody pretty recently added the much needed detai
On Thu, 25 Feb 2016 13:44:30 - Nick Fisk wrote:
>
>
> > -Original Message-
> > From: ceph-users [mailto:ceph-users-boun...@lists.ceph.com] On Behalf
> > Of Jason Dillaman
> > Sent: 25 February 2016 01:30
> > To: Christian Balzer
> > Cc: ceph-us...@ceph.com
> > Subject: Re: [ceph-use
On Thu, 25 Feb 2016 16:41:59 -0500 (EST) Shinobu Kinjo wrote:
> > Just beware of HBA compatibility, even in passthrough mode some crappy
> > firmwares can try and be smart about what you can do (LSI-Avago, I'm
> > looking your way for crippling TRIM, seriously WTH).
>
> This is very good to know.
On Thu, 25 Feb 2016 10:07:37 -0500 (EST) Jason Dillaman wrote:
> > > Let's start from the top. Where are you stuck with [1]? I have
> > > noticed that after evicting all the objects with RBD that one object
> > > for each active RBD is still left, I think this is the head object.
> > Precisely.
>
Hello,
On Wed, 24 Feb 2016 23:01:43 -0700 Robert LeBlanc wrote:
> With my S3500 drives in my test cluster, the latest master branch gave me
> an almost 2x increase in performance compare to just a month or two ago.
> There looks to be some really nice things coming in Jewel around SSD
> performa
Hello,
On Wed, 24 Feb 2016 22:56:15 -0700 Robert LeBlanc wrote:
> We are moving to the Intel S3610, from our testing it is a good balance
> between price, performance and longevity. But as with all things, do your
> testing ahead of time. This will be our third model of SSDs for our
> cluster. T
On Fri, Feb 26, 2016 at 3:10 PM, Christian Balzer wrote:
> Then we come to a typical problem for fast evolving SW like Ceph, things
> that are not present in older versions.
I was going to post on this too (I had similar frustrations), and would
like to propose that a move to splitting the docu
Hello,
On Fri, 26 Feb 2016 15:59:51 +1100 Nigel Williams wrote:
> On Fri, Feb 26, 2016 at 3:10 PM, Christian Balzer wrote:
>
> > Then we come to a typical problem for fast evolving SW like Ceph,
> > things that are not present in older versions.
>
>
> I was going to post on this too (I had s
The docs are already split by version, although it doesn't help that
it isn't linked in an obvious manner.
http://docs.ceph.com/docs/master/rados/operations/cache-tiering/
http://docs.ceph.com/docs/hammer/rados/operations/cache-tiering/
Updating the documentation takes a lot of effort by all in
On Fri, Feb 26, 2016 at 4:09 PM, Adam Tygart wrote:
> The docs are already split by version, although it doesn't help that
> it isn't linked in an obvious manner.
>
> http://docs.ceph.com/docs/master/rados/operations/cache-tiering/
Is there any reason to keep this "master" (version-less variant)
Unfortunately, what seems to happen as users (and developers) get more
in tune with software projects, we forget what is and isn't common
knowledge.
Perhaps said "wall of text" should be a glossary of terms. A
definition list, something that can be open in one tab, and define any
ceph-specific or
Hello,
On Thu, 25 Feb 2016 23:09:52 -0600 Adam Tygart wrote:
> The docs are already split by version, although it doesn't help that
> it isn't linked in an obvious manner.
>
> http://docs.ceph.com/docs/master/rados/operations/cache-tiering/
>
> http://docs.ceph.com/docs/hammer/rados/operations
My guess would be that if you are already running hammer on the client it
is already using the new watcher API. This would be a fix on the OSDs to
allow the object to be moved because the current client is smart enough to
try again. It would be watchers per object.
Sent from a mobile device, pleas
I was only testing one SSD per node and it used 3.5-4.5 cores on my 8 core
Atom boxes. I've also set these boxes to only 4 GB of RAM to reduce the
effects of page cache. So no, I still had some headroom, but I was also
running fio on my nodes too. I don't remember how much idle I had overall,
but t
We replaced 32 S3500s with 48 Micron M600s in our production cluster. The
S3500s were only doing journals because they were too small and we still
ate 3-4% of their life in a couple of months. We started having high wait
times on the M600s so we got 6 S3610s, 6 M500dcs, and 6 500 GB M600s (they
hav
Hello,
On my crappy test cluster (Debian Jessie, Hammer 0.94.6) I'm seeing rados
bench crashing doing "seq" runs.
As I'm testing cache tiers at the moment I also tried it with a normal,
replicated pool with the same result.
After creating some benchmark objects with:
---
rados -p data bench 20
rbd engine with fsync=1 seems stuck.
Jobs: 1 (f=1): [w(1)] [0.0% done] [0KB/0KB/0KB /s] [0/0/0 iops] [eta
1244d:10h:39m:18s]
But fio using /dev/rbd0 sync=1 direct=1 ioengine=libaio iodepth=64, get
very high iops ~35K, similar to direct wirte.
I'm confused with that result, IMHO, ceph could just i
Since fio /dev/rbd0 sync=1 works well, it doesn't matter with ceph server,
just related to librbd (rbd_aio_flush) implement?
2016-02-26 14:50 GMT+08:00 Huan Zhang :
> rbd engine with fsync=1 seems stuck.
> Jobs: 1 (f=1): [w(1)] [0.0% done] [0KB/0KB/0KB /s] [0/0/0 iops] [eta
> 1244d:10h:39m:18s]
>
51 matches
Mail list logo
