Just from memory, I think these values are only used for the initial pool
creations when the cluster is first set up.
We have been moving for a while to making users specify pg_num explicitly
on every pool create, and you should do so. :)
-Greg
On Mon, Jan 26, 2015 at 7:38 AM Jason Anderson <
jaso
t; is out of date? create_pool in rados.py
> (https://github.com/ceph/ceph/blob/master/src/pybind/rados.py#L535) only
> requires a pool_name….doesn’t even offer pg_num as an optional argument.
>
>
>
> Thank you,
>
> -Jason
>
>
>
> From: Gregory Farnum [mailto:g...@gregs4
On Mon, Jan 26, 2015 at 2:13 PM, Brian Rak wrote:
> I have an existing cluster where all the hosts were just added directly, for
> example:
>
> # ceph osd tree
> # idweight type name up/down reweight
> -1 60.06 root default
> ...
> -14 1.82host OSD75
> 12 1.8
On Mon, Jan 26, 2015 at 6:47 PM, Kim Vandry wrote:
> Hello Ceph users,
>
> In our application, we found that we have a use case for appending to a
> rados object in such a way that the client knows afterwards at what offset
> the append happened, even while there may be other concurrent clients do
On Wed, Jan 28, 2015 at 5:24 AM, John Spray wrote:
> We don't implement the GETFLAGS and SETFLAGS ioctls used for +i.
>
> Adding the ioctls is pretty easy, but then we need somewhere to put
> the flags. Currently we don't store a "flags" attribute on inodes,
> but maybe we could borrow the high b
Armstrong
wrote:
> Hey folks,
>
> Any update on this fix getting merged? We suspect other crashes based on
> this bug.
>
> Thanks,
>
> Chris
>
> On Tue, Jan 13, 2015 at 7:09 AM, Gregory Farnum wrote:
>>
>> Awesome, thanks for the bug report and the fix, guys
On Wed, Jan 28, 2015 at 10:06 AM, Sage Weil wrote:
> On Wed, 28 Jan 2015, John Spray wrote:
>> On Wed, Jan 28, 2015 at 5:23 PM, Gregory Farnum wrote:
>> > My concern is whether we as the FS are responsible for doing anything
>> > more than storing and returning t
I don't think there's any way to force the OSDs to do that. What
exactly are you trying to do?
-Greg
On Fri, Jan 30, 2015 at 4:02 AM, Mudit Verma wrote:
> Hi All,
>
> We are working on a project where we are planning to use Ceph as storage.
> However, for one experiment we are required to disable
On Sun, Dec 8, 2013 at 7:16 AM, Oliver Schulz wrote:
> Hello Ceph-Gurus,
>
> a short while ago I reported some trouble we had with our cluster
> suddenly going into a state of "blocked requests".
>
> We did a few tests, and we can reproduce the problem:
> During / after deleting of a substantial c
On Mon, Dec 9, 2013 at 1:17 AM, Robert van Leeuwen
wrote:
>> your client writes the file to one osd, and before this osd acknowledges
>> your write request,
>> it ensure that it is copied to other osd(s).
>
> I think this behaviour depends on how you configure you POOL:
>
> osd pool default min s
On Wed, Dec 4, 2013 at 7:15 AM, Mr.Salvatore Rapisarda
wrote:
> Hi,
>
> i have a ceph cluster with 3 nodes on Ubuntu 12.04.3 LTS and ceph version
> 0.72.1
>
> My configuration is the follow:
>
> * 3 MON
> - XRVCLNOSTK001=10.170.0.110
> - XRVCLNOSTK002=10.170.0.111
> - XRVOSTKMNG001=10.170.0.
[ Re-added the list since I don't have log files. ;) ]
On Mon, Dec 9, 2013 at 5:52 AM, Oliver Schulz wrote:
> Hi Greg,
>
> I'll send this privately, maybe better not to post log-files, etc.
> to the list. :-)
>
>
>> Nobody's reported it before, but I think the CephFS MDS is sending out
>> too man
On Wed, Dec 11, 2013 at 6:13 PM, Sherry Shahbazi wrote:
>
> Hi all,
>
> I was wondering if u could answer my question regarding cache pool:
> If I have got two servers with 1 SSD in front of each of them, what CRUSH
> map would be like?
>
> For example:
> If I have defined the following CRUSH map:
On Mon, Dec 16, 2013 at 11:08 AM, Dan van der Ster
wrote:
> Hi,
>
> Sorry to revive this old thread, but I wanted to update you on the current
> pains we're going through related to clients' nproc (and now nofile)
> ulimits. When I started this thread we were using RBD for Glance images
> only, bu
On Mon, Dec 16, 2013 at 4:35 AM, Gandalf Corvotempesta
wrote:
> 2013/11/7 Kyle Bader :
>> Ceph handles it's own logs vs using syslog so I think your going to have to
>> write to tmpfs and have a logger ship it somewhere else quickly. I have a
>> feeling Ceph logs will eat a USB device alive, espec
On Thu, Dec 12, 2013 at 10:58 PM, Jeppesen, Nelson
wrote:
> I have an issue with incomplete pgs, I’ve tried repairing it but no such
> luck. Any ideas what to check?
Have you looked at
http://ceph.com/docs/master/rados/troubleshooting/troubleshooting-pg?
In particular, what's the output of "ceph
On Thu, Dec 19, 2013 at 12:39 AM, Christian Balzer wrote:
>
> Hello,
>
> In my "Sanity check" thread I postulated yesterday that to get the same
> redundancy and resilience for disk failures (excluding other factors) as
> my proposed setup (2 nodes, 2x 11 3TB HDs RAID6 per node, 2
> global hotspar
"mon initial members" is a race prevention mechanism whose purpose is
to prevent your monitors from forming separate quorums when they're
brought up by automated software provisioning systems (by not allowing
monitors to form a quorum unless everybody in the list is a member).
If you want to add ot
t;> -----Original Message-
>> From: Gregory Farnum [mailto:g...@inktank.com]
>> Sent: Thursday, December 19, 2013 10:19 PM
>> To: Don Talton (dotalton)
>> Cc: ceph-users@lists.ceph.com
>> Subject: Re: [ceph-users] ceph-deploy issues with initial mons that are
They need to be able to send messages to each other, but routing can work
however you like.
On Sunday, January 5, 2014, James Harper wrote:
> Is there any requirement that the monitors have to be on the same subnet
> as each other, and/or the osd public network? It's going to simplify things
> gr
On Mon, Dec 30, 2013 at 11:14 PM, Kuo Hugo wrote:
>
> Hi all,
>
> I have several question about osd scrub.
>
> Does the scrub job run in the background automatically? Is it working
> periodically ?
Yes, the OSDs will periodically scrub the PGs they host based on load
and the min/max scrub interv
On Fri, Jan 3, 2014 at 2:02 AM, Andrei Mikhailovsky wrote:
> Hi guys,
>
> Could someone explain what's the new perf stats show and if the numbers are
> reasonable on my cluster?
>
> I am concerned about the high fs_commit_latency, which seems to be above
> 150ms for all osds. I've tried to find th
I have a vague memory of this being something that happened in an
outdated version of the ceph tool. Are you running an older binary on
the node in question?
-Greg
Software Engineer #42 @ http://inktank.com | http://ceph.com
On Sat, Jan 4, 2014 at 4:34 PM, Zeb Palmer wrote:
> I have a small ceph
On Thu, Jan 2, 2014 at 2:18 PM, Alex Pearson wrote:
> Hi All,
> Victory! Found the issue, it was a mistake on my part, however it does raise
> another questions...
>
> The issue was:
> root@osh1:~# ceph --cluster apics auth list
> installed auth entries:
>
> client.cuckoo
> key: AQBjTbl
On Wed, Dec 25, 2013 at 6:13 PM, vernon1...@126.com wrote:
> Hello, my Mon's always HEALTH_WARN, and I run ceph health detail, it show
> me like this:
>
> HEALTH_WARN
> mon.2 addr 192.168.0.7:6789/0 has 30% avail disk space -- low disk space!
>
> I want to know how to set this warning level? I ha
[Hrm, this email was in my spam folder.]
At a quick glance, you're probably running into some issues because
you've got two racks of very different weights. Things will probably
get better if you enable the optimal "crush tunables"; check out the
docs on that and see if you can switch to them.
-Gr
On Thu, Jan 2, 2014 at 12:40 AM, Ray Lv wrote:
> Hi there,
>
> Noted that there is a Blueprint item about S3 object versioning in radosgw
> for Firefly at
> http://wiki.ceph.com/Planning/Blueprints/Firefly/rgw%3A_object_versioning
> And Sage has announced v0.74 release for Firefly. Do you guys kno
Assuming the one who lost its filesystem is totally gone, mark it
lost. That will tell the OSDs to give up on whatever data it might
have had and you should be good to go (modulo whatever data you might
have lost from only having it on the dead OSD during the reboot).
-Greg
Software Engineer #42 @
Oh, sorry, you did do that. Hrm.
What osdmap epoch did your lost node (0, I assume) disappear in? What
version of Ceph are you running? That pg stat isn't making a lot of
sense to me.
Software Engineer #42 @ http://inktank.com | http://ceph.com
On Tue, Jan 7, 2014 at 2:45 PM, Gregory F
On Thu, Jan 9, 2014 at 6:27 AM, Dan Van Der Ster
wrote:
> Here’s a more direct question. Given this osd tree:
>
> # ceph osd tree |head
> # idweight type name up/down reweight
> -1 2952root default
> -2 2952room 0513-R-0050
> -3 262.1 ra
Yep!
On Thu, Jan 9, 2014 at 11:01 AM, Dan Van Der Ster
wrote:
> Thanks Greg. One thought I had is that I might try just crush rm'ing the OSD
> instead of or just after marking it out... That should avoid the double
> rebalance, right?
>
> Cheers, Dan
>
> On Jan 9, 20
On Tuesday, January 14, 2014, Christian Balzer wrote:
>
> Hello,
>
> In http://ceph.com/docs/next/rbd/rbd-config-ref/ it is said that:
>
> "The kernel driver for Ceph block devices can use the Linux page cache to
> improve performance."
>
> Is there anywhere that provides more details about this?
Hrm, at first glance that looks like the on-disk state got corrupted
somehow. If it's only one OSD which has this issue, I'd turn it off
and mark it out. Then if the cluster recovers properly, wipe it and
put it back in as a new OSD.
-Greg
On Wed, Jan 15, 2014 at 1:49 AM, Rottmann, Jonas (centron
On Friday, January 17, 2014, Iban Cabrillo wrote:
> Dear,
> we are studying the possibility to migrate our FS in the next year to
> cephfs. I know that it is not prepare for production environments yet, but
> we are planning to play with it in the next months deploying a basic
> testbed.
> Re
Almost! The primary OSD sends out the data to its replicas simultaneously
with putting it into the journal.
-Greg
On Monday, January 20, 2014, Tim Zhang wrote:
> Hi guys,
> I wonder how does store objects. Consider the writing obj process, IMO,
> osd first get obj data from client, then the pri
On Wed, Jan 22, 2014 at 8:04 AM, Dan Ryder (daryder) wrote:
>
> Hello,
>
>
>
> I am wondering if there is any detailed documentation for obtaining I/O
> statistics for a Ceph cluster.
>
> The important metrics I’m looking for are: the number of operations, size of
> operations, and latency of op
On Wed, Jan 22, 2014 at 8:35 AM, zorg wrote:
> Hi,
> After reading the thread
> http://lists.ceph.com/pipermail/ceph-users-ceph.com/2013-June/002358.html
>
> We have done this crush map to make thing work.
> srv1 and srv1ssd are the same physical server (same srv2,3,4)
> we split it in the crush t
On Wed, Jan 22, 2014 at 9:13 AM, Caius Howcroft
wrote:
> Hi all,
>
>
> I want to double the number of pgs available for a pool, however I
> want to reduce as much as possible the resulting I/O storm (I have
> quite a bit of data in these pools).
>
> What is the best way of doing this? Is it using
On Sun, Jan 19, 2014 at 9:00 PM, Sherry Shahbazi wrote:
> Hi all,
>
> I have three pools, which I want to mount Pool 0 with CephFS.
> When I try to set the layout by changing the pool to to 0 (cephfs
> /mnt/oruafs/pool0/ set_layout -p 0), it would not be set to pool 0 while I
> am able to set it t
[Re-added the list.]
On Wed, Jan 22, 2014 at 4:15 AM, Tim Zhang wrote:
> Hi Gregory,
> I find in the src code:ReplicatedPG.cc in the function do_pg_op() I see this
> two flag:
> CEPH_OSD_FLAG_ACK and CEPH_OSD_FLAG_ONDISK,
> what's the differenrence between this two flag?
In some circumstances (w
On Tue, Jan 21, 2014 at 8:26 AM, Sylvain Munaut
wrote:
> Hi,
>
> I noticed in the documentation that the OSD should use 3 ports per OSD
> daemon running and so when I setup the cluster, I originally opened
> enough port to accomodate this (with a small margin so that restart
> could proceed even i
On Wed, Jan 22, 2014 at 3:23 AM, Dmitry Lysenko wrote:
> Good day.
>
> Some time ago i change pg_num like this
> http://www.sebastien-han.fr/blog/2013/03/12/ceph-change-pg-number-on-the-fly/:
>
> ceph osd pool create one-new 500
> rados cppool one one-new
Unfortunately, this command is not copyi
On Wed, Jan 22, 2014 at 3:50 PM, bf wrote:
>
>
> Gregory Farnum writes:
>
>>
>> On Wed, Jan 22, 2014 at 9:13 AM, Caius Howcroft
>> > I want to double the number of pgs available for a pool, however I
>> > want to reduce as much as possible the resulting
On Wed, Jan 22, 2014 at 3:23 PM, Karol Kozubal wrote:
> Hi Everyone,
>
> I have a few questions concerning mounting cephfs with ceph-fuse in fstab at
> boot. I am currently successfully mounting cephfs using ceph-fuse on 6
> clients. I use the following command, where the ip is my mon address:
>
On Wed, Jan 22, 2014 at 10:43 PM, Schlacta, Christ wrote:
> can ceph handle a configuration where a custer node is not "always on", but
> rather gets booted periodically to sync to the cluster, and is also
> sometimes up full time as demand requires? I ask because I want to put an
> OSD on each o
On Thu, Jan 23, 2014 at 3:35 AM, bf wrote:
> Gregory Farnum writes:
>
>
>> Yes, Ceph does all the heavy lifting. Multiple PGs with the same OSDs
>> can happen (eg, if you only have two OSDs, all PGs will be on both),
>> but it behaves about as well as is possible with
[ Returning list to thread. ]
On Wed, Jan 22, 2014 at 11:37 PM, Dmitry Lysenko wrote:
> 22.01.2014 13:01, Gregory Farnum пишет:
>
>
>> On Wed, Jan 22, 2014 at 3:23 AM, Dmitry Lysenko wrote:
>> > Good day.
>> >
>> > Some time ago i change pg_num like t
On Thu, Jan 23, 2014 at 8:07 AM, Arne Wiebalck wrote:
>
> On Jan 23, 2014, at 4:18 PM, Gregory Farnum
> wrote:
>
>> On Wed, Jan 22, 2014 at 3:23 PM, Karol Kozubal
>> wrote:
>>> Hi Everyone,
>>>
>>> I have a few questions concerning mounti
On Thu, Jan 23, 2014 at 2:21 PM, Schlacta, Christ wrote:
> What guarantees does ceph place on data integrity? Zfs uses a Merkel tree to
> guarantee the integrity of all data and metadata on disk and will ultimately
> refuse to return "duff" data to an end user consumer.
>
> I know ceph provides so
On Thu, Jan 23, 2014 at 5:24 PM, Stuart Longland wrote:
> Hi all,
>
> I'm in the process of setting up a storage cluster for production use.
> At the moment I have it in development and am testing the robustness of
> the cluster. One key thing I'm conscious of is single points of
> failure. Thus
're giving blood.”
>
> Phone: +33 (0)1 49 70 99 72
> Mail: sebastien@enovance.com
> Address : 10, rue de la Victoire - 75009 Paris
> Web : www.enovance.com - Twitter : @enovance
>
> On 22 Jan 2014, at 19:01, Gregory Farnum >
> wrote:
>
> > On Tue, Jan 21
Looks like you got lost over the Christmas holidays; sorry!
I'm not an expert on running rgw but it sounds like garbage collection
isn't running or something. What version are you on, and have you done
anything to set it up?
-Greg
Software Engineer #42 @ http://inktank.com | http://ceph.com
On Su
These aren't really consumerized yet, so you pretty much have to
google and see if somebody's already discussed them or go through the
code. Not sure where they are on the priority list for docs.
-Greg
Software Engineer #42 @ http://inktank.com | http://ceph.com
On Sun, Jan 26, 2014 at 7:34 PM, D
This isn't a topic I know a ton about, but:
It is not possible to boot from CephFS, but will be soon (search for
"[PATCH 1/4] init: Add a new root device option, the Ceph file
system").
I think it is possible to boot from rbd (there is native kernel
support for it as a block device, for starters),
On Mon, Jan 27, 2014 at 9:05 PM, Stuart Longland wrote:
> On 25/01/14 16:41, Stuart Longland wrote:
>> Hi Gregory,
>> On 24/01/14 12:20, Gregory Farnum wrote:
>>> Did the cluster actually detect the node as down? (You could check
>>> this by looking at the ceph
On Tue, Jan 28, 2014 at 6:43 PM, Stuart Longland wrote:
> Hi Gregory,
> On 28/01/14 15:51, Gregory Farnum wrote:
>>> I do note ntp doesn't seem to be doing its job, but that's a side issue.
>> Actually, that could be it. If you take down one of the monitors and
&g
On Mon, Feb 3, 2014 at 10:43 AM, Craig Lewis wrote:
> I've been noticing somethings strange with my RGW federation. I added some
> statistics to radosgw-agent to try and get some insight
> (https://github.com/ceph/radosgw-agent/pull/7), but that just showed me that
> I don't understand how replic
On Tue, Feb 4, 2014 at 9:29 AM, Gruher, Joseph R
wrote:
>
>
>>-Original Message-
>>From: ceph-users-boun...@lists.ceph.com [mailto:ceph-users-
>>boun...@lists.ceph.com] On Behalf Of Mark Nelson
>>Sent: Monday, February 03, 2014 6:48 PM
>>To: ceph-users@lists.ceph.com
>>Subject: Re: [ceph-u
ceph-syn is a synthetic client against the full CephFS distributed
filesystem. It's really more of a developer tool than even a usable
benchmark at this time.
-Greg
Software Engineer #42 @ http://inktank.com | http://ceph.com
On Wed, Feb 5, 2014 at 10:39 AM, Sahana wrote:
> Hi ,
>
> I have few q
There are a few more options that are in current dev releases for
directing traffic to replicas, but it remains pretty specialized and
probably won't be supported past the direct librados client layer for
Firefly (unless somebody's prioritized it for RGW or RBD that I
haven't heard about).
-Greg
So
On Tuesday, February 11, 2014, Hyangtack Lee wrote:
> I'm new to Ceph, and looking for a new storage to replace legacy system.
>
> My system has a lot of files accessing temporarily for 2 or 3 days.
> Those files are uploaded from many clients everyday, and batch job deletes
> unused files everyd
This is harder than it sounds, but you can get a broad picture by looking
for all the instances of the client request string (I believe it's the one
from the incoming message, but I'm not at a computer to check). It outputs
all the key points with that tag, and you can look at what's going on in
th
If you look at http://ceph.com/docs/master/install/install-ceph-gateway,
the fcgi package you have installed is labeled as "W/Out 100
Continue". You need to add a separate "gitbuilder" repository if you
want it with 100-continue (supporting chunked transfers); it's all
described on that page.
-Greg
Check out
http://ceph.com/docs/master/rados/operations/placement-groups/#get-statistics-for-stuck-pgs
and http://ceph.com/docs/master/rados/troubleshooting/troubleshooting-pg/.
What does the dump of the PG say is going on?
-Greg
Software Engineer #42 @ http://inktank.com | http://ceph.com
On Sun
Did you maybe upgrade that box to v0.67.6? This sounds like one of the
bugs Sage mentioned in it.
-Greg
Software Engineer #42 @ http://inktank.com | http://ceph.com
On Sun, Feb 16, 2014 at 4:23 AM, Wido den Hollander wrote:
> Hi,
>
> Yesterday I got a notification that a RGW setup was having iss
e
able to see the big points of progress. But really, to read these logs
you need to spend the time to understand them; there's no quick fix
for it.
-Greg
Software Engineer #42 @ http://inktank.com | http://ceph.com
>
> Thank you,
> Mihaly
>
> 2014-02-12 17:28 GMT+01:00 Gregor
On Wed, Feb 19, 2014 at 1:31 PM, mike smith
wrote:
> I am trying to learn about Ceph and have been looking at the documentation
> and speaking to colleagues who work with it and had a question that I could
> not get the answer to. As I understand it, the Crush map is updated every
> time a disk i
On Wed, Feb 19, 2014 at 2:50 AM, Dane Elwell wrote:
> Hello list,
>
> We've recently moved our 588TB Ceph cluster into production by moving
> VMs on there, but this morning we started receiving the following
> message:
>
> cluster etc
> health HEALTH_WARN 20 requests are blocked >
Arne,
Sorry this got dropped -- I had it marked in my mail but didn't have
the chance to think about it seriously when you sent it. Does this
still happen after the updatedb config change you guys made recently?
-Greg
Software Engineer #42 @ http://inktank.com | http://ceph.com
On Fri, Jan 31, 20
On Tue, Feb 18, 2014 at 7:24 AM, Guang Yang wrote:
> Hi ceph-users,
> We are using Ceph (radosgw) to store user generated images, as GET latency
> is critical for us, most recently I did some investigation over the GET path
> to understand where time spend.
>
> I first confirmed that the latency c
I created a ticket: http://tracker.ceph.com/issues/7487
But my guess is that this is a result of having 0 CRUSH weight for the
entire tree while linking them up. Can you give the OSD a weight and
see if it works after that?
-Greg
Software Engineer #42 @ http://inktank.com | http://ceph.com
On Tu
On Mon, Feb 17, 2014 at 12:29 AM, Alex Pearson wrote:
> Hi All,
> I've been looking, but haven't been able to find any detailed documentation
> about the journal usage on OSDs. Does anyone have any detailed docs they
> could share? My initial questions are:
Hmm, I'm not sure if we have any se
On Thu, Jan 30, 2014 at 1:17 PM, Derek Yarnell wrote:
> Hi,
>
> So I am trying to remove OSDs from one of our 6 ceph OSDs, this is a
> brand new cluster and no data is yet on it. I was following the manual
> procedure[1] with the following script. I removed OSDs 0-3 but I am
> seeing ceph not fu
On Thu, Feb 20, 2014 at 7:52 AM, Pavel V. Kaygorodov wrote:
> Hi!
>
> May be it is a dumb question, but anyway:
>
> If I lose all monitors (mon data dirs), does it possible to recover cluster
> with data from OSDs only?
Short version: no.
Long version: probably, if you're willing to spend lots o
[Re-added the list]
On Thu, Feb 20, 2014 at 8:09 AM, Pavel V. Kaygorodov wrote:
> Hi!
>
>> I created a ticket: http://tracker.ceph.com/issues/7487
>>
>> But my guess is that this is a result of having 0 CRUSH weight for the
>> entire tree while linking them up. Can you give the OSD a weight and
>
On Thu, Feb 20, 2014 at 4:26 AM, Michael wrote:
> Hi All,
>
> Have a log full of -
>
> "log [ERR] : 1.9 log bound mismatch, info (46784'1236417,46797'1239418]
> actual [46784'1235968,46797'1239418]"
Do you mean that error message is showing up for a lot of different
PGs? The specific error indica
On Tue, Feb 18, 2014 at 8:21 AM, Dan van der Ster
wrote:
> Hi,
> Today I've noticed an interesting result of not have hashpspool
> enabled on a number of pools -- backfilling is delayed.
>
> Take for example the following case: a PG from each of 5 different
> pools (details below) are all mapped t
On Thu, Feb 20, 2014 at 12:48 PM, Dan van der Ster
wrote:
> Hi,
>
> On Thu, Feb 20, 2014 at 7:47 PM, Gregory Farnum wrote:
>>
>> On Tue, Feb 18, 2014 at 8:21 AM, Dan van der Ster
>> wrote:
>> > Hi,
>> > Today I've noticed an interesting result
It depends on how long ago (in terms of data writes) it disappeared.
Each PG has a log of the changes that have been made (by default I
think it's 3000? Maybe just 1k), and if an OSD goes away and comes
back while the logs still overlap it will just sync up the changed
objects. Otherwise it has to
I haven't done the math, but it's probably a result of how the df
command interprets the output of the statfs syscall. We changed the
fr_size and block_size units we report to make it work more
consistently across different systems "recently"; I don't know if that
change was before or after the ker
in the case where the log no longer overlaps?
>
> I guess the key question is - do I have to worry about it, or will it
> always "do the right thing"?
>
> Tim.
>
> On Fri, Feb 21, 2014 at 11:57:09AM -0800, Gregory Farnum wrote:
>> It depends on how long ago (in terms
.818 x:6803/128164 1220 : [WRN] slow request
>> 30.446683 seconds old, received at 2014-02-21 16:25:44.754914:
>> osd_op(client.16735018.1:22524842 10352bf.0355 [write 0~4194304
>> [12@0],startsync 0~0] 0.c36d4557 snapc 1=[] e42655) v4 currently waiting for
>> subop
n 20 03:36:48
> UTC
>
> Markus
> Am 21.02.2014 20:59, schrieb Gregory Farnum:
>
>> I haven't done the math, but it's probably a result of how the df
>> command interprets the output of the statfs syscall. We changed the
>> fr_size and block_size units
Okay, well, let's try and track some of these down. What's the content
of the "ceph.layout" xattr on the directory you're running this test
in? Can you verify that pool 0 is the data pool used by CephFS, and
that all reported slow ops are in that pool? Can you record the IO
patterns on an OSD while
On Mon, Feb 24, 2014 at 11:48 PM, Markus Goldberg
wrote:
> Hi Gregory,
> here we go:
>
> root@bd-a:/mnt/myceph#
> root@bd-a:/mnt/myceph# ls -la
> insgesamt 4
> drwxr-xr-x 1 root root 25928099891213 Feb 24 14:14 .
> drwxr-xr-x 4 root root 4096 Aug 30 10:34 ..
> drwx-- 1 root root 2592
With the reweight-by-utilization applied, CRUSH is failing to generate
mappings of enough OSDs, so the system is falling back to keeping
around copies that already exist, even though they aren't located on
the correct CRUSH-mapped OSDs (since there aren't enough OSDs).
Are your OSDs correctly weigh
On Tue, Feb 25, 2014 at 7:13 PM, Guang wrote:
> Hello,
> Most recently when looking at PG's folder splitting, I found that there was
> only one sub folder in the top 3 / 4 levels and start having 16 sub folders
> starting from level 6, what is the design consideration behind this?
>
> For example,
osd/ceph-0
> ^^^ wrong ^^
> Am 25.02.2014 18:55, schrieb Gregory Farnum:
>
> [Re-adding the list.]
>
> Yeah, that pg dump indicates that each OSD believes it is storing
> about 30GB (which could include a lot of stuff besides the raw RADOS
> usage) a
Your OSDs aren't supposed to be listed in the config file, but they
should show up under /var/lib/ceph. Probably your OSD disks aren't
being mounted for some reason (that would be the bug). Try mounting
them and seeing what blocked the mount.
-Greg
Software Engineer #42 @ http://inktank.com | http:
On Wed, Feb 26, 2014 at 6:10 AM, David Champion wrote:
> I have a 1.6 TB collecton of 8 million files in CephFS, distributed up
> to 8-10 directories deep. (Never mind why - this design decision is out
> of my hands and not in scope.) I need to expose this data on multiple
> application servers.
On Wed, Feb 26, 2014 at 10:37 AM, David Champion wrote:
> Thanks, Greg, for the response.
>
> * On 26 Feb 2014, Gregory Farnum wrote:
>> >
>> > 1. Place the 8m files in a disk image. Mount the disk image (read-only)
>> > to provide access to the 8m files,
On Thu, Feb 27, 2014 at 4:25 PM, Marc wrote:
> Hi,
>
> I was handed a Ceph cluster that had just lost quorum due to 2/3 mons
> (b,c) running out of disk space (using up 15GB each). We were trying to
> rescue this cluster without service downtime. As such we freed up some
> space to keep mon b runn
On Fri, Feb 28, 2014 at 6:14 AM, Sage Weil wrote:
> On Wed, 26 Feb 2014, haiquan...@sina.com wrote:
>> Hi ,
>>
>> Recently, we are testing use CTDB+Cephfs+Samba/NFS HA cluster , but
>> it's not success, so the Cephfs if support CTDB cluster file ? if it
>> support , could you pls offer a g
By default your filesystem data is stored in the "data" pool, ID 0.
You can change to a different pool (for files going forward, not
existing ones) by setting the root directory's layout via the
ceph.layout.pool virtual xattr, but it doesn't look like you've done
that yet.
Until then, you've got tw
On Thu, Feb 27, 2014 at 9:29 PM, Michael Sevilla wrote:
> I'm looking for the debug messages in Client.cc, which uses ldout
> (library debugging). I increased the client debug level for all
> daemons (i.e. under [global] in ceph.conf) and verified that it got
> set:
>
> $ ceph --admin-daemon /var/
On Wed, Feb 26, 2014 at 11:39 AM, David Champion wrote:
> * On 26 Feb 2014, Gregory Farnum wrote:
>> >> > q1. CephFS has a tunable for max file size, currently set to 1TB. If
>> >> > I want to change this, what needs to be done or redone? Do I have to
>>
RBD itself will behave fine with whenever you take the snapshot. The
thing to worry about is that it's a snapshot at the block device
layer, not the filesystem layer, so if you don't quiesce IO and sync
to disk the filesystem might not be entirely happy with you for the
same reasons that it won't b
On Sunday, March 2, 2014, Diedrich Ehlerding <
diedrich.ehlerd...@ts.fujitsu.com> wrote:
> Gregory Farnum wrote:
>
> >
> > Your OSDs aren't supposed to be listed in the config file, but they
> > should show up under /var/lib/ceph. Probably your OSD disks aren&
The apply latency is how long it's taking for the backing filesystem to ack
(not sync to disk) writes from the OSD. Either it's getting a lot more
writes than the other OSDs (you can check by seeing how many PGs are mapped
to each) and then just apply standard local fs debugging techniques to that
The setting is calculated per-OSD, and if any OSD hits the hard limit
the whole cluster transitions to the full state and stops accepting
writes until the situation is resolved.
-Greg
Software Engineer #42 @ http://inktank.com | http://ceph.com
On Tue, Mar 4, 2014 at 9:58 AM, Barnes, Thomas J
wr
901 - 1000 of 2358 matches
Mail list logo
