On Mon, 14 Mar 2016 23:38:24 -0700 Blade Doyle wrote:
> On Mon, Mar 14, 2016 at 3:48 PM, Christian Balzer wrote:
>
> >
> > Hello,
> >
> > On Mon, 14 Mar 2016 09:16:13 -0700 Blade Doyle wrote:
> >
> > > Hi Ceph Community,
> > >
> > > I am trying to use "ceph -w" output to monitor my ceph cluster.
Hi there,
I setup ceph cluster with disable cephx cluster authen and enable cephx client
authen as follow :
auth_cluster_required = none
auth_service_required = cephx
auth_client_required = cephx
I can run command such as `ceph -s`, `rados -p rbd put` but I can not run
command `rbd ls`, `rbd crea
Hi,
We have a 3 ceph clusters (Hammer 0.94.5) on same physical nodes Using LXC on
debian Wheezy. Each physical node has 12 4To 7200 RPM hard drive, 2x200Gb SSD
MLC, 2 x 10 Gb ethernet. On each physical drive we have an lxc container for 1
OSD and the journal is on SSD partition.
One of our cep
Thanks Christian .
Still
"So yes, your numbers are normal for single client, low depth reads, as many
threads in this ML confirm."
we're facing very high latency ( i expect much less latency from ssd
cluster ) :
clat percentiles (usec):
| 1.00th=[ 350], 5.00th=[ 390], 10.00th=[ 414]
On Tue, Mar 15, 2016 at 6:38 AM, Blade Doyle wrote:
>
>
> On Mon, Mar 14, 2016 at 3:48 PM, Christian Balzer wrote:
>>
>>
>> Hello,
>>
>> On Mon, 14 Mar 2016 09:16:13 -0700 Blade Doyle wrote:
>>
>> > Hi Ceph Community,
>> >
>> > I am trying to use "ceph -w" output to monitor my ceph cluster. The
Hi Guys .
On a full ssd cluster , is it meaningful to put the journal on a different
drive ? does it have any impact on performance ?
Thanks
*Yair Magnezi *
*Storage & Data Protection // KenshooOffice +972 7 32862423 // Mobile
+972 50 575-2955__
Hi, can the ceph client(by librbd) io continued if connection to primary osd lost? Thanks
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Yes, if you can manage *cost* , separating journal on a different device should
improve write performance. But, you need to evaluate how many osd journals you
can dedicate to a single OSD as at some point it will be bottlenecked by that
journal device BW.
Thanks & Regards
Somnath
From: ceph-us
Hi all,
The documentation [0] gives us the following formula for calculating
the number of PG if the cluster is bigger than 50 OSDs:
(OSDs * 100)
Total PGs =
pool size
When we have mixed storage server (HDD disks and SSD disks) and we
have
you can find in http://ceph.com/pgcalc/
2016-03-15 23:41 GMT+08:00 Martin Palma :
> Hi all,
>
> The documentation [0] gives us the following formula for calculating
> the number of PG if the cluster is bigger than 50 OSDs:
>
> (OSDs * 100)
> Total PGs =
>
Hello Martin,
The proper way is to perform the following process:
For all Pools utilizing the same bucket of OSDs:
(Pool1_pg_num * Pool1_size) + (Pool2_pg_num * Pool2_size) + ...
(Pool(n)_pg_num * Pool(n)_size)
-
Hi,
The ceph-disk (10.0.4 version) command seems to have problems operating on a
Redhat 7 system, it uses the partprobe command unconditionally to update the
partition table, I had to change this to partx -u to get past this.
@@ -1321,13 +1321,13 @@
processed, i.e. the 95-ceph-osd.rules a
there are not any monitors running on the new nodes. the monitors are on
separate nodes and running the 0.94.5 release. i spent some time thinking
about this last night as well and my thoughts went to the recency patches.
i wouldn't think that caused this but its the only thing that seems close.
m
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256
There are no monitors on the new node.
It doesn't look like there has been any new corruption since we
stopped changing the cache modes. Upon closer inspection, some files
have been changed such that binary files are now ASCII files and visa
versa.
Thank you both for the quick replay, and I found my answer "Number of
OSDs which this Pool will have PGs in. Typically, this is the entire
Cluster OSD count, but could be less based on CRUSH rules. (e.g.
Separate SSD and SATA disk sets)"
@Michael: So the ration of PGs per OSDs should between 100 a
It seems like ceph-disk is often breaking on centos/redhat systems. Does it
have automated tests in the ceph release structure?
-Ben
On Tue, Mar 15, 2016 at 8:52 AM, Stephen Lord
wrote:
>
> Hi,
>
> The ceph-disk (10.0.4 version) command seems to have problems operating on
> a Redhat 7 system,
Wanting to play around with Ceph, I have a single-node Ceph with 1
monitor and 3 OSDs running on a VM. I am loving the flexibility that
Ceph provides (and perhaps just the novelty of it). I've been
planning for some time to build a NAS for home use and seriously
thinking about running Ceph on rea
There's a ceph-disk suite from last August that Loïc set up, but based
on the qa list it wasn't running for a while and isn't in great shape.
:/ I know there are some CentOS7 boxes in the sepia lab but it might
not be enough for a small and infrequently-run test to reliably get
tested against them.
Ceph-deploy suite and also selinux suite(which isn't merged yet) indirectly
tests ceph-disk and has been run on Jewel as well. I guess the issue
Stephen is seeing is on multipath device
which I believe is a known issue.
On Tue, Mar 15, 2016 at 11:42 AM, Gregory Farnum wrote:
> There's a ceph-dis
Not multipath if you mean using the multipath driver, just trying to setup OSDs
which use a data disk and a journal ssd. If I run just a disk based OSD and
only specify one device to ceph-deploy then it usually works although sometimes
has to retry. In the case where I am using it to carve an SS
Do you mind giving the full failed logs somewhere in fpaste.org along with
some os version details?
There are some known issues on RHEL, If you use 'osd prepare' and 'osd
activate'(specifying just the journal partition here) it might work better.
On Tue, Mar 15, 2016 at 12:05 PM, Stephen Lord
w
I would have to nuke my cluster right now, and I do not have a spare one..
The procedure though is literally this, given a 3 node redhat 7.2 cluster,
ceph00, ceph01 and ceph02
ceph-deploy install --testing ceph00 ceph01 ceph02
ceph-deploy new ceph00 ceph01 ceph02
ceph-deploy mon create ceph00
Thanks for the steps that should be enough to test it out, I hope you got
the latest ceph-deploy either from pip or throught github.
On Tue, Mar 15, 2016 at 12:29 PM, Stephen Lord
wrote:
> I would have to nuke my cluster right now, and I do not have a spare one..
>
> The procedure though is lite
My ceph-deploy came from the download.ceph.com site and it is 1.5.31-0. This
code is in ceph itself though, the deploy logic is where the code appears to do
the right thing ;-)
Steve
> On Mar 15, 2016, at 2:38 PM, Vasu Kulkarni wrote:
>
> Thanks for the steps that should be enough to test it
Hi Loris,
If i'm not mistaken, there are no rbd ACL in cephx.
Why not 1 pool/client and pool quota ?
David.
2016-02-12 3:34 GMT+01:00 Loris Cuoghi :
> Hi!
>
> We are on version 9.2.0, 5 mons and 80 OSDS distributed on 10 hosts.
>
> How could we twist cephx capabilities so to forbid our KVM+QEMU+l
Interesting !
Is it safe to do this ?
Perhaps "rados" is considered an internal command while rbd is a
client librados ?
In MonClient.cc :
if (!cct->_conf->auth_supported.empty())
method = cct->_conf->auth_supported;
else if (entity_name.get_type() == CEPH_ENTITY_TYPE_OSD ||
Hi David,
One pool per virtualization host would make it impossible to live
migrate a VM. :)
Thanks,
Loris
Le 15/03/2016 22:11, David Casier a écrit :
> Hi Loris,
> If i'm not mistaken, there are no rbd ACL in cephx.
> Why not 1 pool/client and pool quota ?
>
> David.
>
> 2016-02-12 3:34 GMT+
Hi,
Maybe (not tested) :
[osd ]allow * object_prefix ?
2016-03-15 22:18 GMT+01:00 Loris Cuoghi :
>
> Hi David,
>
> One pool per virtualization host would make it impossible to live
> migrate a VM. :)
>
> Thanks,
>
> Loris
>
>
> Le 15/03/2016 22:11, David Casier a écrit :
>> Hi Loris,
>> If i'm
So, one key per RBD.
Or, dynamically enable/disable access to each RBD in each hypervisor's key.
Uhm, something doesn't scale here. :P
(I wonder if there's any limit to a key's capabilities string...)
But, as it appears, I share your view that it is the only available
approach right now.
Anyone w
On Tue, Mar 15, 2016 at 2:44 PM, Loris Cuoghi wrote:
> So, one key per RBD.
> Or, dynamically enable/disable access to each RBD in each hypervisor's key.
> Uhm, something doesn't scale here. :P
> (I wonder if there's any limit to a key's capabilities string...)
>
> But, as it appears, I share your
Ok, a branch merged to master which should fix this
(https://github.com/ceph/ceph/pull/8136). It'll be backported in due
course. The problem is that that patch won't clean orphaned files
that already exist.
Let me explain a bit about what the orphaned files look like. The
problem is files with
Hello,
I've tried to install the ceph using ceph-deploy as usual.
[ceph@octopus conf]$ ceph-deploy install --mon --mds --testing octopus
*install* was completed without any surprising.
But *mon create-initial* was failed:
### Take1 ###
Log
[ceph@octopus conf]$ ceph-deploy mon create-i
[back on list]
ceph-objectstore-tool has a whole bunch of machinery for modifying an
offline objectstore. It would be the easiest place to put it -- you
could add a
ceph-objectstore-tool --op filestore-repair-orphan-links [--dry-run] ...
command which would mount the filestore in a special mode
One more question.did we hit the bug because we had hardware issues
during the remapping or would it have happened regardless of the hardware
issues? e.g. I'm not planning to add any additional hardware soon, but
would the bug pop again on an (unpatched) system not subject to any
remapping?
Hi,
Let me start. Ceph is amazing, no it really is!
But a hypervisor reading and writing all its data off the network off the
network will add some latency to read and writes.
So the hypervisor could do with a local cache, possible SSD or even NVMe.
Spent a while looking into this but it seems
The bug is entirely independent of hardware issues -- entirely a ceph
bug. xfs doesn't let us specify an ordering when reading a directory,
so we have to keep directory sizes small. That means that when one of
those pg collection subfolders has 320 files in it, we split it into
up to 16 smaller d
The good news is such a feature is in the early stage of design [1]. Hopefully
this is a feature that will land in the Kraken release timeframe.
[1]
http://tracker.ceph.com/projects/ceph/wiki/Rbd_-_ordered_crash-consistent_write-back_caching_extension
--
Jason Dillaman
- Original Me
Thanks.
Reassuring but I could do with something today :)
-Original Message-
From: Jason Dillaman [mailto:dilla...@redhat.com]
Sent: 16 March 2016 01:25
To: Daniel Niasoff
Cc: ceph-users@lists.ceph.com
Subject: Re: [ceph-users] Local SSD cache for ceph on each compute node.
The good ne
Perhaps something like this?
mon 'allow r' osd 'allow class-read object_prefix rbd_children, allow r
class-read object_prefix rbd_directory, allow rwx object_prefix rbd_header.,
allow rwx object_prefix rbd_data., allow rwx object_prefix rbd_id.'
As Greg mentioned, this won't stop you from just
Ah, actually, I think there will be duplicates only around half the
time -- either the old link or the new link could be orphaned
depending on which xfs decides to list first. Only if the old link is
orphaned will it match the name of the object once it's recreated. I
should be able to find time
Indeed, well understood.
As a shorter term workaround, if you have control over the VMs, you could
always just slice out an LVM volume from local SSD/NVMe and pass it through to
the guest. Within the guest, use dm-cache (or similar) to add a cache
front-end to your RBD volume. Others have als
I am using openstack so need this to be fully automated and apply to all my VMs.
If I could do what you mention at the hypervisor level that would me much
easier.
The options that you mention I guess are for very specific use cases and need
to be configured on a per vm basis whilst I am looking
On 16 March 2016 at 04:34, Edward Wingate wrote:
> Given my resources,
> I'd still only run a single node with 3 OSDs and replica count of 2.
> I'd then have a VM mount the a Ceph RBD to serve Samba/NFS shares.
>
Fun & instructive to play with ceph that way, but not really a good use of
it - ce
Hi,
I find this to be discussed here before, but couldn¹t find any solution
hence the mail. In RGW, for a bucket holding objects in the range of ~
millions, one can find it to take for ever to delete the bucket(via
radosgw-admin). I understand the gc(and its parameters) that would reclaim
the spac
Hi all,
Hi, I try to find an api that can list all s3 user like command
'radosgw-admin metadata list user'.
But I can not find any document related. Have anyone know how to get this
information？
Any comments will be much appreciated!
Best wishes,
Mika
___
45 matches
Mail list logo
