Hi Christian,
> Hello,
>
> Ceph 0.94.5 for the record.
>
> As some may remember, I phased in a 2TB cache tier 5 weeks ago.
>
> About now it has reached about 60% usage, which is what I have the
> cache_target_dirty_ratio set to.
>
> And for the last 3 days I could see some writes (op_in_byte
Hi.
I have small question about monitoring performance at ceph cluster.
We have cluster with 5 nodes and 8 drives on each node, and 5 monitor on
every node. For monitoring cluster we use zabbix. It asked every node
for 30 second about current ceph operation and get different result from
every
> Hi.
> I have small question about monitoring performance at ceph cluster.
>
> We have cluster with 5 nodes and 8 drives on each node, and 5 monitor on
> every node. For monitoring cluster we use zabbix. It asked every node for
30
> second about current ceph operation and get different result f
Hello!
On Thu, Apr 07, 2016 at 10:20:41AM +0100, nick wrote:
>> Hi.
>> I have small question about monitoring performance at ceph cluster.
>>
>> We have cluster with 5 nodes and 8 drives on each node, and 5 monitor on
>> every node. For monitoring cluster we use zabbix. It asked every node for
>
Hi.
I use this for my zabbix environment:
https://github.com/thelan/ceph-zabbix/
It works really well for me.
Regards
David
On 07.04.2016 11:20, Nick Fisk wrote:
Hi.
I have small question about monitoring performance at ceph cluster.
We have cluster with 5 nodes and 8 drives on each n
Hi Christian,
thank you for answering, i appriciate your time !
---
Its used for RBD hosted vm's and also cephfs hosted vm's.
Well the basic problem is/was that single OSD's simply go out/down.
Ending in SATA BUS error's for the VM's which have to be rebooted, if
they anyway can, because as lon
Hi all,
I've setup a testing/development Ceph cluster consisting of 5 Dell
PowerEdge R720xd servers (256GB RAM, 2x 8-core Xeon E5-2650 @ 2.60 GHz,
dual-port 10Gb Ethernet, 2x 900GB + 12x 4TB disks) running CentOS 6.5 and
Ceph Hammer 0.94.6. All servers use one 900GB disk for the root partition
and
I'd appreciate any critique on the following plan.
Before I detail the plan, here are my current questions.
---
1) Am I under-powering the CPU on the proposed OSD node configuration?
2) Will latency of roughly 300 micro seconds introduced by
I would strongly consider your journaling setup, (you do mention that you will
revisit this) but we have found that co-locating journals does impact
performance and usually separating them on flash is a good idea. Also not sure
of your networking setup which can also have significant impact.
Fr
We use ceph on a 3 server Debian proxmox cluster that has 4 x4TB disks each
in a shared Ceph cluster. The io wait is much too high (around 9%). The
default file system is XFS and I found a suggestion that by adding this line
osd mount options xfs = rw,noatime,inode64
into the global area of our
I am trying to get cephfs mounted as a user other than client.admin in
order to avoid putting the admin keyring on the client host. When I try to
connect with that user and keyring I am getting the following error:
*mount error 13 = Permission denied*
I am following the exact instructions that K
Hi Sergio
On 04/07/2016 07:00 AM, Sergio A. de Carvalho Jr. wrote:
Hi all,
I've setup a testing/development Ceph cluster consisting of 5 Dell
PowerEdge R720xd servers (256GB RAM, 2x 8-core Xeon E5-2650 @ 2.60 GHz,
dual-port 10Gb Ethernet, 2x 900GB + 12x 4TB disks) running CentOS 6.5
and Ceph Ha
On Thu, Apr 7, 2016 at 10:02 AM, Nate Curry wrote:
> I am trying to get cephfs mounted as a user other than client.admin in order
> to avoid putting the admin keyring on the client host. When I try to
> connect with that user and keyring I am getting the following error:
>
> mount error 13 = Perm
Hi all,
We've pushed 10.1.1, a second release candidate for Jewel. This fixes
another round of bugs, and we are getting pretty close to a final release.
There are a few known issues to watch out for:
- Old CephFS clusters will mangle the layouts with this release; the
fix was committed just
Thanks, Alan.
Unfortunately, we currently don't have much flexibility in terms of the
hardware we can get so adding SSDs might not be possible in the near
future. What is the best practice here, allocating, for each OSD, one disk
just for data and one disk just for thd journal? Since the journals
Hi Oliver,
Have you tried tuning some of the cluster settings to fix the IO errors
in the VMs?
We found some of the same issues when reweighting, backfilling and
removing large snapshots. By minimizing the number of concurrent
backfills and prioritizing client IO we can now add/remove OSDs w
If you're at all concerned with performance and the budget is set, drop a
storage node and replace some osds with SSDs in the other nodes. We use 32x
4TB + 4x SSDs storage nodes and have 192GB of memory; 128GB wasn't enough. If
you try to do this setup without SSD journals then you are going t
Thanks, Mark.
Yes, we're using XFS and 3-replication, although we might switch to
2-replication since we're not too worried about resiliency.
I did some test on single disks with dd, and am able to get about 152 MB/s
writes and 191 MB/s reads from a single disk. I also run the same test on
all 13
Hi Sergio, yes I think you have also answered most of your own points –
The main thing is to try and avoid excessive seeks on the HDDS, it would help
to separate the journal and data but since HDDs are heavily dependent on seek
and latency delays, it would not help to have multiple journals on a
Hi All,
first I wanted to say hello, as I am new to the list.
Secondly, we want to use ceph for VM disks and cephfs for our source
code, image data, login directories, etc.
I would like to know, if striping would improve performance if we would
set something like the following and move away from
You are correct sir. I modified the user capabilities by adding the mds
cap with the 'allow r' permission using the following command.
*ceph auth caps client.cephfs mon 'allow r' mds 'allow r' osd 'allow rwx
pool=cephfs_metadata,allow rwx pool=cephfs_data'*
Thanks,
*Nate Curry*
On Thu, Apr 7
Hi Chris,
thank you very much for your advice !
Currently, we have already running:
osd_op_threads = 8
osd_max_backfills = 1
osd_recovery_max_active = 1
I will add your suggestions !
For sure there is a lot of space in tweaking the config, which is
basically very basic.
--
Mit freundlichen
All,
I have 4 nodes each with 5 OSDs.
I recently upgraded to infernalis via ceph-deploy. It went mostly ok but one of
my nodes cannot mount any OSDs.
When I look at the status of the service, I see:
Apr 07 12:22:06 borg02 ceph-osd[3868]: 9: (ceph::__ceph_assert_fail(char
const*, char const*,
Hi Cephers,
I've setup a production environment Ceph cluster with the Jewel release
(10.1.0 (96ae8bd25f31862dbd5302f304ebf8bf1166aba6)) consisting of 3 MON
Servers and 6 OSD Servers:
3x MON Servers:
2x Intel Xeon E5-2630v3@2.40Ghz
384GB RAM
2x 200G Intel DC3700 in RAID-1 for OS
1x InfiniBand Conn
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256
Ceph is not able to use native Infiniband protocols yet and so it is
only leveraging IPoIB at the moment. The most likely reason you are
only getting ~10 Gb performance is that IPoIB heavily leverages
multicast in Infiniband (if you do so research in
also jewel does not supposed to get more 'performance', since it used
bluestore in order to store metadata. Or do I need to specify during
install to use bluestore?
Thanks,
*German*
2016-04-07 16:55 GMT-03:00 Robert LeBlanc :
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA256
>
> Ceph is not
On 04/07/2016 02:43 PM, German Anders wrote:
Hi Cephers,
I've setup a production environment Ceph cluster with the Jewel release
(10.1.0 (96ae8bd25f31862dbd5302f304ebf8bf1166aba6)) consisting of 3 MON
Servers and 6 OSD Servers:
3x MON Servers:
2x Intel Xeon E5-2630v3@2.40Ghz
384GB RAM
2x 200G I
On Thu, Apr 7, 2016 at 10:09 PM, German Anders wrote:
> also jewel does not supposed to get more 'performance', since it used
> bluestore in order to store metadata. Or do I need to specify during install
> to use bluestore?
Do the words "enable experimental unrecoverable data corrupting
features
On 04/07/2016 03:09 PM, German Anders wrote:
also jewel does not supposed to get more 'performance', since it used
bluestore in order to store metadata. Or do I need to specify during
install to use bluestore?
Bluestore is not used by default in jewel. It's just tech preview for
now. It also
Well, that bug is a completely different backtrace, so it's probably
safe to say that it's not related. It looks like you are missing
maps, which suggests you have inconsistent osd filesystems. Did you
powercycle this node with barriers disabled in xfs? Posting a log
from startup to crash would
We have a 3 node ceph cluster. How do we safely reboot any node without
causing any damage to data in the storage cluster? Or what are the best
practices for this? Do we need to run any commands on the node that we are
going to reboot before hitting the reboot command?
__
There was a discussion regarding to rebooting the ceph cluster before. [1]
[1]
http://lists.ceph.com/pipermail/ceph-users-ceph.com/2013-December/036397.html
Cheers,
Shinobu
On Fri, Apr 8, 2016 at 7:30 AM, Mad Th wrote:
> We have a 3 node ceph cluster. How do we safely reboot any node without
Hi Christian,
thanks for your input. I don't think the PG count is my issue. I've actually
have too many PGs for the size of my cluster consisting of just 18 osds spread
amongst 2 osd servers and 3 mons:
root@arh-ibstorage1-ib:~# ceph -s
health HEALTH_WARN
1 near full osd(s)
There was a discussion before regarding to the situation where you are
facing now. [1]
Would you have a look, if it's helpful or not for you.
[1]
http://lists.ceph.com/pipermail/ceph-users-ceph.com/2016-February/007622.html
Cheers,
Shinobu
___
ceph-use
Hello Nick,
On Thu, 7 Apr 2016 10:03:27 +0100 Nick Fisk wrote:
> Hi Christian,
>
>
> > Hello,
> >
> > Ceph 0.94.5 for the record.
> >
> > As some may remember, I phased in a 2TB cache tier 5 weeks ago.
> >
> > About now it has reached about 60% usage, which is what I have the
> > cache_tar
Hi Jason,
Sorry to bother you. A question about object map.
object map tracks which blocks of the image are actually allocated.
I can see one use case is for read optimization. For read, it checks the
object existence first,if the object doesn't exist locally, it sends read
directly to parent. so t
Hello,
On Thu, 7 Apr 2016 09:24:46 -0500 Brady Deetz wrote:
> I'd appreciate any critique on the following plan.
>
> Before I detail the plan, here are my current questions.
Traditionally they come below the plan/details. ^o^
> ---
> 1) A
hi all:
I have a question about monitor node, what is the maximum numbers of monitor
node and how many monitor node need in a production environment? The official
docs only recommends that monitor need odd number, so how do I calculate the
numbers of monitor in a cluster, for example, a cl
Hello,
On Fri, 8 Apr 2016 13:55:03 +0800 (CST) powerhd wrote:
>
>
> hi all:
> I have a question about monitor node, what is the maximum numbers of
> monitor node and how many monitor node need in a production
> environment? The official docs only recommends that monitor need odd
> number,
intel recommends that deploy three monitors if your nodes within 200.
And if the number of your osds is bigger than 100,you should deploy
monitor in separate node.
2016-04-08 13:55 GMT+08:00 powerhd :
>
> hi all:
> I have a question about monitor node, what is the maximum numbers of
> monitor n
thank you very much, this clear my confused.
在2016年04月08 14时13分, "Christian Balzer"写道:
Hello,
On Fri, 8 Apr 2016 13:55:03 +0800 (CST) powerhd wrote:
>
>
> hi all:
> I have a question about monitor node, what is the maximum numbers of
> monitor node and how many monitor node need in a
it there a link or something? this day I have an assignment to deploy a ceph
cluster with about 20 node, any reference from experienced people may help me a
lot.
在2016年04月08 14时14分, "lin zhou"写道:
intel recommends that deploy three monitors if your nodes within 200.
And if the number of your o
42 matches
Mail list logo
