Re: [ceph-users] ubuntu 14.04 or centos 7

Hi, On Tue, Dec 29, 2015 at 3:58 PM, min fang wrote: > centos 7 and ubuntu 14.04. Which > one is better? thanks. this is a question that often feels like flamebait, and too general. But probably the best answer is: it depends on personal preference, skill-level related to maintenance, use-case,

Re: [ceph-users] cephfs, low performances

Hi, On 28/12/2015 09:04, Yan, Zheng wrote: >> Ok, so in a client node, I have mounted cephfs (via ceph-fuse) and a rados >> block device formatted in XFS. If I have well understood, cephfs uses sync >> IO (not async IO) and, with ceph-fuse, cephfs can't make O_DIRECT IO. So, I >> have tested this

[ceph-users] Create one millon empty files with cephfs

Hi, We create one millon empty files through filebench, here is the test env: MDS: one MDS MON: one MON OSD: two OSD , each with one Intel P3700; data on OSD with 2x replica NetWork: all the nodes are connectted through 10 gigabit network We use more than one client to create files,

Re: [ceph-users] ubuntu 14.04 or centos 7

If you need to get Ceph up&running as part of some enterprise project, where you can't touch it after it gets in production without PM approval and a Change Request then CentOS is what fits in that box. You are unlikely to break anything during upgrades and you get new hardware support and (not

Re: [ceph-users] Ceph & Hbase

Hi, Sorry to reask, but I'm not reaching any further. I think i'm missing some really obvious permission configuration, as I was testing the hadoop command and it only works with one user (the one I use to install ceph). This works ubuntu@cephmaster:~/ceph-cluster$ hadoop fs -put ceph.conf /

Re: [ceph-users] [rados-java] SIGSEGV librados.so Ubuntu

On 28-12-15 20:41, KeesBoog wrote: > Hi Wido and Florent, > > After going back into the error-report with my new knowledge, I think I > found the problem. > > It is the finalizer in com.ceph.rados.Rados.java > Commenting out the finalize() method solved the problem. > I have no idea why it caus

Re: [ceph-users] Tuning ZFS + QEMU/KVM + Ceph RBD’s

A bunch of ideas. I'm not an expert in either ceph or zfs so take with appropriately sized boulders of salt. You might want to ask this in the zfs list as well, especially if your ceph cluster is serving non-zfs workloads you're more likely to want to tune zfs for ceph rather than the other way ar

Re: [ceph-users] My OSDs are down and not coming UP

I checked it out. osd daemon at node2 estabilish connection with mon deamon at node1: node1: tcp6 0 0 ::2:1612::50:6789 ::2:1612::60:42134 SPOJENO 11384/ceph-mon node1: tcp6 0 0 ::2:1612::60:42134 ::2:1612::50:6789 SPOJENO 13843

Re: [ceph-users] My OSDs are down and not coming UP

Again, my hunch is something wrong with the network..'ping' is working doesn't mean network is good for ceph. 1. Run traceroute as I mentioned to osd nodes and mon nodes 2. Disable the firewall if you are not already. Thanks & Regards Somnath From: ceph-users [mailto:ceph-users-boun...@lists.c

Re: [ceph-users] My OSDs are down and not coming UP

Hi, network is OK, all nodes are in one VLAN, in one switch, in one rack. tracepath6 node2 1?: [LOCALHOST]0.030ms pmtu 1500 1: node2 0.634ms reached 1: node2 0.296ms re

[ceph-users] ceph-fuse inconsistent filesystem view from different clients

Hi guys, I have 16 OSD/1MON/1MDS ceph cluster serving CephFS. The FS is mounted on 11 clients using ceph-fuse. In some cases there are multiple ceph-fuse processes per client, each with its own '-r' option. The problem is that some of the clients get significantly out-of-sync. Flushing caches, '

Re: [ceph-users] My OSDs are down and not coming UP

Has the cluster ever worked? Are you sure that "mon initial members = 0" is correct? How do the OSDs know where to look for MONs? Jan > On 29 Dec 2015, at 21:41, Ing. Martin Samek > wrote: > > Hi, > > network is OK, all nodes are in one VLAN, in one switch, in o

Re: [ceph-users] My OSDs are down and not coming UP

Hi, No, never. It is my first attempt, first ceph cluster i try ever run. im not sure, if "mon initial members" should contain mon servers ids or hostnames ? MS: Dne 30.12.2015 v 00:04 Jan Schermer napsal(a): Has the cluster ever worked? Are you sure that "mon initial members = 0" is corre

Re: [ceph-users] My OSDs are down and not coming UP

It should be monitor host names..If you are deploying with ceph-deploy it should be added in the conf file automatically..How are you creating your cluster ? Did you change conf file after installing ? From: Ing. Martin Samek [mailto:samek...@fel.cvut.cz] Sent: Tuesday, December 29, 2015 3:09 PM

Re: [ceph-users] My OSDs are down and not coming UP

I'm deploying ceph cluster manually following different guides. I didn't use ceph-deploy yet. MS: Dne 30.12.2015 v 00:13 Somnath Roy napsal(a): It should be monitor host names..If you are deploying with ceph-deploy it should be added in the conf file automatically..How are you creating your

Re: [ceph-users] My OSDs are down and not coming UP

May be try commenting out mon_initial_members (or give mon host name) and see..It is certainly not correct as Jan pointed out.. From: Ing. Martin Samek [mailto:samek...@fel.cvut.cz] Sent: Tuesday, December 29, 2015 3:16 PM To: Somnath Roy; Jan Schermer Cc: ceph-users@lists.ceph.com Subject: Re: [

Re: [ceph-users] My OSDs are down and not coming UP

Just try putting something like the following in ceph.conf: [global] mon_host =::2:1612::50 ::2:1612::30 mon_initial_members = node-1 node-2 Also, I just noticed you have two MONs? It should always be an odd number. Not sure if they can ever get quorum now? Jan > On 30 Dec 20

Re: [ceph-users] My OSDs are down and not coming UP

OK, ill try. At I have only one MON. MS: Dne 30.12.2015 v 00:31 Jan Schermer napsal(a): Just try putting something like the following in ceph.conf: [global] mon_host =::2:1612::50 ::2:1612::30 mon_initial_members = node-1 node-2 Also, I just noticed you have two MONs? It shou

Re: [ceph-users] My OSDs are down and not coming UP

Jan, Two monitors should be just fine as long as both can agree to every action. Even number of monitors are not useful as the total number of failures tolerated will be the same as odd numbers. For example, 1,2 will not tolerate any failure and 3,4 both can tolerate 2 failures. Thanks & Regards

Re: [ceph-users] My OSDs are down and not coming UP

I made small leap ahead. Not it seem to be some problem with autentication may be. 2015-12-30 00:43:34.041121 7f72773fd700 0 cephx server osd.0: couldn't find entity name: osd.0 2015-12-30 00:44:12.835810 7f7277bfe700 0 mon.ares1@0(leader).data_health(1) update_stats avail 53% total 28662 M

Re: [ceph-users] My OSDs are down and not coming UP

I thank both of you. Now I have 3 OSDs UP and running. Messy ceph.conf was the issue :) MS: Dne 30.12.2015 v 00:31 Jan Schermer napsal(a): Just try putting something like the following in ceph.conf: [global] mon_host =::2:1612::50 ::2:1612::30 mon_initial_members = node-1 node

Re: [ceph-users] OSD size and performance

Hi Team, Anybody please clarify the below quires. Regards Prabu On Tue, 29 Dec 2015 13:03:45 +0530 gjprabu wrote Hi Team, We are using ceph with 3 osd and 2 replicas. Each osd size is 13TB and current data is reached to 2.5TB

Re: [ceph-users] OSD size and performance

FYI , we are using 8TB SSD drive as OSD and not seeing any problem so far. Failure domain could be a concern for bigger OSDs. Thanks & Regards Somnath From: ceph-users [mailto:ceph-users-boun...@lists.ceph.com] On Behalf Of gjprabu Sent: Tuesday, December 29, 2015 9:38 PM To: ceph-users Cc: Siva

Re: [ceph-users] ceph-fuse inconsistent filesystem view from different clients

On Wed, Dec 30, 2015 at 5:59 AM, Mykola Dvornik wrote: > Hi guys, > > I have 16 OSD/1MON/1MDS ceph cluster serving CephFS. > > The FS is mounted on 11 clients using ceph-fuse. In some cases there are > multiple ceph-fuse processes per client, each with its own '-r' option. > > The problem is that

Re: [ceph-users] Ceph & Hbase

On Tue, Dec 29, 2015 at 10:42 PM, Jose M wrote: > Hi, > > > Sorry to reask, but I'm not reaching any further. I think i'm missing some > really obvious permission configuration, as I was testing the hadoop command > and it only works with one user (the one I use to install ceph). > > > This works