[ceph-users] cephforum.com

2013-10-10 Thread james
Hello! Anyone else think a web forum for ceph could work? I'm thinking simple vbulletin or phpBB site. To me it seems this would increase accessibility to the great info (&minds) on here... but obviously it would need those great minds to work :) Happy to host... Cheers :) __

Re: [ceph-users] SSD pool write performance

2013-10-11 Thread james
Just a thought; did you try setting noop scheduler for the SSDs? I guess the journal is written uncached (?) So maybe sticking the SSDs behind BBWC might help by reducing write latency to near zero. Also maybe wear rate might be lower on the SSD too (if journal IO straddles physical cells).

Re: [ceph-users] bit correctness and checksumming

2013-10-16 Thread james
Does Ceph log anywhere corrected(/caught) silent corruption - would be interesting to know how much a problem this is, in a large scale deployment. Something to gather in the league table mentioned at the London Ceph day? Just thinking out-loud (please shout me down...) - if the FS itself

Re: [ceph-users] bit correctness and checksumming

2013-10-16 Thread james
Very interesting link. I don't suppose there is any data available separating 4K and 512-byte sectored drives? On 2013-10-16 18:43, Tim Bell wrote: At CERN, we have had cases in the past of silent corruptions. It is good to be able to identify the devices causing them and swap them out. It's

Re: [ceph-users] RBD & Windows Failover Clustering

2013-10-22 Thread james
RBD can be re-published via iSCSI using a gateway host to sit in between, for example using targetcli. On 2013-10-22 13:15, Damien Churchill wrote: Hi, I was wondering if anyone has had any experience in attempting to use a RBD volume as a clustered drive in Windows Failover Clustering? I'm

Re: [ceph-users] saucy salamander support?

2013-10-22 Thread james
Off topic perhaps but I'm finding it pretty buggy just now - not sure I'd want it underpinning Ceph, at the moment. On 2013-10-22 16:51, Mike Lowe wrote: And a +1 from me as well. It would appear that ubuntu has picked up the 0.67.4 source and included a build of it in their official repo, so

Re: [ceph-users] Hardware: SFP+ or 10GBase-T

2013-10-24 Thread james
On 2013-10-24 15:08, Nathan Stratton wrote: 9 - Samsung 840 EVO 120 GB SSD (1 root 8 ceph) The EVO is a TLC drive with durability of about 1,100 write cycles. Whether that is or isn't a problem in your environment of course is a separate question - I'm just pointing it out :) If they are

[ceph-users] Seagate Kinetic

2013-10-28 Thread james
Not brand-new, but I've not seen it mentioned on here so far. Seagate Kinetic essentially enables HDDs to present themselves directly over Ethernet as Swift object storage: http://www.seagate.com/solutions/cloud/data-center-cloud/platforms/?cmpid=friendly-_-pr-kinetic-us If the CPUs on these

Re: [ceph-users] Seagate Kinetic

2013-10-29 Thread james
That's unfortunate; hopefully 2nd-gens will improve and open things up. Some numbers: - Commercial grid-style SAN is maybe £1.70 per usable GB - Ceph cluster of about 1PB built on Dell hardware is maybe £1.25 per usable GB - Bare drives like WD RE4 3TB are about £0.21/GB (assuming 1/3rd capac

[ceph-users] Multicast

2013-11-02 Thread james
Hi All I was wondering whether multicast could be used for the replication traffic? It just seemed that the outbound network bandwidth from the source could be halved. Cheers ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.c

Re: [ceph-users] Multicast

2013-11-02 Thread james
It would be an interesting exercise though. Depending on network layout (no cluster network) the client could multicast to all replicas and potentially reduce latency by half. I suspect that the client participating in the replication goes against the internal workings of ceph though and wou

Re: [ceph-users] About memory usage of ceph-mon on arm

2013-11-05 Thread james
We recently discussed briefly the Seagate Ethernet drives, which were basically dismissed as too limited. But what about moving an ARM SBC to the drive tray, complete with an mSATA SSD slot? A proper SBC could implement full Ubuntu single-drive failure domains that also solve the journal is

Re: [ceph-users] USB pendrive as boot disk

2013-11-05 Thread james
It has been reported that the system is heavy on the OS during recovery; I believe the current recommendation is 5:1 OSD disks to SSDs and separate OS mirror. On 2013-11-05 21:33, Gandalf Corvotempesta wrote: Hi, what do you think to use a USB pendrive as boot disk for OSDs nodes? Pendrive are

Re: [ceph-users] Running on disks that lose their head

2013-11-05 Thread james
It is cool - and it's interesting that more and more access to the inner workings of the drives would be useful, given ATA controller history (an evolution of the WD1010 MFM controller) having hidden steadily more, to maintain compatibility with the old CHS addressing (later LBA). The streami

Re: [ceph-users] USB pendrive as boot disk

2013-11-06 Thread james
Why? Recovery is made from OSDs/SSD, why ceph is heavy on OS disks? There is nothing usefull to read from that disks during a recovery. See this thread: http://lists.ceph.com/pipermail/ceph-users-ceph.com/2013-October/005378.html ___ ceph-users maili

Re: [ceph-users] Running on disks that lose their head

2013-11-06 Thread james
On 2013-11-06 09:33, Sage Weil wrote: This make me think we really need to build or integrate with some generic SMART reporting infrastructure so that we can identify disks that are failing or going to fail. It could be of use especially for SSD devices used for journals. Unfortunately ther

Re: [ceph-users] ceph cluster performance

2013-11-06 Thread james
On 2013-11-06 20:25, Mike Dawson wrote: We just fixed a performance issue on our cluster related to spikes of high latency on some of our SSDs used for osd journals. In our case, the slow SSDs showed spikes of 100x higher latency than expected. Many SSDs show this behaviour when 100% prov

Re: [ceph-users] Manual Installation steps without ceph-deploy

2013-11-06 Thread james
I also had some difficulty with ceph-deploy on CentOS. I eventually moved to Ubuntu 13.04 - and haven't looked back. On 2013-11-06 21:35, Trivedi, Narendra wrote: Hi All, I did a fresh install of Ceph (this might be like 10th or 11th install) on 4 new VMs (one admin, one MON and two OSDs) bui

Re: [ceph-users] Ceph User Committee

2013-11-06 Thread james
On 2013-11-07 01:03, Alek Paunov wrote: On the other side, I think, the Ceph community is able to help further with the wider and smoother Ceph adoption (further than current mailing list participation in the support) This was my thinking behind a forum format - most sysadmins, and especially

Re: [ceph-users] USB pendrive as boot disk

2013-11-06 Thread james
On 2013-11-07 01:02, Mark Kirkwood wrote: The SSD failures I've seen have all been firmware bugs rather than flash wearout. This has the effect that a RAID1 pair are likley to fail at the same time! Very interesting... and good reason to use two different drives perhaps. The SuperMicro 2U 12

Re: [ceph-users] ceph cluster performance

2013-11-07 Thread james
On 2013-11-07 17:47, Gruher, Joseph R wrote: I wonder how effective trim would be on a Ceph journal area. If the journal empties and is then trimmed the next write cycle should be faster, but if the journal is active all the time the benefits would be lost almost immediately, as those cells ar

Re: [ceph-users] Running on disks that lose their head

2013-11-07 Thread james
On 2013-11-06 09:33, Sage Weil wrote: On Wed, 6 Nov 2013, Loic Dachary wrote: Hi Ceph, People from Western Digital suggested ways to better take advantage of the disk error reporting... when one head out of ten fails : disks can keep working with the nine remaining heads. Losing 1/10 of the

Re: [ceph-users] Ceph Block Storage QoS

2013-11-07 Thread james
On 2013-11-08 03:20, Haomai Wang wrote: On Fri, Nov 8, 2013 at 9:31 AM, Josh Durgin wrote: I just list commands below to help users to understand: cinder qos-create high_read_low_write consumer="front-end" read_iops_sec=1000 write_iops_sec=10 Does this have any normalisation of the IO uni

Re: [ceph-users] ceph on peta scale

2015-01-14 Thread James
op of it. > -Greg cockroachDB might be what you are looking for? http://cockroachdb.org/ hth, James ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

[ceph-users] Spark/Mesos on top of Ceph/Btrfs

2015-01-14 Thread James
Hello, I was wondering if anyone has Mesos running on top of Ceph? I want to test/use Ceph if lieu of HDFS. I'm working on Gentoo, but any experiences with Mesos on Ceph are of keen interest to me as related to performance, stability and any difficulties experienced.

Re: [ceph-users] Spark/Mesos on top of Ceph/Btrfs

2015-01-14 Thread James
p of Btrfs is the most challenging part of this journey so far. I use openrc on gentoo, and have no interest in systemd, just so you know. James [1] https://amplab.cs.berkeley.edu/ [2] http://dune.mathematik.uni-freiburg.de/ [3] http://www.opengeosys.org/ [4] http://w

[ceph-users] Ceph-btrfs layout

2015-01-20 Thread James
COW operations are still troublesome with ceph? So using a raid1 on each node with btrfs will allow me to turn off COW if/when those sorts of issues arise. What I need help with right now is setting up the UUID based /etc/fstab and suggestions on exactly how to configure ceph(fs). My desire is to keep the btrfs-gentoo installs stable but to be able to use ansible or other (ceph based tools) to reconfigure ceph or recover from ceph failures. James ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

[ceph-users] ceph + openrc Long term

2015-11-30 Thread James
Hello, So I run systems using gentoo's openrc. Ceph is interesting, but in the long term will it be mandatory to use systemd to keep using ceph? Will there continue to be a supported branch that works with openrc? Long range guidance is keenly appreciated.

[ceph-users] Raw data size used seems incorrect (version Jewel, 10.2.2)

2016-09-06 Thread james
Hi, Not sure if anyone can help clarify or provide any suggestion on how to troubleshoot this We have a ceph cluster recently build up with ceph version Jewel, 10.2.2. Based on "ceph -s" it shows that the data size is around 3TB but rawdata used is only around 6TB, as the ceph is set with 3

[ceph-users] Some query about using "bcache" as backend of Ceph

2016-10-27 Thread james
Hi, Is there anyone in the community has experience of using "bcache" as backend of Ceph? Nowadays, maybe most Ceph solution are based on full-SSD or full-HDD as backend data disks. So in order to balance the cost and performance/capacity, we are trying the hybrid solution with "bcache". It uti

[ceph-users] Unable to update Swift ACL's on existing containers

2014-05-22 Thread James Page
10.98.191.31/swift/v1/61853c5a-e1d4-11e3-b125-2c768a4f56ac 401 Unauthorized AccessDenied Any ideas? - -- James Page Ubuntu and Debian Developer james.p...@ubuntu.com jamesp...@debian.org -BEGIN PGP SIGNATURE- Version: GnuPG v1 Comment: Using GnuPG with Thunderbird - http://www.e

Re: [ceph-users] Unable to update Swift ACL's on existing containers

2014-05-23 Thread James Page
nd tested - LGTM. Thanks for the quick fix. Cheers James - -- James Page Ubuntu and Debian Developer james.p...@ubuntu.com jamesp...@debian.org -BEGIN PGP SIGNATURE- Version: GnuPG v1 Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQIcBAEBCAAGBQJTf06vAAoJEL/srsu

Re: [ceph-users] v0.67.9 Dumpling released

2014-06-03 Thread James Page
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Hi Sage http://ceph.com/download/ceph-0.67.9.tar.gz appears to be missing - any chance it can be posted so I can SRU it for Ubuntu? Cheers James On 21/05/14 21:02, Sage Weil wrote: > This Dumpling point release fixes several minor bugs. The m

[ceph-users] inconsistent pgs

2014-07-05 Thread James Harper
re the 'out' node stored the only copy of some data, because there were down pgs until I got all the OSD's running again Anyway, after all that dust settled, I now have 5 pgs inconsistent from scrub errors (it was 4 when I started writing this email... I assume more will be

Re: [ceph-users] inconsistent pgs

2014-07-05 Thread James Harper
G::RecoveryState::Crashed::Crashed(boost::statechart::state::my_context)' thread 7fdcb9600700 time 2014-07-05 21:14:57.260547 osd/PG.cc: 5307: FAILED assert(0 == "we got a bad state machine event") (sometimes that would appear in the 'out' node too). Even after the reba

Re: [ceph-users] Release notes for firefly not very clear wrt the tunables

2014-07-07 Thread James Harper
.15 isn't that common, there > is not even any longterm release above it). > As per an email I sent a few days ago - setting tunables to optimal (or whatever the setting is that makes the warning go away) caused a cascade of OSD crashes. Maybe tho

Re: [ceph-users] inconsistent pgs

2014-07-07 Thread James Harper
p until that point. I haven't touched the tunables since then so I still get the constant warning. I'd kind of prefer to stick with the deb's from ceph.com - I hadn't noticed that they were included in Jessie until it was too late, and qemu now depends on them so I guess I&

Re: [ceph-users] inconsistent pgs

2014-07-07 Thread James Harper
the pg's on them were still running (and that there should have been other OSDs with replicas)? My setup is with 2 replicas. Thanks James ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] inconsistent pgs

2014-07-07 Thread James Harper
tely the case (all copies of a pg on a single osd or a single node)? Thanks James ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] inconsistent pgs

2014-07-07 Thread James Harper
ty then no pg should be replicated to an osd in the same node. Probably the same for other levels in the hierarchy (rack, etc) too. Is there a health check I can run that can tell me that my cluster is all as it should be? Thanks James ___ ceph-us

Re: [ceph-users] Using large SSD cache tier instead of SSD journals?

2014-07-08 Thread James Harper
;s in the kernel since 3.10 I think. It would be interesting to see comparisons between no ssd, journal on ssd, and bcache with ssd (with journal on same fs as osd) James ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] Using large SSD cache tier instead of SSD journals?

2014-07-08 Thread James Harper
> Hi James, > > Yes, I've checked bcache, but as far as I can tell you need to manually > configure and register the backing devices and attach them to the cache > device, which is not really suitable to dynamic environment (like RBD devices > for cloud VMs). > You wo

[ceph-users] logrotate

2014-07-10 Thread James Eckersall
Hi, I've just upgraded a ceph cluster from Ubuntu 12.04 with 0.72.1 to Ubuntu 14.04 with 0.80.1. I've noticed that the log rotation doesn't appear to work correctly. The OSD's are just not logging to the current ceph-osd-X.log file. If I restart the OSD's or run "service ceph-osd reload id=X", th

[ceph-users] logrotate

2014-07-10 Thread James Eckersall
Hi, I've just upgraded a ceph cluster from Ubuntu 12.04 with 0.73.1 to Ubuntu 14.04 with 0.80.1. I've noticed that the log rotation doesn't appear to work correctly. The OSD's are just not logging to the current ceph-osd-X.log file. If I restart the OSD's, they start logging, but then overnight,

Re: [ceph-users] logrotate

2014-07-11 Thread James Eckersall
hen I don't have a "done" file in the mounted directory for any of my osd's. My mon's all have the done file and logrotate is working fine for those. So my question is, what is the purpose of the "done" file and should I just create one for each of my osd's ?

Re: [ceph-users] logrotate

2014-07-11 Thread James Eckersall
J On 11 July 2014 15:04, Sage Weil wrote: > On Fri, 11 Jul 2014, James Eckersall wrote: > > Upon further investigation, it looks like this part of the ceph logrotate > > script is causing me the problem: > > > > if [ -e "/var/lib/ceph/$daemon/$f/done"

Re: [ceph-users] v0.80.4 Firefly released

2014-07-16 Thread James Harper
Can you offer some comments on what the impact is likely to be to the data in an affected cluster? Should all data now be treated with suspicion and restored back to before the firefly upgrade? James > -Original Message- > From: ceph-users [mailto:ceph-users-boun...@lists.ceph.c

Re: [ceph-users] v0.80.4 Firefly released

2014-07-16 Thread James Harper
minute - 1.55.0 is >= 1.49.0, but libboost-system1.49.0 is a completely different package to libboost-system1.55.0 so they can't be compared. Debian has 0.80 in Jessie but it is too old. It's going to be a bit of a pain for a while. Ceph maintainers: Could we get ceph packages

[ceph-users] health_err on osd full

2014-07-18 Thread James Eckersall
Hi, I have a ceph cluster running on 0.80.1 with 80 OSD's. I've had fairly uneven distribution of the data and have been keeping it ticking along with "ceph osd reweight XX 0.x" commands on a few OSD's while I try and increase the pg count of the pools to hopefully better balance the data. Tonig

Re: [ceph-users] health_err on osd full

2014-07-18 Thread James Eckersall
ily rectified. J -Original Message- From: ceph-users [mailto:ceph-users-boun...@lists.ceph.com] On Behalf Of Gregory Farnum Sent: 18 July 2014 23:25 To: James Eckersall Cc: ceph-users Subject: Re: [ceph-users] health_err on osd full Yes, that's expected behavior. Since the cluster ca

Re: [ceph-users] health_err on osd full

2014-07-18 Thread James Eckersall
Thanks Greg. I appreciate the advice, and very quick replies too :) On 18 July 2014 23:35, Gregory Farnum wrote: > On Fri, Jul 18, 2014 at 3:29 PM, James Eckersall > wrote: > > Thanks Greg. > > > > Can I suggest that the documentation makes this much clearer? It m

[ceph-users] question about FileStore read()/write()

2014-07-22 Thread James Pan
hanks very much. Best regards, James Pan ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

[ceph-users] ceph metrics

2014-07-28 Thread James Eckersall
Hi, I'm trying to understand what a lot of the values mean that are reported by "perf dump" on the ceph admin socket. I have a collectd plugin which sends all of these values to graphite. Does anyone have a cross-reference list that explains what they are in more detail? You can glean so much f

[ceph-users] GPF kernel panics

2014-07-31 Thread James Eckersall
Hi, I've had a fun time with ceph this week. We have a cluster with 4 OSD (20 OSD's per) servers, 3 mons and a server mapping ~200 rbd's and presenting cifs shares. We're using cephx and the export node has its own cephx auth key. I made a change to the key last week, adding rwx access to anothe

Re: [ceph-users] GPF kernel panics

2014-07-31 Thread James Eckersall
cking. Thanks J On 31 July 2014 09:12, Ilya Dryomov wrote: > On Thu, Jul 31, 2014 at 11:44 AM, James Eckersall > wrote: > > Hi, > > > > I've had a fun time with ceph this week. > > We have a cluster with 4 OSD (20 OSD's per) servers, 3 mons and a server >

Re: [ceph-users] GPF kernel panics

2014-07-31 Thread James Eckersall
roaching the maximum amount of kernel mappings, > which is somewhat shy of 250 in any kernel below 3.14? > > If you can easily upgrade to 3.14 see if that fixes it. > > Christian > > On Thu, 31 Jul 2014 09:37:05 +0100 James Eckersall wrote: > > > Hi, > > > >

Re: [ceph-users] GPF kernel panics

2014-07-31 Thread James Eckersall
yet - 20 hours ish and counting). Now to figure out the best way to get a 3.14 kernel in Ubuntu Trusty :) On 31 July 2014 10:23, Christian Balzer wrote: > On Thu, 31 Jul 2014 10:13:11 +0100 James Eckersall wrote: > > > Hi, > > > > I thought the limit was in relation to c

Re: [ceph-users] GPF kernel panics

2014-08-04 Thread James Eckersall
=hosting_windows_sharedweb, allow rwx pool=infra_systems, allow rwx pool=hosting_linux_sharedweb, allow rwx pool=test Thanks J On 1 August 2014 01:17, Brad Hubbard wrote: > On 07/31/2014 06:37 PM, James Eckersall wrote: > >> Hi, >> >> The stacktraces are very similar. Here is a

[ceph-users] ceph cluster expansion

2014-08-13 Thread James Eckersall
Hi, I'm looking for some advice on my ceph cluster. The current setup is as follows: 3 mon servers 4 storage servers with the following spec: 1x Intel Xeon E5-2640 @2.50GHz 6 core (12 with hyperthreading). 64GB DDR3 RAM 2x SSDSC2BB080G4 for OS LSI MegaRAID 9260-16i with the following drives:

Re: [ceph-users] ceph cluster expansion

2014-08-13 Thread James Eckersall
00 servers backing up mostly web content (millions of small files). J On 13 August 2014 10:28, Christian Balzer wrote: > > Hello, > > On Wed, 13 Aug 2014 09:15:34 +0100 James Eckersall wrote: > > > Hi, > > > > I'm looking for some advice on my ceph cluster.

Re: [ceph-users] ceph cluster expansion

2014-08-13 Thread James Eckersall
13 August 2014 14:06, Christian Balzer wrote: > On Wed, 13 Aug 2014 12:47:22 +0100 James Eckersall wrote: > > > Hi Christian, > > > > We're actually using the following chassis: > > http://rnt.de/en/bf_xxlarge.html > > > Ah yes, one of the Blazeback heritage.

[ceph-users] Fwd: Ceph Filesystem - Production?

2014-08-28 Thread James Devine
On Thu, Aug 28, 2014 at 1:30 PM, Gregory Farnum wrote: > On Thu, Aug 28, 2014 at 10:36 AM, Brian C. Huffman > wrote: > > Is Ceph Filesystem ready for production servers? > > > > The documentation says it's not, but I don't see that mentioned anywhere > > else. > > http://ceph.com/docs/master/cep

Re: [ceph-users] Fwd: Ceph Filesystem - Production?

2014-08-29 Thread James Devine
top output if it happens again. On Thu, Aug 28, 2014 at 8:22 PM, Yan, Zheng wrote: > On Fri, Aug 29, 2014 at 8:36 AM, James Devine wrote: > > > > On Thu, Aug 28, 2014 at 1:30 PM, Gregory Farnum > wrote: > >> > >> On Thu, Aug 28, 2014 at 10:36 AM, Brian C.

Re: [ceph-users] Fwd: Ceph Filesystem - Production?

2014-09-04 Thread James Devine
hasn't gone up but has been slowly coming back down. Same symptoms, the mount on the client is unresponsive and a cat on /sys/kernel/debug/ceph/*/mdsc had a whole list of entries. A umount and remount seems to fix it. On Fri, Aug 29, 2014 at 11:26 AM, James Devine wrote: > I am runnin

Re: [ceph-users] Fwd: Ceph Filesystem - Production?

2014-09-04 Thread James Devine
I'm using 3.13.0-35-generic on Ubuntu 14.04.1 On Thu, Sep 4, 2014 at 6:08 PM, Yan, Zheng wrote: > On Fri, Sep 5, 2014 at 3:24 AM, James Devine wrote: > > It took a week to happen again, I had hopes that it was fixed but alas > it is > > not. Looking at top logs on the

Re: [ceph-users] Fwd: Ceph Filesystem - Production?

2014-09-05 Thread James Devine
No messages in dmesg, I've updated the two clients to 3.16, we'll see if that fixes this issue. On Fri, Sep 5, 2014 at 12:28 AM, Yan, Zheng wrote: > On Fri, Sep 5, 2014 at 8:42 AM, James Devine wrote: > > I'm using 3.13.0-35-generic on Ubuntu 14.04.1 > > > &g

Re: [ceph-users] Ceph Filesystem - Production?

2014-09-09 Thread James Devine
because it had 3.9. Once we upgraded the kernel on the client node, > we were able to mount it fine. FWIW, you need kernel 3.14 and above. > > --jiten > > On Sep 5, 2014, at 6:55 AM, James Devine wrote: > > No messages in dmesg, I've updated the two clients to 3.16,

[ceph-users] monitor quorum

2014-09-17 Thread James Eckersall
"rank": 2, "name": "ceph-mon-03", "addr": "10.1.1.66:6789\/0"}]}} { "name": "ceph-mon-03", "rank": 2, "state": "electing", "election_epoch": 931, "quorum": [], "outside_quorum": [], "extra_probe_peers": [], "sync_provider": [], "monmap": { "epoch": 2, "fsid": "XXX", "modified": "0.00", "created": "0.00", "mons": [ { "rank": 0, "name": "ceph-mon-01", "addr": "10.1.1.64:6789\/0"}, { "rank": 1, "name": "ceph-mon-02", "addr": "10.1.1.65:6789\/0"}, { "rank": 2, "name": "ceph-mon-03", "addr": "10.1.1.66:6789\/0"}]}} Any help or advice is appreciated. Regards James ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] monitor quorum

2014-09-17 Thread James Eckersall
Hi, Thanks for the advice. I feel pretty dumb as it does indeed look like a simple networking issue. You know how you check things 5 times and miss the most obvious one... J On 17 September 2014 16:04, Florian Haas wrote: > On Wed, Sep 17, 2014 at 1:58 PM, James Eckersall > wrote:

Re: [ceph-users] monitor quorum

2014-09-17 Thread James Eckersall
her there is something else that can be done to fix this. With hindsight, I would have stopped the mon service before relocating the nic cable, but I expected the mon to survive a short network outage which it doesn't seem to have done :( On 17 September 2014 16:21, James Eckersall wrote

Re: [ceph-users] monitor quorum

2014-09-18 Thread James Eckersall
on.ceph-mon-03@2(electing).elector(947) election timer expired J On 17 September 2014 17:05, James Eckersall wrote: > Hi, > > Now I feel dumb for jumping to the conclusion that it was a simple > networking issue - it isn't. > I've just checked connectivity properly and I can

Re: [ceph-users] monitor quorum

2014-09-19 Thread James Eckersall
}, { "rank": 2, "name": "ceph-mon-03", "addr": "10.1.1.66:6789\/0"}]}} I'm really struggling to know what to do now, since even removing this monitor and re-creating it didn't seem to fix the proble

Re: [ceph-users] who is using radosgw with civetweb?

2015-02-26 Thread James Page
u in our 15.01 charm release - mod-fastcgi was causing so many headaches! So far in our internal QA cloud its been very reliable running with civetweb - we run three units of the ceph-radosgw charm frontend by haproxy + VIP. I'd +1 switching focus to this approach. Cheers James - -- James

[ceph-users] ceph-create-keys hanging when executed on openSUSE 13.2

2015-03-01 Thread James Oakley
ceph-mon log: http://pastebin.com/ndaYLPYa ceph-create-keys output: http://pastebin.com/wXT1U1wb Does anybody have an idea what might be wrong here? -- James Oakley jf...@funktronics.ca ___ ceph-users mailing list ceph-users@lists.ceph.com

Re: [ceph-users] Heads up: libvirt produces unusable images from RBD pool on Ubuntu trusty

2015-04-22 Thread James Page
ning ;)'. Nice spot - and this is not the first time I've seen a bug due to incorrect specification of the stripe size for rbd images. - -- James Page Ubuntu and Debian Developer james.p...@ubuntu.com jamesp...@debian.org -BEGIN PGP SIGNATURE- Version: GnuPG v2 iQIcBAEBCA

Re: [ceph-users] Packages for Debian jessie, Ubuntu vivid etc

2015-04-22 Thread James Page
e on firefly at this time). trusty and utopic are firefly (0.80.9). vivid is hammer (0.94.1) - this is also available for trusty via the Kilo Cloud Archive - see [0]. [0] https://wiki.ubuntu.com/ServerTeam/CloudArchive - -- James Page Ubuntu and Debian Developer james.p...@ubuntu.com jamesp...@

[ceph-users] ceph-dokan mount error

2015-04-30 Thread James Devine
So I am trying to get ceph-dokan to work. Upon running it with ./ceph-dokan.exe -c ceph.conf -l e it indicates there was a mount error and the monitor it connects to logs cephx server client.admin: unexpected key: req.key=0 expected_key=d7901d515f6b0c61 According to the debug output attached ceph

Re: [ceph-users] ceph-dokan mount error

2015-04-30 Thread James Devine
Yup I think you are correct, I see this listed under issues https://github.com/ketor/ceph-dokan/issues/5 On Thu, Apr 30, 2015 at 12:58 PM, Gregory Farnum wrote: > On Thu, Apr 30, 2015 at 9:49 AM, James Devine wrote: >> So I am trying to get ceph-dokan to work. Upon running it with

[ceph-users] HEALTH_WARN low disk space

2013-07-13 Thread James Harper
n has failed to clean up over time... I can't imagine that mon needs 3GB of historic data. Is there a 'clean up' command, or can I just delete them manually? Thanks James ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.c

Re: [ceph-users] HEALTH_WARN low disk space

2013-07-14 Thread James Harper
> > On 07/14/2013 04:27 AM, James Harper wrote: > > My cluster is in HEALTH_WARN state because one of my monitors has low > disk space on /var/lib/ceph. Looking into this in more detail, there are a > bunch of .sst files dating back to Jul 7, and then a lot more at Jun 30 and &

[ceph-users] 1 x raid0 or 2 x disk

2013-07-21 Thread James Harper
rsal thing? :) Thanks James ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

[ceph-users] network layout

2013-07-29 Thread James Harper
dwidth of the "public" network? Or would it work better with eth[1-3] all in a single LACP bonded interface and put the public and cluster traffic together on that? Thanks James ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] kernel BUG at net/ceph/osd_client.c:2103

2013-08-04 Thread James Harper
What VM? If Xen, have you tried the rbd tap client? James > -Original Message- > From: ceph-users-boun...@lists.ceph.com [mailto:ceph-users- > boun...@lists.ceph.com] On Behalf Of Olivier Bonvalet > Sent: Monday, 5 August 2013 11:07 AM > To: ceph-users@lists.ceph.com &

Re: [ceph-users] kernel BUG at net/ceph/osd_client.c:2103

2013-08-05 Thread James Harper
k with the rbd version and follow the other instructions (fix xend restriction). xen-utils-4.1 installs its own tap-ctl too... I seem to remember removing that. As for maturity, that's certainly a valid concern. I've had less trouble with rbd ta

Re: [ceph-users] Block device storage

2013-08-05 Thread James Harper
her node. This is basically what I'm doing for all my VMs. Of course neither approach protects you if the database itself got corrupt (or the VM broke in the VM case), only if the active node itself failed. James ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] Large storage nodes - best practices

2013-08-05 Thread James Harper
Obviously you still have a 50% reduction in disk space, but you have the advantage that your filesystem never sees the bad disk and all the problems that can cause. James ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] Large storage nodes - best practices

2013-08-05 Thread James Harper
oughput as the writes are done in parallel, and you can get increased performance on read I think. I don't use RAID1 on my setup, but then I don't have 24-36 disks per node! James ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

[ceph-users] fuse or kernel fs?

2013-08-06 Thread James Harper
Are the fuse and kernel filesystem drivers about the same or is one definitely better than the other? Thanks James ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] [list admin] - membership disabled due to bounces

2013-08-11 Thread James Harper
ating a problem with your account. Probably the only thing to do is to white list the address and put up with the spam. James > -Original Message- > From: ceph-users-boun...@lists.ceph.com [mailto:ceph-users- > boun...@lists.ceph.com] On Behalf Of Alex Bligh > Sent: Sunday,

Re: [ceph-users] qemu-1.4.0 and onwards, linux kernel 3.2.x, ceph-RBD, heavy I/O leads to kernel_hung_tasks_timout_secs message and unresponsive qemu-process, [Qemu-devel] [Bug 1207686]

2013-08-13 Thread James Harper
ly I can't get gdb to work with pthreads so I can't get a backtrace. James ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] v0.67 Dumpling released

2013-08-14 Thread James Harper
> Hi, > is it ok to upgrade from 0.66 to 0.67 by just running 'apt-get upgrade' > and rebooting the nodes one by one ? Is a full reboot required? James ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/list

Re: [ceph-users] v0.61.8 Cuttlefish released

2013-08-19 Thread James Harper
> * librados: fix async aio completion wakeup > * librados: fix aio completion locking > * librados: fix rare deadlock during shutdown Could any of these be causing the segfaults I'm seeing in tapdisk rbd? Are these fixes in dumpling? Thanks James

Re: [ceph-users] v0.61.8 Cuttlefish released

2013-08-19 Thread James Harper
> On Mon, 19 Aug 2013, James Harper wrote: > > > > > > We've made another point release for Cuttlefish. This release contains a > > > number of fixes that are generally not individually critical, but do trip > > > up users from time to time, are non-int

Re: [ceph-users] Multiple CephFS filesystems per cluster

2013-08-21 Thread James Harper
my ceph fs I have a directory called /installmedia, and I mount that on /var/installmedia, so I do: ceph-fuse -r /installmedia /var/installmedia James ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] The whole cluster hangs when changing MTU to 9216

2013-08-26 Thread James Harper
> > Centos 6.4 > Ceph Cuttlefish 0.61.7, or 0.61.8. > > I changed the MTU to 9216(or 9000), then restarted all the cluster nodes. > The whole cluster hung, with messages in the mon log as below: Does tcpdump report any tcp or ip checksum errors? (tcpdump -v -s0 -i http://lists.ceph.com/listinfo

[ceph-users] Ceph + Xen - RBD io hang

2013-08-27 Thread James Dingwall
here is some unfavourable interaction between ceph/rbd and blkback but I haven't found anything in the dom0 logs so I would like to know if anyone has some suggestions about where to start trying to hunt this down. Thanks, James ___ ceph-users mailing

Re: [ceph-users] 1 particular ceph-mon never jobs on 0.67.2

2013-08-27 Thread James Page
unately. This has come up > before, actually. Previously we would wait for any interface to > come up and then start, but that broke with multi-nic machines, and > I ended up just making things start in runlevel [2345]. > > James, do you know what should be done to make the job wait for

Re: [ceph-users] Ceph + Xen - RBD io hang

2013-08-28 Thread James Dingwall
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. As per the suggestion of Olivier I tried with 3.9.11 for dom0 but with this kernel I didn't get any writes to succeed from the domU at all. The complete set of commands that I executed are given below. Regards,

Re: [ceph-users] How to force lost PGs

2013-09-03 Thread James Harper
a lot of extra metadata. This would allow a filesystem with such capabilities to best recover from the data loss. James ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] Ceph and RAID

2013-10-03 Thread James Harper
so you have gone from 3x to 4x in terms of raw storage vs useable storage. Disks fail more than entire nodes, so depending on your requirements, a 33% increase in storage may be a reasonable tradeoff. James ___ ceph-users mailing list ceph-users@lists.ce

  1   2   3   4   >