Hi Jan,
Responses inline below
> -Original Message-
> From: ceph-users [mailto:ceph-users-boun...@lists.ceph.com] On Behalf Of
> Jan Schermer
> Sent: 25 May 2015 21:14
> To: Nick Fisk
> Cc: ceph-users@lists.ceph.com
> Subject: Re: [ceph-users] Synchronous writes - tuning and some thoughts
Hi all,I have some questions about using pure SSD pool. The typical usage of
SSD in ceph is using it as journal for SATA OSDs. If we have an pure SSD pool,
do we still need journaling in SSD itself? If so, all data will be stored twice
per SSD. Typically, the journal partition is much smaller th
Hello,
Slow requests, blocked requests and blocked ops occur quite often
in our cluster; too often, I'd say: several times during one day.
I must say we are running some tests, but we are far from pushing
the cluster to the limit (or at least, that's what I believe).
Every time a blocked request/
What queue depth are you testing at?
You will struggle to get much more than about 500iops for a single threaded
write, no matter what the backing disk is.
Nick
From: ceph-users [mailto:ceph-users-boun...@lists.ceph.com] On Behalf Of
lixuehui...@126.com
Sent: 27 May 2015 00:55
To: Vas
Hello,
On Wed, 27 May 2015 18:52:29 +0800 zhenhua.zhang wrote:
> Hi all,I have some questions about using pure SSD pool.
There are many threads about this here, a little research aka google will
be quite helpful.
> The typical
> usage of SSD in ceph is using it as journal for SATA OSDs. If we
Hello,
On Wed, 27 May 2015 12:54:04 +0200 Xavier Serrano wrote:
> Hello,
>
> Slow requests, blocked requests and blocked ops occur quite often
> in our cluster; too often, I'd say: several times during one day.
> I must say we are running some tests, but we are far from pushing
> the cluster to
Hi!
>Does this make sense to you? Any other thoughts?
Yes, we're use:
>osd max backfills = 2
>osd recovery max active = 2
on 5 node 58 osds cluster. The duration of one OSD full recovery is ~4 hours.
Such tuning do not harm client io - we observe only 20-30% performance
degradation.
Megov Igor
Hi
Thanks for your comments
I'll indeed put the OS Controller on, when we get our replacement CPU's and try
what you described here.
If there isn't any guide for this yet, should there be?
Br,
Tuomas
-Original Message-
From: ceph-users [mailto:ceph-users-boun...@lists.ceph.com] On Beh
Garg, Pankaj writes:
>
>
>
> Hi,
>
> What block size does ceph use, and what is the most optimal size? I’m
assuming it uses whatever the file system has been formatted with.
>
> Thanks
> Pankaj
>
>
>
> ___
> ceph-users mailing list
> ceph-user
Hello,
On Wed May 27 21:20:49 2015, Christian Balzer wrote:
>
> Hello,
>
> On Wed, 27 May 2015 12:54:04 +0200 Xavier Serrano wrote:
>
> > Hello,
> >
> > Slow requests, blocked requests and blocked ops occur quite often
> > in our cluster; too often, I'd say: several times during one day.
> >
Just as a reminder, our monthly Ceph Tech Talk will be happening today
(instead of its usual time on a Thurs) at 1p EDT (a little over 3
hours from now). Florian from Hastexo will be discussing placement
groups which is especially important if you are planning a cluster or
looking to expand.
Whil
We are also running a full backup sync to cephfs, using multiple
distributed rsync streams (with zkrsync), and also ran in this issue
today on Hammer 0.94.1 .
After setting the beacon higer, and eventually clearing the journal, it
stabilized again.
We were using ceph-fuse to mount the cephfs,
George,
I will let Christian provide you the details. As far as I know, it was enough
to just do a ‘ls’ on all of the attached drives.
we are using Qemu 2.0:
$ dpkg -l | grep qemu
ii ipxe-qemu 1.0.0+git-2013.c3d1e78-2ubuntu1
all PXE boot firmware - ROM
Hi Nick,
responses inline, again ;-)
Thanks
Jan
> On 27 May 2015, at 12:29, Nick Fisk wrote:
>
> Hi Jan,
>
> Responses inline below
>
>> -Original Message-
>> From: ceph-users [mailto:ceph-users-boun...@lists.ceph.com] On Behalf Of
>> Jan Schermer
>> Sent: 25 May 2015 21:14
>> To: Ni
I was under the impression that "ceph-disk activate" would take care of
setting OSD weights. In fact, the documentation for adding OSDs, the "short
form", only talks about running ceph-disk prepare and activate:
http://ceph.com/docs/master/install/manual-deployment/#adding-osds
This is also how t
On 05/27/2015 09:33 AM, Jan Schermer wrote:
Hi Nick,
responses inline, again ;-)
Thanks
Jan
On 27 May 2015, at 12:29, Nick Fisk wrote:
Hi Jan,
Responses inline below
-Original Message-
From: ceph-users [mailto:ceph-users-boun...@lists.ceph.com] On Behalf Of
Jan Schermer
Sent: 25
On 05/23/2015 08:26 AM, Abhishek L wrote:
Gregory Farnum writes:
On Thu, May 21, 2015 at 8:24 AM, Kenneth Waegeman
wrote:
Hi,
Some strange issue wrt boolean values in the config:
this works:
osd_crush_update_on_start = 0 -> osd not updated
osd_crush_update_on_start = 1 -> osd updated
I
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256
With all the talk of tcmalloc and jemalloc, I decided to do some
testing og the different memory allocating technologies between KVM
and Ceph. These tests were done a pre-production system so I've tried
to remove some the variance with many runs and
On Thu, May 28, 2015 at 1:40 AM, Robert LeBlanc wrote:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA256
>
> With all the talk of tcmalloc and jemalloc, I decided to do some
> testing og the different memory allocating technologies between KVM
> and Ceph. These tests were done a pre-production s
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256
The workload is on average, 17KB per read request and 13KB per write
request with 73% read abd 27% write. This is a web hosting workload.
-
Robert LeBlanc
GPG Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1
On Wed, M
On 05/27/2015 12:40 PM, Robert LeBlanc wrote:
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256
With all the talk of tcmalloc and jemalloc, I decided to do some
testing og the different memory allocating technologies between KVM
and Ceph. These tests were done a pre-production system so I've tried
On Wed, May 27, 2015 at 6:49 AM, Kenneth Waegeman
wrote:
> We are also running a full backup sync to cephfs, using multiple distributed
> rsync streams (with zkrsync), and also ran in this issue today on Hammer
> 0.94.1 .
> After setting the beacon higer, and eventually clearing the journal, it
>
Sorry for the delay; I've been traveling.
On Sun, May 17, 2015 at 3:49 PM, Francois Lafont wrote:
> Hi,
>
> Sorry for my late answer.
>
> Gregory Farnum wrote:
>
>>> 1. Is this kind of freeze normal? Can I avoid these freezes with a
>>> more recent version of the kernel in the client?
>>
>> Yes,
Hi,
Is there a particular verion of Ceph-Deploy that should be used with Hammer
release? This is a brand new cluster.
I'm getting the following error when running command : ceph-deploy mon
create-initial
[ceph_deploy.conf][DEBUG ] found configuration file at:
/home/cephuser/.cephdeploy.conf
[ce
On Sun, May 17, 2015 at 5:08 PM, Francois Lafont wrote:
> Hi,
>
> Wido den Hollander wrote:
>
>> Aren't snapshots something that should protect you against removal? IF
>> snapshots work properly in CephFS you could create a snapshot every hour.
>
> Are you talking about the .snap/ directory in a c
Glad you figured it out!
In the future you can also do repairs based on the underlying RADOS
objects. Generally speaking errors like this mean that the replicas
are storing objects that don't match, but if you go to each OSD
storing the object and find the raw file you will generally find that
two
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256
On Wed, May 27, 2015 at 2:06 PM, Mark Nelson wrote:
>> Compiling Ceph entirely with jemalloc overall had a negative
>> performance impact. This may be due to dynamically linking to RocksDB
>> instead of the default static linking.
>
>
> Is it possi
The max target limit is a hard limit: the OSDs won't let more than
that amount of data in the cache tier. They will start flushing and
evicting based on the percentage ratios you can set (I don't remember
the exact parameter names) and you may need to set these more
aggressively for your given work
On Tue, May 19, 2015 at 7:35 PM, John Peebles wrote:
> Hi,
>
> I'm hoping for advice on whether Ceph could be used in an atypical use case.
> Specifically, I have about ~20TB of files that need replicated to 2
> different sites. Each site has its own internal gigabit ethernet network.
> However, t
I seem to be getting these errors in the Monitor Log :
2015-05-27 21:17:41.908839 3ff907368e0 -1
erasure_code_init(jerasure,/usr/lib/aarch64-linux-gnu/ceph/erasure-code): (5)
Input/output error
2015-05-27 21:17:41.978113 3ff969168e0 0 ceph version 0.94.1
(e4bfad3a3c51054df7e537a724c8d0bf9be972f
On 05/27/2015 04:00 PM, Robert LeBlanc wrote:
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256
On Wed, May 27, 2015 at 2:06 PM, Mark Nelson wrote:
Compiling Ceph entirely with jemalloc overall had a negative
performance impact. This may be due to dynamically linking to RocksDB
instead of the
Are you running this on ARM ?
If not, it should not go for loading this library.
Thanks & Regards
Somnath
From: ceph-users [mailto:ceph-users-boun...@lists.ceph.com] On Behalf Of Garg,
Pankaj
Sent: Wednesday, May 27, 2015 2:26 PM
To: Garg, Pankaj; ceph-users@lists.ceph.com
Subject: Re: [ceph-use
Yes I am on ARM.
-Pankaj
On May 27, 2015 3:58 PM, Somnath Roy wrote:
Are you running this on ARM ?
If not, it should not go for loading this library.
Thanks & Regards
Somnath
From: ceph-users [mailto:ceph-users-bounces@lists.ceph.com]
On Behalf Of Garg, Pankaj
Sent: W
If you are trying to install the ceph repo hammer binaries, I don’t think it is
built for ARM. Both binary and the .so needs to be built in ARM to make this
work I guess.
Try to build hammer code base in your ARM server and then retry.
Thanks & Regards
Somnath
From: Pankaj Garg [mailto:pankaj.g
Actually the ARM binaries do exist and I have been using for previous releases.
Somehow this library is the one that doesn’t load.
Anyway I did compile my own Ceph for ARM, and now getting the following issue:
[ceph_deploy.gatherkeys][WARNIN] Unable to find
/etc/ceph/ceph.client.admin.keyring on
On Mon, May 18, 2015 at 9:34 AM, Brian Rak wrote:
> We just enabled a small cache pool on one of our clusters (v 0.94.1) and
> have run into some issues:
>
> 1) Cache population appears to happen via the public network (not the
> cluster network). We're seeing basically no traffic on the cluster
On Wed, 27 May 2015 14:06:43 -0700 Gregory Farnum wrote:
> On Tue, May 19, 2015 at 7:35 PM, John Peebles wrote:
> > Hi,
> >
> > I'm hoping for advice on whether Ceph could be used in an atypical use
> > case. Specifically, I have about ~20TB of files that need replicated
> > to 2 different sites.
On Wed, May 27, 2015 at 6:57 PM, Christian Balzer wrote:
> On Wed, 27 May 2015 14:06:43 -0700 Gregory Farnum wrote:
>
>> On Tue, May 19, 2015 at 7:35 PM, John Peebles wrote:
>> > Hi,
>> >
>> > I'm hoping for advice on whether Ceph could be used in an atypical use
>> > case. Specifically, I have a
On Wed, 27 May 2015 15:38:26 +0200 Xavier Serrano wrote:
> Hello,
>
> On Wed May 27 21:20:49 2015, Christian Balzer wrote:
>
> >
> > Hello,
> >
> > On Wed, 27 May 2015 12:54:04 +0200 Xavier Serrano wrote:
> >
> > > Hello,
> > >
> > > Slow requests, blocked requests and blocked ops occur quit
Hi Guys,
Running the install-deps.sh script on Debian Squeeze results in the
package 'cryptsetup-bin' not being found (and 'cryptsetup' not being
used).
This is due to the pipe character being deleted.
To fix this, I replaced this line:
-e 's/\|//g;' \
with this line:
-e 's/\s*\|\s*/
On Wed, May 27, 2015 at 3:36 AM, Patrick McGarry wrote:
> Due to popular demand we are expanding the Ceph lists to include a
> Chinese-language list to allow for direct communications for all of
> our friends in China.
>
> ceph...@lists.ceph.com
>
> It was decided that there are many fragmented di
The description of the logging abruptly ending and the journal being
bad really sounds like part of the disk is going back in time. I'm not
sure if XFS internally is set up in such a way that something like
losing part of its journal would allow that?
If any of the OSD developers have the time it'
Jens-Christian Fischer writes:
>
> I think we (i.e. Christian) found the problem:
> We created a test VM with 9 mounted RBD volumes (no NFS server). As soon as
he hit all disks, we started to experience these 120 second timeouts. We
realized that the QEMU process on the hypervisor is opening a
43 matches
Mail list logo
