on the cache pool OSDs with zero length.
Can anybody unbaffle me? ^o^
Christian
--
Christian BalzerNetwork/Systems Engineer
ch...@gol.com Global OnLine Japan/Rakuten Communications
http://www.gol.com/
___
ceph-users mai
Hello Nick,
On Fri, 26 Feb 2016 09:46:03 - Nick Fisk wrote:
> Hi Christian,
>
> > -Original Message-
> > From: ceph-users [mailto:ceph-users-boun...@lists.ceph.com] On Behalf
> > Of Christian Balzer
> > Sent: 26 February 2016 09:07
> > To: c
prise level" product.
On the top of my list is of course the lack of object checksums and
resulting inability of Ceph to recover automatically and safely from scrub
errors. All that redundancy, only for having to play Russian roulette when
comes to fix data corruption.
Christian
--
gt; nodes, and they're working very well. Latency is consistently 1/10th of
> any SATA SSD that I've tested. I'm keen to test some of the larger 2.5"
> NVME SSDs coming to market for use as an OSD. Those are hitting around
> $1.20. I'd also like to try using an m.2
Christian
> Cheers,
> Shinobu
>
> ----- Original Message -
> From: "Christian Balzer"
> To: ceph-users@lists.ceph.com
> Sent: Thursday, February 25, 2016 12:10:41 PM
> Subject: [ceph-users] Observations with a SSD based pool under Hammer
>
>
> Hello,
>
>
; > As a matter of fact, I just “virsh destroyed” the VM, to see if I
> > > could
> > start
> > > it again… and I cant :
> >
> > > # virsh start yyy
> > > error: Failed to start domain yyy
> > > error: internal error: process exited while conne
should have been deleted on upgrade by Infernalis, isn't it ?
> >>>>
> >>>> What are the official recommendations about this ? Should I have to
> >>>> delete old "ceph" script myself and enable all new services ? (and
> >>>
rom osd.4 since back 2016-02-29 05:55:30.452505 front
> 2016-02-29 05:55:30.452505 (cutoff 2016-02-29 06:08:39.230753)
> 2016-02-29 06:08:59.230773 7fd5c5425700 -1 osd.3 14877 heartbeat_check:
> no reply from osd.7 since back 2016-02-29 05:41:52.790422 front
> 2016-02-29 05:41:52.790422 (c
eighted out OSDs or empty ones, it is much less impactful. If you get
> your upgrade done before ours, I'd like to know how it went. I'll be
> posting the results from ours when it is done.
>
> Sent from a mobile device, please excuse any typos.
> On Feb 24, 2016 5:43 PM, &q
Talking to myself again ^o^, see below:
On Sat, 27 Feb 2016 01:48:49 +0900 Christian Balzer wrote:
>
> Hello Nick,
>
> On Fri, 26 Feb 2016 09:46:03 - Nick Fisk wrote:
>
> > Hi Christian,
> >
> > > -Original Message-
> >
ity clause, please notify the
> sender immediately by return email and delete or destroy all copies of
> the email. Any confidentiality, privilege or copyright is not waived or
> lost because this email has been sent to you by mistake.
>
ule is 4.x something or other and LSA, now
> Avago has released P9 through P12 in the past year. When I first
> started using them, I was on the P9 firmware and kernel module, which
> I built from the sources they supply. At this point most of my infra
> is on the P10 version. I've
subject to eviction, even lukewarm ones get to
stay.
Note that for objects that more or less belong in the cache we're using
less than 15% of its capacity.
Christian
--
Christian BalzerNetwork/Systems Engineer
ch...@gol.com Global OnLine Japan/Rakuten Communi
tely recommended,
given the problems the older stuff has.
Christian
> Cheers,
> Shinobu
>
>
> On Fri, Mar 4, 2016 at 3:17 PM, Christian Balzer wrote:
> >
> > Hello,
> >
> > On Mon, 29 Feb 2016 15:00:08 -0800 Heath Albritton wrote:
> >
> >> > Did
your help.
> Confidentiality: This email and any attachments are confidential and may
> be subject to copyright, legal or some other professional privilege.
> They are intended solely for the attention and use of the named
> addressee(s). They may only be copied, distributed or disclose
00000
>total used 2080570792 25030755
>
> It a bug or predictable action?
>
If you didn't set the cache to forward mode first, it will fill up again
immediately.
Christian
--
Christian BalzerNetwork/System
On Sat, 5 Mar 2016 06:08:49 +0100 Francois Lafont wrote:
> Hello,
>
> On 04/03/2016 09:17, Christian Balzer wrote:
>
> > Unlike the subject may suggest, I'm mostly going to try and explain how
> > things work with cache tiers, as far as I understand them.
> >
rget ratio.
Lastly, while we have perf counters like "tier_dirty", a gauge for dirty
and clean objects/bytes would be quite useful to me at least.
And clearly the cache tier agent already has those numbers.
Right now I'm guestimating that most of my cache objects are actually
clean
very well known and predictable IO patterns (nearly 98% writes only).
Though when configuring Ceph for a broad variance of uses, classic deploys
scale a lot better.
Christian
> Very much appreciate your insight and advice.
>
> Cheers,
> Adrian
>
>
>
> >
> >
>
;└─9586 /usr/bin/ceph-osd -f --cluster ceph --id 14 --setuser
> ceph ...
>
> ceph-osd ID's are automatically found, but ceph-mon ID's are not !
>
> "systemctl start ceph.target" is not starting MONs on my systems !
>
&
Hello nick,
On Mon, 7 Mar 2016 08:30:52 - Nick Fisk wrote:
> Hi Christian,
>
> > -Original Message-
> > From: ceph-users [mailto:ceph-users-boun...@lists.ceph.com] On Behalf
> > Of Christian Balzer
> > Sent: 07 March 2016 02:22
> > To: ceph
t tools to figure this out is running atop on your storage
nodes.
Christian
> Does anyone meet the similar issue? How to tune the performance and make
> it stable?
>
> Thanks!
>
> Peifeng Lin
>
>
>
--
Christian BalzerNetwork/Systems Engineer
can improve things, but I guess your cluster
has more fundamental problems than this.
http://lists.ceph.com/pipermail/ceph-users-ceph.com/2014-April/028552.html
Christian
--
Christian BalzerNetwork/Systems Engineer
ch...@gol.com Global OnLine Japan/Rakuten Communicati
, nothing Ceph related.
But the HW controller cache nicely masks the garbage collection spikes and
slowness of SYNC writes of these SSDs in medium load scenarios.
In short, HW cache should always help, but it can do only so much (for so
long) so unless you already have HW with it or can get it dir
pgs: 920
> active+clean; 427 GB data, 917 GB used, 1413 GB / 2456 GB avail; 4832 B/s
> rd, 33827 B/s wr, 2 op/s
> 2016-03-14 09:00:14.825715 mon.0 [INF] pgmap v4110217: 920 pgs: 920
> active+clean; 427 GB data, 917 GB used, 1413 GB / 2456 GB avail; 7151 B/s
> rd, 111 kB/s wr, 22 op/s
>
Hello,
On Mon, 14 Mar 2016 15:51:11 +0200 Yair Magnezi wrote:
> On Fri, Mar 11, 2016 at 2:01 AM, Christian Balzer wrote:
>
> >
> > Hello,
> >
> > As alway there are many similar threads in here, googling and reading
> > up stuff are good for you.
> &
nfernalis) exposed the bug(s) when running with EC
backing pools.
Some cache pool members acting upon the recency and others not might
confuse things, but you'd think that this is a per OSD (PG) thing and
objects not promoted being acted upon accordingly.
Those new nodes had no mon
On Mon, 14 Mar 2016 23:38:24 -0700 Blade Doyle wrote:
> On Mon, Mar 14, 2016 at 3:48 PM, Christian Balzer wrote:
>
> >
> > Hello,
> >
> > On Mon, 14 Mar 2016 09:16:13 -0700 Blade Doyle wrote:
> >
> > > Hi Ceph Community,
> > >
> > >
zTLa
> =Wk/a
> -END PGP SIGNATURE-
> --------
> Robert LeBlanc
> PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1
>
>
> On Mon, Mar 14, 2016 at 9:35 PM, Christian Balzer wrote:
> >
> > Hello,
> >
> > On Mon, 14 Mar 2016
+972 7 32862423 //
> Mobile +972 50 575-2955__*
>
>
>
> On Tue, Mar 15, 2016 at 2:28 AM, Christian Balzer wrote:
>
> >
> > Hello,
> >
> > On Mon, 14 Mar 2016 15:51:11 +0200 Yair Magnezi wrote:
> >
> &
Hello,
On Wed, 16 Mar 2016 16:22:06 + Stephen Harker wrote:
> On 2016-02-17 11:07, Christian Balzer wrote:
> >
> > On Wed, 17 Feb 2016 10:04:11 +0100 Piotr Wachowicz wrote:
> >
> >> > > Let's consider both cases:
> >> > > Journals
ocation group is using RAID1/10/5/6). I've never seen this kind
> of behavior documented for Ceph.
>
Ditto.
And if/when Ceph has reliable checksumming (in the storage layer) it
should definitely get auto-repair abilities as well.
Christian
--
Christian BalzerNetwork/Systems Engineer
he tier is a client from the Ceph
perspective.
Unfortunate, but AFAIK there are no plans to change this behavior.
> I appreciate any pointers you might have for me.
>
You will also want to definitely read the recent thread titled
"data corruption with hammer".
Christian
--
;
Nope, but Infernalis breaks things with EC pools at least as well, AFAIK.
Jewel should be fine.
> One other question I have is would it make sense to run RAID-0 for
> improved write performance if that's something I value over more OSDs
> pe
t;> I appreciate any pointers you might have for me.
> > >
> > >You will also want to definitely read the recent thread titled
> > >"data corruption with hammer".
> > >Not sure this is relevant for the version I'm running?
> > >
noo Engineering AG
> Kirschgartenstrasse 5
> CH-4051 Basel
>
> Tel: +41 61 228 94 44
> Fax: +41 61 228 94 49
>
> erik.schwa...@canoo.com
> http://www.canoo.com
--
Christian BalzerNetwork/Systems Engineer
ch...@gol.com Global OnLine J
gt; host cf03 {
> id -4 # do not change unnecessarily
> # weight 5.000
> alg straw
> hash 0 # rjenkins1
> item osd.5 weight 1.000
> item osd.6 weight 1.000
> item osd.7 weight 1.000
> item osd.13 weight 1.000
> item
with write performance about 300 M/s;
> 5 MONs; 1 MDS. Ceph version 0.94.6
> (e832001feaf8c176593e0325c8298e3f16dfb403).
>
> Thanks :)
--
Christian BalzerNetwork/Systems Engineer
ch...@gol.com Global OnLine Japan/Rakuten Communications
http://www.gol.com/
ZE AVAIL RAW USED %RAW USED
> 2456G 1492G 839G 34.16
> POOLS:
> NAMEID USED %USED MAX AVAIL OBJECTS
> rbd 0139G 5.66 185G 36499
> cephfs_data 1
ckfill/recovery done directly to the
> > storage tier?
> >
> > Thanks and regards,
> >
> > Mike
> > ___
> > ceph-users mailing list
> > ceph-users@lists.ceph.com
> > http://lists.ceph.com/listinfo.cgi/ce
:6789/0
> }
> election epoch 28, quorum 0,1,2,3,4
> GGZ-YG-S0311-PLATFORM-138,1,2,3,4
> mdsmap e55: 1/1/1 up {0=1=up:active}
> osdmap e1290: 20 osds: 20 up, 20 in
> pgmap v7180: 1000 pgs, 2 pools, 14925 MB data, 3851 objects
> 37827 M
>
> ___
> ceph-users mailing list
> ceph-users@lists.ceph.com
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>
--
Christian BalzerNetwork/Systems Engineer
ch...@gol.com Global OnLine
much memory does your test machine have) when it
gets flushed.
Christian
--
Christian BalzerNetwork/Systems Engineer
ch...@gol.com Global OnLine Japan/Rakuten Communications
http://www.gol.com/
___
ceph-users mailing li
om/hnuzhoulin/836f9e633b90041e89ad
>
> so what's the recommend steps.
That depends on how you deleted it, but at this point your data is likely
to be mostly stale anyway, so I'd start from scratch.
Christian
--
Christian BalzerNetwork/Systems Engineer
ch
nd objects but we haven't probed all sources,
>
> detail see:
> http://lists.ceph.com/pipermail/ceph-users-ceph.com/2016-March/008452.html
>
> Thanks again.
>
> > >
> > > I try to using ceph-osd to add it,but it can not start.log are paste
> > >
:00 lin zhou :
> > Thanks.I try this method just like ceph document say.
> > But I just test osd.6 in this way,and the leveldb of osd.6 is
> > broken.so it can not start.
> >
> > When I try this for other osd,it works.
> >
> > 2016-03-29 8:22 GMT+08:00 C
Hello,
On Tue, 29 Mar 2016 10:32:35 +0200 Jacek Jarosiewicz wrote:
> On 03/25/2016 04:39 AM, Christian Balzer wrote:
> >
> > Hello,
> >
> >>
> >> ID WEIGHT REWEIGHT SIZE USEAVAIL %USE VAR
> >>0 1.0 1.0 5585G 2653G 29
e to it as possible,
don't expect anybody who doesn't need format 2 features (like me in our
largest use case) to make that change.
Excessive down time, massive data movement, loss of sparse allocation if
you're not careful are all factors against migration to format 2.
Regard
going to argue about that)
AND puts your signature on top as well, thus causing my mail client to not
quote anything below it.
Christian
--
Christian BalzerNetwork/Systems Engineer
ch...@gol.com Global OnLine Japan/R
stian
> Thanks in advance for any advice,
>
> Shain
>
> *NPR* | Shain Miley | Manager of Infrastructure, Digital Media |
> smi...@npr.org | 202.513.3649
--
Christian BalzerNetwork/Systems Engineer
ch...@gol.com Global OnLine Japan/Rakuten C
Hello,
On Wed, 30 Mar 2016 12:19:57 +0800 lin zhou wrote:
> 2016-03-29 14:50 GMT+08:00 Christian Balzer :
> >
> > Hello,
> >
> > On Tue, 29 Mar 2016 14:00:44 +0800 lin zhou wrote:
> >
> >> Hi,Christian.
> >> When I re-add these OSD(0,3,9,1
s, minb=795KB/s, maxb=795KB/s,
> mint=100032msec, maxt=100032msec
>
> Disk stats (read/write):
> sda: ios=864/28988, merge=0/5738, ticks=31932/1061860,
> in_queue=1093892, util=99.99%
> root@node-65:~#
>
> the lifetime of this SSD is over.
>
> Thanks so much,Chris
_head/DIR_A/DIR_8/DIR_2/DIR_4/default.64340.11\\u\\ushadow\\u.VR0pEp1Nea8buLSqa9TGhLFZQ6co3KO\\u106__head_C5CF428A__c
>
> [root@soi-ceph3 ceph]# md5sum
> /var/lib/ceph/osd/ceph-52/current/12.28a_head/DIR_A/DIR_8/DIR_2/DIR_4/default.64340.11\\u\\ushadow\\u.VR0pEp1Nea8buLSqa9TGhLFZQ6co3KO\\u106__head_C5CF428A__c
> \fb57b
hen I tested it.
>
It can work quite well, depending on your work load (cache size vs. really
hot objects) and the cache mode chosen.
I certainly solved my overload cluster problems described in the thread
"Reducing the impact of OSD restarts (noout ain't uptosnuff)"
with a ca
On Wed, 30 Mar 2016 15:50:07 + Calvin Morrow wrote:
> On Wed, Mar 30, 2016 at 1:27 AM Christian Balzer wrote:
>
> >
> > Hello,
> >
> > On Tue, 29 Mar 2016 18:10:33 + Calvin Morrow wrote:
> >
> > > Ceph cluster with 60 OSDs, Giant 0.87.2. O
ss the same) of course also causes
data movement, but done right (see the thread) the impact can be minimized.
Christian
> Many thanks
>
> Andrei
--
Christian BalzerNetwork/Systems Engineer
ch...@gol.com Global OnLine Japan/Rakuten Communications
http:/
ots of small writes and
> reads. Which of these would be correct?
>
> Also is it a simple process to update PGs on a live system without
> affecting service?
> ___
> ceph-users mailing list
> ceph-users@lists.ceph.com
> http://lists.
86107
>
>
> Am 06.04.2016 um 16:41 schrieb d...@integrityhost.com:
> > Will changing the replication size from 2 to 3 cause huge I/O resources
> > to be used, or does this happen quietly in the background?
> >
> >
> > On 2016-04-06 00:40, Christian Balzer wr
re are flushes but no activity on the HDD OSDs as far as Ceph
seems to be concerned.
I can however match the flushes to actual disk activity on the HDDs
(gathered by collectd), which are otherwise totally dormant.
Can somebody shed some light on this, is it a known problem, in need of
a bug
all at the same time.
>
You want at least 2 cache tier servers from the start and well known,
well tested (LSI timeouts!) SSDs in them.
Christian
> I am happy for any kind of advice.
>
> Thank you !
>
--
Christian BalzerNetwork/Systems Engineer
ch...@gol.com
eed clean objects as things are full" type of flushes, as
this counter is non-zero on my test cluster where I indeed created those
situations.
Christian
> > As, in there are flushes but no activity on the HDD OSDs as far as Ceph
> seems
> > to be concerned.
> > I can
y,
except for the lust for fast, durable storage.
If I were you, I'd get 2 dedicated MON nodes (with few, fastish cores) and
32-64GB RAM, then put the other 3 on your MDS nodes which seem to have
plenty resources to go around.
You will want the dedicated MONs to have the lowest IPs in your net
eturn, so more is not better.
When deploying MONs you should of course make sure that you distribute
them correctly across your failure domains, not all on the same switch, in
the same rack, etc.
Christian
--
Christian BalzerNetwork/Systems Engineer
ch...@gol.com
t to that point, its a long road. Last night i was doing a reweight of
> 3 OSD's from 1.0 to 0.9 ending up in one hdd was going down/out, so i
> had to restart the osd. ( with again IO errors in some of the vm's ).
>
> So based on your article, the cache tier solved your problem, and
On Fri, 8 Apr 2016 07:39:18 + Maxime Guyot wrote:
> Hello,
>
> On 08/04/16 04:47, "ceph-users on behalf of Christian Balzer"
> wrote:
>
> >
> >> 11 OSD nodes:
> >> -SuperMicro 6047R-E1R36L
> >> --2x E5-2603v2
> >Vast
[re-added the ML]
On Fri, 8 Apr 2016 08:30:21 -0500 Brady Deetz wrote:
> On Thu, Apr 7, 2016 at 9:47 PM, Christian Balzer wrote:
>
> >
> > Hello,
> >
> > On Thu, 7 Apr 2016 09:24:46 -0500 Brady Deetz wrote:
> >
> > > I'd appreciate any cr
ff, making it a more linear stream going to and from the cold
> HDDs.
>
That's still a tad random once the cache gets full and starts flushing and
at least 4MB(one object) per write.
But yes, at least for me the backing storage has no issues now with both
promotions and flushing happ
the same data.
>
> I want to do this using commands. (Don't want to modify crush map using
> decompile and recompile path)
>
> Any guidance in this regard will be helpful.
>
>
> Thanks.
>
> Daleep Singh Bais
--
Christian BalzerNetwork/Syst
night, so you can see where my lack of enthusiasm for OSD recreation comes
from.
> If there significant ext4 users that are unwilling
> to recreate their OSDs, now would be the time to speak up.
>
Consider that done.
Christian
--
Christian BalzerNetwork/Systems Engineer
full advantage of the IOPS availalbe.
Christian
--
Christian BalzerNetwork/Systems Engineer
ch...@gol.com Global OnLine Japan/Rakuten Communications
http://www.gol.com/
___
ceph-users mailing list
ceph-users@lists.ceph.c
. Which is just understandable if the
> system will see a 300% Idle Waiting Time for just long enough.
>
>
> --
>
> And yes, as it seems, for clusters which are very busy, especially with
> low hardware ressources, needs much more than the standard config
> can/will d
00's at 365MB/s write speed you'd
be only slightly below your estimated HDD speed of 840MB/s combined and
your network speed of 1GB/s.
I'd look into the 400GB model OR if you're happy with 3DWPD the 3610
model(s).
> 12x 6TB NL SAS drives
> 1x dual port 10 Gb EDA/SFP+ OCP
lti-DC EC
> option, do cross city replication using snapshots
>
The later, I seem to remember that there was work in progress to do this
(snapshot replication) in an automated fashion.
> Just want to get a feel for what I need to consider when we start
> building at this scale.
Hello,
On Mon, 11 Apr 2016 21:12:14 -0400 (EDT) Sage Weil wrote:
> On Tue, 12 Apr 2016, Christian Balzer wrote:
> >
> > Hello,
> >
> > What a lovely missive to start off my working day...
> >
> > On Mon, 11 Apr 2016 17:39:37 -0400 (EDT) Sage Weil wrot
t; > >
> > > >
> > > > --
> > > > Mit freundlichen Grüßen
> > > >
> > > > David Riedl
> > > >
> > > >
> > > >
> > > > WINGcon GmbH Wireless New Generation - Consulting & Soluti
run your Ceph cluster with 2 mons.
> > You want to run 3.
>
> Thus if there are two servers only (which used to use drdb) what would
> be the best solution? Just grab another Linux server and install a ceph
> cluster node without OSDs and a monitor only?
>
Yes, even
On Tue, 12 Apr 2016 10:53:50 +0200 Alwin Antreich wrote:
>
> On 04/12/2016 01:48 AM, Christian Balzer wrote:
> > On Mon, 11 Apr 2016 09:25:35 -0400 (EDT) Jason Dillaman wrote:
> >
> > > In general, RBD "fancy" striping can help under certain workloads
&
seeing is definitely at the far end of what one would expect with
Ceph.
Christian
> Thanks for your help
>
> Andrei
>
> - Original Message -
> > From: "Shinobu Kinjo"
> > To: "Andrei Mikhailovsky"
> > Cc: "Christian Balzer" ,
p testing on ext4.
> >
> > Is this reasonable? If there significant ext4 users that are
> > unwilling to recreate their OSDs, now would be the time to speak up.
> >
> > Thanks!
> > sage
> >
> > ___
> > c
CephFS is using the same/similar hashing to create object names as it
does with RBD images I'd imagine you're OK.
Christian
--
Christian BalzerNetwork/Systems Engineer
ch...@gol.com Global OnLine Japan/Rakuten Communications
http://www.gol.com/
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
rger than the default 256Byte Ext4 inodes (2KB in my
case) as it isn't purely academic for me.
Or maybe other people like "Michael Metz-Martini" who need Ext4 for
performance reasons and can't obviously go to BlueStore yet.
>
> On Tue, 12 Apr 2016, Christian Balzer wrote:
for some more thoughts.
As for CephFS, I think fsck is upcoming in Jewel, but don't quote me on
that, use google and the Ceph Release page.
Christian
--
Christian BalzerNetwork/Systems Engineer
ch...@gol.com Global OnLine
On Wed, 13 Apr 2016 08:30:52 -0400 (EDT) Sage Weil wrote:
> On Wed, 13 Apr 2016, Christian Balzer wrote:
> > > > Recently we discovered an issue with the long object name handling
> > > > that is not fixable without rewriting a significant chunk of
> > > > F
Hello,
[reducing MLs to ceph-user]
On Wed, 13 Apr 2016 14:51:58 +0200 Michael Metz-Martini | SpeedPartner
GmbH wrote:
> Hi,
>
> Am 13.04.2016 um 04:29 schrieb Christian Balzer:
> > On Tue, 12 Apr 2016 09:00:19 +0200 Michael Metz-Martini | SpeedPartner
> > GmbH wrote:
>
Hello,
[reduced to ceph-users]
On Thu, 14 Apr 2016 11:43:07 +0200 Steffen Weißgerber wrote:
>
>
> >>> Christian Balzer schrieb am Dienstag, 12. April 2016
> >>> um 01:39:
>
> > Hello,
> >
>
> Hi,
>
> > I'm officially
On Thu, 14 Apr 2016 19:39:01 +0200 Michael Metz-Martini | SpeedPartner
GmbH wrote:
> Hi,
>
> Am 14.04.2016 um 03:32 schrieb Christian Balzer:
[massive snip]
Thanks for that tree/du output, it matches what I expected.
You'd think XFS wouldn't be that intimidated by direc
Hello,
On Fri, 15 Apr 2016 07:02:13 +0200 Michael Metz-Martini | SpeedPartner
GmbH wrote:
> Hi,
>
> Am 15.04.2016 um 03:07 schrieb Christian Balzer:
> >> We thought this was a good idea so that we can change the replication
> >> size different for doc_root and raw-
Hello,
On Fri, 15 Apr 2016 08:20:45 +0200 Michael Metz-Martini | SpeedPartner
GmbH wrote:
> Hi,
>
> Am 15.04.2016 um 07:43 schrieb Christian Balzer:
> > On Fri, 15 Apr 2016 07:02:13 +0200 Michael Metz-Martini | SpeedPartner
> > GmbH wrote:
> >> Am 15.04.2016 um
On Mon, 18 Apr 2016 11:46:18 -0700 Gregory Farnum wrote:
> On Sun, Apr 17, 2016 at 9:05 PM, Christian Balzer wrote:
> >
> > Hello,
> >
> > On Fri, 15 Apr 2016 08:20:45 +0200 Michael Metz-Martini | SpeedPartner
> > GmbH wrote:
> >
> >> Hi,
> >
ctor.
Which is something that cache tiers can deal with very well (or
sufficiently large and well designed "plain" clusters).
Another thing to think about is using the "readforward" cache mode,
leaving your cache tier free to just handle writes and thus giving it more
extra config options look to be around how fast promotion
> into the cache can go, not how fast you can get things out of it :-(
>
> I have been using readforward and that is working OK, there is
> sufficient read bandwidth that it does not matter if data is coming from
> the cache pool
per HDD is a mighty good result.
Remember that this isn't a long sequential write, it is 4MB objects with
all the associated FS journal, Ceph metadata and leveldb updates and thus
seeks.
> Can anyone give me a fairly decent idea on how to tune this properly?
There are more knobs like "f
oth cases UUID is the fail-safe way to go.
> *recreate the journal (for the existing osds)*
> *start the osds*
> *unset noout*
Christian
--
Christian BalzerNetwork/Systems Engineer
ch...@gol.com Global OnLine Japan/Rakuten Communications
http://www.gol.com
> 87 1 97 70%
> 92 1 104 79%
> 93 0.9464 102 75%
> 98 0.9082 113 80%
> 99 1 108 77%
> 104 1 107 79%
> 105 1 109 77%
> ___
0.00 94.47
> > >
> > > Device: rrqm/s wrqm/s r/s w/s rkB/swkB/s
> > > avgrq-sz avgqu-sz await r_await w_await svctm %util
> > > sda 0.00 0.000.003.50 0.0017.00
> > > 9.71 0.001.29
you received this
> message in error, please contact the sender and destroy all copies of
> this email and any attachment(s).
>
>
> On Mon, Dec 10, 2018 at 8:57 PM Christian Balzer wrote:
> >
> > Hello,
> >
> > On Mon, 10 Dec 2018 20:43:40 -0500 Tyler Bishop
sibly in the logs.
> The systems were never powered off or anything during the conversion
> from filestore to bluestore.
>
So anything mentioned as well as kernel changes don't apply.
I shall point the the bluestore devs then. >.>
Christian
--
Christian BalzerNetwork/
to this email is prohibited. If you received
> > this message in error, please contact the sender and destroy all
> > copies of this email and any attachment(s).
> >
> >
> > On Mon, Dec 10, 2018 at 8:57 PM Christian Balzer > <mailto:ch...@gol.com>> wrot
g time to repairs and/or a small cluster (is there
even enough space to rebalance a node worth of data?) things may be
different.
I always set "mon_osd_down_out_subtree_limit = host" (and monitor things
of course) since I reckon a down node can often be brought back way faste
;> The Cluster is recovering and remapping fine, but still has some objects
> >> to process.
> >>
> >> My question: May I just switch the server back on and in best case, the 24
> >> OSDs get back online and recovering will do the job without problems.
> &g
1101 - 1200 of 1226 matches
Mail list logo
