t idea how this (lots of RAM)
will apply to or help with Bluestore
Christian
>
>
> >
> > > And
> > > I don't have enough hardware to setup a test cluster of any significant
> > > size to run some actual testing.
> > >
> > You may want to set
e HDDs cluster may be feasible.
Christian
> Thanks,
> Nick
>
> On Tue, Aug 22, 2017 at 6:56 PM, Christian Balzer wrote:
>
> >
> > Hello,
> >
> > On Tue, 22 Aug 2017 16:51:47 +0800 Nick Tan wrote:
> >
> > > Hi Christian,
> >
On Wed, 23 Aug 2017 16:48:12 +0530 M Ranga Swami Reddy wrote:
> On Mon, Aug 21, 2017 at 5:37 PM, Christian Balzer wrote:
> > On Mon, 21 Aug 2017 17:13:10 +0530 M Ranga Swami Reddy wrote:
> >
> >> Thank you.
> >> Here I have NVMes from Intel. but as the supp
PE nodes and less disks per node and this ca influence in the
> throughput of the journal.
>
How busy are your NVMe journals during that test on the Dells and HPs
respectively?
Same for the HDDs.
Again, run longer, larger tests to get something that will actually
register, also atop with shor
Hello,
On Thu, 24 Aug 2017 14:49:24 -0300 Guilherme Steinmüller wrote:
> Hello Christian.
>
> First of all, thanks for your considerations, I really appreciate it.
>
> 2017-08-23 21:34 GMT-03:00 Christian Balzer :
>
> >
> > Hello,
> >
> > On
04-60004msec
>
>
> This implies that for good IO performance only data with blocksize > 128k (I
> guess > 1M) should be used.
> Can anybody confirm this?
>
> THX
> ___
> ceph-users mailing list
> ceph-users@lists.ceph.co
863a 960GB for example.
There are people here who have actually done this, hopefully some will
speak up.
Christian
--
Christian BalzerNetwork/Systems Engineer
ch...@gol.com Rakuten Communications
___
ceph-users mailing l
Hello,
On Thu, 7 Sep 2017 08:03:31 +0200 Stefan Priebe - Profihost AG wrote:
> Hello,
> Am 07.09.2017 um 03:53 schrieb Christian Balzer:
> >
> > Hello,
> >
> > On Wed, 6 Sep 2017 09:09:54 -0400 Alex Gorbachev wrote:
> >
> >> We are planning a Je
s that these may be consumer models again.
> >>> >
> >>> > Samsung also makes DC grade SSDs and NVMEs, as Adrian pointed out.
> >>> >
> >>> >> Btw, if we split this SSD with multiple OSD (for ex: 1 SSD with 4 or 2
> >>> &
>> 2017-09-26 14:28:56.217892 0.fc
> >> 2017-09-26 14:28:49.665678 0.e3
> >> 2017-09-26 14:11:04.718698 0.1d6
> >> 2017-09-26 14:09:44.975028 0.72
> >> 2017-09-26 14:06:17.945012 0.8a
> >> 2017-09-26 13:54:44.199792 0.ec
> >>
> >&g
3 3
> > 0723953 10541k
> >
> > total_objects355409
> > total_used 2847G
> > total_avail 262T
> > total_space 265T
> >
> > However, the data pool is completely empty! So it seems that data has only
> > been written t
alse
> > nopgchange: false
> > nosizechange: false
> > write_fadvise_dontneed: false
> > noscrub: false
> > nodeep-scrub: false
> > hit_set_type: bloom
> > hit_set_period: 14400
> > hit_set_count: 12
> > hit_set_fpp: 0.05
> > use_gmt_hitset
00
> >> > 31 hdd 7.27829 osd.31 up 1.0 1.0
> >> > 32 hdd 7.27829 osd.32 up 1.0 1.0
> >> > 33 hdd 7.27829 osd.33 up 1.0 1.0
> >> > 34 hdd 7.27829 osd.34
t-go and flakiness is
being kind for sometimes completely indeterministic behavior.
Since there never was an (non-disruptive) upgrade process from non-GPT
based OSDs to GPT based ones, I wonder what changed minds here.
Not that the GPT based users won't appreciate it.
Christian
> sage
___
> ceph-users mailing list
> ceph-users@lists.ceph.com
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>
--
Christian BalzerNetwork/Systems Engineer
ch...@gol.com Rakuten Communications
o be happening on roughly a monthly basis.
>
> Thing is I have no idea what the bounce is or where it is coming from.
> I've tried emailing ceph-users-ow...@lists.ceph.com and the contact
> listed in Mailman (l...@redhat.com) to get more info but haven't
> received any respo
digest messages without any original
attachments (in Mailman lingo, it “scrubs” the messages of attachments).
However, Mailman also includes links to the original attachments that the
recipient can click on.
---
Christian
> Cheers,
>
> On 16 October 2017 at 13:54, Christian Balzer
of these and you're well on your way out.
The default mailman settings and logic require 5 bounces to trigger
unsubscription and 7 days of NO bounces to reset the counter.
Christian
On Mon, 16 Oct 2017 12:23:25 +0900 Christian Balzer wrote:
> On Mon, 16 Oct 2017 14:15:22 +1100 Blair B
h or lvm path or whatever. The only downside for
> this approach that I see is that disks themselves would no longer be
> transferable between the hosts as those few files that describe the OSD are
> no longer on the disk itself.
>
If the OS is on a RAID1 the chances of things b
013848
> >>>>>> Cleaning up (deleting benchmark objects)
> >>>>>> Clean up completed and total clean up time :20.166559
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >&
lers
> > and we just tracked down 10 of our nodes that had >100ms await pretty much
> > always were the only 10 nodes in the cluster with failed batteries on the
> > raid controllers.
> >
> > On Thu, Oct 19, 2017, 8:15 PM Christian Balzer wrote:
> >
> >&g
'
>
> Adding --debug shows the ultimate culprit is just the above rocksdb
> error again.
>
> Q: Is there some way in which I can tell rockdb to truncate or delete /
> skip the respective log entries? Or can I get access to rocksdb('s
> files) in some other way to j
___
> ceph-users mailing list
> ceph-users@lists.ceph.com
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>
--
Christian BalzerNetwork/Systems Engineer
ch...@gol.com Rakuten Communications
__
sde1
> root@virt3:~# ceph-disk list | grep /dev/sdf | grep osd
> /dev/sdc1 ceph data, active, cluster ceph, osd.9, block /dev/sdc2,
> block.db /dev/sdf1
> /dev/sdd1 ceph data, active, cluster ceph, osd.10, block /dev/sdd2,
> block.db /dev/sdf2
>
>
>
n doubt by destroying and re-creating.
And if you're looking for a less invasive procedure, docs and the ML
archive, but AFAIK there is nothing but re-creation at this time.
Christian
>
> On Mon, Nov 20, 2017 at 1:44 PM, Christian Balzer wrote:
>
> > On Mon, 20 Nov 2017 12:38:
list
> ceph-users@lists.ceph.com
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>
--
Christian BalzerNetwork/Systems Engineer
ch...@gol.com Rakuten Communications
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
> 196 1.7 osd.196 up 1.0 1.0
> > 197 1.7 osd.197 up 1.0 1.0
> > 198 1.7 osd.198 up 1.0 1.0
> > 199 1.7 osd.199 up 1.0 1.0
> >
On Tue, 21 Nov 2017 10:00:28 +1100 Nigel Williams wrote:
> On 20 November 2017 at 23:36, Christian Balzer wrote:
> > On Mon, 20 Nov 2017 14:02:30 +0200 Rudi Ahlers wrote:
> >> The SATA drives are ST8000NM0055-1RM112
> >>
> > Note that these (while fast) have
On Tue, 21 Nov 2017 10:35:57 +1100 Nigel Williams wrote:
> On 21 November 2017 at 10:07, Christian Balzer wrote:
> > On Tue, 21 Nov 2017 10:00:28 +1100 Nigel Williams wrote:
> >> Is there something in the specifications that gives them away as SSHD?
> >>
> >
, you will REALLY want to monitor a ceph
cluster this way.
--
Christian BalzerNetwork/Systems Engineer
ch...@gol.com Rakuten Communications
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
On Tue, 21 Nov 2017 09:21:58 +0200 Rudi Ahlers wrote:
> On Mon, Nov 20, 2017 at 2:36 PM, Christian Balzer wrote:
>
> > On Mon, 20 Nov 2017 14:02:30 +0200 Rudi Ahlers wrote:
> >
> > > We're planning on installing 12X Virtual Machines with some heavy loads.
>
On Tue, 21 Nov 2017 11:34:51 +0100 Ronny Aasen wrote:
> On 20. nov. 2017 23:06, Christian Balzer wrote:
> > On Mon, 20 Nov 2017 15:53:31 +0100 Ansgar Jazdzewski wrote:
> >
> >> Hi *,
> >>
> >> just on note because we hit it, take a look on your discar
e. Until a reboot was need, when it promptly bricked itself,
data inaccessible, SMART reporting barely that something was there.
So one wonders what caused your SSDs to get their knickers in such a twist.
Are the survivors showing any unusual signs in their SMART output?
Of course what yo
n issue)
and when pushed hard (when it wants/needs to flush to HDD) it will overload
things and doesn't honor I/O priorities as others have mentioned here.
I'm using bcache for now because in my use case the issues above won't
show up, but I'd be wary to use it with Ceph in a cl
__
> >> ceph-users mailing list
> >> ceph-users@lists.ceph.com
> >> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
> >>
> > _______
> > ceph-users mailing list
> > ceph-u
everybody here to verify this by changing
your test cluster to use the other MTU and compare...
> Is the above a good way of measuring our cluster, or is it better more
> reliable ways of measuring it ?
>
See above.
A fio test is definitely a closer thing to reality compared to OSD or
t;
Define terminal server, are we talking Windows Virtual Desktops with RDP?
Windows is quite the hog when it comes to I/O.
Regards,
Christian
--
Christian BalzerNetwork/Systems Engineer
ch...@gol.com Rakuten Communications
___
r ?
>
This is what you get when programmers write docs.
The above is a left-shift operation, see for example:
http://bit-calculator.com/bit-shift-calculator
Now if shrinking that value is beneficial for reducing recovery load,
that's for you to find
ph tell osd.* injectargs '--osd_recovery_sleep .1'
> -
>
>
> Karun Josy
>
> On Tue, Feb 6, 2018 at 1:15 PM, Christian Balzer wrote:
>
> > On Tue, 6 Feb 2018 13:01:12 +0530 Karun Josy wrote:
> >
> > > Hello,
> > >
> > >
,ie 7340032 ?
>
More like 4MB to match things up nicely in the binary world.
Christian
> Karun Josy
>
> On Tue, Feb 6, 2018 at 1:15 PM, Christian Balzer wrote:
>
> > On Tue, 6 Feb 2018 13:01:12 +0530 Karun Josy wrote:
> >
> > > Hello,
> > >
>
Hello,
On Tue, 6 Feb 2018 09:21:22 +0100 Tobias Kropf wrote:
> On 02/06/2018 04:03 AM, Christian Balzer wrote:
> > Hello,
> >
> > On Mon, 5 Feb 2018 22:04:00 +0100 Tobias Kropf wrote:
> >
> >> Hi ceph list,
> >>
> >> we have a hype
Hello,
On Thu, 8 Feb 2018 10:58:43 + Patrik Martinsson wrote:
> Hi Christian,
>
> First of all, thanks for all the great answers and sorry for the late
> reply.
>
You're welcome.
>
> On Tue, 2018-02-06 at 10:47 +0900, Christian Balzer wrote:
> > Hello,
most if
not ALL your hot objects will be in your cache pool and may NEVER be
written to cold storage ever that shouldn't be your primary concern.
Christian
> Thank you.
>
> Florent
>
> ___
> ceph-users mailing list
> ceph-us
from larger CPU caches (e.g. E5-2XXX
> class cpu), or a smaller cache in faster CPUs a better choice (e.g.
> E5-1XXX or E3-1XXXv5)?
>
Nothing solid here really, but I'd suspect that faster code execution will
beat larger caches, as I doubt that the (variable) meta-data will fit in
ther
On Mon, 22 Aug 2016 15:45:52 +0200 Florent B wrote:
> On 08/22/2016 02:48 PM, Christian Balzer wrote:
> > Hello,
> >
> > On Mon, 22 Aug 2016 14:33:51 +0200 Florent B wrote:
> >
> >> Hi,
> >>
> >> I'm looking for informations abou
Hello,
On Mon, 22 Aug 2016 20:34:54 +0100 Nick Fisk wrote:
> > -Original Message-
> > From: Christian Balzer [mailto:ch...@gol.com]
> > Sent: 22 August 2016 03:00
> > To: 'ceph-users'
> > Cc: Nick Fisk
> > Subject: Re: [ceph-us
s again (they brick SSDs that reach max wear-out levels) and a
failed power cap triggers such a rule.
Either way, if you ever encounter this problem, get a replacement ASAP,
and if used as journal SSD, shut down all associated OSDs, flush the
journals and replace it.
Christian
On Wed, 3 Aug
f those OSDs 2 days ago the spread is now 300GB,
most likely NOT helped by the manual adjustments done earlier.
So your nice and evenly distributed cluster during normal state may be
worse off using custom weights when there is a significant OSD loss.
Christian
--
Christian BalzerNetwork/Sy
des.
Not really, the OSD part is only relevant for the respective node, so only
changes (additions/removals) of OSDs that are pertinent to the local node
would need to be reflected in the ceph.conf of that host.
Christian
--
Christian BalzerNetwork/Systems Engineer
ch..
to make up its mind).
Christian
>I saw that it has been redone but with past
> exposure I've been hesitant. I'll give it another shot in a test instance
> and see how it goes.
>
> Thanks for your help as always Mr. Balzer.
>
> On Aug 28, 2016 8:59 PM, "Chr
;
> 51 active+undersized+degraded+remapped+backfill_toofull
>
>6 active+undersized+degraded+remapped
>
>1 active+clean+scrubbing
>
>
>
>
>
> 192.168.113.113,192.168.113.114,192.168.113.72:6789:/ ceph 14T 11T
>
ou don't have enough space within the confines of your settings.
You could increase the osd_backfill_full_ratio but the next stop after
that is the full ratio and when you reach that the cluster will stop
entirely.
So don't do that, (re-)add OSDs.
Christian
>
>
> Regards
>
, still one link of BW if one goes
down).
The later service level can also be achieved by routing (OSPF/BGP) on the
hosts, something that was discussed in here as well.
It's more involved, but can use cheap switches as well.
Christian
--
Christian BalzerNetwork/Systems Engineer
sty amd64 FUSE-based
> >> client
> >> > for the Ceph distributed file system
> >> > ii ceph-mds 0.94.8-1trusty amd64 metadata server
> >> > for the ceph distributed file system
> >> > ii
d+remapped+backfilling
> 2 active+remapped+backfill_toofull
> recovery io 367 MB/s, 96 objects/s
> client io 5699 B/s rd, 23749 B/s wr, 2 op/s rd, 12 op/s wr
--
Christian BalzerNetwork/Systems Engineer
ch...@gol.com Global OnL
0868 osd.4 up 1.0 1.0
> 7 0.90868 osd.7 up 1.0 1.0
> 10 0.90868 osd.10 up 1.0 1.0
> 13 0.90868 osd.13 up 1.0 1.0
> -6 4.54343 host nodeD
> 15 0.90869 osd.15
; Thank you again.
> >
> > I will add 3 more osd today and leave untouched, maybe over weekend.
> >
> > On Thu, Sep 1, 2016 at 1:16 PM, Christian Balzer wrote:
> >>
> >> Hello,
> >>
> >> On Thu, 1 Sep 2016 11:20:33 +0200 Ishmael Tsoaela wrot
hours ago:
>
>
> >> recovery 389973/3096070 objects degraded (12.596%)
> >> recovery 1258984/3096070 objects misplaced (40.664%)
>
> recovery now:
>
> recovery 8917/3217724 objects degraded (0.277%)
> recovery 1120479/3217724 objects misplaced (34.822%)
&g
, so for a sustained,
sequential write test you're looking at the speed of your HDDs.
And that will be spiky of sorts, due to FS journals, seeks for other
writes (replicas), etc.
Christian
> Some more details:
> - ceph version 0.94.7 (d56bdf93ced6b80b07397d57e3fa68fe68304432)
> - L
appen in a rather predictable, roughly 9
seconds interval, suggests also the possibility that the actual journal
got full, but that's not conclusive.
Christian
> Thanks for the quick feedback, and I'll dive into atop and iostat next.
>
> Regards,
> MJ
>
our OSDs, of course.
I tend to graph filestore_journal_bytes with graphite, which is where I got
the numbers in my referred mail from.
> On Sep 6, 2016 8:20 PM, "Christian Balzer" wrote:
>
> >
> > hello,
> >
> > On Tue, 6 Sep 2016 13:38:45 +0200 lists wrote:
> >
or something of that nature.
The only times when a replica of 2 isn't a cry for Murphy to smite you is
with RAID backed OSDs or VERY well monitored and vetted SSDs.
> Thanks in advance,
>
> Shain
>
>
> ___
> ceph-users mailing list
> ceph-users@lis
others might
encounter it, too.
Christian
> This cluster is about 2 or 3 years old at this point and we have not run
> into this issue at all up to this point.
>
> Thanks,
>
> Shain
>
>
> On 09/07/2016 12:00 AM, Christian Balzer wrote:
> > Hello,
> >
&
scrubs even when busy,
otherwise you're looking at trouble when you loose and OSD and the
resulting balancing as well.
Setting these to something sensible:
"osd_scrub_begin_hour": "0",
"osd_scrub_end_hour": "6",
and especially this:
"osd_scrub_slee
Hello,
On Thu, 8 Sep 2016 17:09:27 +0200 (CEST) David DELON wrote:
>
> First, thanks for your answer Christian.
>
C'est rien.
> - Le 8 Sep 16, à 13:30, Christian Balzer ch...@gol.com a écrit :
>
> > Hello,
> >
> > On Thu, 8 Sep 2016 0
nts connecting? I saw this after a Jewel
> > upgrade as well and it was because of very old clients still connecting to
> > the cluster.
> >
> > Wido
> >
> > > Thanks!
> > > ___
> > > ceph-users maili
> > ceph-users mailing list
> > ceph-users@lists.ceph.com
> > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
> _______
> ceph-users mailing list
> ceph-users@lists.ceph.com
> http://lists.ceph.com/listinfo.cgi/
Hello,
On Tue, 20 Sep 2016 14:40:25 +0200 Stefan Priebe - Profihost AG wrote:
> Hi Christian,
>
> Am 20.09.2016 um 13:54 schrieb Christian Balzer:
> > This and the non-permanence of reweight is why I use CRUSH reweight (a
> > more distinct naming would be VERY helpful, to
; http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
> ___
> ceph-users mailing list
> ceph-users@lists.ceph.com
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>
--
Christia
and Cpu.
> You want to get as many inodes into cache as possible.
>
This is the slide show / test in question, btw:
http://www.slideshare.net/XiaoxiChen3/cephfs-jewel-mds-performance-benchmark
> On 26 Sep 2016 2:09 a.m., "Christian Balzer" wrote:
>
> >
> > H
rw-r--r-- 1 root root 0 Sep 9 11:21
> > ./11.1d_head/DIR_D/DIR_1/DIR_0/DIR_7/DIR_9/rbd\udata.2c383a0238e1f29.00019bf7__head_87C9701D__b
>
> But even a 0-byte file takes some space on the disk, might those be the
> reason?
>
> Any feedback welcome.
> Greetings
> -Sascha-
Hello,
On Wed, 28 Sep 2016 19:36:28 +0200 Sascha Vogt wrote:
> Hi Christian,
>
> Am 28.09.2016 um 16:56 schrieb Christian Balzer:
> > 0.94.5 has a well known and documented bug, it doesn't rotate the omap log
> > of the OSDs.
> >
> > Look into "/var
gt; downtime in order to set a limit and cleanup the backlog of stale
> objects from the cache.
>
> Greetings
> -Sascha-
>
> ___
> ceph-users mailing list
> ceph-users@lists.ceph.com
> http://lists.ceph.com/listinfo.cgi/ceph
SSDs (1.6TB).
3000MB/s / 250MB/s (read AND write) = 12 SSDs (good node size)
12 *3 (replication, 2 if you feel brave) = 36 SSDs (bandwidth solved).
Alas for your space needs, 28 of these 12 SSD nodes are required, 336
SSDs total.
Regards,
Christian
--
Christian Balzer
I just love the sound of my own typing...
See inline, below.
On Fri, 30 Sep 2016 12:18:48 +0900 Christian Balzer wrote:
>
> Hello,
>
> On Thu, 29 Sep 2016 20:15:12 +0200 Sascha Vogt wrote:
>
> > Hi Burkhard,
> >
> > On 29/09/16 15:08, Burkhard Linke wrot
st in storage
space and replication overhead that entails.
> Also if the nodes comes back, we are not down, because of
> the replication stuff ;-)
>
Not sure how to parse this sentence.
Do you mean "The design should be able to handle the recovery (backfill)
traffic from a node
become the bottleneck. You also
> > might want to check whether the I/O subsystem is able to drive 24 SSDs
> > (SAS-3 has 12 GBit/s, expander are usually connected with 4 channels
> > -> 6 GB/s).
>
> Our chassis has 12Gb without expander and all drives connected directl
your care about is reducing latency, a single NVMe journal will be
fine (but also be a very obvious SPoF).
Christian
--
Christian BalzerNetwork/Systems Engineer
ch...@gol.com Global OnLine Japan/Rakuten Communications
http://www.gol.com/
read-forward, so cache-miss reads
go directly to the HDDs while time critical writes (and consequently reads
of the same data) go to the NVMes.
Christian
On Thu, 6 Oct 2016 10:04:41 +0900 Christian Balzer wrote:
>
> Hello,
>
> On Wed, 05 Oct 2016 13:43:27 +0200 Denny Fuchs wrot
ve found the X10DRi:
> >
> > https://www.supermicro.com/products/motherboard/xeon/c600/x10dri.cfm
> > <https://www.supermicro.com/products/motherboard/xeon/c600/x10dri.cfm>
> >
> > and I think we use the E5-2637v4 :-)
> >
> > cu denny
> >
> >
> > -
> of disks as the actual RBD images.
> >
> > My understanding is that enabling journaling is going to double the
> > IOPS on the disks, is that correct?
> >
> >
> >
> > Any assistance appreciated
> >
> >
> >
> > Regards,
> >
>
Hello,
On Tue, 11 Oct 2016 08:30:47 +0200 Gandalf Corvotempesta wrote:
> Il 11 ott 2016 3:05 AM, "Christian Balzer" ha scritto:
> > 10Gb/s MC-LAG (white box) switches are also widely available and
> > affordable.
> >
>
> At which models are you refe
> > > I want to ask,
> > > - "Is there any reason *not* have a setup like this, is there any
> > > obvious bottlenecks or flaws that we are missing or could this may
> > > very
> > > well work as good start (and the ability to grow with adding more
&g
> ceph-users mailing list
> ceph-users@lists.ceph.com
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>
--
Christian BalzerNetwork/Systems Engineer
ch...@gol.com Global OnLine Japan/Rakuten Communications
http://www.gol.com/
minf=1570
> IO depths: 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
> submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%,
> >=64=0.0%
> complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%,
> >=64=0.0%
> issued
ored : 105 expected : 102.49
> > device 13: stored : 106 expected : 102.49
> > device 14: stored : 107 expected : 102.49
> > device 15: stored : 107 expected : 102.400009
> > device 16: stored : 101 expected : 102.49
> > device 18: stored : 93 expected : 102.49
> > device 19: s
gt; [simple-write-14]
> numjobs=14
> [simple-write-18]
> numjobs=18
> [simple-write-22]
> numjobs=22
> [simple-write-26]
> numjobs=26
> [simple-write-30]
> numjobs=30
> [simple-write-34]
> numjobs=34
> [simple-write-38]
> numjobs=38
> [simple-write-42]
> nu
for
> cache tiering to do rbd to ec pools ...
>
Well, there's the EC band-aid use case for cache-tiers, but they can be
very helpful otherwise, depending on the size of working set/cache-pool,
configuration of the cache-pool (write-back vs. read-forward) and
specific use case.
Christian
ested by Nick and rerun fio benchmarks. thx will
>
>
> On Tue, Oct 18, 2016 at 9:44 AM, Christian Balzer wrote:
> >
> > Hello,
> >
> > As I had this written mostly already and since it covers some points Nick
> > raised in more detail, here we go.
>
flapping, we had to
> "set noout" the cluster. For now we have no idea what's going on.
>
> Anyone can help us to understand what's happening ?
>
> thanks for your help
>
--
Christian BalzerNetwork/Sy
ive is
> backed up
>
Yes, consumer SSDs on top of a design that channels everything through
them.
Rebuild your cluster along more conventional and conservative lines, don't
use the 850 PROs.
Feel free to run any new design by us.
Christian
--
Christian BalzerNetwork/Systems
ot always the case I guess, and thanks for pointing that
> out.
>
Again, matching things up in terms of speed (network vs journal vs OSD),
endurance and size is both involved and gets costly quickly.
Christian
> Best regards,
> Patrik Martinsson
> Sweden
>
>
> On fre,
.
> 2. Why the impact on the spinning drives is so hard, while we lowered
> the IO priority for it ?
>
That has only a small impact, deep-scrub by its very nature reads all
objects and thus kills I/Os by seeks and polluting caches.
Christian
--
Ch
OPS
>
>
>
> Sent from Mail<https://go.microsoft.com/fwlink/?LinkId=550986> for Windows 10
>
>
>
> From: Jim Kilborn<mailto:j...@kilborns.com>
> Sent: Thursday, October 20, 2016 10:20 AM
> To: Christian Balzer<mailto:ch...@gol.com>;
> ceph-
vailable config options,
> useable by normal admin's ?
>
> Or is it really neccessary to grab through source codes and release
> notes to collect that kind information on your own ?
>
--
Christian BalzerNetwork/Systems Engineer
ch...@gol.com
ly and don't have to worry about the complexity and risks of
cache-tiering.
OTOH you will want a replication of 3, with the resulting latency penalty
(and costs).
Then at the top end of cost and performance, you'd have a SSD cluster with
NVMe journals.
Christian
--
Christian Balzer
why cache tiering is not meeting the original
> expectations technically?
I think that has been answered both by that statement and what Nick wrote,
alas a this point in time I would also concur with Nick that these initial
problems are moot.
Christian
--
Christian BalzerNetwork/Systems Engine
:reed.d...@focusvq.com>
> Sent: Friday, October 21, 2016 10:06 AM
> To: Christian Balzer<mailto:ch...@gol.com>
> Cc: ceph-users@lists.ceph.com<mailto:ceph-users@lists.ceph.com>
> Subject: Re: [ceph-users] New cephfs cluster performance issues- Jewel -
> cache pressure,
fs_fsr willy-nilly, you might want to
verify that fact for yourself and pick a schedule/time based on those
results.
And OSDs only, not within the VMs.
Christian
--
Christian BalzerNetwork/Systems Engineer
ch...@gol.com Global OnLine Japan/Rakuten Communications
; to process the command, I am more interesting in overall cluster.
> >
> >
> >
> > Thanks,
> >
> > Ashley
> >
> >
> > ___
> > ceph-users mailing list
> > ceph-users@lists.ceph.com
>
801 - 900 of 1226 matches
Mail list logo
