earch engine should also provide some
> informations for the install with your favourite OS.
>
> And then, if you really can not work it out, you are welcome to ask here.
>
> But then, anyone here will really need some more technical, ceph
> related, information to be able to help you.
>
e enough copies to be allowed to process
> >> writes, and I'm a little confused that it's also marked active but I
> >> don't quite remember the PG state diagrams involved. You should
> >> consider it down; it should be trying to recover itself though. I
= host" (and monitor your systems
of course).
b) keep your space usage on dual node clusters/pools below 70%.
Regards,
Christian
On Wed, 25 May 2016 14:51:24 +0900 Christian Balzer wrote:
>
> Hello,
>
> Thanks for the update and I totally agree that it should try to do 2
a huge
> failure domain in each OSD server.
>
> As the number of OSDs increase I would imagine the value of backing SSDs
> with NVME journals diminishes.
>
> B
>
> On Tue, May 24, 2016 at 3:28 AM, Christian Balzer wrote:
> >
> > Hello,
> >
> > On F
> 0.09760 osd.7 up 1.0 1.0 9 0.09760
> >> osd.9 up 1.0 1.010 0.09760
> >> osd.10 up 1.0 1.0-4 0.48798 host osd3 2
> >> 0.09760 osd.2 up 1.0 1.0 5 0.09
our new machine and be done.
Nothing particular really, other than doing them one at a time as you
planned already.
Christian
--
Christian BalzerNetwork/Systems Engineer
ch...@gol.com Global OnLine Japan/Rakuten Communications
http://www.gol.com/
__
/main.py", line 3173, in
> activate [admin-node][WARNIN] ' with fsid %s' % ceph_fsid)
> [admin-node][WARNIN] ceph_disk.main.Error: Error: No cluster conf found
> in /etc/ceph with fsid 8f9bf207-6c6a-4764-8b9e-63f70810837b
> [admin-node][ERROR ] RuntimeError: comma
the same config for fio.
> >>
> >> I am confused because EMC ScaleIO can do much more iops what is boring
> >> my boss :)
> >>
> > There are lot of discussion and slides on how to improve/maximize IOPS
> > with Ceph, go search for them.
e more OSD in the running node your cluster will stop.
You won't loose data (if the OSD was just stopped and can be restarted),
but you won't be able to access it anymore until that point.
If your OSD actually died because a HDD failure, then you would loose data
indeed.
Christian
> 26.0
e SAN :-)
>
>
>
> --
> From: *Jack Makenz*
> Date: Mon, May 30, 2016 at 9:05 AM
> To: Nate Curry , ceph-commun...@lists.ceph.com
>
>
> Thanks Nate,
> But as i mentioned before , providing petabytes of storage capacity on
> commodity hardware or enter
; Of Jack Makenz
> > Sent: 30 May 2016 08:40
> > To: Christian Balzer
> > Cc: ceph-users@lists.ceph.com
> > Subject: Re: [ceph-users] Fwd: [Ceph-community] Wasting the Storage
> > capacity when using Ceph based On high-end storage systems
> >
> >
budget, something like the Cumulus based offerings
(Penguin computing, etc).
Christian
>
>
>
> Regards
>
> David
> _______
> ceph-users mailing list
> ceph-users@lists.ceph.com
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>
--
s, I'd look at 2 switches running MC-LAG (in any
> > of the happy variations there are)
> > https://en.wikipedia.org/wiki/MC-LAG
> >
> > And since you're on a budget, something like the Cumulus based
> > offerings (Penguin computing, etc).
> Thanks, I'
:23:05.287753 osd.177 10.31.0.71:6842/10245 445 : cluster
> [WRN] failed to encode map e282673 with expected crc
>
>
> How do I clear these up after the upgrade?All of the filesystems on
> the OSDs are mounted and the keyrings are there..
>
> Th
(with MC-LAG capable switches) will just work [TM].
Christian
--
Christian BalzerNetwork/Systems Engineer
ch...@gol.com Global OnLine Japan/Rakuten Communications
http://www.gol.com/
___
ceph-users mailing list
ceph-user
capable of.
OTOH, I wouldn't expect your 24 HDDs do to much better than 2GB/s either
(at least with filestore and bluestore is a year away at best).
So good enough, especially if you're read heavy.
> 24x 6TB Enterprise SATA
> 2x E5-2660v4
> 1x Dual Port 40Gb Ethernet
Regards
istian
> I have nine similar OSD nodes and updates on the others didn't experience
> this issue.
> I'll update the list if there are any additional issues related to this.
> Best Regards,
> Jeff
>
> On Wed, Jun 1, 2016 at 7:15 PM, Christian Balzer wrote:
>
&g
each of the confidentiality clause, please notify the
> sender immediately by return email and delete or destroy all copies of
> the email. Any confidentiality, privilege or copyright is not waived or
> lost because this email has been sent to you by mistake.
>
On Thu, 2 Jun 2016 11:11:19 -0500 Brady Deetz wrote:
> On Wed, Jun 1, 2016 at 8:18 PM, Christian Balzer wrote:
>
> >
> > Hello,
> >
> > On Wed, 1 Jun 2016 15:50:19 -0500 Brady Deetz wrote:
> >
> > > Question:
> > > I'm curious if there
On Thu, 2 Jun 2016 21:13:41 -0500 Brady Deetz wrote:
> On Thu, Jun 2, 2016 at 8:58 PM, Christian Balzer wrote:
>
> > On Thu, 2 Jun 2016 11:11:19 -0500 Brady Deetz wrote:
> >
> > > On Wed, Jun 1, 2016 at 8:18 PM, Christian Balzer
> > > wrote:
> > >
> 2x 200GB SSD for OS
> > 2x 400GB P3700 for LevelDB
> > 2x E5-2660v4
> > 1x Dual Port 40Gb Ethernet
> >
> > 2x MDS:
> > 128GB RAM
> > 2x 200GB SSD for OS
> > 2x 400GB P3700 for LevelDB (is this necessary?)
> > 2x E5-2660v4
> > 1x Dual Port 40Gb Ethernet
; > In deploying Ceph, I am hoping to see more throughput than the
> >> > > 7320
> >> can
> >> > > currently provide. I'm also looking to get away from traditional
> >> > > file-systems that require forklift upgrades. That's where Ceph
> >> > > really shines for us.
> >&g
.250#SSD
> item osd.3 weight 0.250#SSD
>
> }
>
>
> So, i am not sure must host bucket name be the same with hostname.
>
>
> Or host bucket name does no matter?
>
>
>
> Best regards,
>
> Xiucai
--
Christian BalzerNetwork/Sy
x10) [0xbba9f0]
> const>>10: (()+0x7dc5) [0x7fc3756ffdc5] 11: (clone()+0x6d)
> const>>[0x7fc3741e228d] NOTE: a copy of the executable, or `objdump -rdS
> const>>` is needed to interpret this. 2016-06-08
> const>>06:21:58.355662 7fc34d3e8700 -1 ./include/interval_set.h: In
> con
order to get early warnings.
> >
> > Yes, but SMART is not always reliable.
> >
> True, but it won't harm to have it running anyway.
>
> >
> > All modern RAID controllers are able to read the whole disk (or disks)
> > looking for bad sectors or inconsi
Hello,
On Thu, 9 Jun 2016 08:43:23 +0200 Gandalf Corvotempesta wrote:
> Il 09 giu 2016 02:09, "Christian Balzer" ha scritto:
> > Ceph currently doesn't do any (relevant) checksumming at all, so if a
> > PRIMARY PG suffers from bit-rot this will be undetected
;t seen anything that indicates it will be in the next
version or the one after that.
Maybe Somnath can pipe up here and prove me wrong. ^o^
Google for "Accelio Ceph".
Christian
--
Christian BalzerNetwork/Systems Engineer
ch...@gol.com Global OnLin
Hello,
On Thu, 9 Jun 2016 09:59:04 +0200 Gandalf Corvotempesta wrote:
> 2016-06-09 9:16 GMT+02:00 Christian Balzer :
> > Neither, a journal failure is lethal for the OSD involved and unless
> > you have LOTS of money RAID1 SSDs are a waste.
>
> Ok, so if a journal fa
on your IPoIB,
> as it maps much more sanely to IB GIDs / MGIDs.
>
> ___
> ceph-users mailing list
> ceph-users@lists.ceph.com
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>
--
Christian BalzerNetwork/Syste
> I believe it's one of the reasons the connection setup speeds are kept
> under the requisite 1.2usec limits etc.
> On Jun 10, 2016 4:16 AM, "Daniel Swarbrick" <
> daniel.swarbr...@profitbricks.com> wrote:
>
> On 10/06/16 02:33, Christian Balzer wrote:
> &g
137.210:6803/29731 autoout,exists
> 2fc49cd5-e48c-4189-a67b-229d09378d1c
>
>
>
> What should normally happens in this situation and why it no happen?
> ___
> ceph-users mailing list
> ceph-users@lists.ceph.com
> http://lists.
gt; But it does not.
> >
> > So, is there something specific with cephfs, or is my config just too
> > much crappy and i have no idea what i am doing here ?
> >
> > Anything is highly welcome !
> >
> > Thank you !
> >
> >
> > --
> > Mit freundlichen Gruessen / Best regards
> >
> > Oliver Dzombic
> > IP-Interactive
> >
> > mailto:i...@ip-interactive.de
> >
> > Anschrift:
> >
> > IP Interactive UG ( haftungsbeschraenkt )
> > Zum Sonnenberg 1-3
> > 63571 Gelnhausen
> >
> > HRB 93402 beim Amtsgericht Hanau
> > Geschäftsführung: Oliver Dzombic
> >
> > Steuer Nr.: 35 236 3622 1
> > UST ID: DE274086107
> >
> > ___
> > ceph-users mailing list
> > ceph-users@lists.ceph.com
> > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
> ___
> ceph-users mailing list
> ceph-users@lists.ceph.com
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
--
Christian BalzerNetwork/Systems Engineer
ch...@gol.com Global OnLine Japan/Rakuten Communications
http://www.gol.com/
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
t.
If your cache pool fails, you'll still be stuck with tons of broken PGs, a
dead pool for all intents and purposes.
While running the cache tier "normally" may allow you to do flushes during
off-peak hours if you cache pool is large enough.
Christian
--
Christian Balzer
he count will grow a bit more and then
stop around 380 objects.
> But at no point its read from the cache.
>
> Its always read from the cold pool.
>
How are you determining that?
Christian
--
Christian BalzerNetwork/Systems Engineer
ch...@gol.com
f the cache and the increasing
> object number.
>
> And the non existing improvement of speed.
>
That could be down to your network or other factors on your client.
Christian
--
Christian BalzerNetwork/Systems Engineer
ch...@gol.com Global OnLine Japan
r...
Christian
> > -Original Message-
> > From: ceph-users [mailto:ceph-users-boun...@lists.ceph.com] On Behalf
> > Of Christian Balzer
> > Sent: 14 June 2016 02:00
> > To: ceph-users@lists.ceph.com
> > Subject: Re: [ceph-users] strange cache tier behaviou
#-4:ca00:::scrub_2.53:head#,
> > > (61)
> > No
> > > data available
> > > Error getting attr on : 2.6b_head,#-4:d600:::scrub_2.6b:head#,
> > > (61)
> > No
> > > data available
> > > Error getti
ists.ceph.com
> >> >> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
> >> > ___
> >> > ceph-users mailing list
> >> > ceph-users@lists.ceph.com
> >> > http://lists.ceph.com/listinfo.cgi/c
0.1",
This will allow client I/O to get a foot in and tends to be the biggest
help in Hammer and before. In Jewel the combined I/O queue should help a
lot as well.
"osd_deep_scrub_interval": "604800",
Once that's exceeded, Ceph will deep
d tell you...
>
> For things like databases this is somewhat less impactful - bit rot
> doesn't "flip a bit" but affects larger blocks of data (like one
> sector), so databases usually catch this during read and err instead of
> returning garbage to the client.
>
&g
along (in a year or so), there will be another
performance and HW design shift.
Christian
--
Christian BalzerNetwork/Systems Engineer
ch...@gol.com Global OnLine Japan/Rakuten Communications
http://www.gol.com/
___
ceph-
Hello,
On Wed, 15 Jun 2016 08:48:57 +0200 Gandalf Corvotempesta wrote:
> Il 15 giu 2016 03:27, "Christian Balzer" ha scritto:
> > And that makes deep-scrubbing something of quite limited value.
>
> This is not true.
Did you read what I and Jan wrote?
> If you
On Wed, 15 Jun 2016 09:50:43 +0200 Gandalf Corvotempesta wrote:
> Il 15 giu 2016 09:42, "Christian Balzer" ha scritto:
> >
> > This is why people are using BTRFS and ZFS for filestore (despite the
> > problems they in turn create) and why the roadmap for bluestore
On Wed, 15 Jun 2016 12:46:49 +0200 Gandalf Corvotempesta wrote:
> Il 15 giu 2016 09:58, "Christian Balzer" ha scritto
> > You _do_ know how and where Ceph/RBD store their data?
> >
> > Right now that's on disks/SSDs, formated with a file system.
> >
the relevant Ceph documentation, my
"Cache tier operation clarifications" thread and if you're using Jewel the
current "strange cache tier behaviuor with cephfs" (sic) thread.
Christian
> Thank you for your response in advance.
>
> - Manoj
--
Christian Balze
_______
> > ceph-users mailing list
> > ceph-users@lists.ceph.com
> > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
> ___
> ceph-users mailing list
> ceph-users@lists.ceph.com
> http://lists
money
on running OSFP equal cost multi-path on your storage nodes and get the
same benefits, fully redundant and load-balanced links.
Lastly, if you can't do either of these, run your things in ALB (may not
work) or simple fail-over mode. 10Gb/s is going to be fast enough in
nearly all situatio
Hello,
On Thu, 16 Jun 2016 12:44:51 +0200 Gandalf Corvotempesta wrote:
> 2016-06-16 3:53 GMT+02:00 Christian Balzer :
> > Gandalf, first read:
> > https://www.mail-archive.com/ceph-users@lists.ceph.com/msg29546.html
> >
> > And this thread by Nick:
> > https:
nks,
> >
> > Marcus Strasser
> >
> >
> >
> >
> >
> > Marcus Strasser
> >
> > Linux Systeme
> >
> > Russmedia IT GmbH
> >
> > A-6850 Schwarzach, Gutenbergstr. 1
> >
> >
> >
> > T +43 5572 501-872
> >
//twitter.com/#!/ENAConnects/>
> [http://www.ena.com/signature/linkedin.png]
> <http://www.linkedin.com/company/15330>
> [http://www.ena.com/signature/youtube.png]
> <https://www.youtube.com/user/EducationNetworks>
>
>
>
>
--
Christian Balzer
the "flush", but no data was
actually transfered to the HDD OSD nodes, nor was there anything (newly)
written.
Dazed and confused,
Christian
--
Christian BalzerNetwork/Systems Engineer
ch...@gol.com Global OnLine Japan/Rakuten Communication
n't be utilized beyond what the OSDs need
themselves it makes little sense to put extra RAM into them.
Christian
--
Christian BalzerNetwork/Systems Engineer
ch...@gol.com Global OnLine Japan/Rakuten Communication
hanks again,
Christian
> I'm not familiar enough with Bluestore to comment on the rest.
>
> [1] http://www.spinics.net/lists/ceph-devel/msg29398.html
>
> --
> Adam
>
> On Thu, Jun 16, 2016 at 11:09 PM, Christian Balzer wrote:
> >
> > Hello,
> >
> &
inutes with Debian is likely too course to spot real peaks.
See the atop documentation.
Christian
--
Christian BalzerNetwork/Systems Engineer
ch...@gol.com Global OnLine Japan/Rakuten Communications
http://www.gol.com/
___
ce
lushed
> (i.e., deleted) in the base tier.
> -Greg
>
> On Thursday, June 16, 2016, Christian Balzer wrote:
>
> >
> > Hello devs and other sage(sic) people,
> >
> > Ceph 0.94.5, cache tier in writeback mode.
> >
> > As mentioned before, I'm runn
iend (when they're not giving your mails to the NSA).
Christian
>
> Thanks
> Swami
> ___
> ceph-users mailing list
> ceph-users@lists.ceph.com
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>
--
Christian Balze
Hello,
On Fri, 17 Jun 2016 14:51:08 +0200 Gandalf Corvotempesta wrote:
> 2016-06-17 10:03 GMT+02:00 Christian Balzer :
> > I'm unfamilar with Xen and Xenserver (the later doesn't support RBD,
> > btw), but if you can see all the combined activity of your VMs on your
>
g to very much think about when doing cache tiering.
Christian
--
Christian BalzerNetwork/Systems Engineer
ch...@gol.com Global OnLine Japan/Rakuten Communications
http://www.gol.com/
___
ceph-users mailing list
ceph
My litmus test is that a "ls -R /var/lib/ceph/osd/ceph-nn/ >/dev/null"
should be pretty much instantaneous and not having to access the disk at
all.
More RAM and proper tuning as well as smaller OSDs are all ways forward to
alleviate/prevent this issue.
It would be interesting to see/kno
quests :-/. We corrected it about
> 12:30, then still took a while to settle.
>
> --
> Cheers,
> ~Blairo
>
> This email and any files transmitted with it are confidential and
> intended solely for the individual or entity to whom they are addressed.
> If you have rec
ize = {2 * (expected throughput * filestore max sync
> interval)}
>
> http://comments.gmane.org/gmane.comp.file-systems.ceph.user/28433
>
Thanks for quoting that thread. ^o^
For the OP, read it, because while the above formula certainly is correct,
large journals are nearly alwa
ge is not the intended recipient, you are on notice that any
> distribution of this message, in any form, is strictly prohibited. If
> you have received this message in error, please immediately notify the
> sender and delete or destroy any copy of this message!
>
> :______
Me drives were on different nodes. That
> is the case for our gear as well.
>
> Also, my gear is already in house so I’ve got what I’ve got to work with
> at this point, for good for ill.
>
> Tim Gipson
>
>
> On 6/16/16, 7:47 PM, "Christian Balzer" wrote
e more meat on that question, as in:
- What are you needs (space, IOPS)?
- What are the costs for either solution? (get a quote from HP)
Christian
--
Christian BalzerNetwork/Systems Engineer
ch...@gol.com Global OnLine Japan/Rakuten Communications
http://www.gol
che tiering about your needs,
by monitoring the pools (and their storage) you want to cache, again
with "df detail" (how many writes/reads?), "ceph -w", atop or iostat, etc.
Christian
> Best regards,
>
> Date: Mon, 20 Jun 2016 09:34:05 +0900
> > From: Christ
ipt
into the cluster to test for exactly this problem.
See his original post.
Christian
--
Christian BalzerNetwork/Systems Engineer
ch...@gol.com Global OnLine Japan/Rakuten Communications
http://www.gol.com/
___
ceph-
d (~330 was the previous ceiling). Subsequent
> > modification to 50 / 24 has enabled us to get to 1.1 Billion+
> >
> > Thank you all very much for your support and assistance.
> >
> > Best Regards,
> > Wade
> >
> >
> > On Mon, Jun 20, 201
Hello Blair, hello Wade (see below),
On Thu, 23 Jun 2016 12:55:17 +1000 Blair Bethwaite wrote:
> On 23 June 2016 at 12:37, Christian Balzer wrote:
> > Case in point, my main cluster (RBD images only) with 18 5+TB OSDs on 3
> > servers (64GB RAM each) has 1.8 million 4MB RBD obje
ing here, as hit_set_period only affects
promotions, not flushes or evictions.
And if probably want to set minimum ages, depending on your usage patterns
and cache size.
Christian
> Best regards,
>
> On Thu, Jun 23, 2016 at 7:23 AM, Christian Balzer wrote:
>
> >
> > Hello,
cation to 50 / 16
> >>> enabled our config to get to ~645Mill objects before the behavior in
> >>> question was observed (~330 was the previous ceiling). Subsequent
> >>> modification to 50 / 24 has enabled us to get to 1.1 Billion+
> >>>
> >
;what production will run right now) You all have helped me do that and
> >>thank you again. I will circle back and test btrfs under the same
> >>conditions. I suspect that it will behave similarly but it's only a
> >>day and half's work or so to test.
> >&g
gt;> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
> >
> > ___
> > ceph-users mailing list
> > ceph-users@lists.ceph.com
> > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>
>
s making it clear which OSD is in need of the hammer of
healing.
So ideally with Bluestore and strong checksums Ceph could indeed react to
a read error automatically and fix things right away, w/o waiting for
scrub.
Christian
--
Christian BalzerNetwork/Systems Engineer
ch.
; software which processes user data is written in Java and deployed on
> multiple client-facing servers, so rados integration should be easy.
> Kind regards, Moïn Danai.
--
Christian BalzerNetwork/Systems Engineer
ch...@gol.com Global OnLine Japan/Rakuten
Hello,
On Mon, 27 Jun 2016 09:49:54 +0200 Dan van der Ster wrote:
> On Mon, Jun 27, 2016 at 2:14 AM, Christian Balzer wrote:
> > On Sun, 26 Jun 2016 19:48:18 +0200 Stefan Priebe wrote:
> >
> >> Hi,
> >>
> >> is there any option or chance to have au
like NFS of CephFS.
EXT4 and other normal FS can't do that and you just corrupted your FS on
that image.
So either use CephFS or run OCFS2/GFS2 on your shared image and clients.
Christian
--
Christian BalzerNetwork/Systems Engineer
ch...@gol.com
define auto tiering.
Ceph cache tiering is more of a cache than full multi-level storage
tiering (it has only has 2 levels).
But depending on configuration it can achieve similar improvements.
Christian
--
Christian BalzerNetwork/Systems Engineer
ch...@gol.com
very much depends on your
usage patterns and size of your hot data.
Christian
--
Christian BalzerNetwork/Systems Engineer
ch...@gol.com Global OnLine Japan/Rakuten Communications
http://www.gol.com/
___
ceph-users mailing
Hello,
On Tue, 28 Jun 2016 08:34:26 +0200 Stefan Priebe - Profihost AG wrote:
> Am 27.06.2016 um 02:14 schrieb Christian Balzer:
> > On Sun, 26 Jun 2016 19:48:18 +0200 Stefan Priebe wrote:
> >
> >> Hi,
> >>
> >> is there any option or chance to have au
On Tue, 28 Jun 2016 09:15:50 +0200 Stefan Priebe - Profihost AG wrote:
>
> Am 28.06.2016 um 09:06 schrieb Christian Balzer:
> >
> > Hello,
> >
> > On Tue, 28 Jun 2016 08:34:26 +0200 Stefan Priebe - Profihost AG wrote:
> >
> >> Am 27.06.2016 um 02:
benchmark gives me ~21 IOPS and 76Mb/s (write).
>
Pretty clear indication that the SSD isn't handling sync writes well,
lacking further info.
> It this a normal CPU utilization for osd daemon for such tiny
> performance?
>
> Relevant part of the crush map:
>
Irrelevant
Hello,
re-adding list.
On Tue, 28 Jun 2016 20:52:51 +0300 George Shuklin wrote:
> On 06/28/2016 06:46 PM, Christian Balzer wrote:
> > Hello,
> >
> > On Tue, 28 Jun 2016 18:23:02 +0300 George Shuklin wrote:
> >
> >> Hello.
> >>
> >> I'm
10 GB / 14846 GB avail
> >
> >1005 active+clean
> >
> > 16 incomplete
> >
> > 3 down+incomplete
> >
> >
> > Unfortunately "7 requests blocked" means no virtual machine can boot
>
"Choosing the right profile is important because it cannot be modified
after the pool is created: a new pool with a different profile needs to be
created and all objects from the previous pool moved to the new."
Christian
--
Christian BalzerNetwork/Systems Engineer
planning?
> 12 OSDs per node are a good start and what I aim for usually, 24 are
> feasible if you have some idea what you're doing.
> More than 24 OSDs per node requires quite the insight and significant
> investments in CPU and RAM. Tons of threads about this he
10.2.2-0.el7.x86_64 --> Processing Dependency:
> > libbabeltrace.so.1()(64bit) for package:
> > 1:ceph-common-10.2.2-0.el7.x86_64 ---> Package librados2.x86_64
> > 1:10.2.2-0.el7 will be installed --> Processing Dependency:
> > liblttng-ust.so.0()(64bit) for pa
;=64=0.0% complete : 0=0.0%, 4=100.0%,
> 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0% issued:
> total=r=0/w=3924/d=0, short=r=0/w=0/d=0
> Run status group 0 (all jobs):
> WRITE: io=15696KB, aggrb=258KB/s, minb=258KB/s, maxb=258KB/s,
> mint=60737msec, maxt=60737msec
0043 (amazing fact sheet, very good
> conditions!)
>
Seagate is on my shit-list (for HDDs, not SSDs, mind ya).
And I can't find any trace of somebody having tested them for Ceph
suitability, so you might want to get a sample.
Christian
--
Christian BalzerNetwork/Systems Engin
On Tue, 5 Jul 2016 10:22:37 +0200 Dan van der Ster wrote:
> On Tue, Jul 5, 2016 at 10:04 AM, Dan van der Ster
> wrote:
> > On Tue, Jul 5, 2016 at 9:53 AM, Christian Balzer wrote:
> >>> Unfamiliar: Samsung SM863
> >>>
> >> You might want to rea
sas_ssd_review
doesn't of course address the suitability of these SSDs for Ceph journals,
it raises a number of points with regards to latency and consistency of
IOPS (spikes).
Simply based on this data I wouldn't go for them.
Christian
--
Christian BalzerNetwork/Systems
_
> >> ceph-users mailing list
> >> ceph-users@lists.ceph.com
> >> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
> > ___
> > ceph-users mailing list
> > ceph-users@lists.ceph
re.
This will go away with bluestore, but then you might still want fast (NVMe
even) storage for bluestore rocksdb and hot objects.
Christian
--
Christian BalzerNetwork/Systems Engineer
ch...@gol.com Global OnLine Japan/Rakuten Commun
00
> 607.84 1.36 19.425.60 34.13 4.04 19.60
> sdn 0.50 0.00 23.000.00 2670.00 0.00
> 232.17 0.072.962.960.00 2.43 5.60
>
> Pretty much 10x the latency. I'm seriously impressed with these NVME
> things.
>
>
> > -Original Messa
SDs I'd venture you'll run out of CPU steam
(with small write IOPS) before your journals become the bottleneck.
Christian
> Cheers,
> Zoltan
>
[snip]
--
Christian BalzerNetwork/Systems Engineer
ch...@gol.com Global OnLine Japan/Rakuten Communi
to be very expensive operations.
See also:
http://ceph.com/planet/use-discard-with-krbd-client-since-kernel-3-18/
I would think that the unified queue in Jewel would help with this.
But how much this is also an XFS amplification and thus not helped by
proper queuing above I can't tel
/dev/sds :
> [cephnode3][DEBUG ] /dev/sds1 ceph data, active, cluster ceph, osd.10
> [cephnode3][DEBUG ] /dev/sdt :
> [cephnode3][DEBUG ] /dev/sdt1 ceph data, active, cluster ceph, osd.11
> [cephnode3][DEBUG ] /dev/sdu :
> [cephnode3][DEBUG ] /dev/sdu1 ceph data, active, clust
> scalable performance as OSD hosts increase (and files as well).
> >
> > That's quite the blanket statement and sounds like from A sales
> > brochure. SSDs for OSD journals are always a good idea.
> > Ceph scales first and foremost by adding more storage nodes and OS
: 63.4401
> Average IOPS: 15
> Stddev IOPS: 4
> Max IOPS: 26
> Min IOPS: 1
> Average Latency(s): 1.00785
> Max latency(s): 6.48138
> Min latency(s): 0.038925
>
> On XFS partitions fragmentation no more than 1%
I'd de-frag anyway, just to rule that out.
Whe
8.00 0.00 247.85 8.66 11.26 11.26 0.00 1.18
> 90.00
> sdg 0.00 0.00 761.00 0.00 97408.00 0.00 256.00 7.80 10.22 10.22 0.00 1.01
> 76.80
> sdh 0.00 0.00 801.00 0.00 102344.00 0.00 255.54 8.05 10.05 10.05 0.00 0.96
> 76.80
> sdi 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
e by permanently enabling read-ahead, as
per:
http://docs.ceph.com/docs/hammer/rbd/rbd-config-ref/
Windows may have native settings to do that, but I know zilch about that.
Christian
> >Понедельник, 11 июля 2016, 12:38 +05:00 от Christian Balzer :
> >
> >
> >Hello,
> >
601 - 700 of 1226 matches
Mail list logo
