Lately I don't have a production cluster anymore. But my homelab is ceph
with Rook. I'm running it since the alpha versions. Awesome piece of
software. You'll need Kubernetes familiarity though.
H
On Wed, Jul 2, 2025, 18:17 Anthony Fecarotta wrote:
> Hello, I was wondering if there are any stat
On Thu, Aug 10, 2023, 17:36 Murilo Morais wrote:
> Good afternoon everybody!
>
> I have the following scenario:
> Pool RBD replication x3
> 5 hosts with 12 SAS spinning disks each
>
> I'm using exactly the following line with FIO to test:
> fio -ioengine=libaio -direct=1 -invalidate=1 -name=test
Is rook/CSI still not using efficient rbd object maps ?
It could be that you issued a new benchmark while ceph was busy
(inefficiently) removing the old rbd images. This is quite a stretch but
could be worth exploring.
On Mon, Jul 25, 2022, 21:42 Mark Nelson wrote:
> I don't think so if this i
Hi,
Run a close to the metal benchmark on the disks first, just to see the
theoretical ceiling.
Also, rerun your benchmarks with random write, just to get more honest
numbers as well.
Based on the numbers so far, you seem to be getting 40k client iops @512
threads, due to 3x replication an
Just to add to warm fuzzy feeling, although just in a homelab, I'm using
rook for many years now, it's awesome. I trust* it with the family
photo's on a selfhosted nextcloud. All on K8s/Ceph/RGW.
Hans
* I have backups though ;)
On 3/7/22 09:45, Bo Thorsen wrote:
Hi Nico and Janne,
Thank y
On 11/17/21 8:19 PM, Martin Verges wrote:
There are still alternative solutions without the need for useless
containers and added complexity. Stay away from that crap and you won't
have a hard time. 😜
I don't have a problem with the containers *at all*. And with me
probably a lot of users. But
Strange to change this in a patch version. This could have all kinds of
nasty implications during upgrade. Firewalls being a prominent one. Or
am I misunderstanding the change?
It's also not mentioned prominently in the release notes.
Hans
On 8/20/21 8:54 AM, Stefan Fleischmann wrote:
If you
Try to install a completely new ceph cluster from scratch on fresh installed
LTS Ubuntu by this doc https://docs.ceph.com/en/latest/cephadm/install/ . Many
interesting discoveries await you.
You mean it doesn't work as advertised?
If you did encounter bugs, have you made bug reports?
-H
__
Interesting, your comment implies that it is a replication issue, which
does not stem from a faulty disk. But, couldn't the disk have a bit
flip? Or would you argue that would've shown as a disk read error
somewhere (because of ECC on the disk.)
On 12/4/20 10:51 AM, Dan van der Ster wrote:
No
All good points (also replying to Frank Schilder)
On 11/16/20 2:36 PM, Janne Johansson wrote:
Not trying to say you don't understand this, but rather that people who
run small ceph clusters tend to start out with R=2 or K+1 EC because the
larger faults are easier to imagine.
TBH, I think I did
I think we're deviating from the original thread quite a bit and I would
never argue that in a production environment with plenty OSDs you should
go for R=2 or K+1, so my example cluster which happens to be 2+1 is a
bit unlucky.
However I'm interested in the following
On 11/16/20 11:31 AM, Ja
> With this profile you can only loose one OSD at a time, which is really
> not that redundant.
That's rather situation dependent. I don't have really large disks, so
the repair time isn't that large.
Further, my SLO isn't that high that I need 99.xxx% uptime, if 2 disks
break in the same repair wi
=default
jerasure-per-chunk-alignment=false
k=2
m=1
plugin=jerasure
technique=reed_sol_van
w=8
On 11/11/20 12:46 PM, Hans van den Bogert wrote:
Can you show a `ceph osd tree` ?
On 11/7/20 1:14 AM, seffyr...@gmail.com wrote:
I've inherited a Ceph Octopus cluster that seems like it needs u
Can you show a `ceph osd tree` ?
On 11/7/20 1:14 AM, seffyr...@gmail.com wrote:
I've inherited a Ceph Octopus cluster that seems like it needs urgent
maintenance before data loss begins to happen. I'm the guy with the most Ceph
experience on hand and that's not saying much. I'm experiencing mo
Hoping to learn from this myself, why will the current setup never work?
On 11/11/20 10:29 AM, Robert Sander wrote:
Am 07.11.20 um 01:14 schrieb seffyr...@gmail.com:
I've inherited a Ceph Octopus cluster that seems like it needs urgent
maintenance before data loss begins to happen. I'm the guy
> I already ordered more ram. Can i turn temporary down the RAM usage of
> the OSDs to not get into that vicious cycle and just suffer small but
> stable performance?
Hi,
Look at
https://docs.ceph.com/en/latest/rados/configuration/bluestore-config-ref/#bluestore-config-reference
and then spec
To my knowledge that's a S3 client setting only. Mostly referred to as
'path-style'. Kickin' in open doors here, but you could opt for a
wildcard certificate, that would be more future proof; Path-style will
not be supported anymore, at least not at Amazon[1], so client support
will fade away i
Perhaps not SBCs, but I have 4x HP 6300s and have been running
Kubernetes together with Ceph/Rook for more than 3 years. The HPs can be
picked up around 80-120eu. I learned so much in 3 years, last time I had
that was when I started using Linux. This was money well spent and still
is, it runs n
Perhaps both clusters have the same bottleneck and you perceive them as
equally fast.
Can you provide as much details of your clusters as possible?
Also please show outputs of the tests that you've run.
On 8/31/20 1:02 PM, VELARTIS Philipp Dürhammer wrote:
I have a productive 60 osd's cluster
I don't think it might lead to more client slow requests if you set it
to 4096 in one step, since there is a cap on how many recovery/backfill
requests there can be per OSD at any given time.
I am not sure though, but I am happy to be proved wrong by the senior
members in this list :)
Hans
Please not a simple gmail block 8)
Not everyone wants to use their corporate account or selfhost email or
use a marginally better/worse commercial gmail alternative
On 8/6/20 12:52 PM, Marc Roos wrote:
Can you block gmail.com or so!!!
___
ceph
The Ceph team has chosen to default to N+2 redundancy. This is analogous to
RAID 6 (NOT RAID 1).
I have to fundamentally disagree; Ceph's replica size of 3 behaves more
like RAID1 with 3 mirrors, than it would RAID6. Raid 6 would be more in
line with Ceph's erasure coding of X+2, e.g., 3+
transitioning to an image based snapshot
config has some problems? But that's just guessing.
Thanks for the follow-up though!
Regards,
Hans
On Mon, Jun 8, 2020, 13:38 Jason Dillaman wrote:
> On Sun, Jun 7, 2020 at 8:06 AM Hans van den Bogert
> wrote:
> >
> > Hi lis
Hi list,
I've awaited octopus for a along time to be able to use mirror with
snapshotting, since my setup does not allow for journal based
mirroring. (K8s/Rook 1.3.x with ceph 15.2.2)
However, I seem to be stuck, i've come to the point where on the
cluster on which the (non-active) replicas shoul
What are the specs of your nodes? And what specific harddisks are you using?
On Fri, May 29, 2020, 18:41 Salsa wrote:
> I have a 3 hosts, 10 4TB HDDs per host ceph storage set up. I deined a 3
> replica rbd pool and some images and presented them to a Vmware host via
> ISCSI, but the write perfo
I would second that, there's no winning in this case for your
requirements and single PSU nodes. If there were 3 feeds, then yes; you
could make an extra layer in your crushmap much like you would
incorporate a rack topology in the crushmap.
On 5/28/20 2:42 PM, Chris Palmer wrote:
Immediate t
Hi list,
When reading the documentation for the new way of mirroring [1], some
questions arose, especially with the following sentence:
> Since this mode is not point-in-time consistent, the full snapshot
delta will need to be synced prior to use during a failover scenario.
1) I'm not sure
27 matches
Mail list logo
