[ceph-users] Request for Comments on the Hardware Recommendations page

2021-12-02 Thread John Zachary Dover
Re: https://docs.ceph.com/en/pacific/start/hardware-recommendations/ Anyone who has knowledge and command of the necessary leisure is invited to look at the Hardware Recommendations page and make complaints to me about what they see there. In the next few months, I'll be putting together material

[ceph-users] Re: Request for Comments on the Hardware Recommendations page

2021-12-02 Thread Dan van der Ster
Hi Zac, I have https://github.com/ceph/ceph/pull/43848 ready for merge. -- dan On Thu, Dec 2, 2021 at 9:58 AM John Zachary Dover wrote: > > Re: > https://docs.ceph.com/en/pacific/start/hardware-recommendations/ > > Anyone who has knowledge and command of the necessary leisure is invited to > lo

[ceph-users] crushtool -i; more info from output?

2021-12-02 Thread Simon Oosthoek
Dear ceph-users, We want to optimise our crush rules further and to test adjustments without impact to the cluster, we use crushtool to show the mappings. eg: crushtool -i crushmap.16 --test --num-rep 4 --show-mappings --rule 0|tail -n 10 CRUSH rule 0 x 1014 [121,125,195,197] CRUSH rule 0 x

[ceph-users] Re: Request for Comments on the Hardware Recommendations page

2021-12-02 Thread John Zachary Dover
Dan, I'm retesting that PR because the arm64 makecheck is failing. inb4: Yes, it's absurd that such a thing holds up a docs PR. When that makecheck succeeds, I'll merge this forthwith. Zac On Thu, Dec 2, 2021 at 7:17 PM Dan van der Ster wrote: > Hi Zac, > > I have https://github.com/ceph/ceph

[ceph-users] Re: Request for Comments on the Hardware Recommendations page

2021-12-02 Thread Dan van der Ster
Great, thanks! Re: the other hdd/ssd info, I think we should remove text related to "journals" -- that's FileStore jargon and not relevant for new deployments. On Thu, Dec 2, 2021 at 10:35 AM John Zachary Dover wrote: > > Dan, > > I'm retesting that PR because the arm64 makecheck is failing. inb

[ceph-users] Re: crushtool -i; more info from output?

2021-12-02 Thread Simon Oosthoek
On 02/12/2021 10:20, Simon Oosthoek wrote: Dear ceph-users, We want to optimise our crush rules further and to test adjustments without impact to the cluster, we use crushtool to show the mappings. eg: crushtool -i crushmap.16  --test --num-rep 4 --show-mappings --rule 0|tail -n 10 CRUSH ru

[ceph-users] Re: Best settings bluestore_rocksdb_options for my workload

2021-12-02 Thread Mark Nelson
Hi Istvan, Is that 1-1.2 billion 40KB rgw objects?  If you are running EC 4+2 on a 42 OSD cluster with that many objects (and a heavily write oriented workload), that could be hitting rocksdb pretty hard.  FWIW, you might want to look at the compaction stats provided in the OSD log.  You can

[ceph-users] Re: OSD crashing - Corruption: block checksumo mismatch

2021-12-02 Thread Igor Fedotov
Hi Eneko, I don't think this is a memory H/W issue. This reminds me the following thread: https://lists.ceph.io/hyperkitty/list/ceph-users@ceph.io/thread/DEOBAUXQBUFL6HNBBNJ3LMQUCQC76HLY/ There was apparently a data corruption in RocksDB which poped up during DB compaction only. There were n

[ceph-users] Re: mgrmap.epoch in ceph -s output

2021-12-02 Thread Eugen Block
Hi, just from checking different lab clusters I can see that the output of "mgrmap" has changed between versions. The "epoch" value is present in L and N, but not in O anymore. Zitat von Manuel Holtgrewe : Dear all, when calling `ceph -s --format=json-pretty`, there is no setting `{"mgr

[ceph-users] Re: 16.2.7 pacific QE validation status, RC1 available for testing

2021-12-02 Thread Sebastian Wagner
Am 29.11.21 um 18:23 schrieb Yuri Weinstein: > Details of this release are summarized here: > > https://tracker.ceph.com/issues/53324 > Release Notes - https://github.com/ceph/ceph/pull/44131 > > Seeking approvals for: > > rados - Neha rados/cephadm looks good. Except for https://tracker.ceph.com/

[ceph-users] Re: ceph-mgr constantly dying

2021-12-02 Thread Malte Stroem
Hello Konstantin, cпасибо, thanks a lot. We're going to upgrade the cluster. Best, Malte Am 02.12.21 um 07:43 schrieb Konstantin Shalygin: Hi, The fix was backported to 14.2.10 I suggest to upgrade your clusters to 14.2.22 k Sent from my iPhone On 1 Dec 2021, at 19:56, Malte Stroem wrote

[ceph-users] Re: Best settings bluestore_rocksdb_options for my workload

2021-12-02 Thread Mark Nelson
On 12/2/21 7:14 AM, Szabo, Istvan (Agoda) wrote: Hi Mark, Thank you the quick answer. I'd say the data distribution in the cluster is around 50kb (on 71TB is stored 1.51B objects). I have 3 giant buckets it is important to mention (both properly pre-sharded because the cluster is a 4 cluster

[ceph-users] Re: Is it normal for a orch osd rm drain to take so long?

2021-12-02 Thread Zach Heise (SSCC)
Good morning David, Assuming you need/want to see the data about the other 31 OSDs, 14 is showing: ID CLASS WEIGHT REWEIGHT SIZE RAW USE

[ceph-users] Re: Best settings bluestore_rocksdb_options for my workload

2021-12-02 Thread Mark Nelson
Regarding the drive: maybe.  On one hand yes, it probably has lower write endurance than a write oriented drive (optane for instance has something like 30-60dwpd).  On the other hand, there's been some disagreement on this list regarding how much write endurance actually matters if your workloa

[ceph-users] Re: 16.2.7 pacific QE validation status, RC1 available for testing

2021-12-02 Thread Casey Bodley
On Thu, Dec 2, 2021 at 1:15 PM Yuri Weinstein wrote: > > Update: > > Neha is finalizing reviewing rados and upgrades (I restarted some tests FYI) > Still need rgw approval - Casey, Matt? rgw approved > > Thx > YuriW > > On Thu, Dec 2, 2021 at 6:08 AM Sebastian Wagner wrote: > > > > > > Am 29.11

[ceph-users] Re: 16.2.7 pacific QE validation status, RC1 available for testing

2021-12-02 Thread Neha Ojha
On Mon, Nov 29, 2021 at 9:23 AM Yuri Weinstein wrote: > > Details of this release are summarized here: > > https://tracker.ceph.com/issues/53324 > Release Notes - https://github.com/ceph/ceph/pull/44131 > > Seeking approvals for: > > rados - Neha Approved, known issues: - rados/perf failures, sho

[ceph-users] Re: Is it normal for a orch osd rm drain to take so long?

2021-12-02 Thread David Orman
Hi, It would be good to have the full output. Does iostat show the backing device performing I/O? Additionally, what does ceph -s show for cluster state? Also, can you check the logs on that OSD, and see if anything looks abnormal? David On Thu, Dec 2, 2021 at 1:20 PM Zach Heise (SSCC) wrote:

[ceph-users] Re: Is it normal for a orch osd rm drain to take so long?

2021-12-02 Thread Zach Heise (SSCC)
Can do ceph -s:   cluster:     id:     health: HEALTH_OK     services:     mon: 4 daemons, quorum ceph05,ceph04,ceph01,ceph03 (age 4d)     mgr: ceph01.fblojp(active, since 25h), standbys: ceph03.futetp     mds: 1/1 daemons up, 1 standby     osd: 32 osds: 32 up (since 9d), 31 in (si

[ceph-users] One question for MON process can't start

2021-12-02 Thread yy orange
Experts : Sorry to bother you , I can't start a new thread from website , have to seek your help with email . We met one question : Cell version : 13.2.5-68-g53a67b5 (53a67b54fa57bcdfe2ce45c460e06adf62fe2eb2) mimic (stable) Problem : MON can't startup , the cluster down .