[ceph-users] DB/WALL and RGW index on the same NVME

2024-04-07 Thread Lukasz Borek
Hi! I'm working on a POC cluster setup dedicated to backup app writing objects via s3 (large objects, up to 1TB transferred via multipart upload process). Initial setup is 18 storage nodes (12HDDs + 1 NVME card for DB/WALL) + EC pool. Plan is to use cephadm. I'd like to follow good practice and

[ceph-users] Re: DB/WALL and RGW index on the same NVME

2024-04-08 Thread Lukasz Borek
7fb0a3df-9553-4a76-938d-d23711e67677.34162.1.2 > default.rgw.buckets.index/.dir.7fb0a3df-9553-4a76-938d-d23711e67677.34162.1.2 > mtime 2022-12-20T07:32:11.00-0500, size 0 > > > On Sun, Apr 7, 2024 at 10:06 PM Lukasz Borek wrote: > >> Hi! >> >> I'm

[ceph-users] Re: DB/WALL and RGW index on the same NVME

2024-04-08 Thread Lukasz Borek
NG_ON_PRIMARY UNFOUND DEGRADED RD_OPS RD WR_OPS WR > >> USED COMPR UNDER COMPR > >> default.rgw.buckets.index 0 B 11 0 33 > >>00 0 208 207 KiB 41 20 KiB 0 B > >>0 B > >> >

[ceph-users] tuning for backup target cluster

2024-05-24 Thread Lukasz Borek
Hi Everyone, I'm putting together a HDD cluster with an ECC pool dedicated to the backup environment. Traffic via s3. Version 18.2, 7 OSD nodes, 12 * 12TB HDD + 1NVME each, 4+2 ECC pool. Wondering if there is some general guidance for startup setup/tuning in regards to s3 object size. Files are

[ceph-users] Re: tuning for backup target cluster

2024-05-27 Thread Lukasz Borek
Anthony, Darren Thanks for response. Answering your questions: What is the network you have for this cluster? 25GB/s > Is this a chassis with universal slots, or is that NVMe device maybe M.2 > or rear-cage? 12 * HDD via LSI jbod + 1 PCI NVME. Now it's 1.6TB, for the production plan is to use

[ceph-users] Re: tuning for backup target cluster

2024-06-04 Thread Lukasz Borek
> > I have certainly seen cases where the OMAPS have not stayed within the > RocksDB/WAL NVME space and have been going down to disk. How to monitor OMAPS size and if it does not get out of NVME? The OP's number suggest IIRC like 120GB-ish for WAL+DB, though depending on > workload spillover coul

[ceph-users] Re: tuning for backup target cluster

2024-06-04 Thread Lukasz Borek
> > You could check if your devices support NVMe namespaces and create more > than one namespace on the device. Wow, tricky. Will give it a try. Thanks! Łukasz Borek luk...@borek.org.pl On Tue, 4 Jun 2024 at 16:26, Robert Sander wrote: > Hi, > > On 6/4/24 16:15, Anthony D'Atri wrote: >

[ceph-users] Re: Lifecycle Stuck PROCESSING and UNINITIAL

2024-10-18 Thread Lukasz Borek
Don't think that the root cause has been found. I disabled versioning as I have to manually remove expired objects using s3 client. On Thu, 17 Oct 2024 at 17:50, Reid Guyett wrote: > Hello, > > I am experiencing an issue where it seems all lifecycles are showing either > PROCESSING or UNINITIAL.

[ceph-users] Re: WAL on NVMe/SSD not used after OSD/HDD replace

2024-09-27 Thread Lukasz Borek
Adding --zap to orch command cleans WALL logical volume : ceph orch osd rm 37 --replace *--zap* After replacement, new OSD is correctly created. Tested a few times with 18.2.4. Thanks. On Fri, 27 Sept 2024 at 19:31, Igor Fedotov wrote: > Hi! > > I'm not an expert in the Ceph orchestrator but

[ceph-users] lifecycle for versioned bucket

2024-09-17 Thread Lukasz Borek
Hi, I'm having issue with lifecycle jobs for 18.2.4 cluster with versioning enabled bucket. /# radosgw-admin lc list [ { "bucket": ":mongobackup-prod:c3e0a369-71df-40f5-a5c0-51e859efe0e0.96754.1", "shard": "lc.0", "started": "Thu, 01 Jan 1970 00:00:00 GMT", "s

[ceph-users] Re: About erasure code for larger hdd

2024-12-09 Thread Lukasz Borek
I'd start with 3+2, so you have one node left for recovery in case one fails. 6-node and 90 hdd per node sounds like a long recovery that needs to be tested for sure. On Mon, 9 Dec 2024 at 06:10, Phong Tran Thanh wrote: > Hi community, > > Please help with advice on selecting an erasure coding a

[ceph-users] Re: fqdn in spec

2025-01-08 Thread Lukasz Borek
I never used fqdn this way, but there is an option for cephadm bootstrap command --allow-fqdn-hostname allow hostname that is fully-qualified (contains ".") Worth checking. Not sure what's behind. Thanks On Wed, 8 Jan 2025 at 12:14, Piotr Pisz wrote: > Hi, > > We ad

[ceph-users] Re: Diskprediction_local mgr module removal - Call for feedback

2025-04-10 Thread Lukasz Borek
+1 I wasn't aware that this module is obsolete and was trying to start it a few weeks ago. We develop a home-made solution some time ago to monitor smart data from both HDD (uncorrected errors, grown defect list) and SSD (WLC/TBW). But keeping it up to date with non-unified disk models is a nigh