Hi Anthony,
Thanks for the answer:
The output of 'ceph osd df' is:
ceph osd df tree
ID CLASS WEIGHT REWEIGHT SIZE RAW USE DATA OMAP META
AVAIL %USE VAR PGS STATUS TYPE NAME
-1 167.64825 - 168 TiB 12 TiB 12 TiB 100 MiB 27
GiB 156 TiB 7.13 1.00 - root default
-3 167.64825 - 168 TiB 12 TiB 12 TiB 100 MiB 27
GiB 156 TiB 7.13 1.00 - host sto-core-hpc01
0 ssd 13.97069 1.00000 14 TiB 32 MiB 4.1 MiB 12 KiB 28
MiB 14 TiB 0 0 0 up osd.0
1 ssd 13.97069 1.00000 14 TiB 12 TiB 12 TiB 6 KiB 26
GiB 2.0 TiB 85.53 12.00 129 up osd.1
2 ssd 13.97069 1.00000 14 TiB 32 MiB 4.1 MiB 12 KiB 28
MiB 14 TiB 0 0 0 up osd.2
3 ssd 13.97069 1.00000 14 TiB 1.7 GiB 258 MiB 100 MiB 1.3
GiB 14 TiB 0.01 0.00 16 up osd.3
4 ssd 13.97069 1.00000 14 TiB 32 MiB 4.1 MiB 12 KiB 28
MiB 14 TiB 0 0 0 up osd.4
5 ssd 13.97069 1.00000 14 TiB 32 MiB 4.1 MiB 12 KiB 28
MiB 14 TiB 0 0 0 up osd.5
6 ssd 13.97069 1.00000 14 TiB 32 MiB 4.1 MiB 12 KiB 28
MiB 14 TiB 0 0 0 up osd.6
7 ssd 13.97069 1.00000 14 TiB 32 MiB 4.1 MiB 12 KiB 28
MiB 14 TiB 0 0 0 up osd.7
8 ssd 13.97069 1.00000 14 TiB 68 MiB 4.8 MiB 12 KiB 63
MiB 14 TiB 0 0 1 up osd.8
9 ssd 13.97069 1.00000 14 TiB 32 MiB 4.1 MiB 12 KiB 28
MiB 14 TiB 0 0 0 up osd.9
10 ssd 13.97069 1.00000 14 TiB 32 MiB 4.1 MiB 12 KiB 28
MiB 14 TiB 0 0 0 up osd.10
11 ssd 13.97069 1.00000 14 TiB 68 MiB 4.8 MiB 12 KiB 63
MiB 14 TiB 0 0 1 up osd.11
TOTAL 168 TiB 12 TiB 12 TiB 100 MiB 27
GiB 156 TiB 7.13
So all the date is on osd.1
But I have checked the balancer and seems active:
ceph balancer status
{
"active": true,
"last_optimize_duration": "0:00:00.000368",
"last_optimize_started": "Fri Mar 28 10:55:06 2025",
"mode": "upmap",
"no_optimization_needed": false,
"optimize_result": "Some objects (0.500000) are degraded; try again
later",
"plans": []
}
But the output of the commnad 'ceph config dump|grep balancer' gives me
nothing.
Best,
Mihai
On 2025-03-27 23:06, Anthony D'Atri wrote:
Look at `ceph osd df`. Is the balancer enabled?
On Mar 27, 2025, at 8:50 AM, Mihai Ciubancan
<mihai.ciuban...@eli-np.ro> wrote:
Hello,
My name is Mihai, and I have started using CEPH this mount for a HPC
cluster.
When was lunch in the production the available space shown was 80TB
now is 16TB and I didn't do anything, while I'm having 12 OSD (SSD of
14TB):
sudo ceph osd tree
ID CLASS WEIGHT TYPE NAME STATUS REWEIGHT
PRI-AFF
-1 167.64825 root default
-3 167.64825 host sto-core-hpc01
0 ssd 13.97069 osd.0 up 1.00000
1.00000
1 ssd 13.97069 osd.1 up 1.00000
1.00000
2 ssd 13.97069 osd.2 up 1.00000
1.00000
3 ssd 13.97069 osd.3 up 1.00000
1.00000
4 ssd 13.97069 osd.4 up 1.00000
1.00000
5 ssd 13.97069 osd.5 up 1.00000
1.00000
6 ssd 13.97069 osd.6 up 1.00000
1.00000
7 ssd 13.97069 osd.7 up 1.00000
1.00000
8 ssd 13.97069 osd.8 up 1.00000
1.00000
9 ssd 13.97069 osd.9 up 1.00000
1.00000
10 ssd 13.97069 osd.10 up 1.00000
1.00000
11 ssd 13.97069 osd.11 up 1.00000
1.00000
sudo ceph df detail
--- RAW STORAGE ---
CLASS SIZE AVAIL USED RAW USED %RAW USED
ssd 168 TiB 156 TiB 12 TiB 12 TiB 7.12
TOTAL 168 TiB 156 TiB 12 TiB 12 TiB 7.12
--- POOLS ---
POOL ID PGS STORED (DATA) (OMAP) OBJECTS
USED (DATA) (OMAP) %USED MAX AVAIL QUOTA OBJECTS QUOTA BYTES
DIRTY USED COMPR UNDER COMPR
.mgr 1 1 705 KiB 705 KiB 0 B 2 1.4
MiB 1.4 MiB 0 B 0 8.1 TiB N/A N/A
N/A 0 B 0 B
cephfs.cephfs.meta 2 16 270 MiB 270 MiB 0 B 85.96k 270
MiB 270 MiB 0 B 0 16 TiB N/A N/A
N/A 0 B 0 B
cephfs.cephfs.data 3 129 12 TiB 12 TiB 0 B 3.73M 12
TiB 12 TiB 0 B 42.49 16 TiB N/A N/A
N/A 0 B 0 B
While on the client side I have this:
$ df -h
10.18.31.1:6789:/ 21T 13T 8.1T 61% /data
I don't know where it's gone all the space that was at the beginning.
Someone has any hint?
Best regards,
Mihai
_______________________________________________
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io
_______________________________________________
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io
