> > Can you please share which version you upgraded this cluster from?
19.2.3. I assume this was a test environment it is. Will be in production in a couple months. thanks. On Mon, 10 Nov 2025 at 17:13, Neha Ojha <[email protected]> wrote: > Thank you for testing 20.2.0! Can you please share which version you > upgraded this cluster from? I assume this was a test environment. > > Thanks, > Neha > > On Mon, Nov 10, 2025 at 2:49 AM Lukasz Borek <[email protected]> wrote: > > > > Thanks. > > > > 8 nodes NVME cluster with EC4+2 upgraded. No problem observed. > > > > Let me know if you want me to grab some logs. > > > > I will do some testing later this week. > > > > Thanks. > > > > NAME HOST PORTS > STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID > CONTAINER ID > > ceph-exporter.fast-cephbackup-mon1 fast-cephbackup-mon1 > running (117m) 9m ago 2w 9349k - 20.2.0 > 524f3da27646 4de9da93f7dc > > ceph-exporter.fast-cephbackup-mon2 fast-cephbackup-mon2 > running (117m) 9m ago 2w 9118k - 20.2.0 > 524f3da27646 456692e98ef9 > > ceph-exporter.fast-cephbackup-mon3 fast-cephbackup-mon3 > running (117m) 9m ago 2w 9172k - 20.2.0 > 524f3da27646 20c6764da039 > > ceph-exporter.fast-cephbackup-osd1 fast-cephbackup-osd1 > running (117m) 9m ago 2w 22.3M - 20.2.0 > 524f3da27646 6e5f442f5dde > > ceph-exporter.fast-cephbackup-osd2 fast-cephbackup-osd2 > running (117m) 8m ago 2w 22.3M - 20.2.0 > 524f3da27646 b341d7dd4390 > > ceph-exporter.fast-cephbackup-osd3 fast-cephbackup-osd3 > running (116m) 8m ago 2w 22.4M - 20.2.0 > 524f3da27646 1240b6cc66f7 > > ceph-exporter.fast-cephbackup-osd4 fast-cephbackup-osd4 > running (116m) 6m ago 2w 22.3M - 20.2.0 > 524f3da27646 df81ed883912 > > ceph-exporter.fast-cephbackup-osd5 fast-cephbackup-osd5 > running (116m) 6m ago 2w 22.2M - 20.2.0 > 524f3da27646 1c089585b35d > > ceph-exporter.fast-cephbackup-osd6 fast-cephbackup-osd6 > running (116m) 4m ago 2w 22.5M - 20.2.0 > 524f3da27646 3be6d4d8b1da > > ceph-exporter.fast-cephbackup-osd7 fast-cephbackup-osd7 > running (116m) 3m ago 2w 22.4M - 20.2.0 > 524f3da27646 cb690c2e1936 > > ceph-exporter.fast-cephbackup-osd8 fast-cephbackup-osd8 > running (115m) 81s ago 2w 22.5M - 20.2.0 > 524f3da27646 187336418c0d > > crash.fast-cephbackup-mon1 fast-cephbackup-mon1 > running (2h) 9m ago 4w 7625k - 20.2.0 > 524f3da27646 5bc9c4e61b24 > > crash.fast-cephbackup-mon2 fast-cephbackup-mon2 > running (2h) 9m ago 3w 7629k - 20.2.0 > 524f3da27646 927fc4633bf8 > > crash.fast-cephbackup-mon3 fast-cephbackup-mon3 > running (2h) 9m ago 3w 7633k - 20.2.0 > 524f3da27646 b1d1ad6e5904 > > crash.fast-cephbackup-osd1 fast-cephbackup-osd1 > running (2h) 9m ago 3w 7671k - 20.2.0 > 524f3da27646 a07c9ba448b0 > > crash.fast-cephbackup-osd2 fast-cephbackup-osd2 > running (2h) 8m ago 3w 7646k - 20.2.0 > 524f3da27646 6ddae0f43fb0 > > crash.fast-cephbackup-osd3 fast-cephbackup-osd3 > running (2h) 8m ago 3w 7650k - 20.2.0 > 524f3da27646 41f2c7c9fd33 > > crash.fast-cephbackup-osd4 fast-cephbackup-osd4 > running (2h) 6m ago 3w 7646k - 20.2.0 > 524f3da27646 b4953c96bd1b > > crash.fast-cephbackup-osd5 fast-cephbackup-osd5 > running (2h) 6m ago 3w 7650k - 20.2.0 > 524f3da27646 ad2a9d34e155 > > crash.fast-cephbackup-osd6 fast-cephbackup-osd6 > running (2h) 4m ago 3w 7646k - 20.2.0 > 524f3da27646 89eaa53086c4 > > crash.fast-cephbackup-osd7 fast-cephbackup-osd7 > running (2h) 3m ago 3w 7646k - 20.2.0 > 524f3da27646 87fd08e6b274 > > crash.fast-cephbackup-osd8 fast-cephbackup-osd8 > running (2h) 81s ago 3w 7638k - 20.2.0 > 524f3da27646 881408f9d76d > > mgr.fast-cephbackup-mon1.xfamca fast-cephbackup-mon1 > *:8443,9283,8765 running (3h) 9m ago 2w 507M - 20.2.0 > 524f3da27646 59f0778832bd > > mgr.fast-cephbackup-mon2.gzasaj fast-cephbackup-mon2 > *:8443,9283,8765 running (3h) 9m ago 3w 865M - 20.2.0 > 524f3da27646 862a55610d55 > > mgr.fast-cephbackup-mon3.trmjnk fast-cephbackup-mon3 > *:8443,9283,8765 running (3h) 9m ago 3w 508M - 20.2.0 > 524f3da27646 7c6c8271e0b2 > > mon.fast-cephbackup-mon1 fast-cephbackup-mon1 > running (3h) 9m ago 2w 442M 2048M 20.2.0 > 524f3da27646 b3701a59d934 > > mon.fast-cephbackup-mon2 fast-cephbackup-mon2 > running (3h) 9m ago 3w 440M 2048M 20.2.0 > 524f3da27646 c70c96d6f220 > > mon.fast-cephbackup-mon3 fast-cephbackup-mon3 > running (3h) 9m ago 3w 444M 2048M 20.2.0 > 524f3da27646 e0465231d5bb > > osd.0 fast-cephbackup-osd5 > running (2h) 6m ago 3w 14.2G 21.7G 20.2.0 > 524f3da27646 77d1125f3ded > > osd.1 fast-cephbackup-osd3 > running (2h) 8m ago 3w 15.6G 21.7G 20.2.0 > 524f3da27646 6224a7873350 > > osd.2 fast-cephbackup-osd6 > running (2h) 4m ago 3w 12.6G 21.7G 20.2.0 > 524f3da27646 173a847e73bf > > osd.3 fast-cephbackup-osd7 > running (2h) 3m ago 3w 14.8G 21.7G 20.2.0 > 524f3da27646 515a1b402559 > > osd.4 fast-cephbackup-osd2 > running (2h) 8m ago 3w 12.6G 21.7G 20.2.0 > 524f3da27646 555e60b3d272 > > osd.5 fast-cephbackup-osd1 > running (2h) 9m ago 3w 14.6G 21.7G 20.2.0 > 524f3da27646 15ec0a8364a1 > > osd.6 fast-cephbackup-osd8 > running (2h) 81s ago 3w 14.4G 21.7G 20.2.0 > 524f3da27646 cefe12a100b3 > > osd.7 fast-cephbackup-osd4 > running (2h) 6m ago 3w 13.7G 21.7G 20.2.0 > 524f3da27646 5ac5aa3645f7 > > osd.8 fast-cephbackup-osd5 > running (2h) 6m ago 3w 13.9G 21.7G 20.2.0 > 524f3da27646 3d9b4fc53385 > > osd.9 fast-cephbackup-osd3 > running (2h) 8m ago 3w 14.9G 21.7G 20.2.0 > 524f3da27646 8bd7b4154925 > > osd.10 fast-cephbackup-osd6 > running (2h) 4m ago 3w 14.0G 21.7G 20.2.0 > 524f3da27646 037877a08541 > > osd.11 fast-cephbackup-osd1 > running (2h) 9m ago 3w 14.2G 21.7G 20.2.0 > 524f3da27646 58659b91ef03 > > osd.12 fast-cephbackup-osd7 > running (2h) 3m ago 3w 14.3G 21.7G 20.2.0 > 524f3da27646 c9aec438a8e4 > > osd.13 fast-cephbackup-osd8 > running (2h) 81s ago 3w 15.1G 21.7G 20.2.0 > 524f3da27646 d60262f8462d > > osd.14 fast-cephbackup-osd2 > running (2h) 8m ago 3w 15.8G 21.7G 20.2.0 > 524f3da27646 9bba8fe67e06 > > osd.15 fast-cephbackup-osd4 > running (2h) 6m ago 3w 16.2G 21.7G 20.2.0 > 524f3da27646 847b77855b6f > > osd.16 fast-cephbackup-osd5 > running (2h) 6m ago 3w 13.3G 21.7G 20.2.0 > 524f3da27646 736932c6d76a > > osd.17 fast-cephbackup-osd3 > running (2h) 8m ago 3w 15.9G 21.7G 20.2.0 > 524f3da27646 4092581a7fe4 > > osd.18 fast-cephbackup-osd6 > running (2h) 4m ago 3w 15.2G 21.7G 20.2.0 > 524f3da27646 64c154a002aa > > osd.19 fast-cephbackup-osd1 > running (2h) 9m ago 3w 15.3G 21.7G 20.2.0 > 524f3da27646 00e58326a1ef > > osd.20 fast-cephbackup-osd8 > running (2h) 81s ago 3w 15.6G 21.7G 20.2.0 > 524f3da27646 22bb3db279d5 > > osd.21 fast-cephbackup-osd2 > running (2h) 8m ago 3w 15.0G 21.7G 20.2.0 > 524f3da27646 cc6c38738efd > > osd.22 fast-cephbackup-osd4 > running (2h) 6m ago 3w 14.5G 21.7G 20.2.0 > 524f3da27646 2d13e252b00e > > osd.23 fast-cephbackup-osd7 > running (2h) 3m ago 3w 14.1G 21.7G 20.2.0 > 524f3da27646 f9e17e6a58fa > > osd.24 fast-cephbackup-osd5 > running (2h) 6m ago 3w 14.3G 21.7G 20.2.0 > 524f3da27646 0f8197209d12 > > osd.25 fast-cephbackup-osd3 > running (2h) 8m ago 3w 14.1G 21.7G 20.2.0 > 524f3da27646 5589f732c8e7 > > osd.26 fast-cephbackup-osd6 > running (2h) 4m ago 3w 13.2G 21.7G 20.2.0 > 524f3da27646 936ef5adcd5b > > osd.27 fast-cephbackup-osd5 > running (2h) 6m ago 3w 12.3G 21.7G 20.2.0 > 524f3da27646 0241c249ebab > > osd.28 fast-cephbackup-osd3 > running (2h) 8m ago 3w 14.4G 21.7G 20.2.0 > 524f3da27646 0daec562fbee > > osd.29 fast-cephbackup-osd6 > running (2h) 4m ago 3w 14.3G 21.7G 20.2.0 > 524f3da27646 1c9dd693d8e0 > > osd.30 fast-cephbackup-osd8 > running (2h) 81s ago 3w 13.9G 21.7G 20.2.0 > 524f3da27646 c76825580158 > > osd.31 fast-cephbackup-osd7 > running (2h) 3m ago 3w 14.2G 21.7G 20.2.0 > 524f3da27646 4e49578cf239 > > osd.32 fast-cephbackup-osd1 > running (2h) 9m ago 3w 14.9G 21.7G 20.2.0 > 524f3da27646 5e49d50a8b93 > > osd.33 fast-cephbackup-osd2 > running (2h) 8m ago 3w 12.9G 21.7G 20.2.0 > 524f3da27646 ad90aff39abd > > osd.34 fast-cephbackup-osd4 > running (2h) 6m ago 3w 15.1G 21.7G 20.2.0 > 524f3da27646 c5849cdceb23 > > osd.35 fast-cephbackup-osd5 > running (2h) 6m ago 3w 15.3G 21.7G 20.2.0 > 524f3da27646 6c870213ae6a > > osd.36 fast-cephbackup-osd3 > running (2h) 8m ago 3w 11.0G 21.7G 20.2.0 > 524f3da27646 04ebd6dc1162 > > osd.37 fast-cephbackup-osd6 > running (2h) 4m ago 3w 13.7G 21.7G 20.2.0 > 524f3da27646 f700c3738d68 > > osd.38 fast-cephbackup-osd1 > running (2h) 9m ago 3w 15.5G 21.7G 20.2.0 > 524f3da27646 1669493ed49c > > osd.39 fast-cephbackup-osd8 > running (2h) 81s ago 3w 11.9G 21.7G 20.2.0 > 524f3da27646 bf9c144c445a > > osd.40 fast-cephbackup-osd2 > running (2h) 8m ago 3w 15.1G 21.7G 20.2.0 > 524f3da27646 539b78b9c0e8 > > osd.41 fast-cephbackup-osd7 > running (2h) 3m ago 3w 10.2G 21.7G 20.2.0 > 524f3da27646 be6466b9a097 > > osd.42 fast-cephbackup-osd4 > running (2h) 6m ago 3w 15.0G 21.7G 20.2.0 > 524f3da27646 a5e9904a0bb4 > > osd.43 fast-cephbackup-osd5 > running (2h) 6m ago 3w 16.0G 21.7G 20.2.0 > 524f3da27646 e2f62ba83020 > > osd.44 fast-cephbackup-osd6 > running (2h) 4m ago 3w 15.2G 21.7G 20.2.0 > 524f3da27646 a8072fa05fa7 > > osd.45 fast-cephbackup-osd3 > running (2h) 8m ago 3w 14.6G 21.7G 20.2.0 > 524f3da27646 01d56e9c8a62 > > osd.46 fast-cephbackup-osd1 > running (2h) 9m ago 3w 11.4G 21.7G 20.2.0 > 524f3da27646 59cbbb9896ba > > osd.47 fast-cephbackup-osd2 > running (2h) 8m ago 3w 15.0G 21.7G 20.2.0 > 524f3da27646 d93301e82e43 > > osd.48 fast-cephbackup-osd8 > running (2h) 81s ago 3w 14.7G 21.7G 20.2.0 > 524f3da27646 f61d43751328 > > osd.49 fast-cephbackup-osd7 > running (2h) 3m ago 3w 14.4G 21.7G 20.2.0 > 524f3da27646 52447edc6a9b > > osd.50 fast-cephbackup-osd4 > running (2h) 6m ago 3w 14.9G 21.7G 20.2.0 > 524f3da27646 49b6112a27d8 > > osd.51 fast-cephbackup-osd5 > running (2h) 6m ago 3w 14.0G 21.7G 20.2.0 > 524f3da27646 9229368e0e16 > > osd.52 fast-cephbackup-osd6 > running (2h) 4m ago 3w 14.5G 21.7G 20.2.0 > 524f3da27646 eb677be81e9b > > osd.53 fast-cephbackup-osd3 > running (2h) 8m ago 3w 15.6G 21.7G 20.2.0 > 524f3da27646 d5de2b9fff4c > > osd.54 fast-cephbackup-osd7 > running (2h) 3m ago 3w 15.3G 21.7G 20.2.0 > 524f3da27646 fa076a6e94d0 > > osd.55 fast-cephbackup-osd1 > running (2h) 9m ago 3w 14.4G 21.7G 20.2.0 > 524f3da27646 3c1d2089252d > > osd.56 fast-cephbackup-osd2 > running (2h) 8m ago 3w 14.0G 21.7G 20.2.0 > 524f3da27646 b29067e714b9 > > osd.57 fast-cephbackup-osd8 > running (2h) 81s ago 3w 14.1G 21.7G 20.2.0 > 524f3da27646 a9a5be72ba34 > > osd.58 fast-cephbackup-osd4 > running (2h) 6m ago 3w 15.4G 21.7G 20.2.0 > 524f3da27646 abbf11f1e2ea > > osd.59 fast-cephbackup-osd7 > running (2h) 3m ago 3w 13.6G 21.7G 20.2.0 > 524f3da27646 ca8df0031018 > > osd.60 fast-cephbackup-osd1 > running (2h) 9m ago 3w 15.1G 21.7G 20.2.0 > 524f3da27646 74738c3d930d > > osd.61 fast-cephbackup-osd2 > running (2h) 8m ago 3w 17.4G 21.7G 20.2.0 > 524f3da27646 098aa2c69b0f > > osd.62 fast-cephbackup-osd8 > running (2h) 81s ago 2w 11.6G 21.7G 20.2.0 > 524f3da27646 113f92904b8f > > osd.63 fast-cephbackup-osd4 > running (2h) 6m ago 3w 15.2G 21.7G 20.2.0 > 524f3da27646 70dff6c0798d > > rgw.label.fast-cephbackup-osd1.oyqdoe fast-cephbackup-osd1 *:8000 > running (2h) 9m ago 3w 648M - 20.2.0 > 524f3da27646 d83719a402d5 > > rgw.label.fast-cephbackup-osd2.lzxiwi fast-cephbackup-osd2 *:8000 > running (2h) 8m ago 3w 554M - 20.2.0 > 524f3da27646 49914023f796 > > rgw.label.fast-cephbackup-osd3.bzbawb fast-cephbackup-osd3 *:8000 > running (2h) 8m ago 2w 545M - 20.2.0 > 524f3da27646 81dce830901e > > rgw.label.fast-cephbackup-osd4.epcdwr fast-cephbackup-osd4 *:8000 > running (2h) 6m ago 3w 595M - 20.2.0 > 524f3da27646 b7b04e7728cc > > rgw.label.fast-cephbackup-osd5.fhzzju fast-cephbackup-osd5 *:8000 > running (2h) 6m ago 2w 542M - 20.2.0 > 524f3da27646 845d6fcf68b6 > > rgw.label.fast-cephbackup-osd6.fztzxe fast-cephbackup-osd6 *:8000 > running (119m) 4m ago 2w 648M - 20.2.0 > 524f3da27646 45bb95f840d7 > > rgw.label.fast-cephbackup-osd7.asrxjx fast-cephbackup-osd7 *:8000 > running (119m) 3m ago 2w 853M - 20.2.0 > 524f3da27646 0e6bae7f8a14 > > rgw.label.fast-cephbackup-osd8.avbbsq fast-cephbackup-osd8 *:8000 > running (119m) 81s ago 3w 584M - 20.2.0 > 524f3da27646 bfd48ec8d147 > > > > > > On Sun, 9 Nov 2025 at 23:31, Yuri Weinstein <[email protected]> wrote: > >> > >> Hello > >> > >> 20.2.0 image is pre-released > >> Please don't share it > >> https://quay.io/repository/yuriw/ceph?tab=tags > >> v20.2.0-20251104 > >> > >> Thx for helping! > >> > >> > >> On Sun, Nov 9, 2025 at 1:02 PM Neha Ojha <[email protected]> wrote: > >>> > >>> Thank you, Lukasz! @yuri can you please share the pre-release image? > >>> > >>> Thanks, > >>> Neha > >>> > >>> On Sun, Nov 9, 2025 at 12:49 PM Lukasz Borek <[email protected]> > wrote: > >>>> > >>>> Hi, > >>>> > >>>> Happy to help with testing. I have not yet production cluster with EC > that can be upgraded. > >>>> > >>>> Let me know image detail. > >>>> > >>>> Łukasz Borek > >>>> [email protected] > >>>> > >>>> > >>>> W dniu pt., 7 lis 2025 o 21:12 Neha Ojha <[email protected]> > napisał(a): > >>>>> > >>>>> Special thanks to @Stefan Kooman for testing upgrades for 20.2.0! He > >>>>> has successfully upgraded 2 clusters including one with RGW > >>>>> mutli-site. > >>>>> > >>>>> Thanks, > >>>>> Neha > >>>>> > >>>>> > >>>>> On Thu, Nov 6, 2025 at 12:31 PM Neha Ojha <[email protected]> wrote: > >>>>> > > >>>>> > Hi everyone, > >>>>> > > >>>>> > On Mon, Nov 3, 2025 at 9:28 AM Casey Bodley <[email protected]> > wrote: > >>>>> > > > >>>>> > > we covered two topics today that generated a lot of good > discussion, > >>>>> > > but neither reached a consensus: > >>>>> > > > >>>>> > > AI Code contributions > >>>>> > > * current lack of clear policy on submissions containing > AI-generated > >>>>> > > code. do we want one? what are the concerns/risks? how do we > enforce? > >>>>> > > * requires further discussion > >>>>> > > > >>>>> > > Tentacle updates? > >>>>> > > * merge conflicts in https://github.com/ceph/ceph/pull/66096 > >>>>> > > * lrc upgrade? > >>>>> > > ** very risky with pending lab move next week. after the move, > lrc may > >>>>> > > not be available for upgrade until 2026? > >>>>> > > ** should this block the release or not? requires further > discussion > >>>>> > > >>>>> > Are there any community members who would be willing to help test > >>>>> > Tentacle 20.2.0 upgrades on their test/staging clusters? This will > >>>>> > immensely help us build more confidence in the stability of the > >>>>> > release and expedite GA'ing long-awaited features like Fast EC! > >>>>> > > >>>>> > Thanks, > >>>>> > Neha > >>>>> > > >>>>> > > >>>>> > > _______________________________________________ > >>>>> > > ceph-users mailing list -- [email protected] > >>>>> > > To unsubscribe send an email to [email protected] > >>>>> > > > >>>>> _______________________________________________ > >>>>> ceph-users mailing list -- [email protected] > >>>>> To unsubscribe send an email to [email protected] > > > > > > > > -- > > Łukasz Borek > > [email protected] > > -- Łukasz Borek [email protected] _______________________________________________ ceph-users mailing list -- [email protected] To unsubscribe send an email to [email protected]
