I would also like to thank Ben, who also volunteered to test 20.2.0 upgrades and Fast EC for us! As of today, he has successfully upgraded his pre-prod RGW cluster from 19.2.3 to pre-released 20.2.0 and he continues to work on testing the new Fast EC code with RGW.
Cheers, Neha On Tue, Nov 11, 2025 at 2:51 PM Lukasz Borek <[email protected]> wrote: >> >> Can you please share which version you upgraded this cluster from? > > 19.2.3. > >> I assume this was a test environment > > it is. Will be in production in a couple months. > > thanks. > > On Mon, 10 Nov 2025 at 17:13, Neha Ojha <[email protected]> wrote: >> >> Thank you for testing 20.2.0! Can you please share which version you >> upgraded this cluster from? I assume this was a test environment. >> >> Thanks, >> Neha >> >> On Mon, Nov 10, 2025 at 2:49 AM Lukasz Borek <[email protected]> wrote: >> > >> > Thanks. >> > >> > 8 nodes NVME cluster with EC4+2 upgraded. No problem observed. >> > >> > Let me know if you want me to grab some logs. >> > >> > I will do some testing later this week. >> > >> > Thanks. >> > >> > NAME HOST PORTS >> > STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID >> > CONTAINER ID >> > ceph-exporter.fast-cephbackup-mon1 fast-cephbackup-mon1 >> > running (117m) 9m ago 2w 9349k - 20.2.0 >> > 524f3da27646 4de9da93f7dc >> > ceph-exporter.fast-cephbackup-mon2 fast-cephbackup-mon2 >> > running (117m) 9m ago 2w 9118k - 20.2.0 >> > 524f3da27646 456692e98ef9 >> > ceph-exporter.fast-cephbackup-mon3 fast-cephbackup-mon3 >> > running (117m) 9m ago 2w 9172k - 20.2.0 >> > 524f3da27646 20c6764da039 >> > ceph-exporter.fast-cephbackup-osd1 fast-cephbackup-osd1 >> > running (117m) 9m ago 2w 22.3M - 20.2.0 >> > 524f3da27646 6e5f442f5dde >> > ceph-exporter.fast-cephbackup-osd2 fast-cephbackup-osd2 >> > running (117m) 8m ago 2w 22.3M - 20.2.0 >> > 524f3da27646 b341d7dd4390 >> > ceph-exporter.fast-cephbackup-osd3 fast-cephbackup-osd3 >> > running (116m) 8m ago 2w 22.4M - 20.2.0 >> > 524f3da27646 1240b6cc66f7 >> > ceph-exporter.fast-cephbackup-osd4 fast-cephbackup-osd4 >> > running (116m) 6m ago 2w 22.3M - 20.2.0 >> > 524f3da27646 df81ed883912 >> > ceph-exporter.fast-cephbackup-osd5 fast-cephbackup-osd5 >> > running (116m) 6m ago 2w 22.2M - 20.2.0 >> > 524f3da27646 1c089585b35d >> > ceph-exporter.fast-cephbackup-osd6 fast-cephbackup-osd6 >> > running (116m) 4m ago 2w 22.5M - 20.2.0 >> > 524f3da27646 3be6d4d8b1da >> > ceph-exporter.fast-cephbackup-osd7 fast-cephbackup-osd7 >> > running (116m) 3m ago 2w 22.4M - 20.2.0 >> > 524f3da27646 cb690c2e1936 >> > ceph-exporter.fast-cephbackup-osd8 fast-cephbackup-osd8 >> > running (115m) 81s ago 2w 22.5M - 20.2.0 >> > 524f3da27646 187336418c0d >> > crash.fast-cephbackup-mon1 fast-cephbackup-mon1 >> > running (2h) 9m ago 4w 7625k - 20.2.0 >> > 524f3da27646 5bc9c4e61b24 >> > crash.fast-cephbackup-mon2 fast-cephbackup-mon2 >> > running (2h) 9m ago 3w 7629k - 20.2.0 >> > 524f3da27646 927fc4633bf8 >> > crash.fast-cephbackup-mon3 fast-cephbackup-mon3 >> > running (2h) 9m ago 3w 7633k - 20.2.0 >> > 524f3da27646 b1d1ad6e5904 >> > crash.fast-cephbackup-osd1 fast-cephbackup-osd1 >> > running (2h) 9m ago 3w 7671k - 20.2.0 >> > 524f3da27646 a07c9ba448b0 >> > crash.fast-cephbackup-osd2 fast-cephbackup-osd2 >> > running (2h) 8m ago 3w 7646k - 20.2.0 >> > 524f3da27646 6ddae0f43fb0 >> > crash.fast-cephbackup-osd3 fast-cephbackup-osd3 >> > running (2h) 8m ago 3w 7650k - 20.2.0 >> > 524f3da27646 41f2c7c9fd33 >> > crash.fast-cephbackup-osd4 fast-cephbackup-osd4 >> > running (2h) 6m ago 3w 7646k - 20.2.0 >> > 524f3da27646 b4953c96bd1b >> > crash.fast-cephbackup-osd5 fast-cephbackup-osd5 >> > running (2h) 6m ago 3w 7650k - 20.2.0 >> > 524f3da27646 ad2a9d34e155 >> > crash.fast-cephbackup-osd6 fast-cephbackup-osd6 >> > running (2h) 4m ago 3w 7646k - 20.2.0 >> > 524f3da27646 89eaa53086c4 >> > crash.fast-cephbackup-osd7 fast-cephbackup-osd7 >> > running (2h) 3m ago 3w 7646k - 20.2.0 >> > 524f3da27646 87fd08e6b274 >> > crash.fast-cephbackup-osd8 fast-cephbackup-osd8 >> > running (2h) 81s ago 3w 7638k - 20.2.0 >> > 524f3da27646 881408f9d76d >> > mgr.fast-cephbackup-mon1.xfamca fast-cephbackup-mon1 >> > *:8443,9283,8765 running (3h) 9m ago 2w 507M - 20.2.0 >> > 524f3da27646 59f0778832bd >> > mgr.fast-cephbackup-mon2.gzasaj fast-cephbackup-mon2 >> > *:8443,9283,8765 running (3h) 9m ago 3w 865M - 20.2.0 >> > 524f3da27646 862a55610d55 >> > mgr.fast-cephbackup-mon3.trmjnk fast-cephbackup-mon3 >> > *:8443,9283,8765 running (3h) 9m ago 3w 508M - 20.2.0 >> > 524f3da27646 7c6c8271e0b2 >> > mon.fast-cephbackup-mon1 fast-cephbackup-mon1 >> > running (3h) 9m ago 2w 442M 2048M 20.2.0 >> > 524f3da27646 b3701a59d934 >> > mon.fast-cephbackup-mon2 fast-cephbackup-mon2 >> > running (3h) 9m ago 3w 440M 2048M 20.2.0 >> > 524f3da27646 c70c96d6f220 >> > mon.fast-cephbackup-mon3 fast-cephbackup-mon3 >> > running (3h) 9m ago 3w 444M 2048M 20.2.0 >> > 524f3da27646 e0465231d5bb >> > osd.0 fast-cephbackup-osd5 >> > running (2h) 6m ago 3w 14.2G 21.7G 20.2.0 >> > 524f3da27646 77d1125f3ded >> > osd.1 fast-cephbackup-osd3 >> > running (2h) 8m ago 3w 15.6G 21.7G 20.2.0 >> > 524f3da27646 6224a7873350 >> > osd.2 fast-cephbackup-osd6 >> > running (2h) 4m ago 3w 12.6G 21.7G 20.2.0 >> > 524f3da27646 173a847e73bf >> > osd.3 fast-cephbackup-osd7 >> > running (2h) 3m ago 3w 14.8G 21.7G 20.2.0 >> > 524f3da27646 515a1b402559 >> > osd.4 fast-cephbackup-osd2 >> > running (2h) 8m ago 3w 12.6G 21.7G 20.2.0 >> > 524f3da27646 555e60b3d272 >> > osd.5 fast-cephbackup-osd1 >> > running (2h) 9m ago 3w 14.6G 21.7G 20.2.0 >> > 524f3da27646 15ec0a8364a1 >> > osd.6 fast-cephbackup-osd8 >> > running (2h) 81s ago 3w 14.4G 21.7G 20.2.0 >> > 524f3da27646 cefe12a100b3 >> > osd.7 fast-cephbackup-osd4 >> > running (2h) 6m ago 3w 13.7G 21.7G 20.2.0 >> > 524f3da27646 5ac5aa3645f7 >> > osd.8 fast-cephbackup-osd5 >> > running (2h) 6m ago 3w 13.9G 21.7G 20.2.0 >> > 524f3da27646 3d9b4fc53385 >> > osd.9 fast-cephbackup-osd3 >> > running (2h) 8m ago 3w 14.9G 21.7G 20.2.0 >> > 524f3da27646 8bd7b4154925 >> > osd.10 fast-cephbackup-osd6 >> > running (2h) 4m ago 3w 14.0G 21.7G 20.2.0 >> > 524f3da27646 037877a08541 >> > osd.11 fast-cephbackup-osd1 >> > running (2h) 9m ago 3w 14.2G 21.7G 20.2.0 >> > 524f3da27646 58659b91ef03 >> > osd.12 fast-cephbackup-osd7 >> > running (2h) 3m ago 3w 14.3G 21.7G 20.2.0 >> > 524f3da27646 c9aec438a8e4 >> > osd.13 fast-cephbackup-osd8 >> > running (2h) 81s ago 3w 15.1G 21.7G 20.2.0 >> > 524f3da27646 d60262f8462d >> > osd.14 fast-cephbackup-osd2 >> > running (2h) 8m ago 3w 15.8G 21.7G 20.2.0 >> > 524f3da27646 9bba8fe67e06 >> > osd.15 fast-cephbackup-osd4 >> > running (2h) 6m ago 3w 16.2G 21.7G 20.2.0 >> > 524f3da27646 847b77855b6f >> > osd.16 fast-cephbackup-osd5 >> > running (2h) 6m ago 3w 13.3G 21.7G 20.2.0 >> > 524f3da27646 736932c6d76a >> > osd.17 fast-cephbackup-osd3 >> > running (2h) 8m ago 3w 15.9G 21.7G 20.2.0 >> > 524f3da27646 4092581a7fe4 >> > osd.18 fast-cephbackup-osd6 >> > running (2h) 4m ago 3w 15.2G 21.7G 20.2.0 >> > 524f3da27646 64c154a002aa >> > osd.19 fast-cephbackup-osd1 >> > running (2h) 9m ago 3w 15.3G 21.7G 20.2.0 >> > 524f3da27646 00e58326a1ef >> > osd.20 fast-cephbackup-osd8 >> > running (2h) 81s ago 3w 15.6G 21.7G 20.2.0 >> > 524f3da27646 22bb3db279d5 >> > osd.21 fast-cephbackup-osd2 >> > running (2h) 8m ago 3w 15.0G 21.7G 20.2.0 >> > 524f3da27646 cc6c38738efd >> > osd.22 fast-cephbackup-osd4 >> > running (2h) 6m ago 3w 14.5G 21.7G 20.2.0 >> > 524f3da27646 2d13e252b00e >> > osd.23 fast-cephbackup-osd7 >> > running (2h) 3m ago 3w 14.1G 21.7G 20.2.0 >> > 524f3da27646 f9e17e6a58fa >> > osd.24 fast-cephbackup-osd5 >> > running (2h) 6m ago 3w 14.3G 21.7G 20.2.0 >> > 524f3da27646 0f8197209d12 >> > osd.25 fast-cephbackup-osd3 >> > running (2h) 8m ago 3w 14.1G 21.7G 20.2.0 >> > 524f3da27646 5589f732c8e7 >> > osd.26 fast-cephbackup-osd6 >> > running (2h) 4m ago 3w 13.2G 21.7G 20.2.0 >> > 524f3da27646 936ef5adcd5b >> > osd.27 fast-cephbackup-osd5 >> > running (2h) 6m ago 3w 12.3G 21.7G 20.2.0 >> > 524f3da27646 0241c249ebab >> > osd.28 fast-cephbackup-osd3 >> > running (2h) 8m ago 3w 14.4G 21.7G 20.2.0 >> > 524f3da27646 0daec562fbee >> > osd.29 fast-cephbackup-osd6 >> > running (2h) 4m ago 3w 14.3G 21.7G 20.2.0 >> > 524f3da27646 1c9dd693d8e0 >> > osd.30 fast-cephbackup-osd8 >> > running (2h) 81s ago 3w 13.9G 21.7G 20.2.0 >> > 524f3da27646 c76825580158 >> > osd.31 fast-cephbackup-osd7 >> > running (2h) 3m ago 3w 14.2G 21.7G 20.2.0 >> > 524f3da27646 4e49578cf239 >> > osd.32 fast-cephbackup-osd1 >> > running (2h) 9m ago 3w 14.9G 21.7G 20.2.0 >> > 524f3da27646 5e49d50a8b93 >> > osd.33 fast-cephbackup-osd2 >> > running (2h) 8m ago 3w 12.9G 21.7G 20.2.0 >> > 524f3da27646 ad90aff39abd >> > osd.34 fast-cephbackup-osd4 >> > running (2h) 6m ago 3w 15.1G 21.7G 20.2.0 >> > 524f3da27646 c5849cdceb23 >> > osd.35 fast-cephbackup-osd5 >> > running (2h) 6m ago 3w 15.3G 21.7G 20.2.0 >> > 524f3da27646 6c870213ae6a >> > osd.36 fast-cephbackup-osd3 >> > running (2h) 8m ago 3w 11.0G 21.7G 20.2.0 >> > 524f3da27646 04ebd6dc1162 >> > osd.37 fast-cephbackup-osd6 >> > running (2h) 4m ago 3w 13.7G 21.7G 20.2.0 >> > 524f3da27646 f700c3738d68 >> > osd.38 fast-cephbackup-osd1 >> > running (2h) 9m ago 3w 15.5G 21.7G 20.2.0 >> > 524f3da27646 1669493ed49c >> > osd.39 fast-cephbackup-osd8 >> > running (2h) 81s ago 3w 11.9G 21.7G 20.2.0 >> > 524f3da27646 bf9c144c445a >> > osd.40 fast-cephbackup-osd2 >> > running (2h) 8m ago 3w 15.1G 21.7G 20.2.0 >> > 524f3da27646 539b78b9c0e8 >> > osd.41 fast-cephbackup-osd7 >> > running (2h) 3m ago 3w 10.2G 21.7G 20.2.0 >> > 524f3da27646 be6466b9a097 >> > osd.42 fast-cephbackup-osd4 >> > running (2h) 6m ago 3w 15.0G 21.7G 20.2.0 >> > 524f3da27646 a5e9904a0bb4 >> > osd.43 fast-cephbackup-osd5 >> > running (2h) 6m ago 3w 16.0G 21.7G 20.2.0 >> > 524f3da27646 e2f62ba83020 >> > osd.44 fast-cephbackup-osd6 >> > running (2h) 4m ago 3w 15.2G 21.7G 20.2.0 >> > 524f3da27646 a8072fa05fa7 >> > osd.45 fast-cephbackup-osd3 >> > running (2h) 8m ago 3w 14.6G 21.7G 20.2.0 >> > 524f3da27646 01d56e9c8a62 >> > osd.46 fast-cephbackup-osd1 >> > running (2h) 9m ago 3w 11.4G 21.7G 20.2.0 >> > 524f3da27646 59cbbb9896ba >> > osd.47 fast-cephbackup-osd2 >> > running (2h) 8m ago 3w 15.0G 21.7G 20.2.0 >> > 524f3da27646 d93301e82e43 >> > osd.48 fast-cephbackup-osd8 >> > running (2h) 81s ago 3w 14.7G 21.7G 20.2.0 >> > 524f3da27646 f61d43751328 >> > osd.49 fast-cephbackup-osd7 >> > running (2h) 3m ago 3w 14.4G 21.7G 20.2.0 >> > 524f3da27646 52447edc6a9b >> > osd.50 fast-cephbackup-osd4 >> > running (2h) 6m ago 3w 14.9G 21.7G 20.2.0 >> > 524f3da27646 49b6112a27d8 >> > osd.51 fast-cephbackup-osd5 >> > running (2h) 6m ago 3w 14.0G 21.7G 20.2.0 >> > 524f3da27646 9229368e0e16 >> > osd.52 fast-cephbackup-osd6 >> > running (2h) 4m ago 3w 14.5G 21.7G 20.2.0 >> > 524f3da27646 eb677be81e9b >> > osd.53 fast-cephbackup-osd3 >> > running (2h) 8m ago 3w 15.6G 21.7G 20.2.0 >> > 524f3da27646 d5de2b9fff4c >> > osd.54 fast-cephbackup-osd7 >> > running (2h) 3m ago 3w 15.3G 21.7G 20.2.0 >> > 524f3da27646 fa076a6e94d0 >> > osd.55 fast-cephbackup-osd1 >> > running (2h) 9m ago 3w 14.4G 21.7G 20.2.0 >> > 524f3da27646 3c1d2089252d >> > osd.56 fast-cephbackup-osd2 >> > running (2h) 8m ago 3w 14.0G 21.7G 20.2.0 >> > 524f3da27646 b29067e714b9 >> > osd.57 fast-cephbackup-osd8 >> > running (2h) 81s ago 3w 14.1G 21.7G 20.2.0 >> > 524f3da27646 a9a5be72ba34 >> > osd.58 fast-cephbackup-osd4 >> > running (2h) 6m ago 3w 15.4G 21.7G 20.2.0 >> > 524f3da27646 abbf11f1e2ea >> > osd.59 fast-cephbackup-osd7 >> > running (2h) 3m ago 3w 13.6G 21.7G 20.2.0 >> > 524f3da27646 ca8df0031018 >> > osd.60 fast-cephbackup-osd1 >> > running (2h) 9m ago 3w 15.1G 21.7G 20.2.0 >> > 524f3da27646 74738c3d930d >> > osd.61 fast-cephbackup-osd2 >> > running (2h) 8m ago 3w 17.4G 21.7G 20.2.0 >> > 524f3da27646 098aa2c69b0f >> > osd.62 fast-cephbackup-osd8 >> > running (2h) 81s ago 2w 11.6G 21.7G 20.2.0 >> > 524f3da27646 113f92904b8f >> > osd.63 fast-cephbackup-osd4 >> > running (2h) 6m ago 3w 15.2G 21.7G 20.2.0 >> > 524f3da27646 70dff6c0798d >> > rgw.label.fast-cephbackup-osd1.oyqdoe fast-cephbackup-osd1 *:8000 >> > running (2h) 9m ago 3w 648M - 20.2.0 >> > 524f3da27646 d83719a402d5 >> > rgw.label.fast-cephbackup-osd2.lzxiwi fast-cephbackup-osd2 *:8000 >> > running (2h) 8m ago 3w 554M - 20.2.0 >> > 524f3da27646 49914023f796 >> > rgw.label.fast-cephbackup-osd3.bzbawb fast-cephbackup-osd3 *:8000 >> > running (2h) 8m ago 2w 545M - 20.2.0 >> > 524f3da27646 81dce830901e >> > rgw.label.fast-cephbackup-osd4.epcdwr fast-cephbackup-osd4 *:8000 >> > running (2h) 6m ago 3w 595M - 20.2.0 >> > 524f3da27646 b7b04e7728cc >> > rgw.label.fast-cephbackup-osd5.fhzzju fast-cephbackup-osd5 *:8000 >> > running (2h) 6m ago 2w 542M - 20.2.0 >> > 524f3da27646 845d6fcf68b6 >> > rgw.label.fast-cephbackup-osd6.fztzxe fast-cephbackup-osd6 *:8000 >> > running (119m) 4m ago 2w 648M - 20.2.0 >> > 524f3da27646 45bb95f840d7 >> > rgw.label.fast-cephbackup-osd7.asrxjx fast-cephbackup-osd7 *:8000 >> > running (119m) 3m ago 2w 853M - 20.2.0 >> > 524f3da27646 0e6bae7f8a14 >> > rgw.label.fast-cephbackup-osd8.avbbsq fast-cephbackup-osd8 *:8000 >> > running (119m) 81s ago 3w 584M - 20.2.0 >> > 524f3da27646 bfd48ec8d147 >> > >> > >> > On Sun, 9 Nov 2025 at 23:31, Yuri Weinstein <[email protected]> wrote: >> >> >> >> Hello >> >> >> >> 20.2.0 image is pre-released >> >> Please don't share it >> >> https://quay.io/repository/yuriw/ceph?tab=tags >> >> v20.2.0-20251104 >> >> >> >> Thx for helping! >> >> >> >> >> >> On Sun, Nov 9, 2025 at 1:02 PM Neha Ojha <[email protected]> wrote: >> >>> >> >>> Thank you, Lukasz! @yuri can you please share the pre-release image? >> >>> >> >>> Thanks, >> >>> Neha >> >>> >> >>> On Sun, Nov 9, 2025 at 12:49 PM Lukasz Borek <[email protected]> wrote: >> >>>> >> >>>> Hi, >> >>>> >> >>>> Happy to help with testing. I have not yet production cluster with EC >> >>>> that can be upgraded. >> >>>> >> >>>> Let me know image detail. >> >>>> >> >>>> Łukasz Borek >> >>>> [email protected] >> >>>> >> >>>> >> >>>> W dniu pt., 7 lis 2025 o 21:12 Neha Ojha <[email protected]> napisał(a): >> >>>>> >> >>>>> Special thanks to @Stefan Kooman for testing upgrades for 20.2.0! He >> >>>>> has successfully upgraded 2 clusters including one with RGW >> >>>>> mutli-site. >> >>>>> >> >>>>> Thanks, >> >>>>> Neha >> >>>>> >> >>>>> >> >>>>> On Thu, Nov 6, 2025 at 12:31 PM Neha Ojha <[email protected]> wrote: >> >>>>> > >> >>>>> > Hi everyone, >> >>>>> > >> >>>>> > On Mon, Nov 3, 2025 at 9:28 AM Casey Bodley <[email protected]> >> >>>>> > wrote: >> >>>>> > > >> >>>>> > > we covered two topics today that generated a lot of good >> >>>>> > > discussion, >> >>>>> > > but neither reached a consensus: >> >>>>> > > >> >>>>> > > AI Code contributions >> >>>>> > > * current lack of clear policy on submissions containing >> >>>>> > > AI-generated >> >>>>> > > code. do we want one? what are the concerns/risks? how do we >> >>>>> > > enforce? >> >>>>> > > * requires further discussion >> >>>>> > > >> >>>>> > > Tentacle updates? >> >>>>> > > * merge conflicts in https://github.com/ceph/ceph/pull/66096 >> >>>>> > > * lrc upgrade? >> >>>>> > > ** very risky with pending lab move next week. after the move, lrc >> >>>>> > > may >> >>>>> > > not be available for upgrade until 2026? >> >>>>> > > ** should this block the release or not? requires further >> >>>>> > > discussion >> >>>>> > >> >>>>> > Are there any community members who would be willing to help test >> >>>>> > Tentacle 20.2.0 upgrades on their test/staging clusters? This will >> >>>>> > immensely help us build more confidence in the stability of the >> >>>>> > release and expedite GA'ing long-awaited features like Fast EC! >> >>>>> > >> >>>>> > Thanks, >> >>>>> > Neha >> >>>>> > >> >>>>> > >> >>>>> > > _______________________________________________ >> >>>>> > > ceph-users mailing list -- [email protected] >> >>>>> > > To unsubscribe send an email to [email protected] >> >>>>> > > >> >>>>> _______________________________________________ >> >>>>> ceph-users mailing list -- [email protected] >> >>>>> To unsubscribe send an email to [email protected] >> > >> > >> > >> > -- >> > Łukasz Borek >> > [email protected] >> > > > -- > Łukasz Borek > [email protected] _______________________________________________ ceph-users mailing list -- [email protected] To unsubscribe send an email to [email protected]
