[ceph-users] Re: RGW - S3 bucket browser and/or S3 explorer

[Anthony Fecarotta]

Interesting! Will mess with that today.

Regards,


* Anthony Fecarotta*
 Founder & President
 [image: phone-icon]  anth...@linehaul.ai
 [image: phone-icon]  224-339-1182  [image: phone-icon]  (855) 625-0300
 [image: phone-icon]  1 Mid America Plz Flr 3 Oakbrook Terrace, IL 60181
 [image: phone-icon]  www.linehaul.ai
[image: linehaul.ai logo] 
[image: linkedin icon] 


On Tue, Feb 11, 2025 at 2:21 AM Laimis Juzeliūnas <
laimis.juzeliu...@oxylabs.io> wrote:

> Also Cyberduck works well with RGW: https://cyberduck.io/
> Its just an endpoint difference from AWS since the basis of Ceph S3
> protocol is the same.
>
> Best,
> *Laimis J.*
> laimis.juzeliu...@oxylabs.io
>
> On 11 Feb 2025, at 08:49, Anthony Fecarotta  wrote:
>
> Perhaps this?
> https://www.google.com/url?q=https://s3browser.com/&source=gmail-imap&ust=173986148400&usg=AOvVaw2aGO3QO62i_KoflakkWnLQ
>
> Regards,
>
>
> * Anthony Fecarotta*
> Founder & President
> [image: phone-icon]  anth...@linehaul.ai
> [image: phone-icon]  224-339-1182  [image: phone-icon]  (855) 625-0300
> [image: phone-icon]  1 Mid America Plz Flr 3 Oakbrook Terrace, IL 60181
> [image: phone-icon]  www.linehaul.ai
> [image: linehaul.ai logo] <
> https://www.google.com/url?q=http://www.linehaul.ai/&source=gmail-imap&ust=173986148400&usg=AOvVaw0y1OB3e3H92u-pTQnPc9Zp
> >
> [image: linkedin icon] <
> https://www.google.com/url?q=https://www.linkedin.com/in/anthony-fec/&source=gmail-imap&ust=173986148400&usg=AOvVaw15iLzJ4KQ8zf90Ha_6cnpj
> >
>
>
> On Fri, Feb 7, 2025 at 7:09 PM  wrote:
>
> Hi all.
>
> Is anyone aware of a ("S3 bucket browser" | "S3 explorer"), which works
> with RadosGW static websites?
>
> I've played around with those below, but none of them works (in my
> environment):
>
> -
> https://www.google.com/url?q=https://github.com/qoomon/aws-s3-bucket-browser&source=gmail-imap&ust=173986148400&usg=AOvVaw2GsknVblPKkS6hKZANRkvy
> -
> https://www.google.com/url?q=https://github.com/guimou/ceph-js-s3-explorer&source=gmail-imap&ust=173986148400&usg=AOvVaw1JfZ46YSv_gb5PXnERGlVa
> -
> https://www.google.com/url?q=https://github.com/awslabs/aws-js-s3-explorer&source=gmail-imap&ust=173986148400&usg=AOvVaw2vHZmQ6T15NYIzmh-dnr1s
> -
> https://www.google.com/url?q=https://github.com/awslabs/aws-js-s3-explorer/tree/v2-alpha&source=gmail-imap&ust=173986148400&usg=AOvVaw2_Ma6FVQ2rP5uVdpfzu3FN
>
> Any hints are highly appreciated!
>
> ___
> ceph-users mailing list -- ceph-users@ceph.io
> To unsubscribe send an email to ceph-users-le...@ceph.io
>
> ___
> ceph-users mailing list -- ceph-users@ceph.io
> To unsubscribe send an email to ceph-users-le...@ceph.io
>
>
>
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: RGW - S3 bucket browser and/or S3 explorer

[Rok Jaklič]

What would you like to do?

Serve your bucket objects as static files on web?

On Tue, Feb 11, 2025 at 1:31 PM Anthony Fecarotta 
wrote:

> Interesting! Will mess with that today.
>
> Regards,
>
>
> * Anthony Fecarotta*
>  Founder & President
>  [image: phone-icon]  anth...@linehaul.ai
>  [image: phone-icon]  224-339-1182  [image: phone-icon]  (855) 625-0300
>  [image: phone-icon]  1 Mid America Plz Flr 3 Oakbrook Terrace, IL 60181
>  [image: phone-icon]  www.linehaul.ai
> [image: linehaul.ai logo] 
> [image: linkedin icon] 
>
>
> On Tue, Feb 11, 2025 at 2:21 AM Laimis Juzeliūnas <
> laimis.juzeliu...@oxylabs.io> wrote:
>
> > Also Cyberduck works well with RGW: https://cyberduck.io/
> > Its just an endpoint difference from AWS since the basis of Ceph S3
> > protocol is the same.
> >
> > Best,
> > *Laimis J.*
> > laimis.juzeliu...@oxylabs.io
> >
> > On 11 Feb 2025, at 08:49, Anthony Fecarotta  wrote:
> >
> > Perhaps this?
> >
> https://www.google.com/url?q=https://s3browser.com/&source=gmail-imap&ust=173986148400&usg=AOvVaw2aGO3QO62i_KoflakkWnLQ
> >
> > Regards,
> >
> >
> > * Anthony Fecarotta*
> > Founder & President
> > [image: phone-icon]  anth...@linehaul.ai
> > [image: phone-icon]  224-339-1182  [image: phone-icon]  (855) 625-0300
> > [image: phone-icon]  1 Mid America Plz Flr 3 Oakbrook Terrace, IL 60181
> > [image: phone-icon]  www.linehaul.ai
> > [image: linehaul.ai logo] <
> >
> https://www.google.com/url?q=http://www.linehaul.ai/&source=gmail-imap&ust=173986148400&usg=AOvVaw0y1OB3e3H92u-pTQnPc9Zp
> > >
> > [image: linkedin icon] <
> >
> https://www.google.com/url?q=https://www.linkedin.com/in/anthony-fec/&source=gmail-imap&ust=173986148400&usg=AOvVaw15iLzJ4KQ8zf90Ha_6cnpj
> > >
> >
> >
> > On Fri, Feb 7, 2025 at 7:09 PM  wrote:
> >
> > Hi all.
> >
> > Is anyone aware of a ("S3 bucket browser" | "S3 explorer"), which works
> > with RadosGW static websites?
> >
> > I've played around with those below, but none of them works (in my
> > environment):
> >
> > -
> >
> https://www.google.com/url?q=https://github.com/qoomon/aws-s3-bucket-browser&source=gmail-imap&ust=173986148400&usg=AOvVaw2GsknVblPKkS6hKZANRkvy
> > -
> >
> https://www.google.com/url?q=https://github.com/guimou/ceph-js-s3-explorer&source=gmail-imap&ust=173986148400&usg=AOvVaw1JfZ46YSv_gb5PXnERGlVa
> > -
> >
> https://www.google.com/url?q=https://github.com/awslabs/aws-js-s3-explorer&source=gmail-imap&ust=173986148400&usg=AOvVaw2vHZmQ6T15NYIzmh-dnr1s
> > -
> >
> https://www.google.com/url?q=https://github.com/awslabs/aws-js-s3-explorer/tree/v2-alpha&source=gmail-imap&ust=173986148400&usg=AOvVaw2_Ma6FVQ2rP5uVdpfzu3FN
> >
> > Any hints are highly appreciated!
> >
> > ___
> > ceph-users mailing list -- ceph-users@ceph.io
> > To unsubscribe send an email to ceph-users-le...@ceph.io
> >
> > ___
> > ceph-users mailing list -- ceph-users@ceph.io
> > To unsubscribe send an email to ceph-users-le...@ceph.io
> >
> >
> >
> ___
> ceph-users mailing list -- ceph-users@ceph.io
> To unsubscribe send an email to ceph-users-le...@ceph.io
>
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: RGW - S3 bucket browser and/or S3 explorer

[Laimis Juzeliūnas]

Interesting, I just use the Amazon S3 connection template in the UI and
specify the server endpoint for Ceph S3 with respective keys. The Ceph
endpoint is exposed as a load balancer for all the RGW daemons and it works
straight "out of the box" with Cyberduck.


Laimis J.

On Tue, Feb 11, 2025, 21:41  wrote:

> yes, Cyberduck works pretty ok if you want a "GUI application".
>
> As far as I remember, CD switched to the "domain-path" URLs some releases
> / years ago And the "usual" RGW S3 is still on "path", so you need
> either to tweak CD in the user's "Cyberduck config file" a bit to ensure it
> uses the "path-style" S3 URLs... or use a very old version, which did
> "path" per default...
>
> *Gesendet: *Dienstag, 11. Februar 2025 um 09:20
> *Von: *"Laimis Juzeliūnas" 
> *An: *"Anthony Fecarotta" 
> *CC: *no...@ist-einmalig.de,ceph-users 
> *Betreff: *Re: [ceph-users] RGW - S3 bucket browser and/or S3 explorer
> Also Cyberduck works well with RGW: https://cyberduck.io/
> Its just an endpoint difference from AWS since the basis of Ceph S3
> protocol is the same.
>
> Best,
> *Laimis J.*
> laimis.juzeliu...@oxylabs.io
>
> On 11 Feb 2025, at 08:49, Anthony Fecarotta  wrote:
>
> Perhaps this?
> https://www.google.com/url?q=https://s3browser.com/&source=gmail-imap&ust=173986148400&usg=AOvVaw2aGO3QO62i_KoflakkWnLQ
>
> Regards,
>
>
> * Anthony Fecarotta*
> Founder & President
> [image: phone-icon]  anth...@linehaul.ai
> [image: phone-icon]  224-339-1182  [image: phone-icon]  (855) 625-0300
> [image: phone-icon]  1 Mid America Plz Flr 3 Oakbrook Terrace, IL 60181
> [image: phone-icon]  www.linehaul.ai
> [image: linehaul.ai logo] <
> https://www.google.com/url?q=http://www.linehaul.ai/&source=gmail-imap&ust=173986148400&usg=AOvVaw0y1OB3e3H92u-pTQnPc9Zp
> >
> [image: linkedin icon] <
> https://www.google.com/url?q=https://www.linkedin.com/in/anthony-fec/&source=gmail-imap&ust=173986148400&usg=AOvVaw15iLzJ4KQ8zf90Ha_6cnpj
> >
>
>
> On Fri, Feb 7, 2025 at 7:09 PM  wrote:
>
> Hi all.
>
> Is anyone aware of a ("S3 bucket browser" | "S3 explorer"), which works
> with RadosGW static websites?
>
> I've played around with those below, but none of them works (in my
> environment):
>
> -
> https://www.google.com/url?q=https://github.com/qoomon/aws-s3-bucket-browser&source=gmail-imap&ust=173986148400&usg=AOvVaw2GsknVblPKkS6hKZANRkvy
> -
> https://www.google.com/url?q=https://github.com/guimou/ceph-js-s3-explorer&source=gmail-imap&ust=173986148400&usg=AOvVaw1JfZ46YSv_gb5PXnERGlVa
> -
> https://www.google.com/url?q=https://github.com/awslabs/aws-js-s3-explorer&source=gmail-imap&ust=173986148400&usg=AOvVaw2vHZmQ6T15NYIzmh-dnr1s
> -
> https://www.google.com/url?q=https://github.com/awslabs/aws-js-s3-explorer/tree/v2-alpha&source=gmail-imap&ust=173986148400&usg=AOvVaw2_Ma6FVQ2rP5uVdpfzu3FN
>
> Any hints are highly appreciated!
>
> ___
> ceph-users mailing list -- ceph-users@ceph.io
> To unsubscribe send an email to ceph-users-le...@ceph.io
>
> ___
> ceph-users mailing list -- ceph-users@ceph.io
> To unsubscribe send an email to ceph-users-le...@ceph.io
>
>
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: RGW - S3 bucket browser and/or S3 explorer

[Anthony Fecarotta]

@no...@ist-einmalig.de   you are 100% correct, as I
have just spent an hour finding that out the hard way lol.

Regards,


* Anthony Fecarotta*
 Founder & President
 [image: phone-icon]  anth...@linehaul.ai
 [image: phone-icon]  224-339-1182  [image: phone-icon]  (855) 625-0300
 [image: phone-icon]  1 Mid America Plz Flr 3 Oakbrook Terrace, IL 60181
 [image: phone-icon]  www.linehaul.ai
[image: linehaul.ai logo] 
[image: linkedin icon] 


On Tue, Feb 11, 2025 at 1:41 PM  wrote:

> yes, Cyberduck works pretty ok if you want a "GUI application".
>
> As far as I remember, CD switched to the "domain-path" URLs some releases
> / years ago And the "usual" RGW S3 is still on "path", so you need
> either to tweak CD in the user's "Cyberduck config file" a bit to ensure it
> uses the "path-style" S3 URLs... or use a very old version, which did
> "path" per default...
>
> *Gesendet: *Dienstag, 11. Februar 2025 um 09:20
> *Von: *"Laimis Juzeliūnas" 
> *An: *"Anthony Fecarotta" 
> *CC: *no...@ist-einmalig.de,ceph-users 
> *Betreff: *Re: [ceph-users] RGW - S3 bucket browser and/or S3 explorer
> Also Cyberduck works well with RGW: https://cyberduck.io/
> Its just an endpoint difference from AWS since the basis of Ceph S3
> protocol is the same.
>
> Best,
> *Laimis J.*
> laimis.juzeliu...@oxylabs.io
>
> On 11 Feb 2025, at 08:49, Anthony Fecarotta  wrote:
>
> Perhaps this?
> https://www.google.com/url?q=https://s3browser.com/&source=gmail-imap&ust=173986148400&usg=AOvVaw2aGO3QO62i_KoflakkWnLQ
>
> Regards,
>
>
> * Anthony Fecarotta*
> Founder & President
> [image: phone-icon]  anth...@linehaul.ai
> [image: phone-icon]  224-339-1182  [image: phone-icon]  (855) 625-0300
> [image: phone-icon]  1 Mid America Plz Flr 3 Oakbrook Terrace, IL 60181
> [image: phone-icon]  www.linehaul.ai
> [image: linehaul.ai logo] <
> https://www.google.com/url?q=http://www.linehaul.ai/&source=gmail-imap&ust=173986148400&usg=AOvVaw0y1OB3e3H92u-pTQnPc9Zp
> >
> [image: linkedin icon] <
> https://www.google.com/url?q=https://www.linkedin.com/in/anthony-fec/&source=gmail-imap&ust=173986148400&usg=AOvVaw15iLzJ4KQ8zf90Ha_6cnpj
> >
>
>
> On Fri, Feb 7, 2025 at 7:09 PM  wrote:
>
> Hi all.
>
> Is anyone aware of a ("S3 bucket browser" | "S3 explorer"), which works
> with RadosGW static websites?
>
> I've played around with those below, but none of them works (in my
> environment):
>
> -
> https://www.google.com/url?q=https://github.com/qoomon/aws-s3-bucket-browser&source=gmail-imap&ust=173986148400&usg=AOvVaw2GsknVblPKkS6hKZANRkvy
> -
> https://www.google.com/url?q=https://github.com/guimou/ceph-js-s3-explorer&source=gmail-imap&ust=173986148400&usg=AOvVaw1JfZ46YSv_gb5PXnERGlVa
> -
> https://www.google.com/url?q=https://github.com/awslabs/aws-js-s3-explorer&source=gmail-imap&ust=173986148400&usg=AOvVaw2vHZmQ6T15NYIzmh-dnr1s
> -
> https://www.google.com/url?q=https://github.com/awslabs/aws-js-s3-explorer/tree/v2-alpha&source=gmail-imap&ust=173986148400&usg=AOvVaw2_Ma6FVQ2rP5uVdpfzu3FN
>
> Any hints are highly appreciated!
>
> ___
> ceph-users mailing list -- ceph-users@ceph.io
> To unsubscribe send an email to ceph-users-le...@ceph.io
>
> ___
> ceph-users mailing list -- ceph-users@ceph.io
> To unsubscribe send an email to ceph-users-le...@ceph.io
>
>
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: RGW - S3 bucket browser and/or S3 explorer

[Laimis Juzeliūnas]

Also Cyberduck works well with RGW: https://cyberduck.io/
Its just an endpoint difference from AWS since the basis of Ceph S3 protocol is 
the same.

Best,
Laimis J.
laimis.juzeliu...@oxylabs.io

> On 11 Feb 2025, at 08:49, Anthony Fecarotta  wrote:
> 
> Perhaps this? 
> https://www.google.com/url?q=https://s3browser.com/&source=gmail-imap&ust=173986148400&usg=AOvVaw2aGO3QO62i_KoflakkWnLQ
> 
> Regards,
> 
> 
> * Anthony Fecarotta*
> Founder & President
> [image: phone-icon]  anth...@linehaul.ai
> [image: phone-icon]  224-339-1182  [image: phone-icon]  (855) 625-0300
> [image: phone-icon]  1 Mid America Plz Flr 3 Oakbrook Terrace, IL 60181
> [image: phone-icon]  www.linehaul.ai
> [image: linehaul.ai logo] 
> 
> [image: linkedin icon] 
> 
> 
> 
> On Fri, Feb 7, 2025 at 7:09 PM  wrote:
> 
>> Hi all.
>> 
>> Is anyone aware of a ("S3 bucket browser" | "S3 explorer"), which works
>> with RadosGW static websites?
>> 
>> I've played around with those below, but none of them works (in my
>> environment):
>> 
>> - 
>> https://www.google.com/url?q=https://github.com/qoomon/aws-s3-bucket-browser&source=gmail-imap&ust=173986148400&usg=AOvVaw2GsknVblPKkS6hKZANRkvy
>> - 
>> https://www.google.com/url?q=https://github.com/guimou/ceph-js-s3-explorer&source=gmail-imap&ust=173986148400&usg=AOvVaw1JfZ46YSv_gb5PXnERGlVa
>> - 
>> https://www.google.com/url?q=https://github.com/awslabs/aws-js-s3-explorer&source=gmail-imap&ust=173986148400&usg=AOvVaw2vHZmQ6T15NYIzmh-dnr1s
>> - 
>> https://www.google.com/url?q=https://github.com/awslabs/aws-js-s3-explorer/tree/v2-alpha&source=gmail-imap&ust=173986148400&usg=AOvVaw2_Ma6FVQ2rP5uVdpfzu3FN
>> 
>> Any hints are highly appreciated!
>> 
>> ___
>> ceph-users mailing list -- ceph-users@ceph.io
>> To unsubscribe send an email to ceph-users-le...@ceph.io
>> 
> ___
> ceph-users mailing list -- ceph-users@ceph.io
> To unsubscribe send an email to ceph-users-le...@ceph.io

___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Announcing go-ceph v0.32.0

[Sven Anderson]

We are happy to announce another release of the go-ceph API library. This
is a
regular release following our every-two-months release cadence.

https://github.com/ceph/go-ceph/releases/tag/v0.32.0

The library includes bindings that aim to play a similar role to the
"pybind"
python bindings in the ceph tree but for the Go language. The library also
includes additional APIs that can be used to administer cephfs, rbd, and rgw
subsystems.
There are already a few consumers of this library in the wild, including the
ceph-csi project.

Sven
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Ceph RGW Cloud-Sync Issue

[Mark Selby]

This is a 18.2.4 reef cluster

Looking for help with cloud sync module. Here is my setup

1. Two Ceph clusters with RGW setup in a MultiSite configuration. There is no 
data synchronization between the clusters. No data sync groups, flows, or pipes.

2. We use a multi tenant setup with a small number of tenants (10). We have one 
tenant that is writing to the same bucket in different rgw zones. The names of 
files are unique in each zone.

3. We are using the cloud-sync module to copy data from our ceph clusters to 
aws. I have pasted the commands used to create the sync zone

The cloud-sync module works but for one issue that I can not figure out. For 
some reason another bucket in another tenant is showing up as target to sync 
but was never specified in the original config. I can never get a clean 
radosgw-admin sync status because there are recovering shards from a bucket in 
another tenant that I never intended to sync to the cloud.

The data sync status of the recovering shard is pointing to a bucket that never 
had any cloud sync specified. tenant1/bucket1 was never part of any tier config.

radosgw-admin sync error list --rgw-zone cloud-sync shows a never ending list 
of objects that can not be synced from tenant1/bucket1 there are 4 different 
types of error messages.

"message": "failed to sync bucket instance: (22) Invalid argument"
"message": "failed to sync bucket instance: (39) Directory not empty"
"message": "failed to sync object(16) Device or resource busy"
"message": "failed to sync object(39) Directory not empty"
 One other point is that radosgw-admin sync info --rgw-zone cloud-sync --bucket 
tenant1/bucket1 seems to indicate that that is part of the
cloud-sync config.

What I just want to do is remove tenant1/bucket1 from the sync config and stop 
the data sync such that the shards recover and the error log stop
accumulating messages. I would really like a clean sync status for my cloud 
tier zone. It would also be nice to help understand what is happening here.

All and any help is greatly appreciated.

#
# sync and data sync status
#

radosgw-admin sync status --rgw-zone=cloud-sync
  realm 02b82a81-7b16-4760-ab94-08b79176397e (realm)
  zonegroup 0e44f33d-4e70-4ba6-8c75-794d3d0b69f8 (zonegroup)
   zone 34e5298b-4537-4ada-b0d9-75579dc9640a (cloud-sync)
   current time 2025-02-11T23:56:52Z
zonegroup features enabled: resharding
   disabled: compress-encrypted
  metadata sync syncing
full sync: 0/64 shards
incremental sync: 64/64 shards
metadata is caught up with master
  data sync source: 792edbc3-8516-4014-aad4-8e7b6ebe2757 (zone-1)
syncing
full sync: 0/128 shards
incremental sync: 128/128 shards
1 shards are recovering
recovering shards: [72]
source: f3e72377-2fb1-4ab6-aa0a-76b91d157fb3 (zone-2)
syncing
full sync: 0/128 shards
incremental sync: 128/128 shards
data is behind on 4 shards
behind shards: [72,75,77,78]
oldest incremental change not applied: 
2025-02-11T15:56:34.654956-0800 [75]
10 shards are recovering
recovering shards: [68,69,70,71,72,73,74,76,77,78]

radosgw-admin data sync status --rgw-zone cloud-sync --source-zone zone-2 
--shard-id 68
{
"shard_id": 68,
"marker": {
"status": "incremental-sync",
"marker": ":00357645",
"next_step_marker": "",
"total_entries": 1,
"pos": 0,
"timestamp": "2025-02-11T23:42:34.266680Z"
},
"pending_buckets": [
"tenant1/bucket1:f3e72377-2fb1-4ab6-aa0a-76b91d157fb3.2251620.1:0"
],
"recovering_buckets": [
"tenant1/bucket1:f3e72377-2fb1-4ab6-aa0a-76b91d157fb3.2251620.1:0[0]"
],
"current_time": "2025-02-11T23:58:08Z"
}

#
# cloud sync zone creation
#

radosgw-admin zone create \
  --rgw-zonegroup=us \
  --rgw-zone=cloud-sync \
  --endpoints=https://ceph-server:4430 \
  --tier-type=cloud

radosgw-admin zone modify \
  --rgw-zonegroup=us \
  --rgw-zone=s-cloud-sync \
  --access-key= \
  --secret=

radosgw-admin zone modify \
  --rgw-zonegroup=us \
  --rgw-zone=cloud-sync \
  --tier-config=connections[].id=aws-cloud-sync

radosgw-admin zone modify \
  --rgw-zonegroup=us \
  --rgw-zone=cloud-sync \
  
--tier-config=connections[-1].endpoint=https://s3.amazonaws.com,connections[-1].access_key=,connections[-1].secret=

radosgw-admin zone modify \
  --rgw-zonegroup=us \
  --rgw-zone=cloud-sync \
  --tier-config=profiles[].connection_id=aws-cloud-sync

radosgw-admin zone modify \
  --rgw-zonegroup=us \
  --rgw-zone=cloud-sync \
  
--tier-config=profiles[-1].connection_id=aws-cloud-sync,profiles[-1].source_bucket='tenant0:bu

[ceph-users] Re: RGW - S3 bucket browser and/or S3 explorer

[Rok Jaklič]

You can't have "everything S3 has to offer / supports..." without secrets
in the browser.

Amazon released storage browser react component
 a while ago, maybe
this is what you are looking for; but we've been using really simple and
minimalistic https://github.com/rokj/stree-web (in terms of supported s3
functionality) which suffice our needs for now.

Rok

On Tue, Feb 11, 2025 at 8:31 PM  wrote:

>
> yes.
>
> a nice website showing the "static website" objects with all available
> object meta data, like size, (modification) time, everything S3 has to
> offer / supports...
>
> an extremely great additional thing would be something similar to (unix /
> linux like OS'es) symlinks... or, lets say, the GitHub "latest" URLs for
> the GH releases... so, showing an "(latest) object" and as soon as one
> either selects it in the "web browser gui" or "downloads" it via other
> S3-like ways (s3cmd, aws s3, Cyberduck, ...) the object refered to by this
> "link object" would be shown / downloaded...
> (I'm aware, this is not really officially supported by AWS S3, but let's
> dream on ;) )
>
> and, again, I'm talking about "static websites" only here... not the usual
> full-fledged S3 RGW endpoint, where you i.e. need to provide your
> secrets b'cause for this, I've a working "S3 browser" by now already...
>
> *Gesendet: *Dienstag, 11. Februar 2025 um 13:59
> *Von: *"Rok Jaklič" 
> *An: *"Anthony Fecarotta" 
> *CC: *"Laimis Juzeliūnas" ,
> no...@ist-einmalig.de,ceph-users 
> *Betreff: *[ceph-users] Re: RGW - S3 bucket browser and/or S3 explorer
> What would you like to do?
>
> Serve your bucket objects as static files on web?
>
> On Tue, Feb 11, 2025 at 1:31 PM Anthony Fecarotta 
> wrote:
>
> > Interesting! Will mess with that today.
> >
> > Regards,
> >
> >
> > * Anthony Fecarotta*
> > Founder & President
> > [image: phone-icon] anth...@linehaul.ai
> > [image: phone-icon] 224-339-1182 [image: phone-icon] (855) 625-0300
> > [image: phone-icon] 1 Mid America Plz Flr 3 Oakbrook Terrace, IL 60181
> > [image: phone-icon] www.linehaul.ai
> > [image: linehaul.ai logo] 
> > [image: linkedin icon] 
> >
> >
> > On Tue, Feb 11, 2025 at 2:21 AM Laimis Juzeliūnas <
> > laimis.juzeliu...@oxylabs.io> wrote:
> >
> > > Also Cyberduck works well with RGW: https://cyberduck.io/
> > > Its just an endpoint difference from AWS since the basis of Ceph S3
> > > protocol is the same.
> > >
> > > Best,
> > > *Laimis J.*
> > > laimis.juzeliu...@oxylabs.io
> > >
> > > On 11 Feb 2025, at 08:49, Anthony Fecarotta 
> wrote:
> > >
> > > Perhaps this?
> > >
> >
> https://www.google.com/url?q=https://s3browser.com/&source=gmail-imap&ust=173986148400&usg=AOvVaw2aGO3QO62i_KoflakkWnLQ
> > >
> > > Regards,
> > >
> > >
> > > * Anthony Fecarotta*
> > > Founder & President
> > > [image: phone-icon] anth...@linehaul.ai
> > > [image: phone-icon] 224-339-1182 [image: phone-icon] (855) 625-0300
> > > [image: phone-icon] 1 Mid America Plz Flr 3 Oakbrook Terrace, IL 60181
> > > [image: phone-icon] www.linehaul.ai
> > > [image: linehaul.ai logo] <
> > >
> >
> https://www.google.com/url?q=http://www.linehaul.ai/&source=gmail-imap&ust=173986148400&usg=AOvVaw0y1OB3e3H92u-pTQnPc9Zp
> > > >
> > > [image: linkedin icon] <
> > >
> >
> https://www.google.com/url?q=https://www.linkedin.com/in/anthony-fec/&source=gmail-imap&ust=173986148400&usg=AOvVaw15iLzJ4KQ8zf90Ha_6cnpj
> > > >
> > >
> > >
> > > On Fri, Feb 7, 2025 at 7:09 PM  wrote:
> > >
> > > Hi all.
> > >
> > > Is anyone aware of a ("S3 bucket browser" | "S3 explorer"), which works
> > > with RadosGW static websites?
> > >
> > > I've played around with those below, but none of them works (in my
> > > environment):
> > >
> > > -
> > >
> >
> https://www.google.com/url?q=https://github.com/qoomon/aws-s3-bucket-browser&source=gmail-imap&ust=173986148400&usg=AOvVaw2GsknVblPKkS6hKZANRkvy
> > > -
> > >
> >
> https://www.google.com/url?q=https://github.com/guimou/ceph-js-s3-explorer&source=gmail-imap&ust=173986148400&usg=AOvVaw1JfZ46YSv_gb5PXnERGlVa
> > > -
> > >
> >
> https://www.google.com/url?q=https://github.com/awslabs/aws-js-s3-explorer&source=gmail-imap&ust=173986148400&usg=AOvVaw2vHZmQ6T15NYIzmh-dnr1s
> > > -
> > >
> >
> https://www.google.com/url?q=https://github.com/awslabs/aws-js-s3-explorer/tree/v2-alpha&source=gmail-imap&ust=173986148400&usg=AOvVaw2_Ma6FVQ2rP5uVdpfzu3FN
> > >
> > > Any hints are highly appreciated!
> > >
> > > ___
> > > ceph-users mailing list -- ceph-users@ceph.io
> > > To unsubscribe send an email to ceph-users-le...@ceph.io
> > >
> > > ___
> > > ceph-users mailing list -- ceph-users@ceph.io
> > > To unsubscribe send an email to ceph-users-le...@ceph.io
> > >
> > >
> > >
> > ___
> > ceph-users mailing list -- ceph-

[ceph-users] Re: Grafana certificate issue

[Eugen Block]

Hi,

this has already been asked recently:

https://lists.ceph.io/hyperkitty/list/ceph-users@ceph.io/thread/RX7BREBAQBWFVZZ6ADXC33PZNNT5IY5H/

The way certificates are handled has changed.


Zitat von Devender Singh :


Hello all

Recently upgraded to 19.2.1 and started facing ..

Invalid grafana certificate on hostnode1: Invalid  
certificate key: [('PEM routines', '', 'no start line’)]


Tried suggest steps, even re-deployed grafana but no solution?

1.
 [WRN] CEPHADM_CERT_ERROR: Invalid grafana certificate on host  
node1: Invalid certificate key: [('PEM routines', '', 'no start  
line')]


Detected invalid grafana certificates. Please, use  
the following commands:


  > ceph config-key set  
mgr/cephadm/node1/grafana_crt -i 
  > ceph config-key set  
mgr/cephadm/node1/grafana_key -i 


to set valid key and certificate or reset their  
value to an empty string
in case you want cephadm to generate self-signed  
Grafana certificates.


Once done, run the following command to reconfig the daemon:

  > ceph orch daemon reconfig  
grafana.ld6-converged03n.ld6.netskrt.org



Also, when I exit my host from maintenance, I get below… but when I  
rerun same command, it works fine, why?


2.
# ceph orch host maintenance exit node2
Error EINVAL: Traceback (most recent call last):
  File "/usr/share/ceph/mgr/mgr_module.py", line 1862, in _handle_command
return self.handle_command(inbuf, cmd)
  File "/usr/share/ceph/mgr/orchestrator/_interface.py", line 184,  
in handle_command

return dispatch[cmd['prefix']].call(self, cmd, inbuf)
  File "/usr/share/ceph/mgr/mgr_module.py", line 499, in call
return self.func(mgr, **kwargs)
  File "/usr/share/ceph/mgr/orchestrator/_interface.py", line 120,  
in 
wrapper_copy = lambda *l_args, **l_kwargs: wrapper(*l_args,  
**l_kwargs)  # noqa: E731

  File "/usr/share/ceph/mgr/orchestrator/_interface.py", line 109, in wrapper
return func(*args, **kwargs)
  File "/usr/share/ceph/mgr/orchestrator/module.py", line 805, in  
_host_maintenance_exit

raise_if_exception(completion)
  File "/usr/share/ceph/mgr/orchestrator/_interface.py", line 238,  
in raise_if_exception

e = pickle.loads(c.serialized_exception)
TypeError: __init__() missing 2 required positional arguments:  
'hostname' and ‘addr'



Regards
Dev
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io



___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Grafana certificate issue

[Devender Singh]

Hello all 

Recently upgraded to 19.2.1 and started facing .. 

Invalid grafana certificate on hostnode1: Invalid certificate key: 
[('PEM routines', '', 'no start line’)]

Tried suggest steps, even re-deployed grafana but no solution?

1.
 [WRN] CEPHADM_CERT_ERROR: Invalid grafana certificate on host node1: Invalid 
certificate key: [('PEM routines', '', 'no start line')]

Detected invalid grafana certificates. Please, use the 
following commands:

  > ceph config-key set mgr/cephadm/node1/grafana_crt -i 

  > ceph config-key set mgr/cephadm/node1/grafana_key -i 


to set valid key and certificate or reset their value to an 
empty string
in case you want cephadm to generate self-signed Grafana 
certificates.

Once done, run the following command to reconfig the daemon:

  > ceph orch daemon reconfig 
grafana.ld6-converged03n.ld6.netskrt.org


Also, when I exit my host from maintenance, I get below… but when I rerun same 
command, it works fine, why?

2.
# ceph orch host maintenance exit node2
Error EINVAL: Traceback (most recent call last):
  File "/usr/share/ceph/mgr/mgr_module.py", line 1862, in _handle_command
return self.handle_command(inbuf, cmd)
  File "/usr/share/ceph/mgr/orchestrator/_interface.py", line 184, in 
handle_command
return dispatch[cmd['prefix']].call(self, cmd, inbuf)
  File "/usr/share/ceph/mgr/mgr_module.py", line 499, in call
return self.func(mgr, **kwargs)
  File "/usr/share/ceph/mgr/orchestrator/_interface.py", line 120, in 
wrapper_copy = lambda *l_args, **l_kwargs: wrapper(*l_args, **l_kwargs)  # 
noqa: E731
  File "/usr/share/ceph/mgr/orchestrator/_interface.py", line 109, in wrapper
return func(*args, **kwargs)
  File "/usr/share/ceph/mgr/orchestrator/module.py", line 805, in 
_host_maintenance_exit
raise_if_exception(completion)
  File "/usr/share/ceph/mgr/orchestrator/_interface.py", line 238, in 
raise_if_exception
e = pickle.loads(c.serialized_exception)
TypeError: __init__() missing 2 required positional arguments: 'hostname' and 
‘addr'


Regards
Dev
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: RGW - S3 bucket browser and/or S3 explorer

[notna]

yes.
 
a nice website showing the "static website" objects with all available object meta data, like size, (modification) time, everything S3 has to offer / supports...
 
an extremely great additional thing would be something similar to (unix / linux like OS'es) symlinks... or, lets say, the GitHub "latest" URLs for the GH releases... so, showing an "(latest) object" and as soon as one either selects it in the "web browser gui" or "downloads" it via other S3-like ways (s3cmd, aws s3, Cyberduck, ...) the object refered to by this "link object" would be shown / downloaded... 
(I'm aware, this is not really officially supported by AWS S3, but let's dream on ;) ) 
and, again, I'm talking about "static websites" only here... not the usual full-fledged S3 RGW endpoint, where you i.e. need to provide your secrets b'cause for this, I've a working "S3 browser" by now already...
 


Gesendet: Dienstag, 11. Februar 2025 um 13:59
Von: "Rok Jaklič" 
An: "Anthony Fecarotta" 
CC: "Laimis Juzeliūnas" ,no...@ist-einmalig.de,ceph-users 
Betreff: [ceph-users] Re: RGW - S3 bucket browser and/or S3 explorer

What would you like to do?Serve your bucket objects as static files on web?On Tue, Feb 11, 2025 at 1:31 PM Anthony Fecarotta wrote:> Interesting! Will mess with that today.>> Regards,>>> * Anthony Fecarotta*> Founder & President> [image: phone-icon] anth...@linehaul.ai> [image: phone-icon] 224-339-1182 [image: phone-icon] (855) 625-0300> [image: phone-icon] 1 Mid America Plz Flr 3 Oakbrook Terrace, IL 60181> [image: phone-icon] www.linehaul.ai> [image: linehaul.ai logo] > [image: linkedin icon]  laimis.juzeliu...@oxylabs.io> wrote:>> > Also Cyberduck works well with RGW: https://cyberduck.io/> > Its just an endpoint difference from AWS since the basis of Ceph S3> > protocol is the same.> >> > Best,> > *Laimis J.*> > laimis.juzeliu...@oxylabs.io> >> > On 11 Feb 2025, at 08:49, Anthony Fecarotta  wrote:> >> > Perhaps this?> >> https://www.google.com/url?q=https://s3browser.com/&source=gmail-imap&ust=173986148400&usg=AOvVaw2aGO3QO62i_KoflakkWnLQ> >> > Regards,> >> >> > * Anthony Fecarotta*> > Founder & President> > [image: phone-icon] anth...@linehaul.ai> > [image: phone-icon] 224-339-1182 [image: phone-icon] (855) 625-0300> > [image: phone-icon] 1 Mid America Plz Flr 3 Oakbrook Terrace, IL 60181> > [image: phone-icon] www.linehaul.ai> > [image: linehaul.ai logo] <> >> https://www.google.com/url?q=http://www.linehaul.ai/&source=gmail-imap&ust=173986148400&usg=AOvVaw0y1OB3e3H92u-pTQnPc9Zp> > >> > [image: linkedin icon] <> >> https://www.google.com/url?q=https://www.linkedin.com/in/anthony-fec/&source=gmail-imap&ust=173986148400&usg=AOvVaw15iLzJ4KQ8zf90Ha_6cnpj> > >> >> >> > On Fri, Feb 7, 2025 at 7:09 PM  wrote:> >> > Hi all.> >> > Is anyone aware of a ("S3 bucket browser" | "S3 explorer"), which works> > with RadosGW static websites?> >> > I've played around with those below, but none of them works (in my> > environment):> >> > -> >> https://www.google.com/url?q=https://github.com/qoomon/aws-s3-bucket-browser&source=gmail-imap&ust=173986148400&usg=AOvVaw2GsknVblPKkS6hKZANRkvy> > -> >> https://www.google.com/url?q=https://github.com/guimou/ceph-js-s3-explorer&source=gmail-imap&ust=173986148400&usg=AOvVaw1JfZ46YSv_gb5PXnERGlVa> > -> >> https://www.google.com/url?q=https://github.com/awslabs/aws-js-s3-explorer&source=gmail-imap&ust=173986148400&usg=AOvVaw2vHZmQ6T15NYIzmh-dnr1s> > -> >> https://www.google.com/url?q=https://github.com/awslabs/aws-js-s3-explorer/tree/v2-alpha&source=gmail-imap&ust=173986148400&usg=AOvVaw2_Ma6FVQ2rP5uVdpfzu3FN> >> > Any hints are highly appreciated!> >> > ___> > ceph-users mailing list -- ceph-users@ceph.io> > To unsubscribe send an email to ceph-users-le...@ceph.io> >> > ___> > ceph-users mailing list -- ceph-users@ceph.io> > To unsubscribe send an email to ceph-users-le...@ceph.io> >> >> >> ___> ceph-users mailing list -- ceph-users@ceph.io> To unsubscribe send an email to ceph-users-le...@ceph.io>___ceph-users mailing list -- ceph-users@ceph.ioTo unsubscribe send an email to ceph-users-le...@ceph.io___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: RGW - S3 bucket browser and/or S3 explorer

[notna]

yes, Cyberduck works pretty ok if you want a "GUI application".
 
As far as I remember, CD switched to the "domain-path" URLs some releases / years ago And the "usual" RGW S3 is still on "path", so you need either to tweak CD in the user's "Cyberduck config file" a bit to ensure it uses the "path-style" S3 URLs... or use a very old version, which did "path" per default...


Gesendet: Dienstag, 11. Februar 2025 um 09:20
Von: "Laimis Juzeliūnas" 
An: "Anthony Fecarotta" 
CC: no...@ist-einmalig.de,ceph-users 
Betreff: Re: [ceph-users] RGW - S3 bucket browser and/or S3 explorer

Also Cyberduck works well with RGW: https://cyberduck.io/
Its just an endpoint difference from AWS since the basis of Ceph S3 protocol is the same.
 
Best,
Laimis J.
laimis.juzeliu...@oxylabs.io


On 11 Feb 2025, at 08:49, Anthony Fecarotta  wrote:


Perhaps this? https://www.google.com/url?q=https://s3browser.com/&source=gmail-imap&ust=173986148400&usg=AOvVaw2aGO3QO62i_KoflakkWnLQRegards,* Anthony Fecarotta*Founder & President[image: phone-icon]  anth...@linehaul.ai[image: phone-icon]  224-339-1182  [image: phone-icon]  (855) 625-0300[image: phone-icon]  1 Mid America Plz Flr 3 Oakbrook Terrace, IL 60181[image: phone-icon]  www.linehaul.ai[image: linehaul.ai logo] [image: linkedin icon] On Fri, Feb 7, 2025 at 7:09 PM  wrote:
Hi all.Is anyone aware of a ("S3 bucket browser" | "S3 explorer"), which workswith RadosGW static websites?I've played around with those below, but none of them works (in myenvironment):- https://www.google.com/url?q=https://github.com/qoomon/aws-s3-bucket-browser&source=gmail-imap&ust=173986148400&usg=AOvVaw2GsknVblPKkS6hKZANRkvy- https://www.google.com/url?q=https://github.com/guimou/ceph-js-s3-explorer&source=gmail-imap&ust=173986148400&usg=AOvVaw1JfZ46YSv_gb5PXnERGlVa- https://www.google.com/url?q=https://github.com/awslabs/aws-js-s3-explorer&source=gmail-imap&ust=173986148400&usg=AOvVaw2vHZmQ6T15NYIzmh-dnr1s- https://www.google.com/url?q=https://github.com/awslabs/aws-js-s3-explorer/tree/v2-alpha&source=gmail-imap&ust=173986148400&usg=AOvVaw2_Ma6FVQ2rP5uVdpfzu3FNAny hints are highly appreciated!___ceph-users mailing list -- ceph-users@ceph.ioTo unsubscribe send an email to ceph-users-le...@ceph.io
___ceph-users mailing list -- ceph-users@ceph.ioTo unsubscribe send an email to ceph-users-le...@ceph.io




___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Cephadm cluster setup with unit-dir and data-dir

[Rajmohan Ramamoorthy]

Hi,

I wanted to understand the behaviour of using --unit-dir and --data-dir
while setting up the ceph cluster using cephadm. Can I pass on a reference
to a mount points on a different disk (other than the root disk) for these
options?

The deployment I am trying to achieve is to persist the Ceph containers
systemd unit files and their persistent cluster data on a different disk
than the root disk. This way, the nodes retains its identity even if the
Operating System disk is redeployed on the root disk and the persistent
data (systemd unit files and container data) is retained on the other disk.

Is this possible with cephadm?

I tried pointing unit-dir and data-dir to a different disk mount point but
it failed. The unit files are always created in the /etc/systemd/system
directory in the root disk.


Unit directory: https://
docs.ceph.com/en/latest/man/8/cephadm/?highlight=orch%20device%20zap#cmdoption-cephadm-unit-dir

Data directory: https://
docs.ceph.com/en/latest/man/8/cephadm/?highlight=orch%20device%20zap#cmdoption-cephadm-data-dir



Regards,
Rajmohan R
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: ceph iscsi gateway

[Iban Cabrillo]

Okay Gregory, 
   Bad news for me, I will have to find another way. The truth is that from the 
operational and long-term maintenance point of view it is practically 
transparent. 
This option fitted very well in our system, since the Ceph cluster is easy to 
maintain, while for example the iSCSI cabinets have to be migrated every 
certain time.

   In our case where the use of block access is not intensive, the truth is 
that we were not having a bad performance, 
it is also true that until now I had only exported 4 blocks with this system.

Regards, I

-- 

  Ibán Cabrillo Bartolomé
  Instituto de Física de Cantabria (IFCA-CSIC)
  Santander, Spain
  Tel: +34942200969/+34669930421
  Responsible for advanced computing service (RSC)
=
=
All our suppliers must know and accept IFCA policy available at:

https://confluence.ifca.es/display/IC/Information+Security+Policy+for+External+Suppliers
==
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: ceph iscsi gateway

[Joachim Kraftmayer]

Hi Iban,
have you thought about replacing the iSCSI gateway with the NVMeOF gateway?
Regards, Joachim


  joachim.kraftma...@clyso.com

  www.clyso.com

  Hohenzollernstr. 27, 80801 Munich

Utting | HR: Augsburg | HRB: 25866 | USt. ID-Nr.: DE275430677



Am Mi., 12. Feb. 2025 um 08:53 Uhr schrieb Iban Cabrillo <
cabri...@ifca.unican.es>:

> Okay Gregory,
>Bad news for me, I will have to find another way. The truth is that
> from the operational and long-term maintenance point of view it is
> practically transparent.
> This option fitted very well in our system, since the Ceph cluster is easy
> to maintain, while for example the iSCSI cabinets have to be migrated every
> certain time.
>
>In our case where the use of block access is not intensive, the truth
> is that we were not having a bad performance,
> it is also true that until now I had only exported 4 blocks with this
> system.
>
> Regards, I
>
> --
> 
>   Ibán Cabrillo Bartolomé
>   Instituto de Física de Cantabria (IFCA-CSIC)
>   Santander, Spain
>   Tel: +34942200969/+34669930421
>   Responsible for advanced computing service (RSC)
>
> =
>
> =
> All our suppliers must know and accept IFCA policy available at:
>
>
> https://confluence.ifca.es/display/IC/Information+Security+Policy+for+External+Suppliers
>
> ==
> ___
> ceph-users mailing list -- ceph-users@ceph.io
> To unsubscribe send an email to ceph-users-le...@ceph.io
>
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io