Re: [ceph-users] RGW: removal of support for fastcgi

2017-05-15 Thread Yehuda Sadeh-Weinraub
On Mon, May 15, 2017 at 8:35 AM, Ken Dreyer wrote: > On Fri, May 5, 2017 at 1:51 PM, Yehuda Sadeh-Weinraub > wrote: >> >> TL;DR: Does anyone care if we remove support for fastcgi in rgw? > > Please remove it as soon as possible. The old libfcgi project's code &g

Re: [ceph-users] RGW lifecycle not expiring objects

2017-06-02 Thread Yehuda Sadeh-Weinraub
Have you opened a ceph tracker issue, so that we don't lose track of the problem? Thanks, Yehuda On Fri, Jun 2, 2017 at 3:05 PM, wrote: > Hi Graham. > > We are on Kraken and have the same problem with "lifecycle". Various (other) > tools like s3cmd or CyberDuck do show the applied "expiration"

Re: [ceph-users] Living with huge bucket sizes

2017-06-09 Thread Yehuda Sadeh-Weinraub
On Fri, Jun 9, 2017 at 2:21 AM, Dan van der Ster wrote: > Hi Bryan, > > On Fri, Jun 9, 2017 at 1:55 AM, Bryan Stillwell > wrote: >> This has come up quite a few times before, but since I was only working with >> RBD before I didn't pay too close attention to the conversation. I'm >> looking >>

Re: [ceph-users] s3cmd --disable-multipart

2015-12-10 Thread Yehuda Sadeh-Weinraub
On Thu, Dec 10, 2015 at 11:10 AM, Deneau, Tom wrote: > If using s3cmd to radosgw and using s3cmd's --disable-multipart option, is > there any limit to the size of the object that can be stored thru radosgw? > rgw limits plain uploads to 5GB > Also, is there a recommendation for multipart chunk

Re: [ceph-users] Preventing users from deleting their own bucket in S3

2015-12-10 Thread Yehuda Sadeh-Weinraub
On Thu, Dec 10, 2015 at 11:25 AM, Gregory Farnum wrote: > On Thu, Dec 10, 2015 at 2:26 AM, Xavier Serrano > wrote: >> Hello, >> >> We are using ceph version 0.94.4, with radosgw offering S3 storage >> to our users. >> >> Each user is assigned one bucket (and only one; max_buckets is set to 1). >>

Re: [ceph-users] about federated gateway

2015-12-14 Thread Yehuda Sadeh-Weinraub
On Sun, Dec 13, 2015 at 7:27 AM, 孙方臣 wrote: > Hi, All, > > I'm setting up federated gateway. One is master zone, the other is slave > zone. Radosgw-agent is running in slave zone. I have encountered some > problems, can anybody help answering this: > > 1. When put a object to radosgw, there are t

Re: [ceph-users] v10.0.2 released

2016-01-14 Thread Yehuda Sadeh-Weinraub
On Thu, Jan 14, 2016 at 7:37 AM, Sage Weil wrote: > This development release includes a raft of changes and improvements for > Jewel. Key additions include CephFS scrub/repair improvements, an AIX and > Solaris port of librados, many librbd journaling additions and fixes, > extended per-pool optio

Re: [ceph-users] RGW -- 404 on keys in bucket.list() thousands of multipart ids listed as well.

2016-01-15 Thread Yehuda Sadeh-Weinraub
On Thu, Jan 14, 2016 at 10:51 PM, seapasu...@uchicago.edu wrote: > It looks like the gateway is experiencing a similar race condition to what > we reported before. > > The rados object has a size of 0 bytes but the bucket index shows the object > listed and the object metadata shows a size of > 71

Re: [ceph-users] RGW -- 404 on keys in bucket.list() thousands of multipart ids listed as well.

2016-01-15 Thread Yehuda Sadeh-Weinraub
uot;", > "ns": "", > "object": > "2015\/01\/01\/PAKC\/NWS_NEXRAD_NXL2DP_PAKC_2015010111_20150101115959.tar", > "instance": "" > }, > "hea

Re: [ceph-users] RGW -- 404 on keys in bucket.list() thousands of multipart ids listed as well.

2016-01-15 Thread Yehuda Sadeh-Weinraub
2015010113_20150101135959.tar.2~wksHvto9gRgHUJbhm_TZPXJTZUPXLT2.1_1 > default.384153.1__multipart_2015/01/01/KABR/NWS_NEXRAD_NXL2DP_KABR_2015010113_20150101135959.tar.2~wksHvto9gRgHUJbhm_TZPXJTZUPXLT2.1 > > > > > On 1/15/16 12:05 PM, Yehuda Sadeh-Weinraub wrote: >> >&g

Re: [ceph-users] RGW -- 404 on keys in bucket.list() thousands of multipart ids listed as well.

2016-01-15 Thread Yehuda Sadeh-Weinraub
y I am a bit confused. The successful list that I provided is from a > different object of the same size to show that I could indeed get a list. > Are you saying to copy the working object to the missing object? Sorry for > the confusion. > > > On 1/15/16 3:20 PM, Yehuda Sadeh-We

Re: [ceph-users] RGW -- 404 on keys in bucket.list() thousands of multipart ids listed as well.

2016-01-15 Thread Yehuda Sadeh-Weinraub
9.tar' > .rgw.buckets/default.384153.1_2015/01/01/PAKC/NWS_NEXRAD_NXL2DP_PAKC_2015010111_20150101115959.tar > mtime 2015-11-04 15:29:30.00, size 0 > > Sorry again for the confusion. > > > > On 1/15/16 3:58 PM, Yehuda Sadeh-Weinraub wrote: >> >> Ah, I

Re: [ceph-users] RGW -- 404 on keys in bucket.list() thousands of multipart ids listed as well.

2016-01-19 Thread Yehuda Sadeh-Weinraub
"remote_addr": "", > 17462372 "object_owner": "b05f707271774dbd89674a0736c9406e", > 17462373 "user": "b05f707271774dbd89674a0736c9406e", > 17462374 "operation": "PUT", > 17462

Re: [ceph-users] RGW -- 404 on keys in bucket.list() thousands of multipart ids listed as well.

2016-01-20 Thread Yehuda Sadeh-Weinraub
On Wed, Jan 20, 2016 at 10:43 AM, seapasu...@uchicago.edu wrote: > > > On 1/19/16 4:00 PM, Yehuda Sadeh-Weinraub wrote: >> >> On Fri, Jan 15, 2016 at 5:04 PM, seapasu...@uchicago.edu >> wrote: >>> >>> I have looked all

Re: [ceph-users] RGW -- 404 on keys in bucket.list() thousands of multipart ids listed as well.

2016-01-20 Thread Yehuda Sadeh-Weinraub
? I mean > ideally this should never be possible, right? Even with a complete object > that is 0 bytes it should be downloaded as 0 bytes and have a different > md5sum and not report as 7mb? > > > > On 1/20/16 1:30 PM, Yehuda Sadeh-Weinraub wrote: >> >> On Wed, Jan 20,

Re: [ceph-users] RGW -- 404 on keys in bucket.list() thousands of multipart ids listed as well.

2016-01-20 Thread Yehuda Sadeh-Weinraub
> I'm working on getting the code they used and trying different timeouts in > my multipart upload code. Right now I have not created any new 404 keys > though :-( > > > On 1/20/16 3:44 PM, Yehuda Sadeh-Weinraub wrote: >> >> We'll need to confirm that this is th

Re: [ceph-users] RGW -- 404 on keys in bucket.list() thousands of multipart ids listed as well.

2016-01-21 Thread Yehuda Sadeh-Weinraub
On Thu, Jan 21, 2016 at 4:02 PM, seapasu...@uchicago.edu wrote: > I haven't been able to reproduce the issue on my end but I do not fully > understand how the bug exists or why it is happening. I was finally given > the code they are using to upload the files:: > > http://pastebin.com/N0j86NQJ > >

Re: [ceph-users] How-to doc: hosting a static website on radosgw

2016-01-26 Thread Yehuda Sadeh-Weinraub
On Tue, Jan 26, 2016 at 2:37 PM, Florian Haas wrote: > On Tue, Jan 26, 2016 at 8:56 PM, Wido den Hollander wrote: >> On 01/26/2016 08:29 PM, Florian Haas wrote: >>> Hi everyone, >>> >>> we recently worked a bit on running a full static website just on >>> radosgw (akin to >>> http://docs.aws.amaz

Re: [ceph-users] RGW :: bucket quota not enforced below 1

2016-01-27 Thread Yehuda Sadeh-Weinraub
On Wed, Jan 27, 2016 at 4:18 PM, seapasu...@uchicago.edu wrote: > if you set a RGW user to have abucket quota of 0 buckets you can still > create buckets. The only way I have found to prevent a user from being able > to create buckets is to set the op_mask to read. 1.) it looks like > bucket_polic

Re: [ceph-users] RGW: oddity when creating users via admin api

2016-01-27 Thread Yehuda Sadeh-Weinraub
On Wed, Jan 27, 2016 at 4:20 PM, seapasu...@uchicago.edu wrote: > So when I create a new user with the admin api. If the user already exists > it just generates a new keypair for that user. Shouldn't the admin api > report that the user already exists? I ask because I can end up with > multiple ke

Re: [ceph-users] 411 Content-Length required error

2016-01-27 Thread Yehuda Sadeh-Weinraub
On Wed, Jan 27, 2016 at 3:31 PM, John Hogenmiller wrote: > I did end up switching to civetweb and I also found that rgw content length > compat, which I set to true. I am still getting the 411 Length required > issue. > > I have had more discussions with our testing team, and I am still trying to

Re: [ceph-users] radosgw anonymous write

2016-02-09 Thread Yehuda Sadeh-Weinraub
On Tue, Feb 9, 2016 at 5:15 AM, Jacek Jarosiewicz wrote: > Hi list, > > My setup is: ceph 0.94.5, ubuntu 14.04, tengine (patched nginx). > > I'm trying to migrate from our old file storage (MogileFS) to the new ceph > radosgw. The problem is that the old storage had no access control - no > author

Re: [ceph-users] Idea for speedup RadosGW for buckets with many objects.

2016-02-18 Thread Yehuda Sadeh-Weinraub
On Wed, Feb 17, 2016 at 12:51 PM, Krzysztof Księżyk wrote: > Hi, > > I'm experiencing problem with poor performance of RadosGW while operating on > bucket with many object. That's known issue with LevelDB and can be > partially resolved using shrading but I have one more idea. As I see in ceph > o

Re: [ceph-users] Problem create user RGW

2016-02-24 Thread Yehuda Sadeh-Weinraub
try running: $ radosgw-admin --name client.rgw.servergw001 metadata list user Yehuda On Wed, Feb 24, 2016 at 8:41 AM, Andrea Annoè wrote: > I don’t see any user create in RGW > > > > sudo radosgw-admin metadata list user > > [ > > ] > > > > > > sudo radosgw-admin user create --uid="user1site1"

Re: [ceph-users] radosgw flush_read_list(): d->client_c->handle_data() returned -5

2016-02-27 Thread Yehuda Sadeh-Weinraub
On Wed, Feb 24, 2016 at 5:48 PM, Ben Hines wrote: > Any idea what is going on here? I get these intermittently, especially with > very large file. > > The client is doing RANGE requests on this >51 GB file, incrementally > fetching later chunks. > > 2016-02-24 16:30:59.669561 7fd33b7fe700 1 =

Re: [ceph-users] ceph RGW NFS

2016-03-01 Thread Yehuda Sadeh-Weinraub
On Tue, Mar 1, 2016 at 7:23 AM, Daniel Gryniewicz wrote: > On 02/28/2016 08:36 PM, David Wang wrote: >> >> Hi All, >> How the progress of NFS on RGW? Does it released on Infernalis? The >> contents of NFS on RGW is >> http://tracker.ceph.com/projects/ceph/wiki/RGW_-_NFS >> >> > > The FSAL has

Re: [ceph-users] Problem: silently corrupted RadosGW objects caused by slow requests

2016-03-03 Thread Yehuda Sadeh-Weinraub
On Thu, Feb 25, 2016 at 7:17 AM, Ritter Sławomir wrote: > Hi, > > > > We have two CEPH clusters running on Dumpling 0.67.11 and some of our > "multipart objects" are incompleted. It seems that some slow requests could > cause corruption of related S3 objects. Moveover GETs for that objects are > w

Re: [ceph-users] Problem: silently corrupted RadosGW objects caused by slow requests

2016-03-04 Thread Yehuda Sadeh-Weinraub
On Fri, Mar 4, 2016 at 7:26 AM, Ritter Sławomir wrote: >> From: Robin H. Johnson [mailto:robb...@gentoo.org] >> Sent: Friday, March 04, 2016 12:40 AM >> To: Ritter Sławomir >> Cc: ceph-us...@ceph.com; ceph-devel >> Subject: Re: [ceph-users] Problem: silently corrupted RadosGW objects caused >> by

Re: [ceph-users] rgw bucket deletion woes

2016-03-19 Thread Yehuda Sadeh-Weinraub
On Tue, Mar 15, 2016 at 11:36 PM, Pavan Rallabhandi wrote: > Hi, > > I find this to be discussed here before, but couldn¹t find any solution > hence the mail. In RGW, for a bucket holding objects in the range of ~ > millions, one can find it to take for ever to delete the bucket(via > radosgw-admi

Re: [ceph-users] Can Jewel read Hammer radosgw buckets?

2016-04-23 Thread Yehuda Sadeh-Weinraub
On Sat, Apr 23, 2016 at 6:22 AM, Richard Chan wrote: > Hi Cephers, > > I upgraded to Jewel and noted the is massive radosgw multisite rework > in the release notes. > > Can Jewel radosgw be configured to present existing Hammer buckets? > On a test system, jewel didn't recognise my Hammer buckets

Re: [ceph-users] Can Jewel read Hammer radosgw buckets?

2016-04-25 Thread Yehuda Sadeh-Weinraub
(sorry for resubmission, adding ceph-users) On Mon, Apr 25, 2016 at 9:47 AM, Richard Chan wrote: > Hi Yehuda > > I created a test 3xVM setup with Hammer and one radosgw on the (separate) > admin node; creating one user and buckets. > > I upgraded the VMs to jewel and created a new radosgw on one

Re: [ceph-users] Can Jewel read Hammer radosgw buckets?

2016-04-25 Thread Yehuda Sadeh-Weinraub
I managed to reproduce the issue, and there seem to be multiple problems. Specifically we have an issue when upgrading a default cluster that hasn't had a zone (and region) explicitly configured before. There is another bug that I found (http://tracker.ceph.com/issues/15597) that makes things even

Re: [ceph-users] RGW Swift metadata dropped when S3 bucket versioning enabled

2018-11-29 Thread Yehuda Sadeh-Weinraub
On Wed, Nov 28, 2018 at 10:07 AM Maxime Guyot wrote: > > Hi Florian, > > You assumed correctly, the "test" container (private) was created with the > "openstack container create test", then I am using the S3 API to > enable/disable object versioning on it. > I use the following Python snippet to

<    1   2   3   4   5   6