Re: [ceph-users] [RadosGW] FastCGI: comm with server "/var/www/s3gw.fcgi" aborted: read failed

2013-12-26 Thread Yehuda Sadeh
(0 bytes) received from server "/var/www/s3gw.fcgi" > > > > > 3) No any wait in OSD or RGW's perf dump > > 4) Am I in the wrong Fastcgi module? Don't think so, otherwise all PUTs would have failed. Yehuda > > Good news is that the RadosGW can handle 5

Re: [ceph-users] [RadosGW] FastCGI: comm with server "/var/www/s3gw.fcgi" aborted: read failed

2013-12-26 Thread Yehuda Sadeh
it cpu limit with enough concurrency. Yehuda > > Cheers / Hugo > > > 2013/12/26 Yehuda Sadeh >> >> On Thu, Dec 26, 2013 at 3:00 AM, Kuo Hugo wrote: >> > Hi all, >> > >> > >> > I think the FastCGI module is the latest one on my server. &g

Re: [ceph-users] RGW sends response as "Transfer-Encoding: chunked"

2013-12-29 Thread Yehuda Sadeh
On Sat, Dec 28, 2013 at 4:12 PM, Sam Yaple wrote: > I am testing the RadosGW as a replacement for swift and I am running into an > issue. I am using keystone to authenticate and it is working perfectly > except for the issue below. > > The following command returns from RGW as 'Transfer-Encoding:

Re: [ceph-users] Custom ACL's are not being honoured.

2014-01-07 Thread Yehuda Sadeh
On Tue, Jan 7, 2014 at 2:40 AM, Jaseer Tk wrote: > > Hi all, > > I tried to apply custom ACL(only Read) on an s3 bucket created. The rules > seems to have got applied. But it looks the ACL's not getting honored. > > when I use getacl, I get the result. > === > > Array > ( > [0] => Array >

Re: [ceph-users] Custom ACL's are not being honoured.

2014-01-08 Thread Yehuda Sadeh
s3 interface. > getacl shows my acl is applied. But user is still able to delete. Please > let me know if you want me to share my php code for getacl and putacl. > > > > > -- > Thank you, > Jaseer TK > > > > > > > > > On Tue, Jan 7, 2014 at 9:30

Re: [ceph-users] Custom ACL's are not being honoured.

2014-01-08 Thread Yehuda Sadeh
aseer TK > > > > > > On Wed, Jan 8, 2014 at 10:27 PM, Yehuda Sadeh wrote: >> >> On Tue, Jan 7, 2014 at 11:16 PM, Jaseer Tk wrote: >> > >> > Hi Yehuda, >> > >> > Thanks for response. >> > >> > my setup is on

Re: [ceph-users] Custom ACL's are not being honoured.

2014-01-08 Thread Yehuda Sadeh
.1_vip.txt > [getxattrs,stat] 8.eb952482 e567) v4 -- ?+0 0x7faa8c008eb0 con > 0x7faabc043070 > 2014-01-08 18:06:46.392062 7faad8d3c700 1 -- 10.2.4.203:0/1004169 <== > osd.54 10.2.4.203:6865/22234 1 osd_op_reply(1248 east-1.8607.1_vip.txt > [getxattrs,stat] v0'0 uv59 ondisk = 0) v6 222

Re: [ceph-users] Custom ACL's are not being honoured.

2014-01-08 Thread Yehuda Sadeh
remove the system flag from a user? > > -Hemant > > > On Wed, Jan 8, 2014 at 11:44 PM, Yehuda Sadeh wrote: >> >> Well, user jaseer has the 'system' flag set so it acts like a super user. >> >> On Wed, Jan 8, 2014 at 10:09 AM, Jaseer Tk wrot

Re: [ceph-users] recreate bucket error

2014-01-22 Thread Yehuda Sadeh
On Wed, Jan 22, 2014 at 2:33 AM, Sylvain Munaut wrote: > Hi, > > On Sat, Dec 7, 2013 at 6:34 PM, Yehuda Sadeh wrote: >> Sounds like disabling the cache triggers some bug. I'll open a relevant >> ticket. > > Any news on this ? > > I have the same issue, bu

Re: [ceph-users] RADOS Gateway Issues

2014-01-22 Thread Yehuda Sadeh
On Wed, Jan 22, 2014 at 8:05 AM, Graeme Lambert wrote: > Hi, > > I'm using the aws-sdk-for-php classes for the Ceph RADOS gateway but I'm > getting an intermittent issue with the uploading files. > > I'm attempting to upload an array of objects to Ceph one by one using the > create_object() functi

Re: [ceph-users] RADOS Gateway Issues

2014-01-22 Thread Yehuda Sadeh
er when one of them is stuck: > http://pastebin.com/KVcpAeYT > > Regarding the modules, apache version is 2.2.22-2precise.ceph and the > fastcgi mod version is 2.4.7~0910052141-2~bpo70+1.ceph. > > Best regards > > Graeme > > > On 22/01/14 16:28, Yehuda Sadeh wro

Re: [ceph-users] RADOS Gateway Issues

2014-01-22 Thread Yehuda Sadeh
ards > > Graeme > > > > On 22/01/14 16:38, Graeme Lambert wrote: > > Hi, > > Following discussions with people in the IRC I set debug_ms and this is what > is being looped over and over when one of them is stuck: > http://pastebin.com/KVcpAeYT > > Regarding th

Re: [ceph-users] Ceph S3 Object Gateway Error : 405 Method Not Allowed

2014-01-23 Thread Yehuda Sadeh
On Thu, Jan 23, 2014 at 8:24 AM, David Francheski (dfranche) wrote: > Hi, > > I'm using the latest Emperor Ceph release, and trying to bring up the S3 > Object Gateway. > I have a Ceph cluster deployed on an Ubuntu 13.10 based distribution. > > When I attempt to create a S3 bucket using the "boto"

Re: [ceph-users] Ceph S3 Object Gateway Error : 405 Method Not Allowed

2014-01-23 Thread Yehuda Sadeh
Yehuda > > Thanks again for your help > I really appreciate it ! > > -David > > > On 1/23/14 9:31 AM, "Yehuda Sadeh" wrote: > >>On Thu, Jan 23, 2014 at 8:24 AM, David Francheski (dfranche) >> wrote: >>> Hi, >>> >>> I&#x

Re: [ceph-users] RadosGW S3 API - Bucket Versions

2014-01-24 Thread Yehuda Sadeh
On Thu, Jan 23, 2014 at 4:27 PM, Greg Poirier wrote: > Hello! > > I have a great deal of interest in the ability to version objects in buckets > via the S3 API. Where is this on the roadmap for Ceph? This still needs to be prioritized. There was a discussion about it in the latest CDS, but there

Re: [ceph-users] Radosgw ops and usage logging

2014-01-27 Thread Yehuda Sadeh
On Mon, Jan 27, 2014 at 3:34 PM, Caius Howcroft wrote: > Hi all > > I would like to run op and usage log on a production cluster to aid in some > debugging. The problem is it would have to run for several weeks. Does > anyone have any feel for what the performance hit would be ? Are large ops > lo

Re: [ceph-users] s3 downloaded file verification

2014-01-30 Thread Yehuda Sadeh
On Thu, Jan 30, 2014 at 5:58 AM, Dominik Mostowiec wrote: > Hi, > I'm looking for solution how to verify file downloaded from s3 where > ETag is multiparted ( with '-' ) and don't know how is part size. > > When part size is known, it is possible eg do it with scrip: > https://github.com/Teachnova

Re: [ceph-users] RGW Replication

2014-02-04 Thread Yehuda Sadeh
On Tue, Feb 4, 2014 at 10:07 AM, Craig Lewis wrote: > > > > On 2/3/14 14:34 , Craig Lewis wrote: > > > On 2/3/14 10:51 , Gregory Farnum wrote: > > On Mon, Feb 3, 2014 at 10:43 AM, Craig Lewis > wrote: > > I've been noticing somethings strange with my RGW federation. I added some > statistics to

Re: [ceph-users] RGW Replication

2014-02-04 Thread Yehuda Sadeh
ok | LinkedIn | Blog > > On 2/4/14 11:36 , Yehuda Sadeh wrote: > > Also, verify whether any objects are missing. Start with just counting > the total number of objects in the buckets (radosgw-admin bucket stats > can give you that info). > > Yehuda > > > T

Re: [ceph-users] RGW Replication

2014-02-04 Thread Yehuda Sadeh
On Tue, Feb 4, 2014 at 5:06 PM, Craig Lewis wrote: > > > On 2/4/14 14:43 , Yehuda Sadeh wrote: > > Now that objects are missing in the slave, how do I fix it? radosgw-agent > --sync-scope=full ? > > That would do it, yes. > > > I'm hesitant to do this, at le

Re: [ceph-users] RGW Replication

2014-02-05 Thread Yehuda Sadeh
On Wed, Feb 5, 2014 at 2:21 PM, Josh Durgin wrote: > On 02/05/2014 01:23 PM, Craig Lewis wrote: >> >> >> On 2/4/14 20:02 , Josh Durgin wrote: >>> >>> >>> From the log it looks like you're hitting the default maximum number of >>> entries to be processed at once per shard. This was intended to prev

Re: [ceph-users] Swift APIs not authenticating Rados gateway !!!

2014-02-19 Thread Yehuda Sadeh
On Wed, Feb 19, 2014 at 2:37 AM, Srinivasa Rao Ragolu wrote: > Hi all, > > I have setup cluster successfully and one node using to setup rados gateway. > Machine is Fedora 19(all nodes) > > Steps I followed > > 1) Installed httpd, mod_fastcgi, ceph and ceph-radosgw using link > http://ceph.com/doc

Re: [ceph-users] Ceph GET latency

2014-02-19 Thread Yehuda Sadeh
On Tue, Feb 18, 2014 at 7:24 AM, Guang Yang wrote: > Hi ceph-users, > We are using Ceph (radosgw) to store user generated images, as GET latency > is critical for us, most recently I did some investigation over the GET path > to understand where time spend. > > I first confirmed that the latency c

Re: [ceph-users] radosgw swift client, Auth Get failed

2014-02-19 Thread Yehuda Sadeh
On Wed, Feb 19, 2014 at 6:36 PM, JinHwan Hwang wrote: > After i followed radosgw install and configuration > document(http://ceph.com/docs/next/radosgw/config/), I have got stucked on > using swift client. It just returns me "Auth GET failed", even though i can > see that radosgw already have swif

[ceph-users] Fwd: [rgw] increase the first chunk size

2014-02-20 Thread Yehuda Sadeh
Apparently I missed including the mailing list in the response. -- Forwarded message -- From: Yehuda Sadeh Date: Thu, Feb 20, 2014 at 9:04 AM Subject: Re: [ceph-users] [rgw] increase the first chunk size To: Ray Lv On Thu, Feb 20, 2014 at 5:08 AM, Ray Lv wrote: >

Re: [ceph-users] v0.77 released

2014-02-20 Thread Yehuda Sadeh
On Thu, Feb 20, 2014 at 1:40 PM, Derek Yarnell wrote: > >> * rgw: user quotas (Yehuda Sadeh) > > Hi, > > Now that there will be both bucket and user level quotas (which thank > you yehuda this is critical to what we want to do with ceph in our > organization) can you e

Re: [ceph-users] adding placement pool to radosgw

2014-02-20 Thread Yehuda Sadeh
On Thu, Feb 20, 2014 at 2:18 PM, Michael Nelson wrote: > I am trying to add a placement pool to radosgw (based on > http://comments.gmane.org/gmane.comp.file-systems.ceph.user/4992), but > radosgw keeps complaining in the log that it can't find the placement rule > when I create a bucket using s3c

Re: [ceph-users] ceph access using curl

2014-02-24 Thread Yehuda Sadeh
Try X-Auth-Key and X-Auth-User instead. Should use /auth. Yehuda On Fri, Oct 4, 2013 at 12:37 PM, Snider, Tim wrote: > I'm having pilot error with getting the path correct using curl. > > Bucket listing using "radosgw-admin bucket list" works as does the swift > API. > > Can someone point out my

Re: [ceph-users] ceph access using curl

2014-02-24 Thread Yehuda Sadeh
See my last (and much overdue) response to Tim. Yehuda On Mon, Feb 24, 2014 at 12:14 PM, Liu, Larry wrote: > Tim, has this been resolved? I'm having the same issue. No one seems to have > an answer. Guess we just have to stick to using swift client util pkg. > > From: , Tim > Date: Friday, Oct

Re: [ceph-users] ceph access using curl

2014-02-24 Thread Yehuda Sadeh
HTTP/1.1 204 No Content > < Date: Mon, 24 Feb 2014 20:55:28 GMT > < Server: Apache/2.2.22 (Ubuntu) > < Vary: Accept-Encoding > < Content-Type: text/plain; charset=utf-8 > < > * Connection #0 to host u1304test001.compliant.disney.private left intact > * Closing connecti

Re: [ceph-users] Admin credentials

2014-02-27 Thread Yehuda Sadeh
On Thu, Feb 27, 2014 at 7:53 AM, Erik Tank wrote: > On http://ceph.com/docs/master/radosgw/adminops/ a "configurable 'admin' > resource entry point" is described. I have the URI, however, I can not find > information on how to get/set the 'admin' credentials. > > Any help is greatly appreciated

Re: [ceph-users] [rgw] increase the first chunk size

2014-03-03 Thread Yehuda Sadeh
can get total latency improved much even the first byte > latency increased very little. > > Thanks, > Ray > > On 2/21/14, 4:46 AM, "Yehuda Sadeh" wrote: > >>Apparently I missed including the mailing list in the response. >> >>-- Forwarded me

Re: [ceph-users] [rgw] increase the first chunk size

2014-03-04 Thread Yehuda Sadeh
then set rgw_max_chunk_size to > 1 MB, radosgw can read out the previously put objects. > > Thanks, > Ray > > > On 3/4/14, 9:01 AM, "Yehuda Sadeh" wrote: > >>Created issue #7589, and pushed a wip-7589 that addresses the issue. I >>just ran some basic te

Re: [ceph-users] State of RadosGW and small files

2014-03-05 Thread Yehuda Sadeh
On Wed, Mar 5, 2014 at 12:26 AM, Christopher O'Connell wrote: > Hello, > > There are several older discussions regarding RGW performance with high > volume small files. > > I'm planning on running some tests on our test cluster to benchmark this > performance, but before I do I wanted to ask sever

Re: [ceph-users] Access Denied errors

2014-03-10 Thread Yehuda Sadeh
On Mon, Mar 10, 2014 at 4:26 PM, Steve Carter wrote: > Further, here is the logging output (when I set 'debug rgw log = 20/20' in > ceph.conf). I have removed some information. The server replies with a 403. > Any insight into why? When the account submits a non-admin type request it > work

Re: [ceph-users] neither POSIX nor REST

2014-03-12 Thread Yehuda Sadeh
On Wed, Mar 12, 2014 at 9:58 AM, Lorieri wrote: > Hi, > > I saw radosgw-admin doesn't require radosgw daemon. > > I was wondering if by adding creation of buckets and manipulation of > objects to radosgw-admin (or a library similar to it) ceph would have > a great tool between the low and high lev

Re: [ceph-users] RGW python API failed

2014-03-12 Thread Yehuda Sadeh
On Wed, Mar 12, 2014 at 7:40 PM, wsnote wrote: > I use the RGW python API but never succeeded. > > test-ceph.py: > #!/usr/bin/python > # Filename: test-ceph.py > import boto > import boto.s3.connection > access_key = 'TJLN5ID' > secret_key = 'EcaCMS1\/4b3\/AAM50Q' > > c

Re: [ceph-users] Access Denied errors

2014-03-12 Thread Yehuda Sadeh
ideas on this? Anything I can do to further troubleshoot? > > -Steve > > > > From: "Steve Carter" > To: "Yehuda Sadeh" > Cc: ceph-users@lists.ceph.com > Sent: Tuesday, March 11, 2014 4:42:24 PM > > Subject: Re: [ce

Re: [ceph-users] help with ceph radosgw configure

2014-03-14 Thread Yehuda Sadeh
You might have a default web server set up on apache. Remove it and restart apache. On Fri, Mar 14, 2014 at 8:03 AM, wsnote wrote: > OS: CentOS 6.5 > version: ceph 0.67.7 > > I have configured radosgw and start it. > When I surfed https://hostname:65443/, I thought it should be >

Re: [ceph-users] 答复: why objects are still in .rgw.buckets after deleted

2014-03-19 Thread Yehuda Sadeh
The RGW objects are [for most cases] immutable. Therefore in order to provide read and write consistency we keep most of the data in immutable rados objects, and do the operations on the mutable 'head' part of the object atomically. So this allows us to do stuff like having one user read data of th

Re: [ceph-users] RGW S3 multipart upload - Copy Part

2014-03-21 Thread Yehuda Sadeh
On Fri, Mar 21, 2014 at 6:25 AM, Gudu, Diana (SCC) wrote: > Hi everyone, > > I was wondering if the Copy Part feature of the S3 API is supported or > whether there is a plan to implement this in the near future. There wasn't any plan, haven't seen any demand as you're the first one to bring this

Re: [ceph-users] rados federated gateway - selective replication

2014-03-26 Thread Yehuda Sadeh
On Wed, Mar 26, 2014 at 10:58 AM, Dinu Vlad wrote: > I'm trying to figure out a way to configure selective replication of objects > between 2 geographically-separated ceph clusters, via the radosgw-agent. > Ideally that should happen at the bucket level - but as far as I can figure > that seems

Re: [ceph-users] rados federated gateway - selective replication

2014-03-26 Thread Yehuda Sadeh
icated, one not? I would need 2 radosgw instances, each pointing to > a different zone, using different access URLs ... ? > > > On Mar 26, 2014, at 8:07 PM, Yehuda Sadeh wrote: > >> On Wed, Mar 26, 2014 at 10:58 AM, Dinu Vlad wrote: >>> I'm trying to figure o

Re: [ceph-users] ec pools and radosgw

2014-03-27 Thread Yehuda Sadeh
On Wed, Mar 26, 2014 at 4:48 PM, Michael Nelson wrote: > I am playing around with erasure coded pools on 0.78-348 (firefly) and am > attempting to enable EC on the .rgw.buckets pool for radosgw > (fresh install). > > If I use a plain EC profile (no settings changed), uploads of various sizes > wor

Re: [ceph-users] ec pools and radosgw

2014-03-27 Thread Yehuda Sadeh
On Thu, Mar 27, 2014 at 1:17 PM, Michael Nelson wrote: > > > On Thu, 27 Mar 2014, Yehuda Sadeh wrote: > >> On Wed, Mar 26, 2014 at 4:48 PM, Michael Nelson >> wrote: >>> >>> I am playing around with erasure coded pools on 0.78-348 (firefly) and am >&

Re: [ceph-users] radosgw multipart-uploaded downloads fail

2014-04-01 Thread Yehuda Sadeh
On Tue, Apr 1, 2014 at 1:19 AM, Benedikt Fraunhofer wrote: > Hello everyone, > > I can't download anything that's been uploaded as a multipart-upload. > I'm on 0.78 (f6c746c314d7b87b8419b6e584c94bfe4511dbd4) > on a non-ec-pool. > > The upload is acknowledged as beeing ok > > 2014-03-31 14:56:56.7

Re: [ceph-users] radosgw multipart-uploaded downloads fail

2014-04-02 Thread Yehuda Sadeh
On Wed, Apr 2, 2014 at 2:08 AM, Benedikt Fraunhofer wrote: > Hi Yehuda, > > i tried your patch and it feels fine, > except you might need some special handling for those already corrupt uploads, > as trying to delete them gets radosgw in an endless loop and high cpu usage: The problem was with th

Re: [ceph-users] Multi-site Implementation

2014-04-03 Thread Yehuda Sadeh
On Tue, Apr 1, 2014 at 7:34 PM, Shang Wu wrote: > Hi all, > > I have some questions about the Ceph multi-site implementation. > > I am thinking to have Ceph as the storage solution for across three internal > site. I think, with a good internet connection, using the Multi-site object > storage wit

Re: [ceph-users] radosgw multipart-uploaded downloads fail

2014-04-03 Thread Yehuda Sadeh
Ok, we were able to reproduce it. Opened issue #7978, and there's a fix pending for upstream. Thanks, Yehuda On Wed, Apr 2, 2014 at 7:20 AM, Yehuda Sadeh wrote: > On Wed, Apr 2, 2014 at 2:08 AM, Benedikt Fraunhofer > wrote: >> Hi Yehuda, >> >> i tried your patch

Re: [ceph-users] RadosGW: bad request

2014-04-07 Thread Yehuda Sadeh
On Mon, Apr 7, 2014 at 5:38 AM, Gandalf Corvotempesta wrote: > I'm getting these trying to upload any file: > > 2014-04-07 14:33:27.084369 7f5268f86700 5 Getting permissions > id=testuser owner=testuser perm=2 > 2014-04-07 14:33:27.084372 7f5268f86700 10 uid=testuser requested > perm (type)=2, p

Re: [ceph-users] RadosGW Quotas

2014-04-07 Thread Yehuda Sadeh
On Mon, Apr 7, 2014 at 4:34 PM, Derek Yarnell wrote: > Hi, > > Most likely Yehuda can speak to this. So there is some documentation on > the way to set quotas for a user in master (src/rgw/rgw_rest_user.cc > line 712) but these are not in the docs it seems yet. I have started to > incorporate the

Re: [ceph-users] [Ceph-rgw] pool assigment

2014-04-11 Thread Yehuda Sadeh
On Fri, Apr 11, 2014 at 3:20 AM, wrote: > Hi all, > > Context : CEPH dumpling on Ubuntu 12.04 > > I would like to manage as accurately as possible the pools assigned to Rados > gateway > My goal is to apply specific SLA to applications which use http-driven > storage. > I 'd like to store the con

Re: [ceph-users] Radosgw and s3cmd

2014-04-14 Thread Yehuda Sadeh
On Mon, Apr 14, 2014 at 10:12 PM, Shashank Puntamkar wrote: > I am setting up a ceph cluster with amazon s3 like capabilities. > I have configured Ceph Object Gateway, radowgw on Ubuntu12.04 as > described in ceph > documentation.(http://ceph.com/docs/master/radosgw/config/). > > While I am testi

Re: [ceph-users] Radosgw and s3cmd

2014-04-15 Thread Yehuda Sadeh
along with bucket name > (test.server-vm621) in /etc/hosts of server and I am running s3cmd > command on that server only. So far I tried in Local environment on > single node. > Hope you will give hint where the error is? You're probably missing the 'rgw dns name' configurati

Re: [ceph-users] Access denied error

2014-04-16 Thread Yehuda Sadeh
On Tue, Apr 15, 2014 at 11:33 PM, Punit Dambiwal wrote: > Hi, > > Still i am getting the same error,when i run the following :- > > -- > curl -i 'http://xxx.xlinux.com/admin/usage?format=json' -X GET -H > 'Authorization: AWS > YHFQ4D8BM835BCGERHTN:kXpM0XB9UjOadexDu2ZoP8s4nKjuoL

Re: [ceph-users] Access denied error

2014-04-21 Thread Yehuda Sadeh
execution getting an access denied error >> message. Is there any issues in the above header information ? or could you >> please explain how we can authenticate using access keys and access tokens >> ? or how we can pass the header information ? >> >> >> On Thu, A

Re: [ceph-users] RadosGW Quotas

2014-04-22 Thread Yehuda Sadeh
On Tue, Apr 22, 2014 at 7:51 AM, Derek Yarnell wrote: > Hi Yehuda, > > So i finally got to testing the quota stuff per the info from lines > 712-758 in rgw_rest_user.cc. I am using the query parameters version > and getting a 409 error out. Below in the debug log of the gateway > running 0.79. >

Re: [ceph-users] Operation canceled on first operation after RadosGW restart

2014-04-22 Thread Yehuda Sadeh
On Tue, Apr 22, 2014 at 9:18 AM, Evgeni Golov wrote: > Hi, > > we have a small (3x10GB) ceph cluster for testing, running Ceph 0.72.2 > with RadosGW inside an Apache2. All the three nodes are identical: each > has one osd, one mon, one radosgw. > > The application (self-written) uses Perl's Amazon

Re: [ceph-users] Access denied error

2014-04-24 Thread Yehuda Sadeh
b5700 10 failed to authorize request > 2014-04-22 09:36:45.526431 7ff106fb5700 2 req 16:0.000304::GET > /admin/usage:get_usage:http status=403 > 2014-04-22 09:36:45.526571 7ff106fb5700 1 == req done req=0x1ec2b70 > http_status=403 == > 2014-04-22 09:36:53.047233 7ff16effd700

Re: [ceph-users] [Bug]radosgw-agent can't sync files with Chinese filename

2014-04-24 Thread Yehuda Sadeh
Hi, sorry for the late response. I opened a ceph tracker issue for it (#8202). Thanks, Yehuda On Wed, Apr 16, 2014 at 1:00 AM, wsnote wrote: > OS: CentOS 6.5 > Version: Ceph 0.67.7 or 0.79 > > Hello, everyone! > I have configured federated gateways for several. > Now I can sync files from mas

Re: [ceph-users] [Bug]radosgw-agent can't sync files with Chinese filename

2014-04-25 Thread Yehuda Sadeh
to me whether you assume it's the sync agent issue or the gateway issue. My assumption is that the gateway fails to encode it correctly when sending the cross zones copy request. Yehuda > > At 2014-04-25 03:32:02,"Yehuda Sadeh" wrote: > >>Hi, >> >>

Re: [ceph-users] Access denied error

2014-04-25 Thread Yehuda Sadeh
On Fri, Apr 25, 2014 at 1:03 AM, Punit Dambiwal wrote: > Hi Yehuda, > > Thanks for your help...that missing date error gone but still i am getting > the access denied error :- > > - > 2014-04-25 15:52:56.988025 7f00d37c6700 1 == starting new request > req=0x237a090

Re: [ceph-users] Swift API and object versioning

2014-04-27 Thread Yehuda Sadeh
ual features, between S3 > and Swift, considering the stability and robustness, what it would be ? > I ask because I think you have a whole and precise vision about S3 and > Swift, which I havn't, "yet" ;-) > > Cheers ! > > > Le 27/04/2014 17:04, Yehuda Sadeh a

Re: [ceph-users] Swift API and object versioning

2014-04-27 Thread Yehuda Sadeh
We discussed it internally a few days ago, and even created some tickets for future work. The swift object versioning has some differences from the s3 one, and our plan at the moment is have the s3 working first and only then do swift. Yehuda On Sun, Apr 27, 2014 at 3:51 AM, Cedric Lemarchand wr

Re: [ceph-users] Creating a bucket failed

2014-04-28 Thread Yehuda Sadeh
This could happen if your client is uses the bucket through subdomain scheme, but the rgw is not resolving it correctly (either rgw_dns_name is misconfigured, or you were accessing it through different host name). Yehuda On Mon, Apr 28, 2014 at 2:02 PM, Seowon Jung wrote: > Hello, > > I've ins

Re: [ceph-users] List users not listing users

2014-05-08 Thread Yehuda Sadeh
You can use the metadata api to list the users, something along the lines of: GET /admin/metadata/user?format=json Yehuda On Thu, May 8, 2014 at 9:19 PM, Shanil S wrote: > Hi Punit, > > I hope someone in the community forum can solve this issue. I am waiting for > their responses.. > > > On T

Re: [ceph-users] List users not listing users

2014-05-08 Thread Yehuda Sadeh
-- > > I think it may be due to the permission issue, what do you think ? but i am > using the admin user to do this action and users permission as * > > > > On Fri, May 9, 2014 at 9:52 AM, Yehuda Sadeh wrote:

Re: [ceph-users] List users not listing users

2014-05-08 Thread Yehuda Sadeh
t": "", > "placement_tags": [], > "bucket_quota": { "enabled": false, > "max_size_kb": -1, > "max_objects": -1}} > > > > > On Fri, May 9, 2014 at 10:13 AM, Shanil S wrote: > >> Hi Ye

Re: [ceph-users] Lost access to radosgw after crash?

2014-05-13 Thread Yehuda Sadeh
On Tue, May 13, 2014 at 8:52 AM, Brian Rak wrote: > I hit a "bug" where radosgw crashed with > > -101> 2014-05-13 15:26:07.188494 7fde82886820 0 ERROR: FCGX_Accept_r > returned -24 too many files opened. You probably need to adjust your limits. > > 0> 2014-05-13 15:26:07.193772 7fde828868

Re: [ceph-users] Problem with radosgw and some file name characters

2014-05-15 Thread Yehuda Sadeh
Your rewrite rule might be off a bit. Can you provide log with 'debug rgw = 20'? Yehuda On Thu, May 15, 2014 at 8:02 AM, Andrei Mikhailovsky wrote: > Hello guys, > > > I am trying to figure out what is the problem here. > > > Currently running Ubuntu 12.04 with latest updates and radosgw version

Re: [ceph-users] Problem with radosgw and some file name characters

2014-05-16 Thread Yehuda Sadeh
3gw.fcgi?page=$1¶ms=$2&%{QUERY_STRING} > [E=HTTP_AUTHORIZATION:%{HTTP:Authorization},L] > > Or are you talking about something else? > > Cheers > > Andrei > > From: "Yehuda Sadeh" > To: "Andrei Mikhailovsky" >

Re: [ceph-users] How to point custom domains to a bucket and set default page and error page

2014-05-18 Thread Yehuda Sadeh
On Sat, May 17, 2014 at 3:32 AM, wsnote wrote: > Hi, everyone! > I want to use ceph to host static websites. > In Amazon S3, if I want to host test.domain.com, I create a bucket name > test.domain.com, and set a CNAME record: test.domain.com CNAME > test.domain.com.s3.amazonaws.com. > > In ceph, I

Re: [ceph-users] Various file lengths while uploading the same file

2014-05-19 Thread Yehuda Sadeh
On Mon, May 19, 2014 at 12:19 AM, Arthur Tumanyan wrote: > Hi, > I'm writing an fastcgi application, which just gets data from stdin and > appends it to ceph. > Please,have a look: > > while (0 != (written = fread(buffer, sizeof (char), sizeof > buffer, stdin))) { >

Re: [ceph-users] Problem with radosgw and some file name characters

2014-05-21 Thread Yehuda Sadeh
7;re using and the way it signs the requests. Can you try a different S3 client, see whether it works or not? Are you by any chance going through some kind of a load balancer that rewrites the urls? Yehuda > > From: "Andrei Mikhailovsky" > To:

Re: [ceph-users] RadosGW Admin API

2013-02-26 Thread Yehuda Sadeh
On Tue, Feb 26, 2013 at 2:12 PM, Gandalf Corvotempesta wrote: > Hi, > I would like to access to some usage stats (like bytes sent and > received) through api. > I'm following this guide: > http://ceph.com/docs/master/radosgw/admin/ > but there isn't any example. How can I set the admin endpoint? w

Re: [ceph-users] RadosGW Admin API

2013-02-27 Thread Yehuda Sadeh
On Wed, Feb 27, 2013 at 12:19 AM, Gandalf Corvotempesta wrote: > 2013/2/26 Yehuda Sadeh : >> The admin endpoint is 'admin' by default. You set it through the 'rgw >> admin entry' configurable. > > What do you mean with "endpoint"? Actually I&#

Re: [ceph-users] radosgw-admin subuser access level

2013-02-27 Thread Yehuda Sadeh
On Wed, Feb 27, 2013 at 3:44 AM, Mihály Árva-Tóth wrote: > Hello, > > When I create a subuser to access objects via swift, I have to specify > access level.I found 'full' in every documentations. What types can I add > else? read, write, readwrite, full > > radosgw-admin subuser create --uid=joh

Re: [ceph-users] radosgw-admin subuser access level

2013-02-28 Thread Yehuda Sadeh
On Thu, Feb 28, 2013 at 12:09 AM, Mihály Árva-Tóth wrote: > 2013/2/27 Yehuda Sadeh >> >> On Wed, Feb 27, 2013 at 3:44 AM, Mihály Árva-Tóth >> wrote: >> > Hello, >> > >> > When I create a subuser to access objects via swift, I have to s

Re: [ceph-users] Which web server for radosgw?

2013-02-28 Thread Yehuda Sadeh
On Thu, Feb 28, 2013 at 4:46 PM, Nick Bartos wrote: > What web server is recommended for the radosgw front-end? From what I > understand nginx buffers content before sending it to fastcgi, so that > may have issues on larger files or streams. It seems like apache and > lighttpd are reasonable al

Re: [ceph-users] Which web server for radosgw?

2013-03-04 Thread Yehuda Sadeh
tcgi >>> >>> >>> On Fri, Mar 1, 2013 at 9:15 AM, Nick Bartos wrote: >>>> Great, thanks. Where can I download the modified mod_fastcgi module? >>>> >>>> On Thu, Feb 28, 2013 at 11:24 PM, Yehuda Sadeh wrote: >>>>> On Thu, Feb

Re: [ceph-users] Radosgw only gives 400s

2013-03-04 Thread Yehuda Sadeh
On Mon, Mar 4, 2013 at 8:49 AM, Nick Bartos wrote: > I'm trying to figure out what's wrong with my radosgw setup. I am > getting HTTP 400 responses when running some basic stat/upload tests, > which work fine when using openstack (folsom) swift. Additionally, I > get HTTP 400 responses when tryi

Re: [ceph-users] Radosgw only gives 400s

2013-03-04 Thread Yehuda Sadeh
On Mon, Mar 4, 2013 at 9:18 AM, Nick Bartos wrote: > Yea my first thought was a rewrite rule issue. Unfortunately the one > you just gave me also ends up with the same result. Here is a log > snippet when trying to list containers and create a new container from > the openstack folsom dashboard:

Re: [ceph-users] Radosgw only gives 400s

2013-03-04 Thread Yehuda Sadeh
: > > Mar 4 17:38:16 node-172-17-0-13 apache2[31127]: [error] [client > 172.17.0.15] chunked Transfer-Encoding forbidden: /swift/v1/test/test > > Looks like this may be an apache issue. Any ideas? > > > On Mon, Mar 4, 2013 at 9:31 AM, Yehuda Sadeh wrote: >> On Mon, Mar

Re: [ceph-users] Radosgw only gives 400s

2013-03-04 Thread Yehuda Sadeh
t's mostly working (well at least I'm not >>> getting 400's). >>> >>> I can list and create containers, but I'm having an issue uploading a file: >>> >>> Mar 4 17:38:16 node-172-17-0-13 apache2[31127]: [error] [client >>> 172.17.0.15]

Re: [ceph-users] radosgw log insanely verbose

2013-03-04 Thread Yehuda Sadeh
On Mon, Mar 4, 2013 at 8:16 PM, Nick Bartos wrote: > I'm trying to figure out how to tone down the radosgw log messages, > preferably just to warnings and errors. Here's an excerpt from a > simple bucket list from the openstack dashboard: > > https://gist.github.com/anonymous/aabe9886a7c236cc2321

Re: [ceph-users] Bucket operations from radosgw

2013-03-05 Thread Yehuda Sadeh
On Tue, Mar 5, 2013 at 3:44 AM, Gandalf Corvotempesta wrote: > Hello, > is possible to get a list of operations made on a bucket (for exmaple, > put file, delete file, and so on) from a radosgw call? Not at the moment. If enabled, the ops log can be accessed through radosgw-admin. Yehuda ___

Re: [ceph-users] radosgw bucket/containers must be >= 3 chars long?

2013-03-05 Thread Yehuda Sadeh
On Mon, Mar 4, 2013 at 10:00 PM, Nick Bartos wrote: > When trying to create a swift container with a length of 1 or 2 > characters, I get a 400 error. When choosing a container name with at > least 3 characters, this does not happen. Is this an arbitrary limit > that can be easily changed? > > M

Re: [ceph-users] Bucket operations from radosgw

2013-03-05 Thread Yehuda Sadeh
On Tue, Mar 5, 2013 at 6:48 AM, Gandalf Corvotempesta wrote: > 2013/3/5 Yehuda Sadeh : >> Not at the moment. If enabled, the ops log can be accessed through >> radosgw-admin. > > Logs are enabled in my configuration but i'm unable to get them from > radosgw-admin:

Re: [ceph-users] Bucket operations from radosgw

2013-03-05 Thread Yehuda Sadeh
On Tue, Mar 5, 2013 at 7:10 AM, Gandalf Corvotempesta wrote: > 2013/3/5 Yehuda Sadeh : >> You're missing the bucket-id param. You can get it via: >> >> # radosgw-admin bucket stats --bucket=mybucket > > Thank you. > I think that you should change this message: &

Re: [ceph-users] bucket/user stats for billing

2013-03-06 Thread Yehuda Sadeh
On Wed, Mar 6, 2013 at 1:46 AM, Gandalf Corvotempesta wrote: > We would like to get buckets or users stats for billing purposes. > We need to calculate used disk space for each user. > > Actually, i'm doing this with radosgw-admin but getting a list of > buckets and then > > radosgw-admin bucket s

Re: [ceph-users] bucket/user stats for billing

2013-03-06 Thread Yehuda Sadeh
On Wed, Mar 6, 2013 at 8:43 AM, Gandalf Corvotempesta wrote: > 2013/3/6 Yehuda Sadeh : >> Not really. But try this: >> >> radosgw-admin --rgw-cache-enabled=false bucket stats --bucket=bucketname > > WOW > > # time radosgw-admin bucket stats --bucket=buc

Re: [ceph-users] bucket/user stats for billing

2013-03-06 Thread Yehuda Sadeh
On Wed, Mar 6, 2013 at 8:57 AM, Gandalf Corvotempesta wrote: > 2013/3/6 Yehuda Sadeh : >> It's not going through the cache initialization phase (registering all >> the watches, etc.). We have an opened issue about fixing that for >> radosgw-admin. > > Any workaro

Re: [ceph-users] bucket/user stats for billing

2013-03-06 Thread Yehuda Sadeh
On Wed, Mar 6, 2013 at 2:25 PM, Gandalf Corvotempesta wrote: > 2013/3/6 Gandalf Corvotempesta : >> Seems perfect. >> Thank you. > > But there is no way to get users list... You can do: rados -p .users.uid ls | grep -v .buckets Not very clean, and not too efficient. I'm working on some api that'

Re: [ceph-users] Wildcard for "rgw keystone accepted roles"

2013-03-06 Thread Yehuda Sadeh
On Wed, Mar 6, 2013 at 4:39 PM, Nick Bartos wrote: > Is there a way to allow all roles (aside from listing them all) access > to radosgw when using keystone? If new roles get added, I want them > to automatically have access. There's no such way currently. Created issue #4635. Yehuda __

Re: [ceph-users] Number of buckets limit per account

2013-03-12 Thread Yehuda Sadeh
On Tue, Mar 12, 2013 at 6:16 AM, Ashish Kumar wrote: > > Hi Guys, > > > > I am configuring CEPH and planning to use just one account for our app. > One schemer will map to one bucket, but before I do this, I need the answer > of following > > questions: > > > > 1) can the number of buckets/account

Re: [ceph-users] [radosgw]create a bucket failed

2013-03-13 Thread Yehuda Sadeh
On Wed, Mar 13, 2013 at 2:46 AM, Jun Jun8 Liu wrote: > > Hi all > > I want to create a bucket using radosgw-admin,but I got a error > -95. my environment is Ubuntu 12, ceph 0.56.3. > > > > root@:~# radosgw-admin bucket link --bucket=testtt > --uid=liuj > >

Re: [ceph-users] Rados Gateway Pools

2013-03-15 Thread Yehuda Sadeh
On Fri, Mar 15, 2013 at 5:06 PM, Mandell Degerness wrote: > How are the pools used by rgw defined? > > Specifically, if I want to ensure that all of the data stored by rgw > uses pools which are replicated 3 times and have a pgnum and a pgpnum > greater than 8, what do I need to set? There are a

Re: [ceph-users] Rados Gateway Pools

2013-03-15 Thread Yehuda Sadeh
On Fri, Mar 15, 2013 at 8:40 PM, Mandell Degerness wrote: > Thanks. One further question: I notice from the docs that there is a > setting for rgw_data which defaults to > "/var/lib/ceph/radosgw/$cluster-$id". What is stored here? How big > should it be? Is it even necessary? The directory d

Re: [ceph-users] Planning for many small files

2013-03-17 Thread Yehuda Sadeh
Not at the moment. We had some discussions about "blind" buckets, it's definitely on our mind, but we're not there yet. Yehuda On Sun, Mar 17, 2013 at 3:09 PM, Rustam Aliyev wrote: > Thanks for detailed explanation. > > Is there any way to disable bucket indexes? We already store index in our >

Re: [ceph-users] using Ceph FS as OpenStack Glance's backend

2013-03-21 Thread Yehuda Sadeh
On Thu, Mar 21, 2013 at 2:12 AM, Sebastien Han wrote: > > Hi, > > Storing the image as an object with RADOS or RGW will result as a single big > object stored somewhere in Ceph. However with RBD the image is spread across > thousands of objects across the entire cluster. At the end, you get way

<    1   2   3   4   5   6   >