i meet same problem with u , but i still can not after i create
.rgw.buckets
.rgw.buckets.index
.log
.intent-log
.usage
still stuck here.
2014-03-13 7:38 GMT+07:00 Greg Poirier <greg.poir...@opower.com>:
> And, I figured out the issue.
>
> The utility I was using to create pools, zones, and regions automatically
> failed to do two things:
>
> - create rgw.buckets and rgw.buckets.index for each zone
> - setup placement pools for each zone
>
> I did both of those, and now everything is working.
>
> Thanks, me, for the commitment to figuring this poo out.
>
>
> On Wed, Mar 12, 2014 at 8:31 PM, Greg Poirier <greg.poir...@opower.com>
> wrote:
>
>> Increasing the logging further, and I notice the following:
>>
>> 2014-03-13 00:27:28.617100 7f6036ffd700 20 rgw_create_bucket returned
>> ret=-1 bucket=test(@.rgw.buckets[us-west-1.15849318.1])
>>
>> But hope that .rgw.buckets doesn't have to exist... and that
>> @.rgw.buckets is perhaps telling of something?
>>
>> I did notice that .us-west-1.rgw.buckets and .us-west-1.rgw.buckets.index
>> weren't created. I created those, restarted radosgw, and still 403 errors.
>>
>>
>> On Wed, Mar 12, 2014 at 8:00 PM, Greg Poirier <greg.poir...@opower.com>
>> wrote:
>>
>>> And the debug log because that last log was obviously not helpful...
>>>
>>> 2014-03-12 23:57:49.497780 7ff97e7dd700 1 ====== starting new request
>>> req=0x23bc650 =====
>>> 2014-03-12 23:57:49.498198 7ff97e7dd700 2 req 1:0.000419::PUT
>>> /test::initializing
>>> 2014-03-12 23:57:49.498233 7ff97e7dd700 10 host=s3.amazonaws.com
>>> rgw_dns_name=us-west-1.domain
>>> 2014-03-12 23:57:49.498366 7ff97e7dd700 10 s->object=<NULL>
>>> s->bucket=test
>>> 2014-03-12 23:57:49.498437 7ff97e7dd700 2 req 1:0.000659:s3:PUT
>>> /test::getting op
>>> 2014-03-12 23:57:49.498448 7ff97e7dd700 2 req 1:0.000670:s3:PUT
>>> /test:create_bucket:authorizing
>>> 2014-03-12 23:57:49.498508 7ff97e7dd700 10 cache get:
>>> name=.us-west-1.users+BLAHBLAHBLAH : miss
>>> 2014-03-12 23:57:49.500852 7ff97e7dd700 10 cache put:
>>> name=.us-west-1.users+BLAHBLAHBLAH
>>> 2014-03-12 23:57:49.500865 7ff97e7dd700 10 adding
>>> .us-west-1.users+BLAHBLAHBLAH to cache LRU end
>>> 2014-03-12 23:57:49.500886 7ff97e7dd700 10 moving
>>> .us-west-1.users+BLAHBLAHBLAH to cache LRU end
>>> 2014-03-12 23:57:49.500889 7ff97e7dd700 10 cache get:
>>> name=.us-west-1.users+BLAHBLAHBLAH : type miss (requested=1, cached=6)
>>> 2014-03-12 23:57:49.500907 7ff97e7dd700 10 moving
>>> .us-west-1.users+BLAHBLAHBLAH to cache LRU end
>>> 2014-03-12 23:57:49.500910 7ff97e7dd700 10 cache get:
>>> name=.us-west-1.users+BLAHBLAHBLAH : hit
>>> 2014-03-12 23:57:49.502663 7ff97e7dd700 10 cache put:
>>> name=.us-west-1.users+BLAHBLAHBLAH
>>> 2014-03-12 23:57:49.502667 7ff97e7dd700 10 moving
>>> .us-west-1.users+BLAHBLAHBLAH to cache LRU end
>>> 2014-03-12 23:57:49.502700 7ff97e7dd700 10 cache get:
>>> name=.us-west-1.users.uid+test : miss
>>> 2014-03-12 23:57:49.505128 7ff97e7dd700 10 cache put:
>>> name=.us-west-1.users.uid+test
>>> 2014-03-12 23:57:49.505138 7ff97e7dd700 10 adding
>>> .us-west-1.users.uid+test to cache LRU end
>>> 2014-03-12 23:57:49.505157 7ff97e7dd700 10 moving
>>> .us-west-1.users.uid+test to cache LRU end
>>> 2014-03-12 23:57:49.505160 7ff97e7dd700 10 cache get:
>>> name=.us-west-1.users.uid+test : type miss (requested=1, cached=6)
>>> 2014-03-12 23:57:49.505176 7ff97e7dd700 10 moving
>>> .us-west-1.users.uid+test to cache LRU end
>>> 2014-03-12 23:57:49.505178 7ff97e7dd700 10 cache get:
>>> name=.us-west-1.users.uid+test : hit
>>> 2014-03-12 23:57:49.507401 7ff97e7dd700 10 cache put:
>>> name=.us-west-1.users.uid+test
>>> 2014-03-12 23:57:49.507406 7ff97e7dd700 10 moving
>>> .us-west-1.users.uid+test to cache LRU end
>>> 2014-03-12 23:57:49.507521 7ff97e7dd700 10 get_canon_resource():
>>> dest=/test
>>> 2014-03-12 23:57:49.507529 7ff97e7dd700 10 auth_hdr:
>>> PUT
>>>
>>> binary/octet-stream
>>> Wed, 12 Mar 2014 23:57:51 GMT
>>> /test
>>> 2014-03-12 23:57:49.507674 7ff97e7dd700 2 req 1:0.009895:s3:PUT
>>> /test:create_bucket:reading permissions
>>> 2014-03-12 23:57:49.507682 7ff97e7dd700 2 req 1:0.009904:s3:PUT
>>> /test:create_bucket:verifying op mask
>>> 2014-03-12 23:57:49.507695 7ff97e7dd700 2 req 1:0.009917:s3:PUT
>>> /test:create_bucket:verifying op permissions
>>> 2014-03-12 23:57:49.509604 7ff97e7dd700 2 req 1:0.011826:s3:PUT
>>> /test:create_bucket:verifying op params
>>> 2014-03-12 23:57:49.509615 7ff97e7dd700 2 req 1:0.011836:s3:PUT
>>> /test:create_bucket:executing
>>> 2014-03-12 23:57:49.509694 7ff97e7dd700 10 cache get:
>>> name=.us-west-1.domain.rgw+test : miss
>>> 2014-03-12 23:57:49.512229 7ff97e7dd700 10 cache put:
>>> name=.us-west-1.domain.rgw+test
>>> 2014-03-12 23:57:49.512259 7ff97e7dd700 10 adding
>>> .us-west-1.domain.rgw+test to cache LRU end
>>> 2014-03-12 23:57:49.512333 7ff97e7dd700 10 cache get:
>>> name=.us-west-1.domain.rgw+.pools.avail : miss
>>> 2014-03-12 23:57:49.518216 7ff97e7dd700 10 cache put:
>>> name=.us-west-1.domain.rgw+.pools.avail
>>> 2014-03-12 23:57:49.518228 7ff97e7dd700 10 adding
>>> .us-west-1.domain.rgw+.pools.avail to cache LRU end
>>> 2014-03-12 23:57:49.518248 7ff97e7dd700 10 moving
>>> .us-west-1.domain.rgw+.pools.avail to cache LRU end
>>> 2014-03-12 23:57:49.518251 7ff97e7dd700 10 cache get:
>>> name=.us-west-1.domain.rgw+.pools.avail : type miss (requested=1, cached=6)
>>> 2014-03-12 23:57:49.518270 7ff97e7dd700 10 moving
>>> .us-west-1.domain.rgw+.pools.avail to cache LRU end
>>> 2014-03-12 23:57:49.518272 7ff97e7dd700 10 cache get:
>>> name=.us-west-1.domain.rgw+.pools.avail : hit
>>> 2014-03-12 23:57:49.520295 7ff97e7dd700 10 cache put:
>>> name=.us-west-1.domain.rgw+.pools.avail
>>> 2014-03-12 23:57:49.520348 7ff97e7dd700 10 moving
>>> .us-west-1.domain.rgw+.pools.avail to cache LRU end
>>> 2014-03-12 23:57:49.522672 7ff97e7dd700 2 req 1:0.024893:s3:PUT
>>> /test:create_bucket:http status=403
>>> 2014-03-12 23:57:49.523204 7ff97e7dd700 1 ====== req done req=0x23bc650
>>> http_status=403 ======
>>>
>>>
>>> On Wed, Mar 12, 2014 at 7:36 PM, Greg Poirier <greg.poir...@opower.com>
>>> wrote:
>>>
>>>> The saga continues...
>>>>
>>>> So, after fiddling with haproxy a bit, I managed to make sure that my
>>>> requests were hitting the RADOS Gateway.
>>>>
>>>> NOW, I get a 403 from my ruby script:
>>>>
>>>> 2014-03-12 23:34:08.289670 7fda9bfbf700 1 ====== starting new request
>>>> req=0x215a780 =====
>>>> 2014-03-12 23:34:08.305105 7fda9bfbf700 1 ====== req done
>>>> req=0x215a780 http_status=403 ======
>>>>
>>>> The aws-s3 gem forces the Host header to be set to s3.amazonaws.com --
>>>> and I am wondering if this could potentially cause a problem. Is that the
>>>> case? Or is radosgw able to just know where it is supposed to look for
>>>> objects based on its configuration? I assume the latter, otherwise we would
>>>> have multi-tenant radosgw instead of needing to have an instance per zone.
>>>>
>>>> So does it ignore the Host header?
>>>>
>>>> Is this 403 related to my problems with the user error I found earlier
>>>> where I'm unable to view the user with the radosgw-admin tool?
>>>>
>>>>
>>>>
>>>> On Wed, Mar 12, 2014 at 1:54 PM, Greg Poirier <greg.poir...@opower.com>
>>>> wrote:
>>>>
>>>>> Also... what are linger_ops?
>>>>>
>>>>> ceph --admin-daemon /var/run/ceph/ceph-client.radosgw.<hostname>.asok
>>>>> objecter_requests
>>>>> { "ops": [],
>>>>> "linger_ops": [
>>>>> { "linger_id": 1,
>>>>> "pg": "7.4322fa9f",
>>>>> "osd": 25,
>>>>> "object_id": "notify.0",
>>>>> "object_locator": "@7",
>>>>> "snapid": "head",
>>>>> "registering": "head",
>>>>> "registered": "head"},
>>>>> { "linger_id": 2,
>>>>> "pg": "7.16dafda0",
>>>>> "osd": 132,
>>>>> "object_id": "notify.1",
>>>>> "object_locator": "@7",
>>>>> "snapid": "head",
>>>>> "registering": "head",
>>>>> "registered": "head"},
>>>>> { "linger_id": 3,
>>>>> "pg": "7.88aa5c95",
>>>>> "osd": 32,
>>>>> "object_id": "notify.2",
>>>>> "object_locator": "@7",
>>>>> "snapid": "head",
>>>>> "registering": "head",
>>>>> "registered": "head"},
>>>>> { "linger_id": 4,
>>>>> "pg": "7.f8c99aee",
>>>>> "osd": 62,
>>>>> "object_id": "notify.3",
>>>>> "object_locator": "@7",
>>>>> "snapid": "head",
>>>>> "registering": "head",
>>>>> "registered": "head"},
>>>>> { "linger_id": 5,
>>>>> "pg": "7.a204812d",
>>>>> "osd": 129,
>>>>> "object_id": "notify.4",
>>>>> "object_locator": "@7",
>>>>> "snapid": "head",
>>>>> "registering": "head",
>>>>> "registered": "head"},
>>>>> { "linger_id": 6,
>>>>> "pg": "7.31099063",
>>>>> "osd": 28,
>>>>> "object_id": "notify.5",
>>>>> "object_locator": "@7",
>>>>> "snapid": "head",
>>>>> "registering": "head",
>>>>> "registered": "head"},
>>>>> { "linger_id": 7,
>>>>> "pg": "7.97c520d4",
>>>>> "osd": 135,
>>>>> "object_id": "notify.6",
>>>>> "object_locator": "@7",
>>>>> "snapid": "head",
>>>>> "registering": "head",
>>>>> "registered": "head"},
>>>>> { "linger_id": 8,
>>>>> "pg": "7.84ada7c9",
>>>>> "osd": 94,
>>>>> "object_id": "notify.7",
>>>>> "object_locator": "@7",
>>>>> "snapid": "head",
>>>>> "registering": "head",
>>>>> "registered": "head"}],
>>>>> "pool_ops": [],
>>>>> "pool_stat_ops": [],
>>>>> "statfs_ops": [],
>>>>> "command_ops": []}
>>>>>
>>>>>
>>>>> On Wed, Mar 12, 2014 at 10:45 AM, Greg Poirier <
>>>>> greg.poir...@opower.com> wrote:
>>>>>
>>>>>> Rados GW and Ceph versions installed:
>>>>>> Version: 0.67.7-1precise
>>>>>>
>>>>>> I create a user:
>>>>>> radosgw-admin --name client.radosgw.<hostname> user create --uid test
>>>>>> --display-name "Test User"
>>>>>>
>>>>>> It outputs some JSON that looks convincing:
>>>>>> { "user_id": "test",
>>>>>> "display_name": "test user",
>>>>>> "email": "",
>>>>>> "suspended": 0,
>>>>>> "max_buckets": 1000,
>>>>>> "auid": 0,
>>>>>> "subusers": [],
>>>>>> "keys": [
>>>>>> { "user": "test",
>>>>>> "access_key": "<snip>",
>>>>>> "secret_key": "<snip>"},
>>>>>> { "user": "test",
>>>>>> "access_key": "<snip>",
>>>>>> "secret_key": "<snip>"}],
>>>>>> "swift_keys": [],
>>>>>> "caps": [],
>>>>>> "op_mask": "read, write, delete",
>>>>>> "default_placement": "",
>>>>>> "placement_tags": []}
>>>>>>
>>>>>> There are two keys because I have tried this twice.
>>>>>>
>>>>>> I can see it in metadata list:
>>>>>> radosgw-admin --name client.radosgw.<hostname> metadata list user
>>>>>> [
>>>>>> "test",
>>>>>> "us-east-2",
>>>>>> "us-west-1"]
>>>>>>
>>>>>> I then try to get user info:
>>>>>>
>>>>>> radosgw-admin --name client.radosgw.<hostname> user info test
>>>>>> could not fetch user info: no user info saved
>>>>>>
>>>>>> I try to create a bucket with the user using Ruby's aws/s3 API:
>>>>>>
>>>>>> equire 'aws/s3'
>>>>>>
>>>>>> AWS::S3::Base.establish_connection!(
>>>>>> access_key_id: '<snip>',
>>>>>> secret_access_key: '<snip>',
>>>>>> use_ssl: true,
>>>>>> server: '<snip>',
>>>>>> persistent: true
>>>>>> )
>>>>>>
>>>>>> AWS::S3::Bucket.create('test')
>>>>>>
>>>>>> file = 'sloth.txt'
>>>>>>
>>>>>> AWS::S3::S3Object.store(file, open(file), 'test')
>>>>>>
>>>>>> bucket = AWS::S3::Bucket.find('test')
>>>>>>
>>>>>> puts bucket
>>>>>>
>>>>>> bucket.each do |object|
>>>>>> puts
>>>>>> "#{object.key}\t#{object.about['content-length']}\t#{object.about['last-modified']}"
>>>>>> end
>>>>>>
>>>>>> And I get the following:
>>>>>> #<AWS::S3::Bucket:0x007f9852854e58>
>>>>>> /Users/greg.poirier/.rvm/gems/ruby-1.9.3-p429/gems/aws-s3-0.6.3/lib/aws/s3/base.rb:235:in
>>>>>> `method_missing': undefined local variable or method `name' for
>>>>>> #<AWS::S3::Bucket:0x007f9852854e58> (NameError)
>>>>>> from
>>>>>> /Users/greg.poirier/.rvm/gems/ruby-1.9.3-p429/gems/aws-s3-0.6.3/lib/aws/s3/bucket.rb:313:in
>>>>>> `reload!'
>>>>>> from
>>>>>> /Users/greg.poirier/.rvm/gems/ruby-1.9.3-p429/gems/aws-s3-0.6.3/lib/aws/s3/bucket.rb:242:in
>>>>>> `objects'
>>>>>> from
>>>>>> /Users/greg.poirier/.rvm/gems/ruby-1.9.3-p429/gems/aws-s3-0.6.3/lib/aws/s3/bucket.rb:253:in
>>>>>> `each'
>>>>>> from test.rb:21:in `<main>'
>>>>>>
>>>>>> The bucket fails to be created:
>>>>>>
>>>>>> radosgw-admin --name client.radosgw.<hostname> bucket list
>>>>>> []
>>>>>>
>>>>>> And also this:
>>>>>>
>>>>>> radosgw-admin --name client.radosgw.<hostname> metadata list bucket
>>>>>> []2014-03-12 17:42:42.221112 7f426b779780 -1 failed to list objects
>>>>>> pool_iterate returned r=-2
>>>>>>
>>>>>>
>>>>>> So clearly there is something going on here. My questions:
>>>>>>
>>>>>> Is this failure to create a bucket related to do the "no user info
>>>>>> saved" error?
>>>>>>
>>>>>> What would cause the "no user info saved" error?
>>>>>>
>>>>>> What may be causing the bucket to not be created?
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
> _______________________________________________
> ceph-users mailing list
> ceph-users@lists.ceph.com
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>
>
_______________________________________________
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
