We have a recent cluster upgraded from Jewel to Luminous. Today we had
a segmentation fault that led to file system degraded. Systemd then
decided to restart the daemon over and over with a different stack trace
(can be seen after the 10k events in the log file[0]).
After trying to fail over to
On 3/17/16 1:41 PM, Marius Vaitiekunas wrote:
> It's possible that somebody changed the owner of some bucket. But all
> objects in that bucket still belongs to this user. That way you can get
> quota exceeded. We had the same situation.
Well the user says he didn't write to any other buckets than
For clusters that were created pre-hammer and want to use ceph-deploy to
create additional rgw instances is there a way to create the
bootstrap-rgw keyring?
http://docs.ceph.com/ceph-deploy/docs/rgw.html
--
Derek T. Yarnell
University of Maryland
Institute for Advanced Computer Studies
_
Hi,
We have a user with a 50GB quota and has now a single bucket with 20GB
of files. They had previous buckets created and removed but the quota
has not decreased. I understand that we do garbage collection but it
has been significantly longer than the defaults that we have not
overridden. They
-deploy releases are following a new release
>> process so I will
>> have to investigate where is the disconnect.
>>
>> Thanks for letting us know.
>>
>> On Tue, Jan 5, 2016 at 10:31 AM, Derek Yarnell wrote:
>>> It looks like the ceph-deploy > 1.5.28 pack
It looks like the ceph-deploy > 1.5.28 packages in the
http://download.ceph.com/rpm-hammer/el6 and
http://download.ceph.com/rpm-hammer/el7 repositories are not being PGP
signed. What happened? This is causing our yum updates to fail but may
be a sign of something much more nefarious?
# rpm -qp -
Hi,
We want to have users that can authenticate to our RGW but have no quota
and can not create buckets. The former is working by setting the
user_quota max_size_kb to 0. The later does not seem to be working by
setting max_buckets to 0 means no quota on the amount of buckets that
can be created
On 2/25/15 2:31 PM, Sage Weil wrote:
> Hey,
>
> We are considering switching to civetweb (the embedded/standalone rgw web
> server) as the primary supported RGW frontend instead of the current
> apache + mod-fastcgi or mod-proxy-fcgi approach. "Supported" here means
> both the primary platform
enabled"', 'true,"max_size_kb":12345,"max_objects":-1}Content-type:
application/json'), ('content-length', '0'), ('server', 'Apache/2.4.6
(Red Hat Enterprise Linux) OpenSSL/1.0.1e-fips mod_wsgi/3.4 Python/2.7.5')]
On 10/28
I have had this issue before, and I don't think I have resolved it. I
have been using the RGW admin api to set quota based on the docs[0].
But I can't seem to be able to get it to cough up and show me the quota
now. Any ideas I get a 200 back but no body, I have tested this on a
Firefly (0.80.5-9
On 11/28/13, 4:18 AM, Dan Van Der Ster wrote:
> Dear users/experts,
> Does anyone know how to use radosgw-admin log show? It seems to not properly
> read the --bucket parameter.
>
> # radosgw-admin log show --bucket=asdf --date=2013-11-28-09
> --bucket-id=default.7750582.1
> error reading log 20
Hi,
I have Ceph cluster and a OpenStack install running Havana (using RDO).
So when I create a volume from a image and boot the instance from it in
the Image Name field when listing the instances in OpenStack is empty.
Is this normal?
Thanks,
derek
--
Derek T. Yarnell
University of Maryland
In
Hi,
Switching to a POST gives a 200 with the user JSON returned though this
does not include any quota information back. I am guessing this is a
bug and will start a ticket for it, unless you think I am doing
something wrong here.
{u'display_name': u' Mr. Test', u'keys': [{u'access_key': u'REDAC
Hi Yehuda,
So i finally got to testing the quota stuff per the info from lines
712-758 in rgw_rest_user.cc. I am using the query parameters version
and getting a 409 error out. Below in the debug log of the gateway
running 0.79.
2014-04-22 10:46:56.852360 7fa46bfff700 20 enqueued request
req=0
Hi,
Most likely Yehuda can speak to this. So there is some documentation on
the way to set quotas for a user in master (src/rgw/rgw_rest_user.cc
line 712) but these are not in the docs it seems yet. I have started to
incorporate them but I don't see anything as an example for how to
document the
> * rgw: user quotas (Yehuda Sadeh)
Hi,
Now that there will be both bucket and user level quotas (which thank
you yehuda this is critical to what we want to do with ceph in our
organization) can you elaborate on the behavior in some scenarios?
If a user is at their user quota but is trying to w
On 1/30/14, 12:16 PM, alistair.whit...@barclays.com wrote:
> radosgw: must specify 'rgw socket path' or 'rgw port' to run as a daemon
>
This is defined in your /etc/ceph/ceph.conf. For example if the host
you are running the radosgw is named 'myhostname' then your ceph.conf
should have a secti
Hi,
So I am trying to remove OSDs from one of our 6 ceph OSDs, this is a
brand new cluster and no data is yet on it. I was following the manual
procedure[1] with the following script. I removed OSDs 0-3 but I am
seeing ceph not fully recovering.
#!/bin/bash
ceph osd out ${1}
/etc/init.d/ceph st
Hi Mikhail,
Sorry did you find an issue with the library I wrote? Let me test the
bucket linking as I didn't have a good unit test for it yet.
Thanks,
derek
On 1/30/14, 12:45 PM, myk...@gmail.com wrote:
> fix: it was not python boto, it was https://github.com/dyarnell/rgwadmin
> but it doesn`t
>> Our best guess so far is that this line is not matching the underlying
>> disk that is getting hotplugged (95-ceph-osd.rules). Is
>> ID_PART_ENTRY_TYPE just the partition UUID or are we not understanding
>> identifier correctly?
>>
>> ENV{ID_PART_ENTRY_TYPE}=="4fbd7e29-9d25-41b8-afd0-062c0ceff
> don't start after a reboot? Is there an error in
> /var/log/ceph/ceph-mon.`hostname`.log?
>
> sage
>
> On Mon, 27 Jan 2014, Derek Yarnell wrote:
>
>> Hi,
>>
>> Would I take this to understand that this may be a known issue with udev
>> on RHEL
to fstab allows the OSDs to mount during boot, and they
> start automatically when they're already mounted during boot.
>
> Hope this helps until a permanent fix is available.
>
> Michael J. Kidd
> Sr. Storage Consultant
> Inktank Professional Services
>
>
So we have a test cluster, and two production clusters all running on
RHEL6.5. Two are running Emperor and one of them running Dumpling. On
all of them our OSDs do not start at boot it seems via the udev rules.
The OSDs were created with ceph-deploy and are all GPT. The OSDs are
visable with `ce
On 1/15/14, 1:35 PM, Dimitri Maziuk wrote:
> On 01/15/2014 10:53 AM, Alexandre DERUMIER wrote:
From what I understand the "flexbay" are inside the box,
typically usefull for OS (SSD) drives, then it lets you use
all the front hotlug slot with larger platter drives.
>>
>> Yes, it's
On 1/15/14, 9:20 AM, Mark Nelson wrote:
> I guess I'd probably look at the R520 in an 8 bay configuration with an
> E5-2407 and 4 1TB data disks per chassis (along with whatever OS disk
> setup you want). That gives you 4 PCIE slots for the extra network
> cards, the option for a hardware raid con
On 12/19/13, 7:51 PM, Sage Weil wrote:
>> If it takes 15 minutes for one of my servers to reboot is there a risk
>> that some sort of needless automatic processing will begin?
>
> By default, we start rebalancing data after 5 minutes. You can adjust
> this (to, say, 15 minutes) with
>
> mon os
On 11/28/13, 4:44 AM, Micha Krause wrote:
> Hi,
>
>> Note this breaks AWS S3 compatibility and is why it is a configurable.
>
> So how does AWS S3 handle Public access to objects?
You have to explicitly set public ACL on each object.
--
Derek T. Yarnell
University of Maryland
Institute for A
On 11/26/13, 3:31 PM, Shain Miley wrote:
> Micha,
>
> Did you ever figure out a work around for this issue?
>
> I also had plans of using s3cmd to put, and recursively set acl's on a
> nightly basis...however we are getting the 403 errors as well during our
> testing.
>
> I was just wonderin
On 11/26/13, 4:04 AM, Mihály Árva-Tóth wrote:
> Hello,
>
> Is there any idea? I don't know this is s3api limitation or missing feature?
>
> Thank you,
> Mihaly
Hi Mihaly,
If all you are looking for is the current size of the bucket this can be
found from the adminops api[1] or when you get do t
On 11/1/13, 2:07 PM, Gruher, Joseph R wrote:
> Adding some debug arguments has generated output which I believe indicates
> the problem is my keyring is missing, but the keyring seems to be here. Why
> would this complain about the keyring and fail to start?
Hi,
Are you sure the user you are s
On 10/30/13, 4:53 AM, Valery Tschopp wrote:
> radosgw-admin pools list
Hi Valery,
That command lists the data placement pool(s) within radosgw (different
than what I was having problems with). My problem was that I didn't
have the correct underlying rados pool(s) created. They are listed here:
On 10/30/13, 5:48 AM, Matt Thompson wrote:
> Hi Derek,
>
> I'm wondering if your radosgw client does not have read-write (rw) caps
> on mon? See the section "Monitor Key CAPS"
> on http://ceph.com/docs/next/radosgw/config/ for more info.
>
Hi Matt,
You are right. I had forgotten that I didn't
On 10/28/13, 11:20 AM, Abhijeet Nakhwa wrote:
> I have installed apache + fastcgi as per the documentation (note: not
> the ceph customized versions). I have created a user using radosgw-admin
> named radosgwadmin (please see attached file radogwadmin-user.txt)
>
> Now using S3 API, I am able to m
Hi Shain,
Yes we have tested and have working S3 Multipart support for files >5GB
(RHEL64/0.67.4).
However, crossftp unless you have pro it would seem does not support
multipart. Dragondisk gives the error that I have seen when using a PUT
and not multipart, EntityTooLarge. My guess is that it
Hi,
So the problem was that '.usage' pool was not created. I haven't
traversed the code well enough yet to know where this pool is supposed
to get created but it wasn't even though the option was on. As soon as
I hand created the pool the radosgw started logging usage.
Thanks,
derek
--
---
De
> Try moving the above configurables to the global section, if it's
> working then you're probably using the wrong section.
Moving sections doesn't seem to change the behavior. My two other test
gateways seems to be working fine with similar configs all running
0.67.4 (slightly patched for ACLs[1
So I have tried to enable usage logging on a new production Ceph RadosGW
cluster but nothing seems to show up.
I have added to the [client.radosgw.] section the following
rgw enable usage log = true
rgw usage log tick interval = 30
rgw usage log flush threshold = 1024
rgw usage max shards = 32
rg
On 10/16/13 4:26 AM, Valery Tschopp wrote:
> Hi Derek,
>
> Thanks for your example.
>
> I've added caps='metadata=*', but I still have an error and get:
>
> send: 'GET /admin/metadata/user?format=json HTTP/1.1\r\nHost:
> objects.bcc.switch.ch\r\nAccept-Encoding: identity\r\nDate: Wed, 16 Oct
> 2
On 10/16/13 5:15 AM, Fabio - NS3 srl wrote:
> Hello,
> when i set a read permission for all users to the bucket i read only the
> content of the bucket but i received "access denied" for all directory
> and sub-directory inside this bucket.
>
> Where i wrong???
Hi Fabio,
This is the default S3 b
On 10/15/13 12:00 PM, Valery Tschopp wrote:
> Hi Derek,
>
> Thanks. Is the 'radosgw-admin metadata list user' also available as REST
> API?
Hi Valery,
Yes, it is. Not well documented I believe right now. I have this
tested and working in my python bindings[1] for the radosgw (get_users).
You
> root@ineri:~# radosgw-admin user info
> could not fetch user info: no user info saved
Hi Valery,
You need to use
radosgw-admin metadata list user
Thanks,
derek
--
---
Derek T. Yarnell
University of Maryland
Institute for Advanced Computer Studies
__
On 10/10/13 2:36 PM, Matt Thompson wrote:
> Accidentally replied to Alexis directly, but this works for me:
>
> url =
> 'http://%s/admin/user?uid=someuser&display-name=somename&format=json' %
> server
> r = requests.put(url, auth=S3Auth(aws_key, secret, server))
>
> Note that "display-name" is a
Yes, there is something afoot, I guess i hadn't gotten far enough to try
creating users just yet. I have these caps,
"caps": [
{ "type": "buckets",
"perm": "*"},
{ "type": "metadata",
"perm": "*"},
{ "type": "usage",
"perm": "*"},
{ "t
> Via cli, i do radosgw user info --uid=xxx
> but with the REST API ?
Hi Alexis,
Here is a simple python example on how to use the admin api. You will
need to get a few packages from the cheese shop (virtualenv + pip makes
this easy).
pip install requests-aws
You will also need to set the ap
On 10/1/13 11:37 AM, Fuchs, Andreas (SwissTXT) wrote:
> What we need to do is to have a key/secret with read write permission and one
> with read only permission to a certain bucket, is this possible? How?
Hi Andi,
Yes this is possible. You can either create accounts for the radosgw
through the
Hi,
So say a usera has full control (and is the owner) of a bucket in s3 and
gives userb 'FULL_CONTROL' on the bucket. Userb writes a file and it
seems that by default the ACL for that key is going to be 'FULL_CONTROL'
for userb. When usera iterates the key even just over the ACL for the
key, th
> You're not specifying which user you're trying to get info for. The
> request needs extra 'uid' param that specifies the target user. Other
> than that you're ok.
>
> Yehuda
>
Hi Yehuda,
But the docs say I should get back a list of users if no uid is specified.
http://ceph.com/docs/master
-uid=derek
{ "user_id": "derek",
"display_name": "Derek Yarnell",
"email": "",
"suspended": 0,
"max_buckets": 1000,
"auid": 0,
"subusers": [],
"keys": [
{ &q
>> It will never log anything to /var/log/ceph/radosgw.log. I am looking
>> for the debug output which I have seen people post, does anyone have a
>> pointer to what could be going on?
>
> You don't need the 'rgw_enable_ops_log' to have debug logs. The
> log_file param should be enough. Do you ha
I have been running a development radosgw instance for awhile and now
looking to start a larger production one. I am trying to track down
some issues with PUT on larger files I am having from certain S3
clients. I am running on the 0.61.4 RPMs on RHEL6 and have the
following in my ceph.conf on th
Hi,
So when bootstrapping radosgw you are not given the option to create the
pools (and therefor set a specific pg_num). There are a lot of pools
created .rgw, .rgw.gc, .rgw.control, .users.uid, .users.email, .users.
I know I can set osd_pool_default_pg_num but that will apply to all
those unifor
On 6/19/13 12:35 AM, Sage Weil wrote:
>> Yes so the init script does create the directory. Even if i manually
>> create the directory before running the initial 'ceph-deploy mon create'
>> I am still seeing the exception in the mon_create function that I
>> originally posted about. Still trying t
On 6/18/13 5:31 PM, Sage Weil wrote:
>> 1) Remove the %ghost directive and allow RPM to install the directory.
>> Potentially leaving orphaned pid/state files after the package is removed.
>>
>> 2) Or the directory needs to be created in the %post section. If it is
>> created in the %post section
Hi,
So the first error below is that /var/run/ceph isn't created when
installing the ceph RPM(s). This is becasuse of line 440 in
ceph.spec.in using the %ghost directive[1] for the file install. My
reading of the behavior will mean that the file or directory in this
case will be included in the
On 6/18/13 10:29 AM, Sage Weil wrote:
> Derek-
>
> Please also try the latest ceph-deploy and cuttlefish branches, which
> fixed several issues with el6 distros. 'git pull' for hte latest
> ceph-deploy (clone from github and ./bootstrap if you were using the
> package) and install with
>
> .
eph hosts cbcbobj00.umiacs.umd.edu
2013-06-13 16:49:42,137 ceph_deploy.mon DEBUG Deploying mon to
cbcbobj00.umiacs.umd.edu
2013-06-13 16:49:42,564 ceph_deploy.mon DEBUG Distro
RedHatEnterpriseServer codename Santiago, will use sysvinit
Thanks,
derek
>
> On Jun 12, 2013, at 7:09 PM, Dere
Hi,
I am trying to run ceph-deploy on a very basic 1 node configuration.
But this is causing an exception that is confusing me,
ceph-deploy mon create cbcbobj00.umiacs.umd.edu
ceph-mon: mon.noname-a 192.168.7.235:6789/0 is local, renaming to
mon.cbcbobj00
ceph-mon: set fsid to a602d0c8-5c6e-442c-
57 matches
Mail list logo
