What kernel are you running, and which format is the RBD image? I thought
we had a special return code for when the kernel doesn't support the
features used by that image, but that could be the problem.
-Greg
On Thursday, October 17, 2013, Omar Marquez wrote:
>
> Strace produces below:
>
> …
>
> futex(0xb5637c, FUTEX_WAKE_OP_PRIVATE, 1, 1, 0xb56378, {FUTEX_OP_SET, 0,
> FUTEX_OP_CMP_GT, 1}) = 1
> futex(0xb562f8, FUTEX_WAKE_PRIVATE, 1) = 1
> add_key(0x424408, 0x7fff82c4e210, 0x7fff82c4e140, 0x22, 0xfffffffe) =
> 607085216
> stat("/sys/bus/rbd", {st_mode=S_IFDIR|0755, st_size=0, ...}) = 0
> *open("/sys/bus/rbd/add", O_WRONLY) = 3*
> *write(3, "10.198.41.6:6789,10.198.41.8:678"..., 96) = -1 EINVAL (Invalid
> argument)*
> close(3) = 0
> rt_sigaction(SIGINT, {SIG_IGN, [], SA_RESTORER, 0x7fbf8a7efa90}, {SIG_DFL,
> [], 0}, 8) = 0
> rt_sigaction(SIGQUIT, {SIG_IGN, [], SA_RESTORER, 0x7fbf8a7efa90},
> {SIG_DFL, [], 0}, 8) = 0
> rt_sigprocmask(SIG_BLOCK, [CHLD], [PIPE], 8) = 0
> clone(child_stack=0, flags=CLONE_PARENT_SETTID|SIGCHLD,
> parent_tidptr=0x7fff82c4e040) = 22
> wait4(22, [{WIFEXITED(s) && WEXITSTATUS(s) == 0}], 0, NULL) = 22
> rt_sigaction(SIGINT, {SIG_DFL, [], SA_RESTORER, 0x7fbf8a7efa90}, NULL, 8)
> = 0
> rt_sigaction(SIGQUIT, {SIG_DFL, [], SA_RESTORER, 0x7fbf8a7efa90}, NULL, 8)
> = 0
> rt_sigprocmask(SIG_SETMASK, [PIPE], NULL, 8) = 0
> write(2, "rbd: add failed: ", 17rbd: add failed: ) = 17
> write(2, "(22) Invalid argument", 21(22) Invalid argument) = 21
> write(2, "\n", 1
> ) = 1
> exit_group(1) = ?
> +++ exited with 1 +++
>
>
> The app is run inside the container with setuid = 0 and the container is
> able to mount all required filesystems … could this still be a capability
> problem ? Also I do not see any call to capset() in the strafe log …
>
> --
> Om
>
>
> From: Kyle Bader <kyle.ba...@gmail.com <javascript:_e({}, 'cvml',
> 'kyle.ba...@gmail.com');>>
> Date: Thursday, October 17, 2013 5:08 PM
> To: Kevin Weiler <kevin.wei...@imc-chicago.com <javascript:_e({}, 'cvml',
> 'kevin.wei...@imc-chicago.com');>>
> Cc: "ceph-users@lists.ceph.com <javascript:_e({}, 'cvml',
> 'ceph-users@lists.ceph.com');>" <ceph-users@lists.ceph.com<javascript:_e({},
> 'cvml', 'ceph-users@lists.ceph.com');>>,
> Omar Marquez <omar.marq...@imc-chicago.com <javascript:_e({}, 'cvml',
> 'omar.marq...@imc-chicago.com');>>, Khalid Goudeaux <
> khalid.goude...@imc-chicago.com <javascript:_e({}, 'cvml',
> 'khalid.goude...@imc-chicago.com');>>
> Subject: Re: [ceph-users] mounting RBD in linux containers
>
> My first guess would be that it's due to LXC dropping capabilities, I'd
> investigate whether CAP_SYS_ADMIN is being dropped. You need CAP_SYS_ADMIN
> for mount and block ioctls, if the container doesn't have those privs a map
> will likely fail. Maybe try tracing the command with strace?
>
> On Thu, Oct 17, 2013 at 2:45 PM, Kevin Weiler <
> kevin.wei...@imc-chicago.com> wrote:
>
> Hi all,
>
> We're trying to mount an rbd image inside of a linux container that has
> been created with docker (https://www.docker.io/). We seem to have access
> to the rbd kernel module from inside the container:
>
> # lsmod | grep ceph
> libceph 218854 1 rbd
> libcrc32c 12603 3 xfs,libceph,dm_persistent_data
>
> And we can query the pool for available rbds and create rbds from inside
> the container:
>
> # rbd -p dockers --id dockers --keyring
> /etc/ceph/ceph.client.dockers.keyring create lxctest --size 51200
> # rbd -p dockers --id dockers --keyring
> /etc/ceph/ceph.client.dockers.keyring ls
> lxctest
>
> But for some reason, we can't seem to map the device to the container:
>
> # rbd -p dockers --id dockers --keyring
> /etc/ceph/ceph.client.dockers.keyring map lxctest
> rbd: add failed: (22) Invalid argument
>
> I don't see anything particularly interesting in dmesg or messages on
> either the container or the host box. Any ideas on how to troubleshoot this?
>
> Thanks!
>
>
> --
>
> *Kevin Weiler*
>
> IT
>
>
>
> IMC Financial Markets | 233 S. Wacker Drive, Suite 4300 | Chicago, IL
> 60606 | http://imc-chicago.com/
>
> Phone: +1 312-204-7439 | Fax: +1 312-244-3301 | E-Mail: *
> kevin.wei...@imc-chicago.com*
>
> ------------------------------
>
> The information in this e-mail is intended only for the person or entity
> to which it is addressed.
>
> It may contain confidential and /or privileged material. If someone other
> than the intended recipient should receive this e-mail, he / she shall not
> be entitled to read, disseminate, disclose or duplicate it.
>
> If you receive this e-mail unintentionally, please inform us immediately
> by "re
>
>
> ------------------------------
>
> The information in this e-mail is intended only for the person or entity
> to which it is addressed.
>
> It may contain confidential and /or privileged material. If someone other
> than the intended recipient should receive this e-mail, he / she shall not
> be entitled to read, disseminate, disclose or duplicate it.
>
> If you receive this e-mail unintentionally, please inform us immediately
> by "reply" and then delete it from your system. Although this information
> has been compiled with great care, neither IMC Financial Markets & Asset
> Management nor any of its related entities shall accept any responsibility
> for any errors, omissions or other inaccuracies in this information or for
> the consequences thereof, nor shall it be bound in any way by the contents
> of this e-mail or its attachments. In the event of incomplete or incorrect
> transmission, please return the e-mail to the sender and permanently delete
> this message and any attachments.
>
> Messages and attachments are scanned for all known viruses. Always scan
> attachments before opening them.
>
--
Software Engineer #42 @ http://inktank.com | http://ceph.com
_______________________________________________
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
