On Sat, August 24, 2013 10:08 am, Konstantin Belousov wrote:
> On Fri, Aug 23, 2013 at 01:49:19PM -0500, Valeri Galtsev wrote:
>>
>> On Fri, August 23, 2013 1:23 pm, Konstantin Belousov wrote:
>> > On Fri, Aug 23, 2013 at 01:05:24PM -0500, Valeri Galtsev wrote:
>> >> On Fri, August 23, 2013 11:05 am, Konstantin Belousov wrote:
>> >> > On Fri, Aug 23, 2013 at 09:24:32AM -0500, Valeri Galtsev wrote:
>> >> >> Dear Experts,
>> >> >> After searching the web, reading FreeBSD Docs, trying some hacks
>> >> found on
>> >> >> some discussion boards... I feel it is not easily possible. Yet,
>> as
>> >> always
>> >> >> there may be some expert who knows how to do it:
>> >> >> How can one have per user quotas inside jail?
>> >> >> Basically, I would like to give users shell access to some server,
>> >> but
>> >> that I prefer to have in jail, where I will mount all filesystems
>> they
>> >> need access to... and the only question is: how do I restrict them so
>> >> one
>> >> >> (or few) user doesn't fill up the whole filesystem. My mind is not
>> >> married
>> >> >> to any particular filesystem, UFS2, XFS, ZFS... - the only thing I
>> >> would
>> >> >> stay away from is NFS exporting on host and then NFS mounting in
>> jail
>> >> (which may be easiest if not the only way quota wise).
>> >> >
>> >> > UFS quotas work regardless of jailed/non-jailed user. The only
>> >> confusing
>> >> > issue is that quotas are per host uid. In other words, if host and
>> >> jail
>> >> user, or two users from different jails has the same uid, you get one
>> >> quota setting applied and accounted for them.
>> >> >
>> >> > Usual mitigation is to ensure that user uids are globally unique.
>> >> >
>> >>
>> >> Thanks, Konstantin.
>> >>
>> >> Still it doesn't work for me. My system is:
>> >>
>> >> 9.1-RELEASE-p5 amd64
>> >>
>> >> Kernel: the same as GENERIC, with one option added:
>> >>
>> >> options QUOTA # Add disk quota support
>> >>
>> >> filesystem with quota enabled is directly mounted (UFS; rw,userquota)
>> >> into
>> >> directory inside jail. User (with the same username and UID) exists
>> on
>> >> the
>> >> host system and in jail. Quotas work on the host system. Quotas don't
>> >> work
>> >> inside jail, so this user can fill up the whole filesystem when
>> logged
>> >> into jail (jail accepts ssh connections with different hostname...)
>> >>
>> >> Apart from that I tried a hack which I lifted from someone's FreeBSD
>> 7
>> >> hack (only the variable name changed since then), namely:
>> >>
>> >> in kernel, in:
>> >>
>> >> /usr/src/sys/kern/vfs_syscalls.c
>> >>
>> >> I kicked out two lines:
>> >>
>> >> if (!prison_allow(td->td_ucred, PR_ALLOW_QUOTAS))
>> >> return (EPERM);
>> >>
>> >> (which basically obliterate that if done from inside jail as far as I
>> >> understand),
>> >>
>> >> rebuilt and installed this kernel; in file
>> >>
>> >> /etc/rc.d/quota
>> >>
>> >> removed line
>> >>
>> >> # KEYWORD: nojail
>> >>
>> >> Yet, I'm still where I was: quotas work outside jail, not inside
>> jail...
>> >>
>> >> So, I'm at loss. I guess I will have to dive into zfs following Aaron
>> >> Kaufman's suggestion... Sigh.
>> >
>> > UFS quotas work per mount. So if jail root is on a filesystem which
>> > has no quotas configured, obviously the thing cannot work.
>> >
>> > You did not provided any details of your configuration, which makes
>> > a diagnostic impossible.
>> >
>>
>> Hi Konstantin,
>>
>> Thanks a lot for helping me! Sorry, my usual fault, not sufficient
>> details...
>>
>> Jails are set up pretty much as in:
>>
>> http://www.freebsd.org/doc/handbook/jails-application.html
>>
>> (directory names and locations are slightly different). Someone
>> mentioned,
>> ezjail does virtually the same too - if that helps.
>>
>> In /jail/mroot there is the structure resembling real system (binaries,
>> libraries,...) except for a few things that have to be writable inside
>> jail; those are replaced with symlinks pointing to these inside
>> subdirectory s:
>>
>> ls -l /jail/mroot
>> total 48
>> drwxr-xr-x 2 root wheel 1024 Aug 19 13:02 bin
>> drwxr-xr-x 7 root wheel 1024 Aug 19 13:03 boot
>> dr-xr-xr-x 2 root wheel 512 Aug 19 13:02 dev
>> lrwxr-xr-x 1 root wheel 5 Aug 19 13:11 etc -> s/etc
>> lrwxr-xr-x 1 root wheel 6 Aug 19 13:11 home -> s/home
>> drwxr-xr-x 3 root wheel 1536 Aug 19 13:03 lib
>> drwxr-xr-x 3 root wheel 512 Aug 19 13:03 libexec
>> drwxr-xr-x 2 root wheel 512 Aug 19 13:02 media
>> drwxr-xr-x 2 root wheel 512 Aug 19 13:02 mnt
>> dr-xr-xr-x 2 root wheel 512 Aug 19 13:02 proc
>> drwxr-xr-x 2 root wheel 2560 Aug 19 13:03 rescue
>> lrwxr-xr-x 1 root wheel 6 Aug 19 13:11 root -> s/root
>> drwxr-xr-x 2 root wheel 512 Aug 19 13:11 s
>> drwxr-xr-x 2 root wheel 2560 Aug 19 13:03 sbin
>> lrwxr-xr-x 1 root wheel 11 Aug 19 13:02 sys -> usr/src/sys
>> lrwxr-xr-x 1 root wheel 5 Aug 19 13:11 tmp -> s/tmp
>> drwxr-xr-x 14 root wheel 512 Aug 19 13:11 usr
>> lrwxr-xr-x 1 root wheel 5 Aug 19 13:11 var -> s/var
>>
>> particular jail lives in its root directory:
>>
>> /jail/shell
>>
>> /jail/mroot is nullfs readonly mounted onto /jail/shell, rw unique for
>> each shell filesystem is mounted into /jail/shell/s (and populated with
>> appropriate /etc, /var ....), filesystem that has to be with quotas is
>> mounted (UFS; rw,userquota) into
>>
>> /jail/shell/s/home
>>
>> This last one is the one in question: quotas on this work when user will
>> ssh to host system and will write to /jail/shell/s/home; quota does not
>> work if user will ssh into jail (which is accessible from network with
>> different hostname). When the user writes into /home in jail (into
>> /s/home
>> actually, symlink points there which on host system is our
>> /jail/shell/s/home), the quotas do not work.
>>
>> I don't quite understand what quota on jail root filesystem (enabled or
>> not enabled) has to do with quota on different filesystem that is
>> mounted
>> inside that filesystem. Outside jail / has no quotas, different
>> filesystem
>> mounted somewhere inside (/jail/shell/s/home or just /home or /var) with
>> quotas and it does honor quotas. Am I missing something trivial or
>> fundamental?
>>
>> Thanks again for helping me! What other details could help?
>
> I decided that I have no desire to try to understand all the layers of
> indirections which are only relevant to you anyway. Instead, I demostrate
> you what I mean by working quotas. Below is the transcript of the simple
> test.
>
> sandy% mount -v /mnt
> ~
> mount: /dev/ada1p4: Operation not permitted
> /dev/ada1p4 on /mnt (ufs, local, with quotas, soft-updates, writes: sync 2
> async 37, reads: sync 7 async 0)
> sandy% sudo repquota -uah | grep kostik
> ~
> kostik -- 14G 0 0 - 461057
> 0 0 -
> sandy% sudo jail -u kostik / test1 127.0.0.1 /bin/sh
> ~
> $ dd if=/dev/zero bs=1m of=/mnt/1/dddd count=1024
> 1024+0 records in
> 1024+0 records out
> 1073741824 bytes transferred in 10.765265 secs (99741328 bytes/sec)
> $ ^D%
> sandy% sudo repquota -uah | grep kostik
> ~
> kostik -- 15G 0 0 - 461058
> 0 0 -
>
> You could see that the accounted space and inodes are properly increased
> after the dd.
>
> IMO, you should make sure that the users operate on the filesystem which
> has quotas enabled. Or, you should provide a simple to reproduce test
> case, among the lines of the script I pasted above, for me to recreate
> the issue locally.
>
Thanks again for helping me! I guess, I understand now what the difference
is. Apparently, you are much better expert, so correct me if I'm wrong.
You run your jail with root of jail filesystems (/) the same as root
filesystem of host (/). Therefore, inside your jail you have access to all
host's /etc/fstab; /dev, ... I'll try to run jail the same way and will
see if in that case quotas will work for me. If yes, then I at least I
will know that my problem is not on the kernel level, but in the
environment accessible inside jail.
I have all jails set up so that one when in jail is not able to access
filesystem outside jail's own root, which is something like
/jail/{$jailname}... therefore host's /etc /dev are not visible for one
inside jail; what they see inside jail as / is /jail/{$jailname} on host.
Thanks again for all your efforts in helping me!!
Sincerely yours,
Valeri
PS I like _very_much_ your username on that machine: kostik ;-) !!
++++++++++++++++++++++++++++++++++++++++
Valeri Galtsev
Sr System Administrator
Department of Astronomy and Astrophysics
Kavli Institute for Cosmological Physics
University of Chicago
Phone: 773-702-4247
++++++++++++++++++++++++++++++++++++++++
_______________________________________________
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"
