Re: new jail(8) ignoring devfs_ruleset?

[Jamie Gritton]

On 05/09/13 03:17, Jeremie Le Hen wrote:

On Thu, Mar 21, 2013 at 06:46:57PM -0600, Jamie Gritton wrote:


It's not fixed anywhere yet - it sometimes works in current, and
sometimes doesn't. I've been meaning to patch it up, but it the problem
is what I think it is, the patching up is a pretty big operation.

It doesn't mean you can't use jails with devfs in 9.1, just that you
can't use them with jail.conf. The old jail rc file that's all
shell-based is still the official jail startup method, and that one
still works. So existing systems will still work as expected, hence no
errata.


Shouldn't we warn the user about that in the manpage though?


Well really we ought to fix it. I guess the man page could have
something in the meantime, about an assumption that when you specify a
devfs ruleset, that the ruleset in question must actually exist at the time.

- Jamie
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

Re: new jail(8) ignoring devfs_ruleset?

[Jamie Gritton]

On 05/09/13 22:42, Dewayne Geraghty wrote:

An ugly workaround to complete the jail closure, when relying on jail.conf, is 
to:

jail -r $JAILNAME
umount /$LOCATION_OF_JAILS/$JAILNAME/dev || true


The only problem with devfs I'm aware of is it not catching the right
ruleset when starting in the rc system. So does this mean you're having
problems unmounting /dev?

What happens when you add a "-v" to the "jail -r"? It should note that
/dev is being umounted.

- Jamie
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

Re: jail already exists

[Jamie Gritton]

On 05/22/13 15:38, Tomasz Jaroszyk wrote:

I use 9.1-PRERELEASE and have problem with jails.
Sometimes one of them after shutdown do not want start again.

szafir% sudo jail -rc reverse
jail: "reverse" not found
jail: reverse: jail 320 already exists
szafir% pgrep -j 320
szafir%

I guess some file was not removed after shutdown but manual of jail(8) did
say nothing about that.

I have been changing jid when that happened but this is annoying.
Any better solution?


If I had to guess at your jail.conf, I'd say that "reverse" has a
hard-coded jid of 320. Of course it'd be better if I didn't have to
guess :-).

It looks like the jail "reverse" doesn't currently exist, hence the "not
found" message. But *some* jail with a jid of 320 does exist. There's a
good chance that the 320 that does exist is the same jail as the
"reverse" that doesn't exist. That's because when removing, a dying jail
is usually disregarded. When adding a jail, a dying jail of the same
name can be disregarded, but not one with the same jid.

So if my guess is correct, the solution is jail(8)'s "-d" flag, or the
"allow.dying" jail parameter. An jail with active TCP connections will
generally sit around a while in the dying state, which can cause
problems with the "jail -rc" restart - but only if you try to force it
to a particular jid.

- Jamie
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

Re: Problem using bz's multi-IP/IPv6/No-IP Jail Patch (7-STABLE)

[Jamie Gritton]

Kage wrote:


Encountering more issues now.  Binding just an IPv6 address to a jail
shows up in jls -v, but when I run ifconfig -a in the jail, I get an
error I've never encountered, and doesn't show up on any Google
search:

[r...@nub:/etc] jls -v
   JID  Hostname  Path
Name  State
CPUSetID
IP Address(es)
 9  jail.template.tld /usr/jails/TEMPLATE
  ALIVE
10
2610:150:c248:dead:beef:c0ff:eec0:deaa

[r...@jail:/] ifconfig -a
ifconfig: socket(family 2,SOCK_DGRAM): Protocol not supported


Recent patches reject sockets in jails that have no addresses in the
socket's family.  So if you jail has no IPv6 addresses, you won't be
able to create any IPv6 sockets.  Likewise your case: if that jail has
no IPv4 addresses, then it's an IPv4-less jail, and IPv4 sockets won't
work (Protocol not supported).  For actual network connections, this
makes sense: you won't be able to bind or connect with this socket, as
there are no IPv4 addresses in the system.

But ifconfig is a different situation.  It just needs a socket of some
sort, and AF_INET has always worked, because any networked system always
has IPv4 support.  But in an IPv4-less system (which an IPv4-less jail
not acts like), this default isn't useful.  Something will need to be
fixed.  I'm not sure if that something is ifconfig or the kernel.

- Jamie
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

Re: Problem using bz's multi-IP/IPv6/No-IP Jail Patch (7-STABLE)

[Jamie Gritton]

I wrote:

Kage wrote:


Encountering more issues now.  Binding just an IPv6 address to a jail
shows up in jls -v, but when I run ifconfig -a in the jail, I get an
error I've never encountered, and doesn't show up on any Google
search:

[r...@nub:/etc] jls -v
   JID  Hostname  Path
Name  State
CPUSetID
IP Address(es)
 9  jail.template.tld /usr/jails/TEMPLATE
  ALIVE
10
2610:150:c248:dead:beef:c0ff:eec0:deaa

[r...@jail:/] ifconfig -a
ifconfig: socket(family 2,SOCK_DGRAM): Protocol not supported


Recent patches reject sockets in jails that have no addresses in the
socket's family.  So if you jail has no IPv6 addresses, you won't be
able to create any IPv6 sockets.  Likewise your case: if that jail has
no IPv4 addresses, then it's an IPv4-less jail, and IPv4 sockets won't
work (Protocol not supported).  For actual network connections, this
makes sense: you won't be able to bind or connect with this socket, as
there are no IPv4 addresses in the system.

But ifconfig is a different situation.  It just needs a socket of some
sort, and AF_INET has always worked, because any networked system always
has IPv4 support.  But in an IPv4-less system (which an IPv4-less jail
not acts like), this default isn't useful.  Something will need to be
fixed.  I'm not sure if that something is ifconfig or the kernel.


Here's a patch for ifconfig.  It allows "ifconfig -a" and a few other
similar informative ifconfig options to run inside an IPv4-less jail
(of course trying to set anything still fails).  Outside of a jail, you
should see no change.  Apply it inside your /usr/src tree, and install
it both in the root system (under /sbin) and in your jails
(/usr/jails/TEMPLATE or wherever).  Just in case I broke something, keep
a copy of the old one :-).  But I've tested it on my own system so I
don't expect anything to be broken.

This is under review and I expect to be able to commit it to Current
shortly, then MFC it a week or so after that.  If you have any trouble
with it, feel free to ask me - I'm the one who broke ifconfig in the
first place.

- Jamie
Index: sbin/ifconfig/ifgroup.c
===
--- isbin/ifconfig/fgroup.c (revision 189318)
+++ sbin/ifconfig/ifgroup.c (working copy)
@@ -131,9 +131,9 @@
int  len, cnt = 0;
int  s;
 
-   s = socket(AF_INET, SOCK_DGRAM, 0);
+   s = socket(AF_LOCAL, SOCK_DGRAM, 0);
if (s == -1)
-   err(1, "socket(AF_INET,SOCK_DGRAM)");
+   err(1, "socket(AF_LOCAL,SOCK_DGRAM)");
bzero(&ifgr, sizeof(ifgr));
strlcpy(ifgr.ifgr_name, groupname, sizeof(ifgr.ifgr_name));
if (ioctl(s, SIOCGIFGMEMB, (caddr_t)&ifgr) == -1) {
Index: sbin/ifconfig/ifclone.c
===
--- sbin/ifconfig/ifclone.c (revision 189318)
+++ sbin/ifconfig/ifclone.c (working copy)
@@ -54,9 +54,9 @@
int idx;
int s;
 
-   s = socket(AF_INET, SOCK_DGRAM, 0);
+   s = socket(AF_LOCAL, SOCK_DGRAM, 0);
if (s == -1)
-   err(1, "socket(AF_INET,SOCK_DGRAM)");
+   err(1, "socket(AF_LOCAL,SOCK_DGRAM)");
 
memset(&ifcr, 0, sizeof(ifcr));
 
Index: sbin/ifconfig/ifconfig.c
===
--- sbin/ifconfig/ifconfig.c(revision 189318)
+++ sbin/ifconfig/ifconfig.c(working copy)
@@ -441,22 +441,23 @@
DEF_CMD("ifdstaddr", 0, setifdstaddr);
 
 static int
-ifconfig(int argc, char *const *argv, int iscreate, const struct afswtch *afp)
+ifconfig(int argc, char *const *argv, int iscreate, const struct afswtch *uafp)
 {
-   const struct afswtch *nafp;
+   const struct afswtch *afp, *nafp;
const struct cmd *p;
struct callback *cb;
int s;
 
strncpy(ifr.ifr_name, name, sizeof ifr.ifr_name);
+   afp = uafp != NULL ? uafp : af_getbyname("inet");
 top:
-   if (afp == NULL)
-   afp = af_getbyname("inet");
ifr.ifr_addr.sa_family =
afp->af_af == AF_LINK || afp->af_af == AF_UNSPEC ?
-   AF_INET : afp->af_af;
+   AF_LOCAL : afp->af_af;
 
-   if ((s = socket(ifr.ifr_addr.sa_family, SOCK_DGRAM, 0)) < 0)
+   if ((s = socket(ifr.ifr_addr.sa_family, SOCK_DGRAM, 0)) < 0 &&
+   (uafp != NULL || errno != EPROTONOSUPPORT ||
+(s = socket(AF_LOCAL, SOCK_DGRAM, 0)) < 0))
err(1, "socket(family %u,SOCK_DGRAM", ifr.ifr_addr.sa_family);
 
while (argc > 0) {
@@ -803,11 +804,12 @@
 
if (afp == NULL) {
allfamilies = 1;
-   afp = af_getbyname("inet");
-   } else
+   ifr.ifr_addr.sa_family = AF_LOCAL;
+   } else {
allfamilies = 0;
-
-   ifr.ifr_addr.sa

Re: Problem using bz's multi-IP/IPv6/No-IP Jail Patch (7-STABLE)

[Jamie Gritton]

I wrote:

Here's a patch for ifconfig.  It allows "ifconfig -a" and a few other
similar informative ifconfig options to run inside an IPv4-less jail
(of course trying to set anything still fails).  Outside of a jail, you
should see no change.  Apply it inside your /usr/src tree, and install
it both in the root system (under /sbin) and in your jails
(/usr/jails/TEMPLATE or wherever).  Just in case I broke something, keep
a copy of the old one :-).  But I've tested it on my own system so I
don't expect anything to be broken.


The patch went in as r189864 on CURRENT and r189970 on STABLE.  But
the fix brought up some other issues, and now the STABLE patch has been
reverted.  It may come back in time for 7.2 if everything looks good in
CURRENT, but that's not a sure thing given the release schedule.

- Jamie
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

Re: New jail framework - the userland side

[Jamie Gritton]

Poul-Henning Kamp wrote:

In message <49fe5387.3020...@freebsd.org>, Jamie Gritton writes:

Hi all.  I recently added some new jail-related system calls to extend 
the current jail system with an nmount-inspired name=value interface.  


I think this is a great move in the right direction, my only concern is
that we should try to share as much of the string-munging code between
the nmount and jail implementations as possible.


Most if it is shared - jail actually calls vfs_getopt and related calls
from the family.  I might want to spin those functions off into their
own subsystem at some point, now that they're officially used outside
of VFS.

I did have to extend things somewhat for jail_get, as nmount is write-
only and only had to deal with one module at a time (the filesystem
type).  Those extensions are available for use elsewhere, as I suspect
filesystems and jails aren't the only place where we could use name-
based extensibility.

- Jamie
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

Re: Hierarchical jails

[Jamie Gritton]

There's still a change to offer your input on the new jails before they
go in!  OK, given the lack of response so far, it's less "still a
chance" than "please?".  Current plans are to have this in place for
8.0, with connections to the ongoing Vimage work.  Hopefully the silence
is approval, and commits will likely be appearing soon.


I wrote:

Here's the first round of hierarchical jails under the new framework.

Instead of creds having either a prison or a NULL pointer, they all have
a prison pointer with the default being the global "prison0" that
contains information about the real environment.  Jailed root may (if
granted permission) create prisons that would be under its place in the
hierarchy, but may not alter (or even see) prisons at its level or
above.

The JID space is flat, i.e. every prison in the system has a unique ID.
The prison name space is hierarchical, with jails having dot-separated
component names.

prison0 contains three fields that were system globals: pr_root,
pr_host, and pr_securelevel.  I've kept the globals rootvnode and
hostname, and take care that when one is changed the other changes too
(not yet true for hostname - read on).  But I've actually removed the
global securelevel, instead forcing people to use securelevel_gt() and
securelevel_ge() (or in very rare cases to check prison0.pr_securelevel
directly).  I chose to do that because while using the global rootvnode
and hostname may be incorrect, using the wrong securelevel is, well,
insecure.  Actually it would be insecure to use the wrong rootvnode too,
but I'm not convinced removing that global is worth the headache.

Other globals are subsumed into prison0, but they were only ever part of
the jail system anyway: the various jail-related permission bits and
such administrative things as prisoncount.

The prison hierarchy keeps track of restrictions placed on prisons, and
will reflect them downward so a child jail is always at least as
restricted as its ancestors.  It doesn't go the other way though: if a
prison's restrictions are loosened, the children stay as they are.

This patch doesn't have anything for userland, and hierarchical jails
won't work without that patch (because jails don't have permission to
create sub-jails by default, and jail(2) can't grant that permission).
A userland patch will follow soon, very similar to the version I posted
here recently.

- Jamie

___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

Re: Hierarchical jails

[Jamie Gritton]

Julian Elischer wrote:

Jamie Gritton wrote:

prison0 contains three fields that were system globals: pr_root,
pr_host, and pr_securelevel.  I've kept the globals rootvnode and
hostname, and take care that when one is changed the other changes too
(not yet true for hostname - read on).  But I've actually removed the
global securelevel, instead forcing people to use securelevel_gt() and
securelevel_ge() (or in very rare cases to check prison0.pr_securelevel
directly).  I chose to do that because while using the global rootvnode
and hostname may be incorrect, using the wrong securelevel is, well,
insecure.  Actually it would be insecure to use the wrong rootvnode too,
but I'm not convinced removing that global is worth the headache.


not sure why you want to keep hostname a true global
It seems to me  that it is an eminently virtalizable property.
though possible a special hostname might exist for the base system
for error messages etc.
kind of like V_hostname an G_hostname :)


It was mostly for the number of times I saw that global being used -
didn't want to upset the order of things too much.  I didn't see nearly
as much use of securelevel with the advent of securelevel_ge() and
securelevel_gt().  But I suppose the G/V_hostname thing has already
gotten that ball rolling.

There is at least one place that uses the global securelevel directly
(i.e. prison0.securelevel).  The same could be done for hostnames, which
does a pretty good job of pointing out that this is the global hostname
being used.  Because you're right - the hostname is at the center of of
what it means to have a jail identity.

Then there's rootvnode, the third global that's superseded by
hierarchical jails.  I could also remove that, allowing the use of
prison0.pr_root for those who need the real root.
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

Re: Hierarchical jails

[Jamie Gritton]

Jilles Tjoelker wrote:

On Thu, May 14, 2009 at 11:12:50AM -0600, Jamie Gritton wrote:

There's still a change to offer your input on the new jails before they
go in!  OK, given the lack of response so far, it's less "still a
chance" than "please?".  Current plans are to have this in place for
8.0, with connections to the ongoing Vimage work.  Hopefully the silence
is approval, and commits will likely be appearing soon.


I have not tried this, but I think this patch may allow jailed roots to
escape. The problem is that there is only one fd_jdir. The escape would
go like: jailed root creates a new jail in a subdirectory, opens its /
and sends the fd to a process in the new jail via a unix domain socket.
When the process calls fchdir on the fd, it will be able to access ..
normally.

With nested chroot, or chroot in jail, this is not possible, because
fd_jdir always contains the first jail or chroot done and will not allow
escaping from it; however, root in a level 2 chroot can escape back to
level 1 using chroot.



Indeed - considering how that was a major design point of jails, I'm not
sure how I missed it.  ".." processing will need to run up the jail
tree.  No big deal on performance and easily done, but embarrassing not
have had that in place already.
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

Re: sysctl variables not propagating to children jails

[Jamie Gritton]

Bjoern A. Zeeb wrote:

On Tue, 9 Jun 2009, Edwin Shao wrote:

Hi,


In the most recent -current, I've noticed that sysctl variables no
longer propagate to jails and thus it is impossible to allow raw
sockets, allow mounting, etc. This might be related to
.

..

Please let me know if I'm doing something stupid!


No, nothing apart from probably not spotting that the problem was
already well understood and there had been workarounds suggested at
the end of the above thread.


That said, expect the problem to be fixed within 24 hours.  You will
only have to rebuild your jail(8) command line tool, once you spot the
commit by:

1) update your source and make sure to have the new version of jail.c
2) cd src/usr.sbin/jail
3) make obj && make depend && make all
4) sudo make install
5) try again.

In case you still see problems afterwards, cry again, loud and in
here - in case that will fix the problem a short note will be
welcome as well;-)


The patch is now in for this - recompile jail(8) with r193929.

- Jamie
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

Re: Switching /etc/rc.d/jail to new syntax (+ new features)

[Jamie Gritton]

Alexander Leidinger wrote:


at http://www.leidinger.net/FreeBSD/current-patches/jail.diff I
have a patch to switch the jail rc script to the new jail
(8-current) syntax. This includes new config options for a jail
(see etc/defaults/rc.conf after patching). The patch also contains
my X-in-a-jail stuff (feel free to ignore this part, it's disabled
by default).

If you do not make any config change, you will be able to see all
mounted filesystems of the entire machine. To get back to the
previous behavior, you have to add a config option:
 jail_XXX_startparams="enforce_statfs=2"

This config option can also take other jail parameters like
allow.sysvipc and other ones described in the jail man-page
(additional parameters need to be space separated).

Feedback welcome.
  

1) it break various things that will no longer work


As mentioned, it "breaks" the statfs part. If there's anything
else, be more specific please.
  

v6, noIP, ...



I didn't change the IP handling in the rc script. Does this mean
jail(8) works differently regarding the address parsing when called
with the new parameters instead of the old options?

I didn't test anything regarding ipv6, but as long as jail(8) doesn't
behave differently with the new calling syntax compared with what we
have in the tree, then the behavior is not differnt from what we have.
If it behaves differently, this can be fixed in the script.
  


There is a difference.  Under the old options, IPv4 and IPv6 addresses 
are mixed
into the single fixed argument, and then are parsed to determine which 
kind they
are - both by jail(8) and rc.d/jail.  Under the new parameter-based 
command line,

IPv4 addresses and IPv6 address go with ip4.addr and ip6.addr respectively.

The rc.d/jail code that brings up addresses on an interface can be modified
to decide which argument the address goes with.

I've given Bjoern a patch based on yours that handles this as well as 
the allow.*
systctls (though I missed the statfs part).  He still has the larger 
disagreements
he mentioned though, so I'm working now toward a more comprehensive 
solution.



2) it's not a poper solution


The proper solution for the statfs part would be, that jail(8)
defaults to =2 if nothing is specified. Alternatively I can get
convinced that we should do a default for it in defaults/rc.conf if
nothing is specied for startparams for a particular jail (like we
have for some other things), but this would not be as good as if
jail(8) would handle it itself.

If you do not talk about the statfs part but in a more generic way,
what would be a proper solution in your eyes?
  

A proper solution would be a proper mgmt system ready for the future
instead of continuting to hack up rc.d/jail via option fo bar baz and
another 17000 of them.
But this is nothing I'll discuss today while things aren't fully
shaken out yet.



And I assume from what you say, that such a new mgmt system will not be
ready for 8.0. Whatever it will be, it sounds like it will be different
from what we have ATM, so I don't think it will be something which will
replace the current approach in 8-stable, but will be available
additionally, if at all.

  

For now what used to work should continue to work and not break.
Everything else on top of that needs to be done properly and not in a
rainy-midnight-drive-by.



This is not a drive-by. I provide a patch for discussion which allows
to use some new features in 8.0 which doesn't break when someone
updates from 7.x. Some small enhancement which doesn't break backwards
compatibility is always better than no improvement at all. It may not
handle all cases, but for this reason I ask people to test it. After
that some things can maybe fixed, and after that it can be evaluated if
it is worth to commit or not.

I don't even urge to rush this in before 8.0. I just offer it now, so
that people can actually use some new features. I had to write this
anyway, as without the new syntax I wouldn't have been able to use my
enhancement to run X in a jail, which I ported to the new syntax. If
people think it is useful for 8.0 and nothing better is available for
8.0, it should be shipped with 8.0 IMO (if nothing breaks), but if it
isn't, I don't care, as I have it for where I need it.

Bye,
Alexander.
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"
  

___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

Re: Switching /etc/rc.d/jail to new syntax (+ new features)

[Jamie Gritton]

Alexander Leidinger wrote:

Quoting Jamie Gritton  (from Mon, 29 Jun 2009 
11:30:49 -0600):



Alexander Leidinger wrote:


at http://www.leidinger.net/FreeBSD/current-patches/jail.diff I
have a patch to switch the jail rc script to the new jail
(8-current) syntax. This includes new config options for a jail
(see etc/defaults/rc.conf after patching). The patch also contains
my X-in-a-jail stuff (feel free to ignore this part, it's disabled
by default).

If you do not make any config change, you will be able to see all
mounted filesystems of the entire machine. To get back to the
previous behavior, you have to add a config option:
jail_XXX_startparams="enforce_statfs=2"

This config option can also take other jail parameters like
allow.sysvipc and other ones described in the jail man-page
(additional parameters need to be space separated).

Feedback welcome.


1) it break various things that will no longer work


As mentioned, it "breaks" the statfs part. If there's anything
else, be more specific please.


v6, noIP, ...



I didn't change the IP handling in the rc script. Does this mean
jail(8) works differently regarding the address parsing when called
with the new parameters instead of the old options?

I didn't test anything regarding ipv6, but as long as jail(8) doesn't
behave differently with the new calling syntax compared with what we
have in the tree, then the behavior is not differnt from what we have.
If it behaves differently, this can be fixed in the script.



There is a difference.  Under the old options, IPv4 and IPv6 addresses 
are mixed
into the single fixed argument, and then are parsed to determine which 
kind they
are - both by jail(8) and rc.d/jail.  Under the new parameter-based 
command line,
IPv4 addresses and IPv6 address go with ip4.addr and ip6.addr 
respectively.


But why are my jails (with only one ipv4 address) starting correctly then?


The problem is that all addresses are put into ip4.addr, so it will
break (only) if you have any IPv6 addresses.

The rc.d/jail code that brings up addresses on an interface can be 
modified

to decide which argument the address goes with.

I've given Bjoern a patch based on yours that handles this as well as 
the allow.*

systctls (though I missed the statfs part).


Do you mind making it available somewhere?


Sure.  I've put it at http://gritton.org/freebsd/jail.rc.diff

- Jamie
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

Jail parameter patch: disable/new/inherit

[Jamie Gritton]

There's a patch to Current at http://gritton.org/freebsd/triple.diff
that makes some small changes to the new parameter based jail system.
I invite any interested in the future direction of jails to review it
before it goes in (hopefully in the next day or two).

This patch deals with jailed subsystems that may or may not be
virtualized.  At first, there was a boolean to describe this
situation: for example in the VIMAGE kernels, the setting "vnet"
parameter would create a jail with a virtual network stack.

But there's more than just virtual or not.  In particular there are
three things that can be done with a particular subsystem:

"disable": Don't use the subsystem at all in a jail.  For example, if
you create a jail with "ip6=disable", that jail won't be able to use
IPv6 sockets, as if it were a system without INET6 defined in the
kernel.

"new": Create a new virtual instance of the subsystem in the jail.
What constitutes a new instance will vary, but it generally means the
jail is treated in some way different from the rest of the system.
Setting "ip6=new" will restrict IPv6 addresses (to the contents of the
list specified by "ip6.addr" which should also be set).  Setting
"host=new" will let a jail set its own hostname (and related data)
separately from the rest of the system.  Setting "vnet=new" will
create a new network stack for the jail.

"inherit": This is the default state, and means the jail is treated
the same as the rest of the system.  There's no difference between a
jailed and non-jailed process as far as that subsystem is concerned.
A jail with "ip6=inherit" would allow the full use of the available
IPv6 addresses.

As yet, this is just a structural/name change.  It will become
important as other features are added to the jail system, including
any modules that want to have jail support.

- Jamie
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

Re: 8.0 still allow creating ipv6 udp socket in jail without ipv6 ip

[Jamie Gritton]

Bjoern A. Zeeb wrote:

On Wed, 29 Jul 2009, Mykola Dzham wrote:

Bjoern A. Zeeb wrote:

On Sat, 25 Jul 2009, Mykola Dzham wrote:

After r188146 creating tcp ipv6 socket in jail without ipv6 ip is not
allowed, but udp socket is allowed.


I cannot really follow what you are trying to say as wrt IPv4 and IPv6
sockets and what about UDP.

Your sample further down is trying to use an IPv4 address on an IPv6
Datagram socket which is an error either way.


Some java programms attempt to use ipv6 sockets, then use ipv4 if
socket(AF_INET6,...) fail. My sample imitate this


Prior to FreeBSD 7.2 IPv6 hadn't been supported at all for jails.

With 7.2 it was possible to create IPv6 sockets (but only shortly and
then fail on bind/connect/...).  With the commit you reference the
"Protocol not supported" came back in case there was no address of
that address family for a given jail.

With 8 the primary syntax for jails has changed and the "backward
compat mode" again allows you to create a socket on a jail even if
no address of the same family was configured for the jail.

This should be addressed by the following patch:
http://people.freebsd.org/~bz/20090727-01-jail8-legacy.diff

Can you give it a try and report if that fixes your problem?


Patch aplied cleanly on r195820 , but jail can not start after patching:

# jail -l -U root -i /usr/home/d/guests/tap2 tap2.my.domain.com 
10.112.0.151 /bin/sh /etc/rc

jail: ip6: unknown boolean value "disable"


r195820 is too old; but Jamie has a better solution; I would suggest
to backout the jail(8) patch and wait for the next two commits of
Jamie to HEAD and then update the machine again.


OK, with r195945 things should be back to disallowing sockets when no 
addresses were assigned for that family.  You'll need to rebuild the 
kernel for the fix, and libjail and possibly jail(8) to get past the 
"unknown boolean value" error.


- Jamie
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

Re: Tutorial for Hierarchical Jails?

[Jamie Gritton]

Edwin Shao wrote:

Hello,
Does anyone have a walkthrough for how to get hierarchical jails to work?
I've been playing around with it for a couple of days and it simply is not
working. I would like to know if anyone has gotten it to work, and if so,
how?

The error I tend to get within a jail (starting another child jail) is:
hyper# ./jail start
Configuring jails:.
Starting jails: cannot start jail "neko":

I'm using very basic steps as outlined in <
http://www.freebsd.org/doc/en/books/handbook/jails-intro.html> and I am
easily getting the jails to work in the non-jailed highest level system.

What I have done to troubleshoot so far:
* Installed from scratch 8.0-RC1 ISO, make buildworld from scratch 8.0-RC1
/usr/src.
* Created very liberal sysctls.
* Tried different combinations of disabling/enabling mounted systems such as
devfs, procfs, etc.
* Tried modifying different module fs to enable the "jail" flag.

This is under a clean install of 8.0-RC1. I'd be happy to provide additional
information for troubleshooting, but I'm not even sure what's going wrong.
It'd probably be more helpful for you to just let me know what you did to
get it wroking.


The main thing you need to do is to set the first-level jail's
children.max parameter.  It defaults to zero, which doesn't allow a jail
to create any child jails (the non-hierarchical default).  It sounds
like you have everything else you need.

- Jamie
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

Re: Tutorial for Hierarchical Jails?

[Jamie Gritton]

Edwin Shao wrote:


When I try to change the parameter, nothing happens:
rescue /etc> sudo sysctl security.jail.param.children.max=1
security.jail.param.children.max: 0 -> 0

rescue /etc> sudo sysctl security.jail.param.children.max
security.jail.param.children.max: 0

Am I doing this incorrectly?


Yes.  Use jail(8) to set the parameters, not sysctl.  The 
security.jail.param.* sysctls are for reference only and have no useful 
values to get or set.  Set it with:


jail -m jid= children.max=1

Run this on the base system, i.e. not inside the jail in question.

- Jamie
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

Re: Tutorial for Hierarchical Jails?

[Jamie Gritton]

The sysctls not only don't get written to, they don't have any useful
information to read either. They only describe the existence and format
of the various jail parameters. Sorry, but there;s no way to set a
default children.max parameter or inherit it from the parent. We've
decided to set the default to the most secure/restrictive in many cases.
Once we've come up with a new jail configuration interface, this won't
be such a hassle.

The devfs errors are probably something that will have to be addressed
in a later revision - I haven't looked in the devfs direction so I'm not
sure about that. The mount error may be related to the first jail's
allow.mount parameter (whose default comes from
security.jail.mount_allowed).

- Jamie

Edwin Shao wrote:

Thanks, that worked for me.

* Using jail to change children.max on the parent does not affect 
`sysctl security.jail.param.children.max` in the child.  Also 
security.jail.param.children.cur never changes either. Not sure if 
that's intended behavior.
* Is there any way to persist the security.jail.param.children.max 
parameter without entering the jail command every time? 
* I get the following output when I create a jail inside a jail:


hyper ~> ezjail-admin start neko
Configuring jails:.
Starting jails:devfs rule: ioctl DEVFSIO_RGETNEXT: Operation not permitted
devfs rule: ioctl DEVFSIO_RGETNEXT: Operation not permitted
/etc/rc.d/jail: WARNING: devfs_set_ruleset: you must specify a ruleset 
number

devfs rule: ioctl DEVFSIO_SAPPLY: Operation not permitted
ln: log: Operation not permitted
mount: proc : Operation not permitted
 neko.

I'm using the same configuration values as in the parent's jail, which 
work. Everything seems to work alright inside the jail, so I assume the 
errors are safe to ignore?


Thanks again!
- Edwin

On Mon, Sep 28, 2009 at 9:11 PM, Bjoern A. Zeeb 
mailto:bzeeb-li...@lists.zabbadoz.net>> 
wrote:


On Mon, 28 Sep 2009, Edwin Shao wrote:

Hi Jamie,
When I try to change the parameter, nothing happens:
rescue /etc> sudo sysctl security.jail.param.children.max=1
security.jail.param.children.max: 0 -> 0

rescue /etc> sudo sysctl security.jail.param.children.max
security.jail.param.children.max: 0

Am I doing this incorrectly?


Yes. It's a parameter to jail(8).  The security.jail.param sysctls can
be seen as a list of possible options valid to jail(8).  See man 8 jail
for the exact details.

/bz

-- 
Bjoern A. Zeeb   What was I talking about and who are you again?




___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

Re: Tutorial for Hierarchical Jails?

[Jamie Gritton]

Does the base system have security.jail.allow_raw_sockets=1? You need to
have that, or set the jail's allow.raw_sockets. You can't set the jail's
permissions from within the jail itself. If you have multiple jail
levels, then both jails need to allow raw sockets - a jail can't allow a
child jail to do what it can't do itself.

- Jamie


Edwin Shao wrote:
One other thing that is odd: hierarchical jails don't seem to inherit 
some sysctls such as allow_raw_socket.


In the host (jail), rc.conf has jail_set_allow_raw_sockets="YES" and 
sysctl.conf has "security.jail.allow_raw_sockets=1", but no child jail 
can ping out:

neko# ping google.com <http://google.com>
ping: socket: Operation not permitted

What is happening in this case? 


Thank you for your time again.


On Tue, Sep 29, 2009 at 12:16 AM, Jamie Gritton <mailto:ja...@freebsd.org>> wrote:


The sysctls not only don't get written to, they don't have any useful
information to read either. They only describe the existence and format
of the various jail parameters. Sorry, but there;s no way to set a
default children.max parameter or inherit it from the parent. We've
decided to set the default to the most secure/restrictive in many cases.
Once we've come up with a new jail configuration interface, this won't
be such a hassle.

The devfs errors are probably something that will have to be addressed
in a later revision - I haven't looked in the devfs direction so I'm not
sure about that. The mount error may be related to the first jail's
allow.mount parameter (whose default comes from
security.jail.mount_allowed).

- Jamie

Edwin Shao wrote:

Thanks, that worked for me.

* Using jail to change children.max on the parent does not
affect `sysctl security.jail.param.children.max` in the child.
 Also security.jail.param.children.cur never changes either. Not
sure if that's intended behavior.
* Is there any way to persist the
security.jail.param.children.max parameter without entering the
jail command every time? * I get the following output when I
create a jail inside a jail:

hyper ~> ezjail-admin start neko
Configuring jails:.
Starting jails:devfs rule: ioctl DEVFSIO_RGETNEXT: Operation not
permitted
devfs rule: ioctl DEVFSIO_RGETNEXT: Operation not permitted
/etc/rc.d/jail: WARNING: devfs_set_ruleset: you must specify a
ruleset number
devfs rule: ioctl DEVFSIO_SAPPLY: Operation not permitted
ln: log: Operation not permitted
mount: proc : Operation not permitted
 neko.

I'm using the same configuration values as in the parent's jail,
which work. Everything seems to work alright inside the jail, so
I assume the errors are safe to ignore?

Thanks again!
- Edwin

On Mon, Sep 28, 2009 at 9:11 PM, Bjoern A. Zeeb
mailto:bzeeb-li...@lists.zabbadoz.net>
<mailto:bzeeb-li...@lists.zabbadoz.net
<mailto:bzeeb-li...@lists.zabbadoz.net>>> wrote:

   On Mon, 28 Sep 2009, Edwin Shao wrote:

   Hi Jamie,
   When I try to change the parameter, nothing happens:
   rescue /etc> sudo sysctl security.jail.param.children.max=1
   security.jail.param.children.max: 0 -> 0

   rescue /etc> sudo sysctl security.jail.param.children.max
   security.jail.param.children.max: 0

   Am I doing this incorrectly?


   Yes. It's a parameter to jail(8).  The security.jail.param
sysctls can
   be seen as a list of possible options valid to jail(8).  See
man 8 jail
   for the exact details.

   /bz

   --Bjoern A. Zeeb   What was I talking about and
who are you again?




___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

Re: Tutorial for Hierarchical Jails?

[Jamie Gritton]

Without going into the current rc system, which isn't up to the task
of hierarchical jails, here's a minimal set of parameters/commands to
create hierarchical jails that can still ping:

# jail -c name=foo host.hostname=foo allow.raw_sockets children.max=99 
ip4.addr=10.20.12.68 persist

# jexec foo /bin/csh
foo# jail -c name=bar host.hostname=bar allow.raw_sockets 
ip4.addr=10.20.12.68 persist

foo# jexec bar /bin/csh
bar# ping gritton.org
PING gritton.org (161.58.222.4): 56 data bytes
64 bytes from 161.58.222.4: icmp_seq=0 ttl=54 time=78.344 ms

- Jamie


Edwin Shao wrote:
The base system has allow_raw_sockets, the first level jail also has 
allow_raw_sockets and has the exact same configuration as the base 
system (I use puppet to manage config files.) I can't set 
allow_raw_sockets anyway for the second-level jail without manually 
invoking the jail command.


On Tue, Sep 29, 2009 at 7:08 AM, Jamie Gritton <mailto:ja...@freebsd.org>> wrote:


Does the base system have security.jail.allow_raw_sockets=1? You need to
have that, or set the jail's allow.raw_sockets. You can't set the jail's
permissions from within the jail itself. If you have multiple jail
levels, then both jails need to allow raw sockets - a jail can't allow a
child jail to do what it can't do itself.

- Jamie


Edwin Shao wrote:

One other thing that is odd: hierarchical jails don't seem to
inherit some sysctls such as allow_raw_socket.

In the host (jail), rc.conf has jail_set_allow_raw_sockets="YES"
and sysctl.conf has "security.jail.allow_raw_sockets=1", but no
child jail can ping out:
neko# ping google.com <http://google.com> <http://google.com>

ping: socket: Operation not permitted

What is happening in this case?
    Thank you for your time again.


On Tue, Sep 29, 2009 at 12:16 AM, Jamie Gritton
mailto:ja...@freebsd.org>
<mailto:ja...@freebsd.org <mailto:ja...@freebsd.org>>> wrote:

   The sysctls not only don't get written to, they don't have
any useful
   information to read either. They only describe the existence
and format
   of the various jail parameters. Sorry, but there;s no way to
set a
   default children.max parameter or inherit it from the parent.
We've
   decided to set the default to the most secure/restrictive in
many cases.
   Once we've come up with a new jail configuration interface,
this won't
   be such a hassle.

   The devfs errors are probably something that will have to be
addressed
   in a later revision - I haven't looked in the devfs direction
so I'm not
   sure about that. The mount error may be related to the first
jail's
   allow.mount parameter (whose default comes from
   security.jail.mount_allowed).

   - Jamie

   Edwin Shao wrote:

   Thanks, that worked for me.

   * Using jail to change children.max on the parent does not
   affect `sysctl security.jail.param.children.max` in the
child.
Also security.jail.param.children.cur never changes
either. Not
   sure if that's intended behavior.
   * Is there any way to persist the
   security.jail.param.children.max parameter without
entering the
   jail command every time? * I get the following output when I
   create a jail inside a jail:

   hyper ~> ezjail-admin start neko
   Configuring jails:.
   Starting jails:devfs rule: ioctl DEVFSIO_RGETNEXT:
Operation not
   permitted
   devfs rule: ioctl DEVFSIO_RGETNEXT: Operation not permitted
   /etc/rc.d/jail: WARNING: devfs_set_ruleset: you must
specify a
   ruleset number
   devfs rule: ioctl DEVFSIO_SAPPLY: Operation not permitted
   ln: log: Operation not permitted
   mount: proc : Operation not permitted
neko.

   I'm using the same configuration values as in the
parent's jail,
   which work. Everything seems to work alright inside the
jail, so
   I assume the errors are safe to ignore?

   Thanks again!
   - Edwin

   On Mon, Sep 28, 2009 at 9:11 PM, Bjoern A. Zeeb
   mailto:bzeeb-li...@lists.zabbadoz.net>
   <mailto:bzeeb-li...@lists.zabbadoz.net
<mailto:bzeeb-li...@lists.zabbadoz.net>>
   <mailto:bzeeb-li...@lists.zabbadoz.net
<mailto:bzeeb-li...@lists.zabbadoz.net>
   

Re: jail(8) allow.socket_af, unknown oid

[Jamie Gritton]

The sysctls that describe available jail parameters don't always have a
type that sysctl(8) understands. In particular, the boolean parameters
are given a sysctl type of "B", and sysctl(8) will ignore them.

These aren't useful sysctls in any normal way - they never have a
meaningful value. The exist only so their types and sizes can be
determined by jail(8) and jail(3).

As per the jail(8) man page, you can use "sysctl -d" to show sysctl
descriptions without the value. Since it's only the values that
sysctl(8) doesn't understand, such parameters as allow.sock_af will then
show up.

Or, in a short answer to your last question: this isn't a tunable in the
normal sysctl way, just a jail parameter.

- Jamie


On 05/25/10 11:54, Glen Barber wrote:

The jail(8) man page has an entry under 'allow.*', allow.socket_af, which
states to allow access to protocol stacks that have not had jail functionality
added to them.

However, though socket_af exists in sys/kern/kern_jail.c, the sysctl itself
does not exist on my system:

 orion# sysctl -a | grep socket
 kern.ipc.maxsockets: 25600
 kern.ipc.numopensockets: 35
 security.jail.allow_raw_sockets: 0
 security.jail.socket_unixiproute_only: 1

Is this sysctl missing, or is it not a tunable?

___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

Re: jail(8) allow.socket_af, unknown oid

[Jamie Gritton]

I think the current situation should be sufficient, where the only
mention of the parameter sysctls are the note that you can see them via
"sysctl -d security.jail.param".

The move toward jail parameters is also a move away from using sysctl
variables for the same purpose. In this new jail order, the only useful
jail-related sysctls are security.jail.jailed and
security.jail.max_af_ips, which are both mentioned in the "Sysctl MIB
Entries" section of the man page. I don't want to worry about the
sysctls that have been obsoleted by jail parameters.

- Jamie


On 05/26/10 11:48, Glen Barber wrote:

Thanks for the explanation. Would there be opposition about a patch for
jail(8) noting which sysctls are tunable by sysctl(8) and which are not?

On 5/26/10 12:57 PM, Jamie Gritton wrote:

On 05/25/10 11:54, Glen Barber wrote:

The jail(8) man page has an entry under 'allow.*', allow.socket_af,
which
states to allow access to protocol stacks that have not had jail
functionality
added to them.

[snip]

Is this sysctl missing, or is it not a tunable?

The sysctls that describe available jail parameters don't always have a
type that sysctl(8) understands. In particular, the boolean parameters
are given a sysctl type of "B", and sysctl(8) will ignore them.

These aren't useful sysctls in any normal way - they never have a
meaningful value. The exist only so their types and sizes can be
determined by jail(8) and jail(3).

As per the jail(8) man page, you can use "sysctl -d" to show sysctl
descriptions without the value. Since it's only the values that
sysctl(8) doesn't understand, such parameters as allow.sock_af will then
show up.

Or, in a short answer to your last question: this isn't a tunable in the
normal sysctl way, just a jail parameter.

___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

Re: jid's not refreshing after jail shutdown

[Jamie Gritton]

On 06/14/10 07:36, Jacob Whatley wrote:


On 8.0 Release, our system is setup so that we specify the jid when a jail
is built, which we make numerical (ie, 1001, 1002, etc..). This works great
for consistency and for setup scripts. However, we are running into a
problem that after shutting down a jail (or group of jails) using 'jail -r
', when we attempt to start the jail(s) anew, we get the error:

jail: jail  already exists

Running jls shows no active jail for that jid. Is there something that we
can do to flush whatever buffer is holding the old jid data, so that we may
restart a dead jail with the same jid? Incidentally, this happens more often
than not, but occasionally and after a few minutes we can run 'jail -c
.' and it will not bomb.


After you remove a jail, it still exists in a hidden state until nothing
in the kernel refers to anymore. There one big thing that can hold on to
a jail for a few minutes are it's been removed: TCP timers. Once the
last tcp connection times out, the jail is let go and you can re-use its ID.

You can see these half-dead jails with the "-d" flag to jls(8).
Unfortunately, you can't shortcut that timeout, or change an existing
jail's jid.

There is one thing you can do, that may fit your needs. With the "-d"
flag to jail(8), you can make changes to such a dying jail, including
setting the "persist" parameter which will bring it back to life, or
running a command which will make it alive until the command completes
(as with a normal newly created jail). Even though the jail appears to
not exist, it actually does and so you'll need to use the "-m" flag to
modify an existing jail. Of course it may have gone away, so you're best
off with "-cm". You also have to be aware that any other parameters
you've set before may be still in that state (if the jail hasn't gone
away and you're bringing it back to life) or may not (if it went away
and you're creating a new one).

- Jamie
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

Re: docs/96807: document security.jail.list sysctl in jail(8)

[Jamie Gritton]

Actually, I suspect the change was made intentionally.
security.jail.list is obsoleted by jail_get(2), which can show jail
parameters that the struct xprison doesn't include. So using either
jail_get(2) or jailparam_get(3) programatically, or jls from the command
line is a better solution than security.jail.list.

For similar reasons, I removed mention of such MIBs as
security.jail.mount_allowed, which while they still exist serve only to
(incompletely) duplication the function of certain jail parameters.

- Jamie


On 06/23/10 09:07, Anton Yuzhaninov wrote:

After this PR: http://www.freebsd.org/cgi/query-pr.cgi?pr=96807
security.jail.list sysctl description was added to jail(8).

but in this commit:
http://svn.freebsd.org/changeset/base/192896
this text was deleted (probably accidentally).

Please return security.jail.list description to jail(8) man page.

___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

Thoughts on jail.config

[Jamie Gritton]

The rc system is becoming increasingly unable to handle the newer jail
features.  We've held off patching /etc/rc.d/jail for new parameters,
with the promise of something better.  Here's my outline of what I hope
will be in fact better than what we have now.

I'm working on extending jail(8) to use a configuration file that would
have everything currently in a $jail_XXX variable in rc.conf.
/etc/rc.d/jail would ideally be reduced to a single "jail -c" call for
startup and "jail -r" for shutdown, though I'm not sure if thing will go
quite that far.

I'm using the state of the art in config files, the C-style already used
by apmd and devd in /etc, as well as many non-core programs.  Each section
would be a jail name, and within the sections would be the jail parameters,
or pseudo-parameters known by the program.

foo {
 host.hostname = "foo.bar";
 path = "/usr/jail/foo";
 ip4.addr = "11.22.33.44";
}

The "name" parameter is implicit.  Adding an actual name explicitly in
the definition may work if you want it different for some reason, though
I haven't yet worked out how well that would work.

You can also have default parameters, defined at the top level or in a
pseudo-jail called "*".  The reason for that is you can also have
defaults that apply only to some jails, as a hierarchical feature.  So
you could have a section "foo.*" that would have parameters for any jail
under "foo".

Parameters can include other parameters as shell-style variables.  This
use useful for defining defaults based on the jail name.  A common use I
expect is:

path = "/usr/jail/$name";

This would allow you to set up default parameters as templates.  This
variable substitution also works the other way.  Consider the global
variable:

$prefix = "10.1.1";

foo {
ip4.addr = "$prefix.3"
}

Note difference to the previous example.  The variable is defined as
"$prefix = ...", not as "prefix = ...".  That means it won't be included
as a jail parameter (since there is no parameter called "prefix").

When this setup, you should be able to fully specify a jail with most of
the work done on the global end, and the per-jail parameters needing
only the parts that actually vary between jails.

In addition to the known jail parameters, there are pseudo-parameters
that don't get passed to jail_set(2), but have some use in setting up
the jail on the userland side.  The current jail(8) already has the
"command" pseudo-parameter, that specifies something to run (typically
"sh /etc/rc") after the jail is created.

I have done very little work with these pseudo-parameters so far, and
they're still mostly up in the air.  From recent conversation on the
jail list, I've added "depend", which can specify that a jail is not to
start until another jail has been set up.

The other pseudo-parameters come from what /etc/rc.d/jail currently
does.  Many of these have to do with commands run at different stages in
the setup.  Here's the current shell settings I have able to pull from
that file:

Commands:

 exec_prestart: run outside jail before create
 exec_start: single command run inside jail upon creation
  same as "command" parameter
 exec_afterstart: run inside jail after create, each in its own "jexec"
 exec_poststart: run outside jail after create (after exec_afterstart)
 exec_prestop: run outside jail before destroy
 exec_stop: run inside jail before destroy
 exec_poststop: run outside jail after destroy

Other:

 interface: interface to create/destroy all jail's IPs on
 fib: setfib ID
 devfs_enable: mount a /dev
  devfs_ruleset: /dev ruleset
 fdescfs_enable: mount a /dev/fs
 procfs_enable: mount a /proc
 mount_enable: mount arbitrary filesystems
  fstab: filesystems to mount
 consolelog: where to redirect start/stop command output

Some of these parameters could use some cleaning up, and are only the
way they are because of the constraints of the sh-based rc system.
Notably "fib" may be better worked into the kernel as a true jail
parameter.  I wouldn't expect a one-for-one transfer of all these
parameters from /etc/rc.d/jail into jail(8), but I'd want to provide
their functionality in whatever way works best.

This is where I start to need input.  What works best?  I was rather
surprised at the proliferation of exec_* specifiers in the rc system
(including the recent request for yet another), and I'm not sure what
the real needs are for such things.  The various filesystem parameters
could probably be (mostly) merged into a single per-jail fstab, or
perhaps a "mount" pseudo-parameter.  Doubtless this config file format
will grow with time, but I'd like it to get as clean a start as
possible.

Right now I don't have code I can share.  I've made the code to read the
config files, but not to do anything with them yet.  But as I firm up
just what configuration options will exist, something runnable should
soon follow.

I'm interested in hearing the needs of jail users, to make sure I do the
right thing

Re: Thoughts on jail.config

[Jamie Gritton]

On 06/24/10 06:43, Alexander Leidinger wrote:


On Wed, 23 Jun 2010 13:48:28 -0600 Jamie Gritton
wrote:

>

The rc system is becoming increasingly unable to handle the newer jail
features.  We've held off patching /etc/rc.d/jail for new parameters,
with the promise of something better.  Here's my outline of what I
hope will be in fact better than what we have now.


I'm not sure from your explanation if your new setup allows ezjail to
mangage jails as easy as it is now. If the new jail command will have
an option to specify a config file, and the jail command only operates
on the jails of this config file and ignores other jails which are
already running (e.g. on a shutdown request), your new system looks
like it is easy to use with ezjail.


Yes, you'll be able to specify a config file via the command line, with
a default of /etc/jail.conf.

Jails that exist outside of the config file's knowledge are a tricky
point, and the problems are really only on a shutdown request. While I
haven't coded this part of things yet, I've considered that I'll need
two different kinds of blanket shutdowns: one for all the jails in the
config file, and another for all jails in the system. The latter would
be the most sensible to use during system shutdown, when it doesn't make
sense to leave any jails running. But orderly shutdown is part of the
config spec (e.g. running "/bin/sh /etc/rc.shutdown"), and it may be
best to assume that if the jails were created outside of the rc system,
they'll be removed in the same way.

So in short, I think it will be compatible with ezjail.


Another point which interests me is how your new way of doing things
will handle things like allow.raw_sockets. Assume I have some kernel
modification which adds allow.XXX, do I need to modify the parsing of
the jail command to handle this, or will this work transparently
without userland modifications?


That will work transparently, as does the current jail(8) command line.
The only time you'd need to modify userland tools for a new jail
parameter is if that parameter has a data type the tools don't
understand. Most parameters operate on numbers or strings, but for
example IP addresses are passed in binary and userland needs to know how
to convert them to/from strings.

- Jamie
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

Re: Thoughts on jail.config

[Jamie Gritton]

On 06/28/10 08:24, Alexander Leidinger wrote:

Quoting Jamie Gritton  (from Thu, 24 Jun 2010
10:30:42 -0600):

On 06/24/10 06:43, Alexander Leidinger wrote:

>>

Jails that exist outside of the config file's knowledge are a tricky
point, and the problems are really only on a shutdown request. While I
haven't coded this part of things yet, I've considered that I'll need
two different kinds of blanket shutdowns: one for all the jails in the
config file, and another for all jails in the system. The latter would
be the most sensible to use during system shutdown, when it doesn't make
sense to leave any jails running. But orderly shutdown is part of the
config spec (e.g. running "/bin/sh /etc/rc.shutdown"), and it may be
best to assume that if the jails were created outside of the rc system,
they'll be removed in the same way.


There are two additional sides:
1) For jails which are created by example via ezjail I agree that it is
within the responsability of the ezjail to shut them down.
2) For jails which are created/started by hand from a custom config file
for testing purposes, I think a "shutdown all remeaining jails even if
there is not entry in the config file" would be good. The problem with
this is, that you need to make assumptions how to do a shutdown, or
record this info in the kernel on creation time (and use this only if no
config with appropriate info is available).


If any jails are left on shutdown by the time rc.d/jail gets to them,
they would have to be summarily killed. I wouldn't want to make
assumptions about scripts and the like in the absence of the config
lines, since I assume there's a reason they weren't started withing the
jail.conf system.

When you remove a jail via jail_remove(2), it sends a SIGKILL to every
process inside it. I could at least first send them a SIGTERM and give
them a little while to clean up first. But I still wouldn't to run a
script that wasn't specified by the jail creator, which is at this point
necessarily unknown.

So yes, I'd have a "shutdown all jails" option for this.

- Jamie
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

Re: Thoughts on jail.config

[Jamie Gritton]

On 06/28/10 08:41, Rodrigo Mosconi wrote:


An idea: if it works like a "jaild"? A daemon management the start-up,
shutdown, console redirection?  All the admins task could be done by a
"jailctl"?


I don't know what work a daemon would have to do. I only see it running
tasks on startup, and then waiting until something tells it on shutdown
to wake up and stop the jails. That "something" would have to be that
jailctl you mention. If there's a jail program running anyway, might as
well keep all functionality in that one program.

- Jamie
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

Re: Thoughts on jail.config

[Jamie Gritton]

On 06/29/10 04:21, Bjoern A. Zeeb wrote:


One functionality I forgot about but was asked for in the past was
"jail reboot" so that an admin could "restart" a jail completly from
within the jail. The question is whether we may want a "jailinit" (an
init running inside the jail) for that or if we want to handle it from
the outside.


I like the idea of a jailinit, and have had success running our own
stuff that way. But in the meantime, I could do the restart via
userspace - just run the shutdown, wait for the jail to go away, and
then start up again. This is in fact something I was planning on.

- Jamie
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

Re: selective jail restriction controlling in rc.conf

[Jamie Gritton]

On 07/04/10 10:10, Harald Schmalzbauer wrote:

Dear freebsd-jail fellows,

I haven't know of that list yet, nor am I subscribesd, but I did some
work for me to extend rc.d/jail to acclompish with some of my needs and
I'd like to share it.
I don't have much knowledge to join seriouse developement, I'm just
"playing". But I'm sure you can understand my intention of the patch and
maybe take some idea.

Here's my original post to freebsd-stable@:

I very much liked the possibillity to easily manage jails via rc.conf.
Unfortunately I was missing some features.
First, there are many security.jail.allow_* sysctl which didn't get
attention.
Second; I needed to allow different things on different jails. For
example only one distinct jail should habe sysvIPC.

Please find attached a patch wich extends rc.d to my needs.
Some jail_start() modifications were neccessary and some cleanups could
be done in the "Configuring jails:" section (not needed any more) amd in
the _ip_multi processing, since that's not needed any more.
One have to seperatly define ip4 and ip6 addresses. The can be with or
without mask, single oder comma seperated list, doesn't matter, thanks
to the jail_handle_ips_option() coder, it just works :)


The new jail(8) syntax is able handle your second concern, allowing
features on only some jails. I'm currently working on an update that
will use a jail.conf file instead of the rc-based shell variables
currently in use; because of that, there are no plans to keep hacking on
the rc variables. As for the first concern, the sysctl.jail.allow_*
sysctls, those are obsoleted by the new jail system as well. While they
will continue to exist in the (at least near) future, they're being
deprecated for just the reason you mention, that they don't allow
per-jail control.

- Jamie
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

Re: libjail issues.

[Jamie Gritton]

On 07/15/10 04:12, Stanislav Uzunchev wrote:

 I have found something very strange to me... It is a problem with static
allocating size of buffer where jail param is going to be coppied, using
jail_getv function from the libjails. Well for example:

buff[size];
jail_getv(0, "name", "1", "host.hostname", buff, NULL);

the result for size = 64; is bsnmp.test, size = 257; is bsnmp.test, size =
256; is bs1 ?

#sysctl -a | grep "security.jail.param.host.hostname:"
256

#jls
JID Hostname
1 bsnmp.test

#jls -n | grep "bs1"
returns no match.

This is really confusing me.


That's a bug in jail_getv, which I'm committing the fix for now.
Unfortunately, it's too late to get it in the 8.1 release, but it will
at least be in future releases.

The issue is that jail_getv wrongly allocated temporary space based on
the length of the parameters passed in, which don't even have values
yet. The different array sizes in your sample code would coincidentally
locate the "buff" array with different garbage contents, leading to
different behavior with the bug.


Also i will take suggestions, what is the best way, to get and set all
value/params using the jailparam struct.
I am trying first to set the name or jid, and after that getting the values
for the rest parameters with jail_getv, but i face some problems setting
jp_value since it is type (void *).


You don't value to set jp_value directly, but use the jailparam_import
function. That will set jp_value correctly based on the parameter's type.

- Jamie
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

First stab at a new jail(8)

[Jamie Gritton]

I've got code for a config-based jail(8) at 
http://people.freebsd.org/~jamie/jail.tbz .
It drops in under /usr/src/usr.sbin, but is a big enough change from the 
current sources that I didn't bother with a diff.


I haven't yet updated the man page for it, so I'll give a quick overview 
here...


Its syntax is an extension of the current jail(8), which is itself an 
extension of the previous jail(8).  In addition to starting a single 
jail with "-c name=value ... command=do something here" or modifying one 
with "-m name=value ...", you can just specify a single jail name on the 
command line and it will operate on that jail from the config file.


So "jail -c foo" will start up the jail "foo" from /etc/jail.conf (or 
whatever file you specify with "-f").  Just saying "jail -c" will start 
up all jails in the config file if they aren't already running.  More 
generally, when running from the config file you can do the following:


jail -c [jailname]
Start the specified jail, or all jails.  Note that  is a 
single argument.  You can't start multiple jails at once this way (e.g. 
jail -c firstjail secondjail), because if you put two arguments, jail(8) 
will think you're specifying a jail on the command line with parameters 
like "firstjail" and "secondjail".  You can start multiple jails with 
simple wildcards: specifying "foo.*" will start jails that start with 
"foo.".  This isn't a regular expression or globbing, but wildcarding 
entire name components.


jail -m [jailname].
Modify parameters of the specified jail, or all jails.  It will set the 
jail to whatever the current parameters in the config file are.  Some 
parameters, like "path", can only be set on jail startup.  If these are 
the same in the config file and the currently running jail, it will 
silently skip them.  If they're different, it will report and error and 
not update the jail.


jail -r 
In this case, the jail name isn't optional, because I thought it would 
be too easy to accidentally remove everything.  If you want to remove 
all jails, you can say "jail-r '*'".  If you specify a wildcard, it will 
apply only to jails in the config file.  But unlike -c and -m, if you 
specify a single jail, it will first look it up in the config file but 
then back up to looking at currently running jails.  Thus "jail -r 47" 
will remove the jail with jid 47 just like it currently does, unless you 
have a jail called "47" in the config file.


jail -R 
Similar to jail -r, but this doesn't use the config file at all.  The 
jailname must be the name or jid of a running jail, or a wildcard.  In 
this case, the wildcards also apply to running jails, so "jail -R '*'" 
will remove all current jails.  Since the config file isn't used, no 
shutdown scripts will be run and every jail will be removed "hard".


jail -cm [jailname]
Like the "-cm" flags of the current version, this will create a jail if 
it doesn't exist, or update it if it does exist.  This will make sure 
every jail in the config file is up and running.


jail -rc [jailname]
This combination of -r and -c restarts jails - first the entire remove 
procedure then the entire create procedure.


jail -mr [jailname]
jail -cmr [jailname]
These variations on -m and -cm will restart a jail if necessary - 
instead of exiting on an error when attempting to change a create-only 
parameter, it will restart the jail.  The -cmr option, which can also 
create new jails, can make sure every jail is in exactly the same state 
as specified in the config file.


I've described the format of the config while a while back, and I'm 
including a small sample config file in the tarball.  Right now it 
handles all the regular jail parameters, as well as some internal 
pseudo-parameters mostly taken from the current rc.d/jail:


allow-dying
Same as the -d flag, allow modifying a dying jail.

depend
Make a jail's startup depend on another jail. The jail won't be started 
until every jail it depends on is successfully started first.  If you 
specify starting a single jail on the command line and it has 
dependencies, its dependencies are implicitly included.  If you modify a 
jail with dependencies, it will make any changes in dependency order, 
but will not actually modify jails that aren't specified.  The 
dependencies apply in reverse order for removing jails.


ip_hostname
Same as the -h flag, implicitly set the jails' ip4.addr and ip6.addr 
parameters based on a DNS lookup of the host.hostname parameter.


exec.prestart
exec.start
command
exec.afterstart
exec.poststart
The same as used in rc.d/jail, except that there's also a "command" 
parameter as in the current jail(8).  The specified programs are 
executed in the order given, and each parameter can specify multiple 
programs (using the "+=" specifier in the config file).  The jail is 
created after exec.prestart is run.  The exec.start, command, and 
exec.afterstart programs are run inside the jail; the exec.prestart and 
exec.poststart programs are run in the h

Re: jail_attach does not chdir to new root?

[Jamie Gritton]

On 11/03/10 00:56, Nikos Vassiliadis wrote:


Out of curiosity, why jail_attach() does not
chdir() to the new root?

It seems like something worth mentioning in jail(2).


I wasn't involved in the early jail stuff, but I'll venture it was
because chroot(2) also doesn't chdir to the new root, and jail
originally grew out of chroot.

The difference is the chroot(2) man page does specifically mention that
the current directory isn't changed, and you're right that jail(2)
should as well.

- Jamie
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

Re: rc.d/jail issues

[Jamie Gritton]

It's in the the subversion tree, under projects/jailconf.

I've got the dependency stuff there (actually turns out to be a large
chunk of the code). I've got it doing almost everything that rc.d/jail
does now, though it doesn't yet handle errors like it should. After I
get that fixed up, I plan on putting it in HEAD. After that, I still
have a todo list mostly of suggestions from others.

Feel free to give me any "todo" suggestions, or any other feedback :-).

- Jamie


On 01/27/11 10:18, Dirk Engling wrote:

On Thu, 27 Jan 2011, Jamie Gritton wrote:


That's where it's headed. I've been slow on progress lately, but I'm
working on a jail(8) that takes a config file instead of rc shell
variables, and takes care of dependency issues among other things.


Looks like I completely missed the discussion that went on in June this
year. If I can be of any assistance working on a new jail(8), I'd be too
happy to know how I can help. Is the progress visible somewhere?

___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

Re: rc.d/jail issues

[Jamie Gritton]

That's where it's headed. I've been slow on progress lately, but I'm
working on a jail(8) that takes a config file instead of rc shell
variables, and takes care of dependency issues among other things.

- Jamie


On 01/27/11 08:42, Dirk Engling wrote:

On Thu, 27 Jan 2011, Paul Schenkeveld wrote:


like to start A before B but shutdown B before A. Would it make sense
to reverse the order in which jails are stopped during shutdown by
reversing the nales in $jail_list?


Yikes, it does indeed make sense, that's why I already do it in the
ezjail utility, besides having jail dependencies worked out by rcorder...

Now, if /etc/rc.d/jail starts to reverse the list ezjail reverses
before, I'm in trouble. Maybe it's time to think about moving some more
jail abstraction - including jail dependencies - to the base system.

___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

Re: rc.d/jail issues

[Jamie Gritton]

No - this is entirely a user-space project. Those are both things I'd
like to add to jails after I get through the mess on the ofhter side of
the kernel divide.

- Jamie

On 01/27/11 13:30, Kostik Belousov wrote:

On Thu, Jan 27, 2011 at 10:37:22AM -0700, Jamie Gritton wrote:

It's in the the subversion tree, under projects/jailconf.

I've got the dependency stuff there (actually turns out to be a large
chunk of the code). I've got it doing almost everything that rc.d/jail
does now, though it doesn't yet handle errors like it should. After I
get that fixed up, I plan on putting it in HEAD. After that, I still
have a todo list mostly of suggestions from others.

Feel free to give me any "todo" suggestions, or any other feedback :-).

Are per-jail init and console in the list ?

___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

Re: [jail][vnet] wlandebug inside jail, operation not permitted

[Jamie Gritton]

This is by design. The "root" in a jail isn't quite a full-featured
root, and in particular can't do anything that affects the hardware in
ways beyond normal non-administrative use.

- Jamie

On 02/16/11 09:34, Monthadar Al Jaberi wrote:

Hej,

I have created a jail with "jail -c jid=1 vnet persist", then moved a
wlan to it "ifconfig wlan0 vnet 1" but when I try to run:
"jexec 1 wlandebug -i wlan0 state" I get :

wlandebug: sysctl-set(net.wlan.0.debug): Operation not permitted
net.wlan.0.debug: 0x22000 =>

I can run wlandebug before moving wlan to a jail, but why cant I do it
inside the jail? I am running everything as root.


thank you in advance,


___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

Re: New jail(8) with configuration files, not yet in head

[Jamie Gritton]

Oh, never good when "(null)" shows up. I'll see what I'm trying to run
that isn't there.

- Jamie


On 07/04/11 15:36, Brandon Gooch wrote:

I did however notice a minor nit in the output when removing a jail.
Here's the scenario:

...

Now, when I go to remove the jail:

# jail -r ports
Stopping cron.
Waiting for PIDS: 29824.
Terminated
.
jail: ports: (null): failed

I don't think it's anything to worry about, as the jail does indeed
get removed. I wonder though if this might be a indicator of some
underlying problem I haven't ran into yet with my simplistic testing.

___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

Re: New jail(8) with configuration files, not yet in head

[Jamie Gritton]

This is code that notes error return codes from processes run under the
jail (e.g. the exec.start script). But in this case, it's reporting an
error from a process that was part of the jail's shutdown. The reason
the command is "(null)" is this wasn't a command started from jail(8)
itself.

I've fixed that dual-use bit of code to stay silent about existing
processes that die as part of jail shutdown. A small patch for that is
attached.

- Jamie


On 07/04/11 15:36, Brandon Gooch wrote:
> I did however notice a minor nit in the output when removing a jail.
> Here's the scenario:
>
> I set out with a new jail.conf(5) file:
>
> exec.start = "/bin/sh /etc/rc";
> exec.stop = "/bin/sh /etc/rc.shutdown";
> exec.clean;
> mount.devfs;
>
> ports {
>  path = "/usr/jails/$name";
>  ip4.addr = 10.1.1.1;
> }
>
> I run the jail creation command:
>
> # jail -c ports
> ports: created
> /etc/rc: WARNING: $hostname is not set -- see rc.conf(5).
> Creating and/or trimming log files.
> ln: /dev/log: Operation not permitted
> Starting syslogd.
> ELF ldconfig path: /lib /usr/lib /usr/lib/compat
> 32-bit compatibility ldconfig path: /usr/lib32
> Clearing /tmp (X related).
> Updating motd:.
> Starting sshd.
> Starting cron.
> Starting ftpd.
>
> Mon Jul  4 21:21:30 UTC 2011
>
> ...and the jail is running:
>
> # jls
> JID  IP Address  Hostname  Path
>   3  10.1.1.1  /usr/jails/ports
>
> Now, when I go to remove the jail:
>
> # jail -r ports
> Stopping cron.
> Waiting for PIDS: 29824.
> Terminated
> .
> jail: ports: (null): failed
>
> I don't think it's anything to worry about, as the jail does indeed
> get removed. I wonder though if this might be a indicator of some
> underlying problem I haven't ran into yet with my simplistic testing.
Index: command.c
===
--- command.c   (revision 223747)
+++ command.c   (working copy)
@@ -150,11 +150,15 @@
if (!(j->flags & JF_SLEEPQ))
return 0;
j->flags &= ~JF_SLEEPQ;
-   if (*j->comparam != IP_STOP_TIMEOUT) {
-   paralimit++;
-   if (!TAILQ_EMPTY(&runnable))
-   requeue(TAILQ_FIRST(&runnable), &ready);
+   if (*j->comparam == IP_STOP_TIMEOUT)
+   {
+   j->flags &= ~JF_TIMEOUT;
+   j->pstatus = 0;
+   return 0;
}
+   paralimit++;
+   if (!TAILQ_EMPTY(&runnable))
+   requeue(TAILQ_FIRST(&runnable), &ready);
error = 0;
if (j->flags & JF_TIMEOUT) {
j->flags &= ~JF_TIMEOUT;
@@ -270,8 +274,8 @@
 
case IP__OP:
if (down) {
-   if (jail_remove(j->jid) == 0 && verbose >= 0 &&
-   (verbose > 0 || (j->flags & JF_STOP
+   (void)jail_remove(j->jid);
+   if (verbose > 0 || (verbose == 0 && (j->flags & JF_STOP
? note_remove : j->name != NULL)))
jail_note(j, "removed\n");
j->jid = -1;
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

Re: New jail(8) with configuration files, not yet in head

[Jamie Gritton]

The delay you're seeing is $rcshutdown_timeout (default 30 seconds)
which will kill rc.shutdown if it doesn't successfully stop all the
jail's processes. After that, it will forcibly kill all (jailed) processes.

jail(8) has a similar thing going where after any stop scripts have run,
it will send SIGTERM to remaining processes and then wait up to
stop.timeout (default 10 seconds) for them to die before removing the
jail anyway. If the jail has a stop script like /etc/rc.shutdown,
there's seldom any further delay.

- Jamie


On 07/04/11 16:13, Brandon Gooch wrote:
> Here's another take on perhaps the same
> underlying issue.
>
> If I've jexec'd into the jail, say something like this:
>
> # jexec testjail /bin/tcsh
>
> and then I try removing the jail from another terminal (jail -r
> testjail), the jail -r command hangs for about 15-20 seconds, I
> suppose waiting for the jexec'd shell to exit, before finally killing
> the jail off outright. I don't see the aforementioned behavior (jail:
> testjail: (null): failed).
>
> Here's what it looks like:
>
> # jail -r ports
> Stopping ftpd.
> Waiting for PIDS: 32845.
> Stopping cron.
> .
> Terminated
> ^T
> load: 0.19  cmd: jail 32860 [kqread] 1.05r 0.00u 0.00s 0% 1696k
> ^T
> load: 0.09  cmd: jail 32475 [kqread] 10.09r 0.00u 0.00s 0% 1692k
>
> ...then testjail is finally "Killed". Some sort of race condition maybe?
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

Re: New jail(8) with configuration files, not yet in head

[Jamie Gritton]

This project came from a desire to improve the jail startup procedure in 
rc.d/jail, which remains stuck handling the old fixed-parameter jails. 
Rather that continue to extend an already unwieldy number of rc.conf 
shell variables, I opted to add a configuration file like other 
subsystems use (e.g. apmd, devd).  The new jail pseudo-parameters added 
to the config file exist mostly to match the existing rc.d/jail 
functionality - the mount.* and exec.* parameters are direct analogs to 
rc.conf shell variables.  Some other parameters match the command-line 
options of the existing jail(8).


I have tried to avoid bloating the command with new features that don't 
have a reasonably broad base; the only really new things I can think of 
are the "depend" parameter (dependency graph of jails, as you mentioned 
in your footnote [2]) and the "vnet.interface" parameter.


I wouldn't want to do away with a config file, as that's a much cleaner 
way to define multiple jails than depending on the rc system or 
requiring a "roll your own" approach that is currently the only way to 
use the newer features.


Since jail creation is so tied to processes running under them, the 
exec.* parameters as originally defined by the rc system are best kept; 
while a single "command" parameter would suffice for many (perhaps most) 
jails, the flexibility to run commands as either system or jail users, 
either before or after jail creation, seems an advantage worth keeping. 
 The other big rc bits - mounting filesystems and preparing network 
interfaces, seem sufficiently broad-based to include for the general 
populace.


There are still featured I'd like to add, mostly in flexibility of the 
config file - including files, better handling of auto-generated JIDs, 
stuff like that.  I don't really see adding anything else that isn't 
tied to the basic definition of jails, and of starting them, stopping 
them, or changing their parameters - all of which the current 
command-lines usage already does, though not as well.


It's clear now that this won't be happening in 9.0.  So none of this is 
in danger of getting pushed through in a hurry.


- Jamie


On 07/18/11 13:08, Paul Schenkeveld wrote:

Hi,

On Sun, Jul 03, 2011 at 11:24:57PM -0600, Jamie Gritton wrote:

I'm hoping to get the latest version of jail(8) in before the door slams
shut on 9.0.  If anyone wants to take a look at the new code and give it
a spin, it may help to ease RE's mind about my tardiness.  The included
diff applies to the current usr.sbin/jail directory.  In addition to the
new program, it adds a jail.conf(5) man page that explains the config
file format (hint: it's a typical C-style block config).


Although I really like this new functionality, there is one issue that
I am concerned about.  Should all this functionality be integrated into
the jail(8) command?

In UNIX tradition we have simple interfaces[1] in the base system and
jail(8) used to fit in quite well.  Your new jail(8) tries to be an all
singing and dancing subsystem for managing jails but there will always
be users that need one more feature[2].  That's why we have ezjail,
jailer, jailadmin and others in ports and over time there will be new
ones with new features.

Jail(8) is the one FreeBSD command to create, modify and destroy jails
and is used by many of these wrappers.  Your new jail(8) covers part
of the functionality of these wrapper but not all.  It may grow new
features as people submit ideas and you or someone else is willing to
code the feature but there will always be wrappers to suit needs not
covered by this jail command.

I don't want to start a bikeshed discussion and will shut up immediately
if I appear to be the only one concerned but if others care like I do,
I'd suggest to put this functionality into a separate program and leave
jail(8) to be the minimalistic interface to just control the jail(2)
system call from a command interpreter.

Again, I'm not against your enhancements, I'm just worried about
pollution of OS primitives.

With kind regards,

Paul Schenkeveld

[1] Chown/chgrp/chmod are nice examples, I could easily think of at
 least a dozen enhancements like only operating on files matching a
 specific user/group or mapping a list of old uids/gids to a list of
 new ones or looking up user info in LDAP.  Sometimes throwing in
 find(1) will help you solve the problem at hand, sometimes you need
 to write a script or program.

[2] My favorites: starting jails in parallel, dependency graph of jails,
 monitoring jails and restarting them when they die, migration of
 jails to other hosts and populating new jails from a release DVD,
 /usr/src or the source repository and provisioning jails from a
 database.
___
freebsd-jail@freebsd.org mailing list
http:

Re: debugging frequent kernel panics on 8.2-RELEASE

[Jamie Gritton]

On 08/20/11 19:19, Steven Hartland wrote:

- Original Message - From: "Andriy Gapon" 


on 20/08/2011 23:24 Steven Hartland said the following:

- Original Message - From: "Steven Hartland"

Looking through the code I believe I may have noticed a scenario
which could
trigger the problem.

Given the following code:-

static void
prison_deref(struct prison *pr, int flags)
{
struct prison *ppr, *tpr;
int vfslocked;

if (!(flags & PD_LOCKED))
mtx_lock(&pr->pr_mtx);
/* Decrement the user references in a separate loop. */
if (flags & PD_DEUREF) {
for (tpr = pr;; tpr = tpr->pr_parent) {
if (tpr != pr)
mtx_lock(&tpr->pr_mtx);
if (--tpr->pr_uref > 0)
break;
KASSERT(tpr != &prison0, ("prison0 pr_uref=0"));
mtx_unlock(&tpr->pr_mtx);
}
/* Done if there were only user references to remove. */
if (!(flags & PD_DEREF)) {
mtx_unlock(&tpr->pr_mtx);
if (flags & PD_LIST_SLOCKED)
sx_sunlock(&allprison_lock);
else if (flags & PD_LIST_XLOCKED)
sx_xunlock(&allprison_lock);
return;
}
if (tpr != pr) {
mtx_unlock(&tpr->pr_mtx);
mtx_lock(&pr->pr_mtx);
}
}

If you take a scenario of a simple one level prison setup running a
single
process
where a prison has just been stopped.

In the above code pr_uref of the processes prison is decremented. As
this is the
last process then pr_uref will hit 0 and the loop continues instead
of breaking
early.

Now at the end of the loop iteration the mtx is unlocked so other
process can
now manipulate the jail, this is where I think the problem may be.

If we now have another process come in and attach to the jail but
then instantly
exit, this process may allow another kernel thread to hit this same
bit of code
and so two process for the same prison get into the section which
decrements
prison0's pr_uref, instead of only one.

In essence I think we can get the following flow where 1# = process1
and 2# = process2
1#1. prison1.pr_uref = 1 (single process jail)
1#2. prison_deref( prison1,...
1#3. prison1.pr_uref-- (prison1.pr_uref = 0)
1#3. prison1.mtx_unlock <-- this now allows others to change
prison1.pr_uref
1#3. prison0.pr_uref--
2#1. process1.attach( prison1 ) (prison1.pr_uref = 1)
2#2. process1.exit
2#3. prison_deref( prison1,...
2#4. prison1.pr_uref-- (prison1.pr_uref = 0)
2#5. prison1.mtx_unlock <-- this now allows others to change
prison1.pr_uref
2#5. prison0.pr_uref-- (prison1.pr_ref has now been decremented
twice by prison1)

It seems like the action on the parent prison to decrement the
pr_uref is
happening too early, while the jail can still be used and without
the lock on
the child jails mtx, so causing a race condition.

I think the fix is to the move the decrement of parent prison
pr_uref's down
so it only takes place if the jail is "really" being removed. Either
that or
to change the locking semantics so that once the lock is aquired in
this
prison_deref its not unlocked until the function completes.

What do people think?


After reviewing the changes to prison_deref in commit which added
hierarchical
jails, the removal of the lock by the inital loop on the passed in
prison may
be unintentional.
http://www.freebsd.org/cgi/cvsweb.cgi/src/sys/kern/kern_jail.c.diff?r1=1.101;r2=1.102;f=h



If so the following may be all that's needed to fix this issue:-

diff -u sys/kern/kern_jail.c.orig sys/kern/kern_jail.c
--- sys/kern/kern_jail.c.orig 2011-08-20 21:17:14.856618854 +0100
+++ sys/kern/kern_jail.c 2011-08-20 21:18:35.307201425 +0100
@@ -2455,7 +2455,8 @@
if (--tpr->pr_uref > 0)
break;
KASSERT(tpr != &prison0, ("prison0 pr_uref=0"));
- mtx_unlock(&tpr->pr_mtx);
+ if (tpr != pr)
+ mtx_unlock(&tpr->pr_mtx);
}
/* Done if there were only user references to remove. */
if (!(flags & PD_DEREF)) {


Not sure if this would fly as is - please double check the later block
where
pr->pr_mtx is re-locked.


Your right, and its actually more complex than that. Although changing
it to
not unlock in the middle of prison_deref fixes that race condition it
doesn't
prevent pr_uref being incorrectly decremented each time the jail gets into
the dying state, which is really the problem we are seeing.

If hierarchical prisons are used there seems to be an additional problem
where the counter of all prisons in the hierarchy are decremented, but as
far as I can tell only the immediate parent is ever incremented, so another
reference problem there as well I think.

The following patch I believe fixes both of these issues.

I've testing with debug added and confirmed prison0's pr_uref is maintained
correctly even when a jail hits dying state multiple times.

It essentially reverts the changes to the "if (flags & PD_DEUREF)" by
192895 and moves it to after the jail has been actually removed.

diff -u sys/kern/kern_jail.c.orig sys/kern/kern_jail.c
--- sys/kern/kern_jail.c.orig 2011-08-20 21:17:14.856618854 +0100
+++ sys/kern/kern_jail.c 2011-08-21 01:56:58.429894825 +0100
@@ -2449,27 +2449,16 @@
mtx_lock(&pr->pr_mtx);
/* Decrement the user references in a separate loop. */
if (flags & PD_DEUREF) {
- for

Re: debugging frequent kernel panics on 8.2-RELEASE

[Jamie Gritton]

On 08/21/11 05:01, Steven Hartland wrote:

- Original Message - From: "Jamie Gritton" 

The problem isn't with the conditional locking of tpr in prison_deref.
That locking is actually correct, and there's no race condition.


Are you sure? I do think that unlocking the mtx half way through the
call allows the above scenario to create a race condition, all be it
very briefly, when ignoring the overriding issue.

In addition if the code where changed to so that the pr_uref++ also
maintained the parents uref this would definitely lead to a potential
problems in my mind, especially if you had more than one child prison,
of a given parent, entering the dying state at any one time.

In this case I believe you would have to acquire the locks of all
the parent prisons before it would be safe to precede.


Lock order requires that I unlock the child if I want to lock the
parent. While that does allow periods where neither is locked, it's safe
in this case. There may be multiple processes dying in one jail, or in
multiple children of a single jail. But as long as a parent jail is
locked while decrementing pr_uref, then only one of these simultaneous
prison_deref calls would set pr_uref to zero and continue in the loop to
that prison's parent. This might be mixed with pr_uref being incremented
elsewhere, but that's not a problem either as long as the jail in
question is locked.


The trouble lies in the resurrection of dead jails, as Andriy has noted
(though not just attaching, but also by setting its persist flag causes
the same problem).


I not sure that persistent prisons actually suffer from this in any
different way tbh, as they have an additional uref increment so would
never hit this case unless they have been actively removed and hence
unpersisted first.


Right - both the attach and persist cases are only a problem when a jail
has disappeared. There are various ways for a jail to be removed,
potentially to be kept around but in the dying state, but only two
related ways for it to be resurrected: attaching a new process or
setting the persist flag, both via jail_set with the JAIL_DYING flag passed.


There are two possible fixes to this. One is the patch you've given,
which only decrements a parent jail's pr_uref when the child jail
completely goes away (as opposed to when it loses its last uref). This
provides symmetry with the current way pr_uref is incremented on the
parent, which is only when a jail is created.

The other fix is to increment a parent's pr_uref when a jail is
resurrected, which will match the current logic in prison_deref. I like
the external semantics of this solution: a jail isn't visible if it is
not persistent and has no processes and no *visible* sub-jails, as
opposed to having no sub-jails at all. But this solution ends up pretty
complicated - there are a few places where pr_uref is incremented, where
I might need to increment parent jails' pr_uref as well, much like the
current tpr loop in prison_deref decrements them.


Ahh yes in the hierarchical case my patch would indeed mean that none
persistent parent jails would remain visible even when its last child
jail is in a dying state.

As you say making this not the case would likely require replacing all
instances of pr_uref++ with a prison_uref method that implements the
opposite of the loop in prison_dref should the prisons pr_uref be 0 when
called.


Yes, that's the problem. Maybe not all instances, but at least most have
enough times a jail is unlocked that we can't assume the pr_uref hasn't
been set to zero somewhere else, and so we need to do that loop.


Your solution removes code instead of adding it, which is generally a
good thing. While it does change the semantics of pr_uref in the
hierarchical case at least from what I thought it was, those semantics
haven't been working properly anyway.


Good to know my interpretation was correct, even if I was missing the
visibility factor in the hierarchical case :)


Bjoern, I'm adding you to the CC list for this because the whole pr_uref
thing was your idea (though it was pr_nprocs at the time), so you might
care about the hierarchical semantics of it - or you may not. Also, this
is a panic-inducing bug in current and may interest you for that reason.


 From an admin perspective the current jail dying state does cause
confusion when your not aware of its existence. You ask a jail to stop it
appears to have completed that request, but really hasn't, an generally
due to just a lingering tcp connection.

With the introduction of hierarchical jails that gets a little worse
where a whole series of jails could disappear from normal view only to
be resurrected shortly after. Something to bear in mind when deciding
which solution of the two presented to use.


The good news is that the only time a jail (or perhaps a whole set of
jails) can only come back from the dead when the administra

Re: bin/165515: [jail][patch] "jail: unknown parameter: allow.nomount" when starting jail

[Jamie Gritton]

The allow.mount parameter recently changed in a subtle way - it's now
a node (to e.g. allow.mount.devfs) as well as a parameter in its own
right. This confused libjail which knows how to handle such parameters
as long as they're not boolean.

I'm including my proposed fix to libjail. This this fix, allow.nomount
should once again work, as should allow.mount and other things such as
allow.quotas/allow.noquotas should work as they did before.

- Jamie


On 02/27/12 18:21, Glen Barber wrote:



Number: 165515
Category:   bin
Synopsis:   [jail][patch] "jail: unknown parameter: allow.nomount" when 
starting jail
Confidential:   no
Severity:   non-critical
Priority:   low
Responsible:freebsd-bugs
State:  open
Quarter:
Keywords:
Date-Required:
Class:  sw-bug
Submitter-Id:   current-users
Arrival-Date:   Tue Feb 28 01:30:07 UTC 2012
Closed-Date:
Last-Modified:
Originator: Glen Barber
Release:10.0-CURRENT r232116M
Organization:
Environment:

FreeBSD nucleus 10.0-CURRENT FreeBSD 10.0-CURRENT #7 r232116M: Sun Feb 26 
14:12:14 EST 2012 root@nucleus:/usr/obj/usr/src/sys/NUCLEUS  amd64

Description:

On a recent -CURRENT, the jail(8) rc script fails to start jails.  This seems 
to be caused by looking up a sysctl that does not exist.

How-To-Repeat:

Create a minimalistic jail setup as follows:

  nucleus# cat /etc/rc.conf.local
  #
  jail_enable="YES"
  jail_parallel_start="YES"
  jail_list="cfm"
  jail_cfm_hostname="cfm"
  jail_cfm_rootdir="/jails/cfm"
  jail_cfm_ip="172.16.1.2"

Start the jail with the rc(8) script:

  nucleus# sh -x /etc/rc.d/jail start cfm
  + . /etc/rc.subr
  [...]
  + eval '_x="$jail_cfm_ip_multi0"'
  + _x=''
  + break
  + echo ' cannot start jail "cfm": '
   cannot start jail "cfm":
  + tail +2 /tmp/jail.bJIDqW5x/jail.52395
  jail: unknown parameter: allow.nomount


Fix:

The attached patch fixes it for me.


Patch attached with submission follows:

Index: usr.sbin/jail/jail.c
===
--- usr.sbin/jail/jail.c(revision 232116)
+++ usr.sbin/jail/jail.c(working copy)
@@ -84,7 +84,7 @@
{ "security.jail.chflags_allowed",
  "allow.nochflags", "allow.chflags" },
{ "security.jail.mount_allowed",
- "allow.nomount", "allow.mount" },
+ "allow.mount", "allow.nomount" },
{ "security.jail.socket_unixiproute_only",
  "allow.socket_af", "allow.nosocket_af" },
  };



Release-Note:
Audit-Trail:
Unformatted:

___
freebsd-b...@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-bugs
To unsubscribe, send any mail to "freebsd-bugs-unsubscr...@freebsd.org"
Index: lib/libjail/jail.c
===
--- lib/libjail/jail.c  (revision 232240)
+++ lib/libjail/jail.c  (working copy)
@@ -885,36 +885,20 @@
 * the "no" counterpart to a boolean.
 */
nname = nononame(jp->jp_name);
-   if (nname != NULL) {
-   snprintf(desc.s, sizeof(desc.s), SJPARAM ".%s", nname);
-   free(nname);
-   miblen = sizeof(mib) - 2 * sizeof(int);
-   if (sysctl(mib, 2, mib + 2, &miblen, desc.s,
-   strlen(desc.s)) >= 0) {
-   mib[1] = 4;
-   desclen = sizeof(desc);
-   if (sysctl(mib, (miblen / sizeof(int)) + 2,
-   &desc, &desclen, NULL, 0) < 0) {
-   snprintf(jail_errmsg,
-   JAIL_ERRMSGLEN,
-   "sysctl(0.4.%s): %s", desc.s,
-   strerror(errno));
-   return (-1);
-   }
-   if ((desc.i & CTLTYPE) == CTLTYPE_INT &&
-   desc.s[0] == 'B') {
-   jp->jp_ctltype = desc.i;
-   jp->jp_flags |= JP_NOBOOL;
-   jp->jp_valuelen = sizeof(int);
-   return (0);
-   }
-   }
+   if (nname == NULL) {
+   unknown_parameter:
+   snprintf(jail_errmsg, JAIL_ERRMSGLEN,
+   "unknown parameter: %s", jp->jp_name);
+   errno = ENOENT;
+   return (-1);
}
-   unknown_parameter:
-   snprintf(jail_errmsg, JAIL_ERRMSGLEN,
-   "unknown parameter: %s", jp->jp_name);
-   errno = ENOENT;
-   return (-1);
+   snprintf(desc.s, sizeof(desc.s), SJPARAM ".%s

Re: bin/165515: [jail][patch] "jail: unknown parameter: allow.nomount" when starting jail

[Jamie Gritton]

The following reply was made to PR bin/165515; it has been noted by GNATS.

From: Jamie Gritton 
To: Glen Barber 
Cc: freebsd-gnats-sub...@freebsd.org, freebsd-jail@FreeBSD.org,
Martin Matuska 
Subject: Re: bin/165515: [jail][patch] "jail: unknown parameter: allow.nomount"
 when starting jail
Date: Tue, 28 Feb 2012 15:36:20 -0700

 This is a multi-part message in MIME format.
 --090403000502010508070509
 Content-Type: text/plain; charset=ISO-8859-1; format=flowed
 Content-Transfer-Encoding: 7bit
 
 The allow.mount parameter recently changed in a subtle way - it's now
 a node (to e.g. allow.mount.devfs) as well as a parameter in its own
 right. This confused libjail which knows how to handle such parameters
 as long as they're not boolean.
 
 I'm including my proposed fix to libjail. This this fix, allow.nomount
 should once again work, as should allow.mount and other things such as
 allow.quotas/allow.noquotas should work as they did before.
 
 - Jamie
 
 
 On 02/27/12 18:21, Glen Barber wrote:
 >
 >> Number: 165515
 >> Category:   bin
 >> Synopsis:   [jail][patch] "jail: unknown parameter: allow.nomount" when 
 >> starting jail
 >> Confidential:   no
 >> Severity:   non-critical
 >> Priority:   low
 >> Responsible:freebsd-bugs
 >> State:  open
 >> Quarter:
 >> Keywords:
 >> Date-Required:
 >> Class:  sw-bug
 >> Submitter-Id:   current-users
 >> Arrival-Date:   Tue Feb 28 01:30:07 UTC 2012
 >> Closed-Date:
 >> Last-Modified:
 >> Originator: Glen Barber
 >> Release:10.0-CURRENT r232116M
 >> Organization:
 >> Environment:
 > FreeBSD nucleus 10.0-CURRENT FreeBSD 10.0-CURRENT #7 r232116M: Sun Feb 26 
 > 14:12:14 EST 2012 root@nucleus:/usr/obj/usr/src/sys/NUCLEUS  amd64
 >> Description:
 > On a recent -CURRENT, the jail(8) rc script fails to start jails.  This 
 > seems to be caused by looking up a sysctl that does not exist.
 >> How-To-Repeat:
 > Create a minimalistic jail setup as follows:
 >
 >   nucleus# cat /etc/rc.conf.local
 >   #
 >   jail_enable="YES"
 >   jail_parallel_start="YES"
 >   jail_list="cfm"
 >   jail_cfm_hostname="cfm"
 >   jail_cfm_rootdir="/jails/cfm"
 >   jail_cfm_ip="172.16.1.2"
 >
 > Start the jail with the rc(8) script:
 >
 >   nucleus# sh -x /etc/rc.d/jail start cfm
 >   + . /etc/rc.subr
 >   [...]
 >   + eval '_x="$jail_cfm_ip_multi0"'
 >   + _x=''
 >   + break
 >   + echo ' cannot start jail "cfm": '
 >cannot start jail "cfm":
 >   + tail +2 /tmp/jail.bJIDqW5x/jail.52395
 >   jail: unknown parameter: allow.nomount
 >
 >> Fix:
 > The attached patch fixes it for me.
 >
 >
 > Patch attached with submission follows:
 >
 > Index: usr.sbin/jail/jail.c
 > ===
 > --- usr.sbin/jail/jail.c (revision 232116)
 > +++ usr.sbin/jail/jail.c (working copy)
 > @@ -84,7 +84,7 @@
 >  { "security.jail.chflags_allowed",
 >"allow.nochflags", "allow.chflags" },
 >  { "security.jail.mount_allowed",
 > -  "allow.nomount", "allow.mount" },
 > +  "allow.mount", "allow.nomount" },
 >  { "security.jail.socket_unixiproute_only",
 >"allow.socket_af", "allow.nosocket_af" },
 >   };
 >
 >
 >> Release-Note:
 >> Audit-Trail:
 >> Unformatted:
 > ___
 > freebsd-b...@freebsd.org mailing list
 > http://lists.freebsd.org/mailman/listinfo/freebsd-bugs
 > To unsubscribe, send any mail to "freebsd-bugs-unsubscr...@freebsd.org"
 
 --090403000502010508070509
 Content-Type: text/plain;
  name="libjail.diff"
 Content-Transfer-Encoding: 7bit
 Content-Disposition: attachment;
  filename="libjail.diff"
 
 Index: lib/libjail/jail.c
 ===
 --- lib/libjail/jail.c (revision 232240)
 +++ lib/libjail/jail.c (working copy)
 @@ -885,36 +885,20 @@
 * the "no" counterpart to a boolean.
 */
nname = nononame(jp->jp_name);
 -  if (nname != NULL) {
 -  snprintf(desc.s, sizeof(desc.s), SJPARAM ".%s", nname);
 -  free(nname);
 -  miblen = sizeof(mib) - 2 * sizeof(int);
 -  if (sysctl(mib, 2, mib + 2, &miblen, desc.s,
 -  strlen(desc.s)) >= 0)

Re: bin/165515: [jail][patch] "jail: unknown parameter: allow.nomount" when starting jail

[Jamie Gritton]

On 02/28/12 17:30, Martin Matuska wrote:

On 28.2.2012 23:36, Jamie Gritton wrote:

The allow.mount parameter recently changed in a subtle way - it's now
a node (to e.g. allow.mount.devfs) as well as a parameter in its own
right. This confused libjail which knows how to handle such parameters
as long as they're not boolean.

I'm including my proposed fix to libjail. This this fix, allow.nomount
should once again work, as should allow.mount and other things such as
allow.quotas/allow.noquotas should work as they did before.

- Jamie


Thanks. I might MFC this together with all the other new jail stuff to
9-STABLE in a week or so.
Or do we need more testing?

For 8-STABLE, all of this is a no-op, because the prerequisites for all
these changes date back to the VOP_VPTOCNP(9) change by kib.


I've checked that the expected parameters get past the gauntlet, and
that some things that look like them don't (such as host.nohostname).
What I haven't checked, and should be done before I commit - since
you've got it all set up, could you see if these do as you expect?

allow.mount
allow.nomount
allow.mount.devfs
allow.mount.nodevfs
allow.chflags
allow.nochflags

That covers the newly fixed allow.mount, one of your new per-FS allows,
and an existing allow that should work as it did before. Those (except
the last) are all the kind of thing you've been testing anyway so it
shouldn't be too much of an imposition :-). Beyond that, I don't see the
need for 9-specific testing since it has the exact same libjail code.

- Jamie
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

Re: bin/165515: [jail][patch] "jail: unknown parameter: allow.nomount" when starting jail

[Jamie Gritton]

The following reply was made to PR bin/165515; it has been noted by GNATS.

From: Jamie Gritton 
To: Martin Matuska 
Cc: Glen Barber , freebsd-gnats-sub...@freebsd.org,
freebsd-jail@FreeBSD.org
Subject: Re: bin/165515: [jail][patch] "jail: unknown parameter: allow.nomount"
 when starting jail
Date: Tue, 28 Feb 2012 22:56:38 -0700

 On 02/28/12 17:30, Martin Matuska wrote:
 > On 28.2.2012 23:36, Jamie Gritton wrote:
 >> The allow.mount parameter recently changed in a subtle way - it's now
 >> a node (to e.g. allow.mount.devfs) as well as a parameter in its own
 >> right. This confused libjail which knows how to handle such parameters
 >> as long as they're not boolean.
 >>
 >> I'm including my proposed fix to libjail. This this fix, allow.nomount
 >> should once again work, as should allow.mount and other things such as
 >> allow.quotas/allow.noquotas should work as they did before.
 >>
 >> - Jamie
 >
 > Thanks. I might MFC this together with all the other new jail stuff to
 > 9-STABLE in a week or so.
 > Or do we need more testing?
 >
 > For 8-STABLE, all of this is a no-op, because the prerequisites for all
 > these changes date back to the VOP_VPTOCNP(9) change by kib.
 
 I've checked that the expected parameters get past the gauntlet, and
 that some things that look like them don't (such as host.nohostname).
 What I haven't checked, and should be done before I commit - since
 you've got it all set up, could you see if these do as you expect?
 
 allow.mount
 allow.nomount
 allow.mount.devfs
 allow.mount.nodevfs
 allow.chflags
 allow.nochflags
 
 That covers the newly fixed allow.mount, one of your new per-FS allows,
 and an existing allow that should work as it did before. Those (except
 the last) are all the kind of thing you've been testing anyway so it
 shouldn't be too much of an imposition :-). Beyond that, I don't see the
 need for 9-specific testing since it has the exact same libjail code.
 
 - Jamie
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

Re: jail name is interpreted as jid when numeric

[Jamie Gritton]

It might seem clear with the dot-separated names that asd.asd.1 isn't
the same as jail 1. But looking from the viewpoint of asd.asd, that jail
would simply be "1". As jails may be referred to by either number or
name, it made sense to exclude jails whose name was a number, except in
the special case of it being the same as the jid. Otherwise there's the
confusion of a jail having two different numbers (one really being a
name), or of a number referring to two different jails.

So while this does seem to break the dot-separate namespace concept,
it's necessary because jails aren't always referred to by the full
hierarchical name.

- Jamie


On 03/22/12 11:18, Nikos Vassiliadis wrote:

I found this somehow strange behavior and I am reporting it
just to hear your opinions.


lab# jail -c name=asd persist
lab# jail -c name=asd.asd persist


asd is interpreted as jail asd.asd


lab# jail -c name=asd.asd.1 persist
jail: jail 1 already exists


1 is interpreted as jail 1

This has to do with the fact that a numeric name is interpreted as jid
and this breaks the dot-seperated hierarchical jails concept.
I find this behavior somehow strange. Is it intended or it's bug?

Thanks for your insights, Nikos

___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

New jail(8) committed

[Jamie Gritton]

I've finally put my jail(8) changes into HEAD.  This new version of jail 
can create jails from a configuration file - see jail.conf(5) for the 
format, as well as some additions to jail(8).  This doesn't mean you 
*have* to use jail.conf, but it's a better way to manage jails than the 
existing rc.conf method.


The plan is for rc.d/jail to start and stop jails as specified in 
/etc/jail.conf, but that's not happening yet - right now it still uses 
the old system, which jail(8) is still compatible with.


There's still more that I want to do with this, but it's time it was 
part of current.


Enjoy :-).

- Jamie
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

Re: New jail(8) committed

[Jamie Gritton]

I thought I had changed that to 10.0 - and I would then further change
it back to 9.1 assuming it got MFC'd. I had originally intended for 9.0
but missed the deadline, and I thought I had fixed that. Maybe editing
the man page while I already had the svn commit running (in the editor)
wasn't sufficient. I'll make sure I've got the right numbers in there
(10.0 until it's actually MFC'd).

- Jamie


On 04/27/12 10:17, Subbsd wrote:

On Fri, Apr 27, 2012 at 12:07 AM, Jamie Gritton  wrote:

I've finally put my jail(8) changes into HEAD.


http://svnweb.freebsd.org/base/head/usr.sbin/jail/jail.conf.5?view=markup&pathrev=234712
( The .Nm file was added in .Fx 9.0 . )

Looks like man file in HISTORY section is a bit incorrect. According
to the MFC, this file will be in FreeBSD 9.1+   ?

___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

Re: New jail(8) committed

[Jamie Gritton]

The main reason I didn't consider a jail.d approach is just that I
haven't - such things are a little off the radar for me. It seemed very
natural to use a configuration file format that other programs already
use (e.g. named, apmd, devd). I suppose it's true that the "foo.d"
approach is also in use by other programs, though I mostly seem to see
those on Linux. And if I did opt for a directory approach, the files
within the directory would still need a format - you can't get away from
the fact that a config file needs a format.

It would be nice to have a general parser for C-style config files, and
I looked for such a library when I started on this. But such a library
doesn't seem to exist.Perhaps it's time to make one.

- Jamie

On 04/27/12 22:14, Dirk Engling wrote:

On 26.04.12 22:07, Jamie Gritton wrote:


I've finally put my jail(8) changes into HEAD.  This new version of jail
can create jails from a configuration file - see jail.conf(5) for the
format, as well as some additions to jail(8).  This doesn't mean you
*have* to use jail.conf, but it's a better way to manage jails than the
existing rc.conf method.


Out of curiosity, why did you settle for a /etc/jail.conf instead of a
/etc/jail.d/? Your config file format introduces the dependency into an
expensive parser while adding little value. Even worse, the user now has
to struggle with just another format describing the system.

I can foresee that my automated jail management tool ezjail will not be
able to support the jail.conf format due to the lack of a parser. A look
into ezjails config directory structure can give you a hint of how to
achieve some similar clean up with built in tools.

I am not saying, the config directory format is perfect, the current
redundancy in jail_JAILNAME variables is a mess, but inventing a
container format where files would do just fine in my opinion is overkill.

Regards,

   erdgeist

___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

Re: New jail(8) committed

[Jamie Gritton]

I don't know about wrapping a utility around it, but it would be nice to
have in a library. If it could be made to work for not only jail, but
apmd and devd as well, then we could make some existing code cleaner.

- Jamie


On 04/28/12 10:52, Devin Teske wrote:

On Apr 28, 2012, at 7:38 AM, Jamie Gritton wrote:


The main reason I didn't consider a jail.d approach is just that I
haven't - such things are a little off the radar for me. It seemed very
natural to use a configuration file format that other programs already
use (e.g. named, apmd, devd). I suppose it's true that the "foo.d"
approach is also in use by other programs, though I mostly seem to see
those on Linux. And if I did opt for a directory approach, the files
within the directory would still need a format - you can't get away from
the fact that a config file needs a format.

It would be nice to have a general parser for C-style config files, and
I looked for such a library when I started on this. But such a library
doesn't seem to exist.Perhaps it's time to make one.



The config file format that you've chosen is remarkably identical to config 
files for which I've already written parsers.

So, I guess I'm saying that I'm willing to help out in this area.

My parser is written in C, it's very small and light-weight, and it's called 
figpar (con[fig par]ser).

I can dust if off, slap a BSD license on it, wrap a utility around it and we 
could have something like sysrc (which operates on the collection of rc.conf(5) 
files).

Alternatively, I could rewrite it in something like sh(1) if C is not desired.

___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

Re: New jail(8) committed

[Jamie Gritton]

On 04/28/12 00:08, Jason Hellenthal wrote:

On Sat, Apr 28, 2012 at 06:14:07AM +0200, Dirk Engling wrote:

On 26.04.12 22:07, Jamie Gritton wrote:


I've finally put my jail(8) changes into HEAD.  This new version of jail
can create jails from a configuration file - see jail.conf(5) for the
format, as well as some additions to jail(8).  This doesn't mean you
*have* to use jail.conf, but it's a better way to manage jails than the
existing rc.conf method.


Out of curiosity, why did you settle for a /etc/jail.conf instead of a
/etc/jail.d/? Your config file format introduces the dependency into an
expensive parser while adding little value. Even worse, the user now has
to struggle with just another format describing the system.

I can foresee that my automated jail management tool ezjail will not be
able to support the jail.conf format due to the lack of a parser. A look
into ezjails config directory structure can give you a hint of how to
achieve some similar clean up with built in tools.


Since when does a lack of a parser in "YOUR tool" become a problem for
FreeBSD ? just sayin!


To be fair, ezjail is a tool is pretty wide use, and I had no intention
of breaking it - but also no knowledge of its internals. This thing has
been sitting around in the projects directory for a long time now, with
requests for review and comments. It's kind of disheartening to only
hear this the day I committed it to HEAD.

- Jamie
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

Re: New jail(8) committed

[Jamie Gritton]

On 04/29/12 17:37, Mr Dandy wrote:

Looks like the new functionality lost abilities of old /etc/rc.d/jail to carry
out multiple quantity of prestart/stop instruction

Old cool feature:
jail_example_exec_afterstart0=""
jail_example_exec_afterstart1=""
..

In new style it doesn't work:
exec.start0 = '/bin/date';
exec.start1 = '/bin/date';
..


I'll need to make sure I have the documentation in order for this.  You 
can have:


exec.start = '/bin/foo';
exec.start += '/bin/bar';


also it would be great to have "include" for some other places
(/etc/jail.d for example) that it was possible to have one config per jail.


Yes it would.  This is one of the things on my to-do list.

- Jamie
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

Re: jail still broken on 9-STABLE

[Jamie Gritton]

I gave a pretty long MFC period to the new jail stuff since it's such a 
big change.


- Jamie

On 05/15/12 03:09, Christer Solskogen wrote:

jail: unknown parameter: allow.nomount

:-(

___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

Re: Breakage on 9

[Jamie Gritton]

I've got a fix that's going in as soon as it's confirmed. I'm including
the patch here. Backing out r235624 will work, unless you happen to be
on a box where r235624 was necessary.

I had said something before about the long MFC, but that turned out to
be wrong. That was for the new jail(8) code, but the nomount fix is in
the libjail code. Actually, that was fixed a while back, but then I
re-broke it (in a different way) with r235624.

If anyone continues to have problems on 9 with the included patch, let
me know. But I think this will bring it all into working order.

- Jamie

On 05/21/12 12:24, Nathan Schimke wrote:

On 05/21/2012 11:08 AM, Ed Schouten wrote:

Hi,

Today I was bitten by the "jail: unknown parameter: allow.nomount"
bug. It seems this bug is at least more than one month old -- on
9-STABLE! Why hasn't the commit that introduced this bug been backed
out? The code worked before. Now it's broken and we're waiting for an
MFC period? Using a Danish (Dutch?) axe, I was able to at least get my
box working again:


I just hit the same error last night. Reverting r235624 (an MFC made 3
days ago) got my jails working again, but I haven't yet figured out why.

Nathan
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"
Index: jail.c
===
--- jail.c  (revision 235668)
+++ jail.c  (working copy)
@@ -853,7 +853,7 @@
 static int
 jailparam_type(struct jailparam *jp)
 {
-   char *p, *nname;
+   char *p, *name, *nname;
size_t miblen, desclen;
int i, isarray;
struct {
@@ -863,7 +863,8 @@
int mib[CTL_MAXNAME];
 
/* The "lastjid" parameter isn't real. */
-   if (!strcmp(jp->jp_name, "lastjid")) {
+   name = jp->jp_name;
+   if (!strcmp(name, "lastjid")) {
jp->jp_valuelen = sizeof(int);
jp->jp_ctltype = CTLTYPE_INT | CTLFLAG_WR;
return (0);
@@ -872,19 +873,19 @@
/* Find the sysctl that describes the parameter. */
mib[0] = 0;
mib[1] = 3;
-   snprintf(desc.s, sizeof(desc.s), SJPARAM ".%s", jp->jp_name);
+   snprintf(desc.s, sizeof(desc.s), SJPARAM ".%s", name);
miblen = sizeof(mib) - 2 * sizeof(int);
if (sysctl(mib, 2, mib + 2, &miblen, desc.s, strlen(desc.s)) < 0) {
if (errno != ENOENT) {
snprintf(jail_errmsg, JAIL_ERRMSGLEN,
-   "sysctl(0.3.%s): %s", jp->jp_name, strerror(errno));
+   "sysctl(0.3.%s): %s", name, strerror(errno));
return (-1);
}
/*
 * The parameter probably doesn't exist.  But it might be
 * the "no" counterpart to a boolean.
 */
-   nname = nononame(jp->jp_name);
+   nname = nononame(name);
if (nname == NULL) {
unknown_parameter:
snprintf(jail_errmsg, JAIL_ERRMSGLEN,
@@ -892,8 +893,10 @@
errno = ENOENT;
return (-1);
}
-   snprintf(desc.s, sizeof(desc.s), SJPARAM ".%s", nname);
+   name = alloca(strlen(nname) + 1);
+   strcpy(name, nname);
free(nname);
+   snprintf(desc.s, sizeof(desc.s), SJPARAM ".%s", name);
miblen = sizeof(mib) - 2 * sizeof(int);
if (sysctl(mib, 2, mib + 2, &miblen, desc.s,
strlen(desc.s)) < 0)
@@ -906,7 +909,7 @@
if (sysctl(mib, (miblen / sizeof(int)) + 2, &desc, &desclen,
NULL, 0) < 0) {
snprintf(jail_errmsg, JAIL_ERRMSGLEN,
-   "sysctl(0.4.%s): %s", jp->jp_name, strerror(errno));
+   "sysctl(0.4.%s): %s", name, strerror(errno));
return (-1);
}
jp->jp_ctltype = desc.i;
@@ -952,7 +955,7 @@
if (sysctl(mib + 2, miblen / sizeof(int), desc.s, &desclen,
NULL, 0) < 0) {
snprintf(jail_errmsg, JAIL_ERRMSGLEN,
-   "sysctl(" SJPARAM ".%s): %s", jp->jp_name,
+   "sysctl(" SJPARAM ".%s): %s", name,
strerror(errno));
return (-1);
}
@@ -970,7 +973,7 @@
if (sysctl(mib + 2, miblen / sizeof(int),
NULL, &jp->jp_valuelen, NULL, 0) < 0) {
snprintf(jail_errmsg, JAIL_ERRMSGLEN,
-   "sysctl(" SJPARAM ".%s): %s", jp->jp_name,
+   "sysctl(" SJPARAM ".%s): %s", name,
strerror(errno));
return (-

Re: ifconfig not seeing IPv6 in x86 jail

[Jamie Gritton]

A PR sounds appropriate if it's not doing what it should. Just to be
sure: this is an x86 jail on an x86 system? I want to make sure it's
a jail issue and not a 32-bit emulation issue.

- Jamie


On 06/12/12 06:32, Mars G. Miro wrote:

Hi Guys

I've just noticed this now, and I usually build jails (including x86
jails for my x86 systems) for custom-building my base and ports, for
pkg_add'ing later. It works great(!), and I've done this evar since
we've been able to cross-build.

It seems that ifconfig doesn't see the IPv6 address in an x86 jail ?

mars@spry832x86:~> sysctl security.jail.jailed
security.jail.jailed: 1

mars@spry832x86:~> ifconfig
bge0: flags=8843 metric 0 mtu 1500

options=8009b
ether 00:22:19:17:e5:9a
inet 172.17.83.2 netmask 0x broadcast 172.17.83.2
nd6 options=3
bge1: flags=8802 metric 0 mtu 1500

options=8009b
ether 00:22:19:17:e5:9b
lo0: flags=8049 metric 0 mtu 16384
options=3


But I know I'm on IPv6:

mars@spry832x86:~> w
8:28PM up 8 days, 6:27, 1 user, load averages: 0.00, 0.00, 0.00
USER TTY FROM LOGIN@ IDLE WHAT
mars pts/4 2001:470:XXX:2400 8:27PM - w

mars@spry832x86:~> uname -a
FreeBSD spry832x86 8.3-RELEASE-p2 FreeBSD 8.3-RELEASE-p2 #0: Mon Jun 4
13:32:10 PHT 2012 r...@panzer.xxx:/usr/obj/usr/src/sys/PANZER i386
mars@spry832x86:~>

I checked on FreeBSD 8.2 and it's the same. I've yet to check on 9.X tho.

Is this worth a PR ?

Thanks.

___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

Re: [patch] etc/rc.d/jail: allow extra parameters for each jails

[Jamie Gritton]

On 08/17/12 04:28, Bjoern A. Zeeb wrote:

On Thu, 16 Aug 2012, Jun Kuriyama wrote:



Hi,

Here is a patch which I'm using for years in my production
environment.

I usually changes parameters documented in jail(8) for each jails, but
current rc.d/jail has no feature to pass extra parameters at starting
jails.

I hope this patch will not prevent jamie's recent/next work for
rc.d/jail. I'll commit this if there is no objection.


Why not just use his work?


Probably because it's a lot more "next" than "recent".
Soon ... real soon now.

- Jamie
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

Re: IPv6 multicast sent to jail

[Jamie Gritton]

On 08/19/12 11:35, Curtis Villamizar wrote:

I'm trying to run isc-dhcpd using dhcpd -6 in a jail.  No luck.

The following code is run in the jail and doesn't fail.

 if (inet_pton(AF_INET6, All_DHCP_Relay_Agents_and_Servers,
   &mreq.ipv6mr_multiaddr)<= 0) {
 log_fatal("inet_pton: unable to convert '%s'",
   All_DHCP_Relay_Agents_and_Servers);
 }
 mreq.ipv6mr_interface = if_nametoindex(info->name);
 if (setsockopt(sock, IPPROTO_IPV6, IPV6_JOIN_GROUP,
&mreq, sizeof(mreq))<  0) {
 log_fatal("setsockopt: IPV6_JOIN_GROUP: %m");
 }

where All_DHCP_Relay_Agents_and_Servers is defined as "FF02::1:2".

Later dhcpd binds to *.517 which can be seen in netstat -an.

Packets to ff02::1:2.517 are seen on the jailer (as opposed to the
jailee) using tcpdump, but no packets are received by the jailee.

When the same command from the jailer using a chroot to the jailee
directory, the multicast packets are received.

Is there a solution to this other than changing the jail from an
implied "ip6=new" with a specific address to "ip6=inherit".  What I'd
really like is a yet to be invented "ip6=new+multicast".

Using "ip6=inherit" would be OK, adding very little exposure (mostly
DoS attack exposure).  It would be nice if "ip6=inherit" were
supported in the rc.d/jail framework.

Before I go changing anything I'm asking whether allowing the
multicast join and then not passing multicast to the jail is
considered a bug and how it should behave (the join should have failed
or the packets should have arrived).  If the best workaround for now
is "ip6=inherit" would adding jail__ip[46] variables to the
rc files be viewed as a good solution (with a comment in
/etc/defaults/rc.conf indicating that the interaction between setting
addressing using _ip and _ip_multi and setting _ip4 or _ip6 (setting
an address for each family forces "ip[46]=net" for that AF.

Curtis


Offhand, it does sound like a bug. I imagine the solution would be to
reject the join - at least the easy solution to be done first until
something more complicated can be done to make jails play nice with
multicast.

- Jamie
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

Re: misc/170832: jail v2 loses a binding of ip which sets after ips with /"network prefix"

[Jamie Gritton]

On 08/21/12 12:49, Oleg Ginzburg wrote:

Number: 170832
Category:   misc
Synopsis:   jail v2 loses a binding of ip which sets after ips with /"network 
prefix"
Confidential:   no
Severity:   non-critical
Priority:   low
Responsible:freebsd-bugs
State:  open
Quarter:
Keywords:
Date-Required:
Class:  sw-bug
Submitter-Id:   current-users
Arrival-Date:   Tue Aug 21 18:50:10 UTC 2012
Closed-Date:
Last-Modified:
Originator: Oleg Ginzburg
Release:9.1-PRERELEASE, 10-CURRENT
Organization:
Environment:

FreeBSD cbuilder64.my.domain 10.0-CURRENT FreeBSD 10.0-CURRENT #4 r239330: Thu 
Aug 16 22:08:12 MSK 2012 
r...@cbuilder64.my.domain:/usr/obj/usr/src/sys/GENERIC  amd64

Description:

when the jail is created by config file with multiple ips, jail loses all ip assignment 
which sets after ips with /"network prefix"
Nevertheless, all IP are established on the interface with the correct mask

How-To-Repeat:

Have jail config with multiple ips with prefix in the list. Config sample (we 
mean that in /usr/jails/jail1 we have a complete freebsd base environment):

% cat jail1.conf
jail1 {
exec.start = "/bin/sh /etc/rc";
exec.stop = "/bin/sh /etc/rc.shutdown";
exec.clean;
#mount.devfs;
host.hostname = "jail1.my.domain";
path = "/usr/jails/jail1";
allow.raw_sockets;
allow.socket_af;
allow.chflags;
allow.sysvipc;
ip4.addr = 10.0.0.1,10.0.0.2,172.17.0.0/16,10.0.0.3;
interface="em0";
mount.devfs;
devfs_ruleset="4";
}

// Before jail creation, interfaces em0 have:
% ifconfig em0 | grep "inet "
 inet 192.168.1.2 netmask 0xff00 broadcast 192.168.1.255

// Jail start:
% jail -f jail1.conf -c jail1
jail1: created
..

//done. Check for IPs on interface. 172.17.0.0 have correct mask:
ifconfig em0 | grep "inet "
 inet 192.168.1.2 netmask 0xff00 broadcast 192.168.1.255
 inet 10.0.0.1 netmask 0x broadcast 10.0.0.1
 inet 10.0.0.2 netmask 0x broadcast 10.0.0.2
 inet 172.17.0.0 netmask 0x broadcast 172.17.255.255
 inet 10.0.0.3 netmask 0x broadcast 10.0.0.3

//Check for IP in jls:
% jls -v
JID  Hostname  Path
 Name  State
 CPUSetID
 IP Address(es)
  1  jail1.my.domain   /usr/jails/jail1
 jail1 ACTIVE
 2
 10.0.0.1
 10.0.0.2
 172.17.0.0

We have 10.0.0.1/32,10.0.0.2/32 and 172.17.0.0/16 but loose 10.0.0.3


I confused myself on the difference between null-terminated and
length-defined strings, and stuck a '\0' were it didn't belong.
I've committed the fix to HEAD, and I'll have it in 9.1 next week.
I'm also including it here for easy consumption :-).

- Jamie
Index: usr.sbin/jail/config.c
===
--- usr.sbin/jail/config.c  (revision 239600)
+++ usr.sbin/jail/config.c  (revision 239601)
@@ -597,8 +597,7 @@
"ip4.addr: bad netmask \"%s\"", cs);
error = -1; 
}
-   *cs = '\0';
-   s->len = cs - s->s + 1;
+   s->len = cs - s->s;
}
}
}
@@ -621,8 +620,7 @@
cs);
error = -1; 
}
-   *cs = '\0';
-   s->len = cs - s->s + 1;
+   s->len = cs - s->s;
}
}
}
@@ -714,7 +712,7 @@
value = alloca(vallen);
cs = value;
TAILQ_FOREACH_SAFE(s, &p->val, tq, ts) {
-   strcpy(cs, s->s);
+   memcpy(cs, s->s, s->len);
if (ts != NULL) {
cs += s->len + 1;
cs[-1] = ',';
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

Re: IPv6 multicast sent to jail

[Jamie Gritton]

On 08/25/12 14:15, Curtis Villamizar wrote:

In message<503402fe.9080...@freebsd.org>
Jamie Gritton writes:


On 08/19/12 11:35, Curtis Villamizar wrote:

I'm trying to run isc-dhcpd using dhcpd -6 in a jail.  No luck.

The following code is run in the jail and doesn't fail.

  if (inet_pton(AF_INET6, All_DHCP_Relay_Agents_and_Servers,
&mreq.ipv6mr_multiaddr)<= 0) {
  log_fatal("inet_pton: unable to convert '%s'",
All_DHCP_Relay_Agents_and_Servers);
  }
  mreq.ipv6mr_interface = if_nametoindex(info->name);
  if (setsockopt(sock, IPPROTO_IPV6, IPV6_JOIN_GROUP,
 &mreq, sizeof(mreq))<   0) {
  log_fatal("setsockopt: IPV6_JOIN_GROUP: %m");
  }

where All_DHCP_Relay_Agents_and_Servers is defined as "FF02::1:2".

Later dhcpd binds to *.517 which can be seen in netstat -an.

Packets to ff02::1:2.517 are seen on the jailer (as opposed to the
jailee) using tcpdump, but no packets are received by the jailee.

When the same command from the jailer using a chroot to the jailee
directory, the multicast packets are received.

Is there a solution to this other than changing the jail from an
implied "ip6=new" with a specific address to "ip6=inherit".  What I'd
really like is a yet to be invented "ip6=new+multicast".

Using "ip6=inherit" would be OK, adding very little exposure (mostly
DoS attack exposure).  It would be nice if "ip6=inherit" were
supported in the rc.d/jail framework.

Before I go changing anything I'm asking whether allowing the
multicast join and then not passing multicast to the jail is
considered a bug and how it should behave (the join should have failed
or the packets should have arrived).  If the best workaround for now
is "ip6=inherit" would adding jail__ip[46] variables to the
rc files be viewed as a good solution (with a comment in
/etc/defaults/rc.conf indicating that the interaction between setting
addressing using _ip and _ip_multi and setting _ip4 or _ip6 (setting
an address for each family forces "ip[46]=net" for that AF.

Curtis


Offhand, it does sound like a bug. I imagine the solution would be to
reject the join - at least the easy solution to be done first until
something more complicated can be done to make jails play nice with
multicast.

- Jamie



Jamie,

Certainly not the preferred solution.  Best would be a
jail.allow-ipv6multicast sysctl variable with rejecting the join if 0
and accepting the join and passing in multicast if 1.  Same for v4,
though not of immediate concern since DHCPv4 doesn't need it.

If you (or someone) would like to point me in the right direction, I
would be willing to put some time into learning the relevant code and
proposing a fix.  No promises, but I can put some time into it.  Off
list if you prefer.

Curtis


It'll have to be someone besides me - I don't know enough about
multicast myself to be able to do more than keep it out of jails.

- Jamie
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

Re: Quotas inside jails

[Jamie Gritton]

On 08/30/12 17:05, Darek M wrote:

On Thu, Aug 30, 2012 at 5:32 PM, John Nielsen  wrote:

On Aug 30, 2012, at 2:52 PM, Darek M  wrote:


playing around with setting quotas inside a jail.  Configured and
tested them on the host, configured a quota for a jail user, but it
isn't being enforced.  I attempted to set
security.jail.param.allow.quotas to 1, from command line, from
/etc/sysctl.conf, and from /boot/loader.conf, but it remains set to
'0'.

Am I looking at the right sysctl?  If not, where should I be looking?
If yes, why does it appear to be immutable?


I'm assuming you have basically one UFS filesystem for all your jails. Is that 
the case? If so, do you have quotas enabled on the host? See the handbook if 
you haven't already:
http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/quotas.html


Yup, verified that quotas on the host work fine.


I'm doing this on a 9.0-RELEASE system


Another way to set hard quotas for jails is to give each one its own filesystem 
of fixed size. This is trivially easy with zfs--just create a zfs for each jail 
and set the quota property. To use UFS you can create image files of whatever 
size you want, make them md(4) devices, and then newfs(8) and mount(8) them. 
Unlike the method in the handbook, neither of these options requires kernel 
quota support.


But these would be a quota for the entire jail.  I'm interested in
having per-user quotas for users inside a jail.

I'm curious whether the "security.jail.param.allow.quotas" sysctl is
my missing link, and if so, why it is immutable.


The security.jail.param.* sysctls are part of the jail_get/set system
calls, and are all immutable; they server only to define the available
jail parameters.

So the question now comes to the allow.quotas parameter. If you set this
on a jail, then you will indeed be able to manipulate quotas inside the
jail. But the quotas still aren't per-jail - they're keyed only on
UID/GID, and would share with anyone outside the jail using the same
UID/GID. That's fine if the jail has its own filesystem, but not if it
shares with other jails or (especially) with the host system.

- Jamie
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

Re: Quotas inside jails

[Jamie Gritton]

On 08/31/12 14:41, Scott Lambert wrote:

On Thu, Aug 30, 2012 at 07:05:30PM -0400, Darek M wrote:

On Thu, Aug 30, 2012 at 5:32 PM, John Nielsen  wrote:


Another way to set hard quotas for jails is to give each one its
own filesystem of fixed size. This is trivially easy with zfs--just
create a zfs for each jail and set the quota property. To use UFS
you can create image files of whatever size you want, make them
md(4) devices, and then newfs(8) and mount(8) them. Unlike the
method in the handbook, neither of these options requires kernel
quota support.


But these would be a quota for the entire jail.  I'm interested in
having per-user quotas for users inside a jail.

I'm curious whether the "security.jail.param.allow.quotas" sysctl is
my missing link, and if so, why it is immutable.


If using ZFS, you *could* create a file system with quota for each
user's home directory in the jail.  I'm not saying it would be
pretty

With UFS, I think you would have to ensure that UID/GIDs do not
overlap between jails, at least for the users you want to be affected
by quotas.  That could be as ugly as the thousands of ZFS file
systems.


Well, you could if you trusted the jail admins not to use other UID/GIDs
(which he likely isn't even aware of). But the whole point of jails is
that you *don't* have to trust the admin.

- Jamie
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

Re: Fixed Jail ID for ZFS -> need proper mgmt?

[Jamie Gritton]

On 09/04/12 02:55, Bjoern A. Zeeb wrote:

Hi,

I had been talking to someone about jail management and it turns out
people are using jail jid=42 to always have a fixed jail ID. The
reason as I understood is that ZFS datasets are associated by jail id
for delegation? [I admit having no clue about the ZFS side]

If this is true I feel it's a very bad idea as it makes restarting
jails a lot harder in case they remain DYING for say a not fully
closed TCP session.

My memories are: jid are still unique and cannot be re-used, even if
in DYING, names can be re-used and thus are not neccessarily unique.
Jamie, can you confirm this?

Seems we need to sort out one to two problems:

1) can we make sure that the jail management framework can address a
ZFS dataset for delegation somehow and automatically do that as
part of the startup?

2) in the case of (1) it should be possible to address jails by name
as ZFS would be handled automatically and we would not need another
unique identifier I guess?
Otherwise I'd prefer for people to be able to delegate ZFS datasets
to jail names (as well), as long as they are uniquely identifyable
(i.e. there are no 17 jails running with a name of "filesever").

Do we have documentation for the ZFS features in the man pages or
elsewhere btw? If not we should add it.

Does this make sense?

/bz


It's true that a jail left in the DYING state can't be re-created
normally. But it can with the "-d" flag or the "allow.dying" parameter.
In that case, an existing but dying jail will be re-attached to and this
resurrected. So it can be gotten around, and would be a matter of
education. Or perhaps we could change the default behavior to silently
all re-creation of dying jails. Is there any harm in this? I.e. would
there be any difference noticeable to the user if a jail was created
with some old TCP connections attached to it?

- Jamie
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

Re: Fixed Jail ID for ZFS -> need proper mgmt?

[Jamie Gritton]

On 09/04/12 04:20, Bjoern A. Zeeb wrote:

On Tue, 4 Sep 2012, Pawel Jakub Dawidek wrote:


On Tue, Sep 04, 2012 at 11:33:06AM +0200, Martin Matuska wrote:

On 4. 9. 2012 10:55, Bjoern A. Zeeb wrote:

2) in the case of (1) it should be possible to address jails by name
as ZFS would be handled automatically and we would not need another
unique identifier I guess?
Otherwise I'd prefer for people to be able to delegate ZFS datasets
to jail names (as well), as long as they are uniquely identifyable
(i.e. there are no 17 jails running with a name of "filesever").


The binding of a ZFS dataset to a jail has to be exact. So we end up
with id's.
But we could add something like "zfs datasets" to the jail's
configuration file. The jail command would then simply exec "zfs jail
jailid dataset" for each of the datasets on jail creation right before
initiating rc startup and "zfs unjail jailid dataset" for each of the
datasets after jail's rc shutdown but before the jail is destroyed.


Datasets shall not be unjailed. Jailing dataset means that it won't be
mounted in the main system. You need to run 'zfs mount -a' within a
jail, during it start-up to mount its datasets. This is much safer than
mounting anything in jail's directory tree from the main system. We
already had security issues because of that. This is also how it works
in Solaris/IllumOS with zones.

And I can't resist to remind how opposed I was to jail ids in the first
place. Especially because they were dynamically allocated. When they
were introduced I recommended jail names, which we ended up with anyway,
but now we have all this jailid thing to manage in older FreeBSD
versions.

All in all we should move to using jail names, IMHO, the same way zone
names are used in Solaris/IllumOS. When I was adding jail support to ZFS
there were no jail names, only jail hostnames, which weren't an option
really.


I guess we'd need to end up with name and if not uniqe + ID or
something? Really IDs are not the problem as long as they never
appear anywhere in the config file? Just not sure given names are not
unique how to handle it the right way?

/bz


Names are unique. And we don't have the dying-jail problem with them, as
creating a jail with the same name as a dying jail is allowed. OK, that
means that jail names aren't quite unique - but they're at least unique
among the non-dying set.

- Jamie
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

Re: Quotas inside jails

[Jamie Gritton]

On 09/04/12 12:40, Darek M wrote:

On Fri, Aug 31, 2012 at 3:05 PM, Jamie Gritton  wrote:

On 08/30/12 17:05, Darek M wrote:



I'm curious whether the "security.jail.param.allow.quotas" sysctl is
my missing link, and if so, why it is immutable.



The security.jail.param.* sysctls are part of the jail_get/set system
calls, and are all immutable; they server only to define the available
jail parameters.

So the question now comes to the allow.quotas parameter. If you set this
on a jail, then you will indeed be able to manipulate quotas inside the
jail. But the quotas still aren't per-jail - they're keyed only on
UID/GID, and would share with anyone outside the jail using the same
UID/GID. That's fine if the jail has its own filesystem, but not if it
shares with other jails or (especially) with the host system.

- Jamie


Indeed, this looks to be my missing piece.  Using distinct UIDs on
each jail should be easily doable, and would be cleaner than using
zfs, etc..

However, I tried setting "security.jail.param.allow.quotas" to 1
inside the jail via /etc/sysctl.conf and /boot/loader.conf and it
remains at 0.  Am I trying to enable it the wrong way?


Yes. You need to set the "allow.quotas" parameter in the jail. There's
not a good way to do that from rc at this moment, but a "jail -m
jid= allow.quotas" should do the trick after the jail is up and
running.

- Jamie
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

Re: Fixed Jail ID for ZFS -> need proper mgmt?

[Jamie Gritton]

On 09/04/12 14:37, Bjoern A. Zeeb wrote:

On Tue, 4 Sep 2012, Jamie Gritton wrote:


It's true that a jail left in the DYING state can't be re-created
normally. But it can with the "-d" flag or the "allow.dying" parameter.
In that case, an existing but dying jail will be re-attached to and this
resurrected. So it can be gotten around, and would be a matter of
education. Or perhaps we could change the default behavior to silently
all re-creation of dying jails. Is there any harm in this? I.e. would
there be any difference noticeable to the user if a jail was created
with some old TCP connections attached to it?


Yes, really bad and TCP is not the only thing in theory. Assume
your management does not make sure the same users gets the same jail;
you elak a lot of (possibly security related) information. Would also
make it quite hard in terms of auditing etc. to get this right unless
done knowingly and on purpose.


This isn't a ZFS concern anymore it sounds like (if we tie ZFS to
names), but I still wonder about better handling of dying jails. The
other question that comes to mind is, could we make dying jails closer
to nonexistent than they are? Such as stripping them of their jid, so a
jid could be immediately re-used - and a dying jail couldn't be queried
via jail_get. Or perhaps passing off a removed jail's TIME_WAIT tcp
connections to a placeholder jail, possibly to prison0?
I suppose vnet could complicate either of those in ways I'm unaware of.

- Jamie
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

Re: IPv6 multicast sent to jail

[Jamie Gritton]

On 09/05/12 16:51, Bjoern A. Zeeb wrote:

On Wed, 5 Sep 2012, Curtis Villamizar wrote:



In message 
"Bjoern A. Zeeb" writes:


On Sat, 25 Aug 2012, Jamie Gritton wrote:

...

Curtis


Offhand, it does sound like a bug. I imagine the solution would be to
reject the join - at least the easy solution to be done first until
something more complicated can be done to make jails play nice with
multicast.

- Jamie



Jamie,

Certainly not the preferred solution. Best would be a
jail.allow-ipv6multicast sysctl variable with rejecting the join if 0
and accepting the join and passing in multicast if 1. Same for v4,
though not of immediate concern since DHCPv4 doesn't need it.

If you (or someone) would like to point me in the right direction, I
would be willing to put some time into learning the relevant code and
proposing a fix. No promises, but I can put some time into it. Off
list if you prefer.

Curtis


It'll have to be someone besides me - I don't know enough about
multicast myself to be able to do more than keep it out of jails.


sysctl souns bad to me; I think it should actually be grouped by
ip4.* and ip6.*. What dod we currently do for raw sockets? Can we
have a third level easily, as in ip4.raw.*, ip6.mc.*, ... which of
course would kill the classic "allow" thing for raw sockets myabe?

/bz


For raw sockets the sysctl variable is:

security.jail.allow_raw_sockets

One sysctl variable for both inet and inet6 AF. Perhaps a reasonable
name would be:

security.jail.ip4.allow_multicast
security.jail.ip6.allow_multicast

Just to be clear, I was hoping to get some help if I were to make an
attempt to allow ipv6 multicast through, though I suspect that the
code would be very similar for ipv4.


The sysctls are mostly not relevant anymore but yes, if we can get
these options we can look at the code. Defaults to off.
I might be able to help on the v6 trailing end. Jamie could you
prepare the jail options changes for us?


Here's a patch that adds flags for multicast, with the parameters
ip4.multicast and ip6.multicast. They default to false, and don't have
any associated sysctls (which I'd like to phase out). This needs work on
my end, as far making sure permissions are handled correctly for jail
hierarchies, but is enough for starting the work on the multicast side
of things. The check you'll want to make is
prison_flag(cred, PR_IP4_MCAST).

- Jamie
Index: sys/sys/jail.h
===
--- sys/sys/jail.h  (revision 240198)
+++ sys/sys/jail.h  (working copy)
@@ -207,6 +207,8 @@
/* primary jail address. */
 #definePR_IP6_SADDRSEL 0x0100  /* Do IPv6 src addr sel. or use 
the */
/* primary jail address. */
+#definePR_IP4_MCAST0x0200  /* Allow IPv4 multicast */
+#definePR_IP6_MCAST0x0400  /* Allow IPv6 multicast */
 
 /* Internal flag bits */
 #definePR_REMOVE   0x0100  /* In process of being removed 
*/
Index: sys/kern/kern_jail.c
===
--- sys/kern/kern_jail.c(revision 240198)
+++ sys/kern/kern_jail.c(working copy)
@@ -84,14 +84,17 @@
 #ifdef INET
 #ifdef INET6
 #define_PR_IP_SADDRSEL PR_IP4_SADDRSEL|PR_IP6_SADDRSEL
+#define_PR_IP_MCASTPR_IP4_MCAST|PR_IP6_MCAST
 #else
 #define_PR_IP_SADDRSEL PR_IP4_SADDRSEL
+#define_PR_IP_MCASTPR_IP4_MCAST
 #endif
 #else /* !INET */
 #ifdef INET6
 #define_PR_IP_SADDRSEL PR_IP6_SADDRSEL
+#define_PR_IP_MCASTPR_IP6_MCAST
 #else
-#define_PR_IP_SADDRSEL 0
+#define_PR_IP_MCAST0
 #endif
 #endif
 
@@ -108,9 +111,9 @@
.pr_hostuuid= DEFAULT_HOSTUUID,
.pr_children= LIST_HEAD_INITIALIZER(prison0.pr_children),
 #ifdef VIMAGE
-   .pr_flags   = PR_HOST|PR_VNET|_PR_IP_SADDRSEL,
+   .pr_flags   = PR_HOST|PR_VNET|_PR_IP_SADDRSEL|_PR_IP_MCAST,
 #else
-   .pr_flags   = PR_HOST|_PR_IP_SADDRSEL,
+   .pr_flags   = PR_HOST|_PR_IP_SADDRSEL|_PR_IP_MCAST,
 #endif
.pr_allow   = PR_ALLOW_ALL,
 };
@@ -158,9 +161,11 @@
[0] = "persist",
 #ifdef INET
[7] = "ip4.saddrsel",
+   [9] = "ip4.multicast",
 #endif
 #ifdef INET6
[8] = "ip6.saddrsel",
+   [10] = "ip6.multicast",
 #endif
 };
 const size_t pr_flag_names_size = sizeof(pr_flag_names);
@@ -169,9 +174,11 @@
[0] = "nopersist",
 #ifdef INET
[7] = "ip4.nosaddrsel",
+   [9] = "ip4.nomulticast",
 #endif
 #ifdef INET6
[8] = "ip6.nosaddrsel",
+   [10] = "ip6.nomulticast",
 #endif
 };
 const size_t pr_flag_nonames_size = sizeof(pr_flag_nonames);
@@ -232,6 +239,7 @@
 static int jail_default_devfs_rsnum = JAIL_DEFAU

Re: Upgrading FBSD-7.0 --> 7.4 and Jail won't start

[Jamie Gritton]

On 09/08/12 12:15, Jack Stone wrote:

uname -a
FreeBSD mail.sagedata.net 7.0-RELEASE-p9 FreeBSD 7.0-RELEASE-p9 #2: Sun
Jan 18 19:59:27 CST 2009 r...@mail.sagedata.net:/usr/obj/usr/src/sys/SMP
i386

Have been running and upgrading host+jail for years and through several
versions of FreeBSD. However, I got behind on updates and now playing
catchup on an important production server.

Tried to upgrade fbsd-7.0 --> 7.4 followed by upgarde of a jail as in
past. However, on reboot, the host booted up OK, but the jail would not
work. On re-trying to start the jail, only got a crytic error about
missing syscalls. I've never seen this before and don't have a clue what
needs fixing.

My setups for starting jails are in the /etc/rc.conf file and such still
works here in Texas on local servers when upgraded. However, the
production server is in Los Angeles and cannot react to any serious boot
problems without sending the ISP owner downtown to switch my HD back to
a bootable clone before the upgrade attempts. I cannot afford to have
the jail offline as it contains my SSL stuff vital to the business.

I manage the jails using the tools that came with Jail(8).

Can anyone point me in the right direction on this issue? Do I now need
stuff in sysctl.conf for the jail? Or.

Thanks for any help!!


Your mention of sysctl.conf makes me wonder: missing syscalls, or
missing sysctls? There was a change between 7.0 and 7.4 where some
jail-related sysctls were added.

There was also a change to the jail(2) system call, with a new version
of struct jail - but the old version continues to work. I don't think
there were and new syscalls at that point - perhaps other new sysctls?

Is the problem in actually creating the jail, or in processes that run
inside the jail? If the latter, there's really a lot of things that
could be the problem, if you're running an un-updated user space. Not
that they *should* happen, mind you, but still a lot more room for
something to go wrong.

- Jamie
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

Re: Boot-time jails (new jail)

[Jamie Gritton]

When it says rc.conf(5) it means the file and not the man page. And "the
file" is /etc/defaults/rc.conf and not /etc/rc.conf. So it should just
say "Please refer to the ``jail_*'' variables in /etc/defaults/rc.conf."

- Jamie


On 09/21/12 01:11, Matt Burke wrote:

Am I missing something, or if you follow the jail(8) man page (this is 9.1
r239869 btw), do you end up with jails you can only start at boot time via
a 'jail -c' hack in /etc/rc.local?

The man page only has this to say:


It is possible to have jails started at boot time. Please refer to the
``jail_*'' variables in rc.conf(5) for more information.


Yet the rc.conf(5) man page makes no mention of /etc/jail.conf. In fact it
seems the rc.d/jail script hasn't been touched since the new jail MFC and
is incompatible with the new conf file mechanism...

The last entry regarding jails in UPDATING is 20090527.




___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

Recent jail problems [was: ICMP RAW socket error]

[Jamie Gritton]

On 11/09/12 03:38, Beeblebrox wrote:

My jail used to work fine but it seems a recent update broke some things. My
kernel/world has INET6 disblad and there are already 2 threads re the error
that setting causes with jails. Now this error - is it a bug or am I missing
something?

My /etc/sysctl.conf has
security.jail.allow_raw_sockets=1
I even confirm it separately before starting the jail:
# sysctl security.jail.allow_raw_sockets

* I start the jail but isc-dhcpd cannot start: unable to create icmp socket:
Operation not permitted
* I cannot ping from the jail: ping: socket: Operation not permitted
* This error also, but inetd starts: inetd[50866]: madvise() failed:
Operation not permitted

No point in setting up a dhcp debug log, the error is in the network
setting.
/etc/devfs.rules has: add path net unhide,  add path 'net/*' unhide
My system is not exposed to the outside so it does not have pf / nat running
on it.


It is indeed a bug, same as bin/173469, except with allow_raw_sockets
instead of sysvipc_allowed. With the recent change to rc.d/jail, the
jail permission sysctls are no longer used in starting up jails.R

You can get past it for now by setting
jail_xxx_parameters="allow.raw_sockets" in your rc.conf. But that's a
hack fix, and we need something better before any of this goes out to
release.

Here's the problem: I had planned to deprecate those global sysctls in
favor of per-jail parameters, and only "old style" usage of the jail
command paid attention to them so existing setups would work. But I
didn't explicitly say that anywhere, leaving that as part of the new
config file based setup. But the recent patch to allow arbitrary
parameters in rc.d/jail necessitated a switch to the new jail command line.

I think the jail_xxx_parameters patch needs to go, or least rc.d/jail
needs to be a bit more complex, and run the old command line when a jail
doesn't have any "_parameters". That way the old semantics will be
preserved for existing users.

- Jamie
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

Re: Recent jail problems [was: ICMP RAW socket error]

[Jamie Gritton]

On 11/25/12 02:46, Beeblebrox wrote:

I'm probably overlooking certain things with this question, but there seems
to be a number of places to make jail-specific adjustments / settings. The
options available are:

host /etc/rc.conf
host /etc/sysctl.conf
host /etc/devfs.rules
host /etc/jail.conf

Q1: With regards to getting around this problem
(jail_xyz_parameters="allow.raw_sockets"), I tried placing this in
/etc/rc.conf but I also had in /etc/sysctl.conf:
security.jail.allow_raw_sockets=1
security.jail.socket_unixiproute_only=1
Maybe the 2 settings are in conflict?

Q2: As far as I understand, jail related things in rc.conf, sysctl.conf and
devfs.rules should now be migrated to jail.conf - Is that correct?
Thanks.


Jail.conf will replace the jail settings in rc.conf and sysctl.conf, but
it doesn't yet. The rc system still uses the jail program in its old
command-line mode, and not with the conf files. devfs.rules will keep
its jail-related rules though.

- Jamie
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

kern/68189 and kern/169751: what jails are allowed to see in a routing socket

[Jamie Gritton]

I've been looking at PR kern/169751, which was noting that routing 
sockets don't work inside a jail.  It made the point that setting 
security.jail.socket_unixiproute_only or security.jail.allow_raw_sockets 
didn't help things.  It would seem kind of a given from the 
"unixiproute" name that a route socket ought to work.  And indeed, such 
a socket is permitted to be created in such a jail.  It's just using it 
that doesn't work.


I narrowed this failure down to line 816 of sys/net/rtsock.c, which 
explicitly denies jails from reading routes, regardless of the setting 
of the above two sysctls (or the jail allow.* bits they work with).  And 
that bit of code came about in response to PR kern/68189, which noted 
that jails could see interfaces that aren't theirs (i.e. their address 
doesn't live on it).


So we have two PRs that are kind of at cross purposes.  It would be nice 
to keep hiding non-jail interfaces from a jail, but it would also be 
nice to let a jailed process know the route to somewhere - at least 
sometimes.  One solution would be to add a much finer layer of control 
to the jail test in rtsock.c, looking at interfaces and seeing if 
they're somehow connected with one of the jail's IP addresses.  But that 
just seems like a lot of messy corner-case code.


Another way around this, and what I'd like to go with if there are no 
objections, is to allow the route sockets to be used by jails that have 
raw_sockets permission.  I know that's kind of a semantic leap, but it 
seems that a jail that has the power of using raw sockets would be able 
to do pretty much as it pleases with routes anyway if it tried hard 
enough.  Also, it would be consistent to allow such operations on jails 
that aren't IP-restricted, or in VIMAGE jails.


- Jamie
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

Re: kern/68189 and kern/169751: what jails are allowed to see in a routing socket

[Jamie Gritton]

On 01/03/13 02:36, Bjoern A. Zeeb wrote:

On Wed, 2 Jan 2013, Jamie Gritton wrote:


I've been looking at PR kern/169751, which was noting that routing
sockets don't work inside a jail. It made the point that setting
security.jail.socket_unixiproute_only or
security.jail.allow_raw_sockets didn't help things. It would seem kind
of a given from the "unixiproute" name that a route socket ought to
work. And indeed, such a socket is permitted to be created in such a
jail. It's just using it that doesn't work.

I narrowed this failure down to line 816 of sys/net/rtsock.c, which
explicitly denies jails from reading routes, regardless of the setting
of the above two sysctls (or the jail allow.* bits they work with).
And that bit of code came about in response to PR kern/68189, which
noted that jails could see interfaces that aren't theirs (i.e. their
address doesn't live on it).

So we have two PRs that are kind of at cross purposes. It would be
nice to keep hiding non-jail interfaces from a jail, but it would also
be nice to let


jails have no notion of interfaces, only addresses, so by defintiion
there cannot be "non-jail interfaces".


Technically yes. But jails do have IP addresses that are tied to
interfaces. Still, there's too much of a morass that direction.


a jailed process know the route to somewhere - at least sometimes. One
solution would be to add a much finer layer of control to the jail
test in rtsock.c, looking at interfaces and seeing if they're somehow
connected with one of the jail's IP addresses. But that just seems
like a lot of messy corner-case code.

Another way around this, and what I'd like to go with if there are no
objections, is to allow the route sockets to be used by jails that
have raw_sockets permission. I know that's kind of a semantic leap,
but it seems that a jail that has the power of using raw sockets would
be able to do pretty much as it pleases with routes anyway if it tried
hard enough. Also, it would be consistent to allow such operations on
jails that aren't IP-restricted, or in VIMAGE jails.


I have not further looked at the code but the answer is that we should
not further complicate jails after 14 years when we have a perfect
good solution for the problem; vnets are there for exactly this.
Someone with enough interest and time should just finish things (tm) ;-)


I would at least want to open all network things up to jails that have
no network restrictions, because they aren't really "jails in the
network sense."


Meanwhile your suggestion might be ok given simple enough, but I wonder
if a different flag would be helpful still. I would not be able to
"trust" (the little that is possible anyway) raw_sockets anymore if they
suddently could fiddle with the routing table - even read-only, should
that really be enough.
I would explicitly advertise it as 'do not use - will go away again'
feature and it should the moment vnets are declared non-experimental.

Just my 2cts.

/bz


Well I'd rather not introduce something as a stopgap. Either this is
worth doing or it isn't. It does make sense to at least make sure it
works with VNET.

- Jamie
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

Re: problem stoping jails with jail(8), jail.conf and mount.fstab

[Jamie Gritton]

On 02/12/13 07:47, Harald Schmalzbauer wrote:

  Hello,

on 9.1-R, I highly appreciate the new jail(8) and jail.conf
capabilities. Thanks for that extension!

But I have one problem: If I want to stop a jail with 'jaill -r
jailname', I get "umount: unmount of /.jail.jailname failed: Device busy"

It seems to me that the order of fstab.jailname entries are not reverted
by jail(8) when shutting down/umounting.
My C skills don't allow me to verify/fix that in usr.sbin/jail/command.c

Can anybody help please?

Thanks,

-Harry



Yes, that's a serious drawback.  I'll work something up for that.

- Jamie
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

Re: Marking some FS as jailable

[Jamie Gritton]

On 02/14/13 06:27, Baptiste Daroussin wrote:

On Tue, Feb 12, 2013 at 10:06:29PM -0700, Jamie Gritton wrote:

On 02/12/13 12:40, Baptiste Daroussin wrote:


I would like to mark some filesystem as jailable, here is the one I need:
linprocfs, tmpfs and fdescfs, I was planning to do it with adding a
allow.mount.${fs} for each one.

Anyone has an objection?


Would it make sense for linprocfs to use the existing allow.mount.procfs
flag?


Here is a patch that uses allow.mount.procfs for linsysfs and linprocfs.

It also addd a new allow.mount.tmpfs to allow tmpfs.

It seems to work here, can anyone confirm this is the right way to do it?

I'll commit in 2 parts: first lin*fs, second tmpfs related things

http://people.freebsd.org/~bapt/jail-fs.diff


There are some problems. The usage on the mount side of things looks
correct, but it needs more on the jail side. I'm including a patch just
of that part, with a correction in jail.h and further changes in kern_jail.c

- Jamie
Index: sys/jail.h
===
--- sys/jail.h  (revision 246791)
+++ sys/jail.h  (working copy)
@@ -227,7 +227,8 @@
 #definePR_ALLOW_MOUNT_NULLFS   0x0100
 #definePR_ALLOW_MOUNT_ZFS  0x0200
 #definePR_ALLOW_MOUNT_PROCFS   0x0400
-#definePR_ALLOW_ALL0x07ff
+#definePR_ALLOW_MOUNT_TMPFS0x0800
+#definePR_ALLOW_ALL0x0fff
 
 /*
  * OSD methods
Index: kern/kern_jail.c
===
--- kern/kern_jail.c(revision 246791)
+++ kern/kern_jail.c(working copy)
@@ -206,6 +206,7 @@
"allow.mount.nullfs",
"allow.mount.zfs",
"allow.mount.procfs",
+   "allow.mount.tmpfs",
 };
 const size_t pr_allow_names_size = sizeof(pr_allow_names);
 
@@ -221,6 +222,7 @@
"allow.mount.nonullfs",
"allow.mount.nozfs",
"allow.mount.noprocfs",
+   "allow.mount.notmpfs",
 };
 const size_t pr_allow_nonames_size = sizeof(pr_allow_nonames);
 
@@ -4208,6 +4210,10 @@
 CTLTYPE_INT | CTLFLAG_RW | CTLFLAG_MPSAFE,
 NULL, PR_ALLOW_MOUNT_PROCFS, sysctl_jail_default_allow, "I",
 "Processes in jail can mount the procfs file system");
+SYSCTL_PROC(_security_jail, OID_AUTO, mount_tmpfs_allowed,
+CTLTYPE_INT | CTLFLAG_RW | CTLFLAG_MPSAFE,
+NULL, PR_ALLOW_MOUNT_TMPFS, sysctl_jail_default_allow, "I",
+"Processes in jail can mount the tmpfs file system");
 SYSCTL_PROC(_security_jail, OID_AUTO, mount_zfs_allowed,
 CTLTYPE_INT | CTLFLAG_RW | CTLFLAG_MPSAFE,
 NULL, PR_ALLOW_MOUNT_ZFS, sysctl_jail_default_allow, "I",
@@ -4360,6 +4366,8 @@
 "B", "Jail may mount the nullfs file system");
 SYSCTL_JAIL_PARAM(_allow_mount, procfs, CTLTYPE_INT | CTLFLAG_RW,
 "B", "Jail may mount the procfs file system");
+SYSCTL_JAIL_PARAM(_allow_mount, tmpfs, CTLTYPE_INT | CTLFLAG_RW,
+"B", "Jail may mount the tmpfs file system");
 SYSCTL_JAIL_PARAM(_allow_mount, zfs, CTLTYPE_INT | CTLFLAG_RW,
 "B", "Jail may mount the zfs file system");
 
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

Re: Marking some FS as jailable

[Jamie Gritton]

On 02/14/13 07:56, Baptiste Daroussin wrote:

On Thu, Feb 14, 2013 at 07:40:58AM -0700, Jamie Gritton wrote:

On 02/14/13 06:27, Baptiste Daroussin wrote:

On Tue, Feb 12, 2013 at 10:06:29PM -0700, Jamie Gritton wrote:

On 02/12/13 12:40, Baptiste Daroussin wrote:


I would like to mark some filesystem as jailable, here is the one I need:
linprocfs, tmpfs and fdescfs, I was planning to do it with adding a
allow.mount.${fs} for each one.

Anyone has an objection?


Would it make sense for linprocfs to use the existing allow.mount.procfs
flag?


Here is a patch that uses allow.mount.procfs for linsysfs and linprocfs.

It also addd a new allow.mount.tmpfs to allow tmpfs.

It seems to work here, can anyone confirm this is the right way to do it?

I'll commit in 2 parts: first lin*fs, second tmpfs related things

http://people.freebsd.org/~bapt/jail-fs.diff


There are some problems. The usage on the mount side of things looks
correct, but it needs more on the jail side. I'm including a patch just
of that part, with a correction in jail.h and further changes in kern_jail.c


Thank you the patch has been updated with your fixes.


One more bit (literally): PR_ALLOW_ALL in sys/jail.h needs updating.

- Jamie
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

Re: new jail(8) ignoring devfs_ruleset?

[Jamie Gritton]

On 02/15/13 09:27, Harald Schmalzbauer wrote:

  Hello,

like already posted, on 9.1-R, I highly appreciate the new jail(8) and
jail.conf capabilities. Thanks for that extension!

Accidentally I saw that "devfs_ruleset" seems to be ignored.
If I list /dev/ I see all the hosts disk devices etc.
I set "devfs_ruleset = 4;" and "enforce_statfs = 1;" in jail.conf.
   Inside the jail,
sysctl security.jail.devfs_ruleset returnes "1".
But like mentioned, I can access all devices...

Thanks for any help,

-Harry


devfs_ruleset is only used along with mount.devfs - do you also have
that set in jail.conf?

- Jamie
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

Re: new jail(8) ignoring devfs_ruleset?

[Jamie Gritton]

On 02/18/13 01:54, Harald Schmalzbauer wrote:

  schrieb Jamie Gritton am 16.02.2013 00:40 (localtime):

On 02/15/13 09:27, Harald Schmalzbauer wrote:

   Hello,

like already posted, on 9.1-R, I highly appreciate the new jail(8) and
jail.conf capabilities. Thanks for that extension!

Accidentally I saw that "devfs_ruleset" seems to be ignored.
If I list /dev/ I see all the hosts disk devices etc.
I set "devfs_ruleset = 4;" and "enforce_statfs = 1;" in jail.conf.
Inside the jail,
sysctl security.jail.devfs_ruleset returnes "1".
But like mentioned, I can access all devices...

Thanks for any help,

-Harry


devfs_ruleset is only used along with mount.devfs - do you also have
that set in jail.conf?


Thanks for your response.

Yes, I have mount.devfs; set.
Otherwise I wouldn't have any device inside my jail. Verified - and like
intended, right?
Another notable discrepancy: The man page tells that devfs_rulset is "4"
by default.
But when I don't set devfs_rulset in jail.conf at all, inside the jail,
'sysctl security.jail.devfs_ruleset': 0
When set, like mentioned above, it returns the corresponding value, but
it doesn't have any effect.
How gets devfs_rulset handled? Does jail(8) do the whole job? I'd like
to help finding the source, but have missed the whole new jail evolution...
Inside my jails, I don't have a fstab, outside I have them defined and
enabled with "mount" - and noticed the non-reverted umounting.


I found the problem - I noticed you mentioned 9.1-R, and took a look at
devfs(5). On CURRENT, there's a mount option "ruleset", that isn't there
on 9.

So I'll have to get around it by running devfs(8) after the mount. I'll
work on a patch for that.

- Jamie
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

Re: new jail(8) ignoring devfs_ruleset?

[Jamie Gritton]

On 02/18/13 09:29, Mateusz Guzik wrote:

On Mon, Feb 18, 2013 at 09:26:42AM -0700, Jamie Gritton wrote:

On 02/18/13 01:54, Harald Schmalzbauer wrote:

  schrieb Jamie Gritton am 16.02.2013 00:40 (localtime):

On 02/15/13 09:27, Harald Schmalzbauer wrote:

   Hello,

like already posted, on 9.1-R, I highly appreciate the new jail(8) and
jail.conf capabilities. Thanks for that extension!

Accidentally I saw that "devfs_ruleset" seems to be ignored.
If I list /dev/ I see all the hosts disk devices etc.
I set "devfs_ruleset = 4;" and "enforce_statfs = 1;" in jail.conf.
Inside the jail,
sysctl security.jail.devfs_ruleset returnes "1".
But like mentioned, I can access all devices...

Thanks for any help,

-Harry


devfs_ruleset is only used along with mount.devfs - do you also have
that set in jail.conf?


Thanks for your response.

Yes, I have mount.devfs; set.
Otherwise I wouldn't have any device inside my jail. Verified - and like
intended, right?
Another notable discrepancy: The man page tells that devfs_rulset is "4"
by default.
But when I don't set devfs_rulset in jail.conf at all, inside the jail,
'sysctl security.jail.devfs_ruleset': 0
When set, like mentioned above, it returns the corresponding value, but
it doesn't have any effect.
How gets devfs_rulset handled? Does jail(8) do the whole job? I'd like
to help finding the source, but have missed the whole new jail evolution...
Inside my jails, I don't have a fstab, outside I have them defined and
enabled with "mount" - and noticed the non-reverted umounting.


I found the problem - I noticed you mentioned 9.1-R, and took a look at
devfs(5). On CURRENT, there's a mount option "ruleset", that isn't there
on 9.

So I'll have to get around it by running devfs(8) after the mount. I'll
work on a patch for that.



Why not MFC support for that mount option instead?


That may be a better way around it, since either solution will require
an MFC. It'd be nice to have a patch to jail(8) anyway, since just
dropping in a new jail program is easier than dropping in a new kernel.

I'll have to take a look at the devfs code and see if that was a
reasonably small change.

- Jamie
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

Re: new jail(8) ignoring devfs_ruleset?

[Jamie Gritton]

On 02/18/13 09:29, Mateusz Guzik wrote:

On Mon, Feb 18, 2013 at 09:26:42AM -0700, Jamie Gritton wrote:

On 02/18/13 01:54, Harald Schmalzbauer wrote:

  schrieb Jamie Gritton am 16.02.2013 00:40 (localtime):

On 02/15/13 09:27, Harald Schmalzbauer wrote:

   Hello,

like already posted, on 9.1-R, I highly appreciate the new jail(8) and
jail.conf capabilities. Thanks for that extension!

Accidentally I saw that "devfs_ruleset" seems to be ignored.
If I list /dev/ I see all the hosts disk devices etc.
I set "devfs_ruleset = 4;" and "enforce_statfs = 1;" in jail.conf.
Inside the jail,
sysctl security.jail.devfs_ruleset returnes "1".
But like mentioned, I can access all devices...

Thanks for any help,

-Harry


devfs_ruleset is only used along with mount.devfs - do you also have
that set in jail.conf?


Thanks for your response.

Yes, I have mount.devfs; set.
Otherwise I wouldn't have any device inside my jail. Verified - and like
intended, right?
Another notable discrepancy: The man page tells that devfs_rulset is "4"
by default.
But when I don't set devfs_rulset in jail.conf at all, inside the jail,
'sysctl security.jail.devfs_ruleset': 0
When set, like mentioned above, it returns the corresponding value, but
it doesn't have any effect.
How gets devfs_rulset handled? Does jail(8) do the whole job? I'd like
to help finding the source, but have missed the whole new jail evolution...
Inside my jails, I don't have a fstab, outside I have them defined and
enabled with "mount" - and noticed the non-reverted umounting.


I found the problem - I noticed you mentioned 9.1-R, and took a look at
devfs(5). On CURRENT, there's a mount option "ruleset", that isn't there
on 9.

So I'll have to get around it by running devfs(8) after the mount. I'll
work on a patch for that.



Why not MFC support for that mount option instead?


I wasn't quite right about it not being in 9.1. I was looking at my 9.0
desktop, and it's not there. But it was in fact MFCd into 9.1. So I'm
back to saying as long as you use the devfs_ruleset parameter, your
jailed /dev should be correct.

- Jamie
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

Re: vnet jails and rc-scripts

[Jamie Gritton]

On 02/26/13 01:56, Andreas Nilsson wrote:


However I still don't get the purpose of the security.jail.param.*. Are the
to be set in loader.conf/sysctl.conf to influence default config of jails,
or are the supposed to be per-jail ( from inside jail ) carriers of config?
The PR seems to indicate it's not really clear.

Also, man jail says:
"The current set of available parameters can be
  retrieved via ``sysctl -d security.jail.param''.  Any parameters not
set
  will be given default values, often based on the current environment.
  The core parameters are:
"
and then lists some. For example jid. I take that to mean that the value
  of security.jail.param.jid from inside jail should return the jid of the
jail. I just get 0. And security.jail.param.path is 1024, which is not at
all the path of the jail... There seems to be quite a discrepancy between
manpage and implementation.


The bit that the man page says is in fact the entire (user-visible) user
for those sysctls: they're just there to show what parameters are
available, and what types they are. Actually, they also show jail(8) the
same thing, and that's how it knows what parameters exist.

But the parameters don't actually have any useful values. Only their
types, sizes and descriptions are valid.

- Jamie
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

Re: vnet jails and rc-scripts

[Jamie Gritton]

On 02/27/13 01:30, Andreas Nilsson wrote:

On Wed, Feb 27, 2013 at 5:44 AM, Jamie Gritton mailto:ja...@freebsd.org>> wrote:

On 02/26/13 01:56, Andreas Nilsson wrote:

However I still don't get the purpose of the
security.jail.param.*. Are the
to be set in loader.conf/sysctl.conf to influence default config
of jails,
or are the supposed to be per-jail ( from inside jail ) carriers
of config?
The PR seems to indicate it's not really clear.

Also, man jail says:
"The current set of available parameters can be
   retrieved via ``sysctl -d security.jail.param''.  Any
parameters not
set
   will be given default values, often based on the current
environment.
   The core parameters are:
"
and then lists some. For example jid. I take that to mean that
the value
   of security.jail.param.jid from inside jail should return the
jid of the
jail. I just get 0. And security.jail.param.path is 1024, which
is not at
all the path of the jail... There seems to be quite a
discrepancy between
manpage and implementation.


The bit that the man page says is in fact the entire (user-visible) user
for those sysctls: they're just there to show what parameters are
available, and what types they are. Actually, they also show jail(8) the
same thing, and that's how it knows what parameters exist.


Ok. I'm feeling a bit daft here, from within a jail do they say "these
parameters can be set" or "those parameters have been set"?


It's still a matter of "these parameters can be set." Well, if your jail
has been granted permission to create sub-jails. They're read-only
values (or more properly, read-only non-values), so they appear the same
regardless of environment.


But the parameters don't actually have any useful values. Only their
types, sizes and descriptions are valid.

Ok, somewhat disappointing ;) Is there a ongoing effort to teach rc and
friends about difference between jails and vnet jails? Or is it deemed a
security problem that a jail knows the "circumstances of its conception"?


It hasn't really been a problem until vnet jails came along. No, there's
been no effort I know of to teach jails their particulars, but then
neither has there been any particular effort to hide them.

- Jamie
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

Re: IPv4 addresses clash / jails not working after reboot…

[Jamie Gritton]

On 03/07/13 05:29, Yoann Gini wrote:


Le 7 mars 2013 à 10:58, Boris Samorodov  a écrit :


07.03.2013 12:48, Yoann Gini пишет:


I need to share this IP, I’ve only one and I would like to avoid playing with 
NAT…


One IP may be shared but for different services (ports).


That what I’ve understand and what I’ve planned.


If someone have a idea…


Give some more information:
1. OS version, OS arch.


FreeBSD srv0.public.example.com 9.1-RELEASE FreeBSD 9.1-RELEASE #0 r243825: Tue 
Dec  4 09:23:10 UTC 2012 
r...@farrell.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC  amd64


2. Jail configuration (at least one) from /etc and LOCALBASE/etc/ezjail.


What do you want in /etc ? Except the fstab, I don’t see any config here, the 
fstab look like that:

/home/jails/basejail /home/jails/front0.public.example.com/basejail nullfs ro 0 0
/usr/ports  /home/jails/front0.public.example.com/usr/ports nullfs 
ro 0 0

And here is the ezjail config

export jail_front0_public_example_com_hostname="front0.public.example.com"
export jail_front0_public_example_com_ip=« IPv6Prefix::80,SharedIPv4,10.42.0.2"
export 
jail_front0_public_example_com_rootdir="/home/jails/front0.public.example.com"
export jail_front0_public_example_com_exec_start="/bin/sh /etc/rc"
export jail_front0_public_example_com_exec_stop=""
export jail_front0_public_example_com_mount_enable="YES"
export jail_front0_public_example_com_devfs_enable="YES"
export jail_front0_public_example_com_devfs_ruleset="devfsrules_jail"
export jail_front0_public_example_com_procfs_enable="YES"
export jail_front0_public_example_com_fdescfs_enable="YES"
export jail_front0_public_example_com_image=""
export jail_front0_public_example_com_imagetype=""
export jail_front0_public_example_com_attachparams=""
export jail_front0_public_example_com_attachblocking=""
export jail_front0_public_example_com_forceblocking=""
export jail_front0_public_example_com_zfs_datasets=""
export jail_front0_public_example_com_cpuset=""
export jail_front0_public_example_com_fib=""


3. What do you want to achieve.


I want a setup with:
— srv0 listen only for SSH on a alternate port for supervision on public IPv4/6 
;
— front0 to handle any public services (web, DNS, e-mail) on public IPv4/6 ;
— service0 to handle internal services (git, redmine, AFP sharepoints…) on 
private IP and SSH on a other alternate port on public IPv4/6 ;
— gateway0 to act as a VPN server and webproxy to secure access to private 
services on service0 and act as a secure gateway to encrypt network traffic for 
road-warriors on public network.

In the end, I will dispatch those services on different server but for now I 
only access to one system, so I would like to prepare the setup to be 
dispatched on different hardware when the budget come.

Actually, if I remove the SharedIPv4 from the jails, it works.

I’ve investigate more on the open socket area and I think the problem come from 
Apache who still lisent on *:* even if I’ve set a Listen directive…


You're allowed to have the same address in multiple jails, but only in
the case of jails that have one address (i.e. one IPv4 address in this
case). Jails with multiple IP addresses can't share any of those
addresses with other jails. I don't know why it should work once and
then not work later though.

The jail config you show has only a single jail, so I also wonder what
it's clashing with - a clash is defined as the same IP address between
two different jails. Are there other jail configs you didn't show?

Also, there's a chance a jail has been removed but is not yet gone
(though I wouldn't expected that case in a reboot situation). Could you
run "jls -dn" immediately after the failed jail start, and tell any
output it gives?

- Jamie
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

Re: jail.conf & cpuset.id

[Jamie Gritton]

On 03/17/13 05:59, Nicolas de Bari Embriz Garcia Rojas wrote:

Hi, all, I am start using the jail.conf for running my jails, in rc.local I 
have this line jail -c this to start my jails at boot time (any better ideas)

Now checking the man pages for the jail I found a option that cough my 
attention, 'cpuset.id'  any idea of how to use it ?

I would like to found a way to prevent a root user within a jail to run a 
'fork-bum' and freeze the host server.


Take a look at cpuset(1). You use that utility (in the host environment)
to change the CPUs available to a jail. Don't worry about the cpuset.id
parameter itself - you don't need it. Just use cpuset's "-j" flag to
specify the jail itself (by jid only). When you're starting jails in rc,
add the appropriate cpuset commands an exec_poststart option. Such as:

jail_backtest_poststart0="cpuset -c -l1,3-7 -j`cat 
/var/run/jail_backtest.id`"


- Jamie
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

Re: new jail(8) ignoring devfs_ruleset?

[Jamie Gritton]

On 03/21/13 17:59, Miroslav Lachman wrote:

Jeremie Le Hen wrote:

On Mon, Feb 18, 2013 at 09:54:42AM +0100, Harald Schmalzbauer wrote:

schrieb Jamie Gritton am 16.02.2013 00:40 (localtime):

On 02/15/13 09:27, Harald Schmalzbauer wrote:

Hello,

like already posted, on 9.1-R, I highly appreciate the new jail(8) and
jail.conf capabilities. Thanks for that extension!

Accidentally I saw that "devfs_ruleset" seems to be ignored.
If I list /dev/ I see all the hosts disk devices etc.
I set "devfs_ruleset = 4;" and "enforce_statfs = 1;" in jail.conf.
Inside the jail,
sysctl security.jail.devfs_ruleset returnes "1".
But like mentioned, I can access all devices...

Thanks for any help,

-Harry


devfs_ruleset is only used along with mount.devfs - do you also have
that set in jail.conf?


Thanks for your response.

Yes, I have mount.devfs; set.
Otherwise I wouldn't have any device inside my jail. Verified - and like
intended, right?
Another notable discrepancy: The man page tells that devfs_rulset is "4"
by default.
But when I don't set devfs_rulset in jail.conf at all, inside the jail,
'sysctl security.jail.devfs_ruleset': 0
When set, like mentioned above, it returns the corresponding value, but
it doesn't have any effect.
How gets devfs_rulset handled? Does jail(8) do the whole job? I'd like
to help finding the source, but have missed the whole new jail
evolution...
Inside my jails, I don't have a fstab, outside I have them defined and
enabled with "mount" - and noticed the non-reverted umounting.


Look at what's in /dev from you jail. There should a few pseudo
devices (see below), but no real devices:

$ ls /dev
crypto log ptmx random stdin urandom zfs
fd null pts stderr stdout zero


I can confirm mentioned problem on my FreeBSD 9.1-RELEASE amd64 GENERIC

I am now testing new jail.conf possibilities and I am seeing all devices
in /dev in jail.

Even if I set all this in my jail.conf

exec.start = "/bin/sh /etc/rc";
exec.stop = "/bin/sh /etc/rc.shutdown";
exec.clean;
mount.devfs;
devfs_ruleset = 4;
allow.set_hostname = false;

path = "/vol0/jail/$name";
exec.consolelog = "/var/log/jail/$name.console";
mount.fstab = "/etc/fstab.$name";

## Jail bali
bali {
host.hostname = "bali.XXX.YY;
ip4.addr = xx.xx.xx.xx;
devfs_ruleset = 4;
}





# jexec 4 tcsh

root@bali:/ # ls -l /dev/
total 4
crw-r--r-- 1 root wheel 0, 35 Mar 1 19:39 acpi
lrwxr-xr-x 1 root wheel 4 Mar 22 00:46 ad10 -> ada3
lrwxr-xr-x 1 root wheel 6 Mar 22 00:46 ad10s1 -> ada3s1
lrwxr-xr-x 1 root wheel 7 Mar 22 00:46 ad10s1a -> ada3s1a
lrwxr-xr-x 1 root wheel 7 Mar 22 00:46 ad10s1b -> ada3s1b
lrwxr-xr-x 1 root wheel 7 Mar 22 00:46 ad10s1d -> ada3s1d
lrwxr-xr-x 1 root wheel 7 Mar 22 00:46 ad10s1e -> ada3s1e
lrwxr-xr-x 1 root wheel 7 Mar 22 00:46 ad10s1f -> ada3s1f
lrwxr-xr-x 1 root wheel 7 Mar 22 00:46 ad10s1g -> ada3s1g
lrwxr-xr-x 1 root wheel 6 Mar 22 00:46 ad10s2 -> ada3s2
lrwxr-xr-x 1 root wheel 7 Mar 22 00:46 ad10s2a -> ada3s2a
lrwxr-xr-x 1 root wheel 7 Mar 22 00:46 ad10s2b -> ada3s2b
lrwxr-xr-x 1 root wheel 7 Mar 22 00:46 ad10s2d -> ada3s2d
lrwxr-xr-x 1 root wheel 7 Mar 22 00:46 ad10s2e -> ada3s2e
lrwxr-xr-x 1 root wheel 4 Mar 22 00:46 ad4 -> ada0
lrwxr-xr-x 1 root wheel 4 Mar 22 00:46 ad6 -> ada1
lrwxr-xr-x 1 root wheel 4 Mar 22 00:46 ad8 -> ada2
lrwxr-xr-x 1 root wheel 6 Mar 22 00:46 ad8s1 -> ada2s1
lrwxr-xr-x 1 root wheel 7 Mar 22 00:46 ad8s1a -> ada2s1a
lrwxr-xr-x 1 root wheel 7 Mar 22 00:46 ad8s1b -> ada2s1b
lrwxr-xr-x 1 root wheel 7 Mar 22 00:46 ad8s1d -> ada2s1d
lrwxr-xr-x 1 root wheel 7 Mar 22 00:46 ad8s1e -> ada2s1e
lrwxr-xr-x 1 root wheel 7 Mar 22 00:46 ad8s1f -> ada2s1f
lrwxr-xr-x 1 root wheel 7 Mar 22 00:46 ad8s1g -> ada2s1g
lrwxr-xr-x 1 root wheel 6 Mar 22 00:46 ad8s2 -> ada2s2
lrwxr-xr-x 1 root wheel 7 Mar 22 00:46 ad8s2a -> ada2s2a
lrwxr-xr-x 1 root wheel 7 Mar 22 00:46 ad8s2b -> ada2s2b
lrwxr-xr-x 1 root wheel 7 Mar 22 00:46 ad8s2d -> ada2s2d
lrwxr-xr-x 1 root wheel 7 Mar 22 00:46 ad8s2e -> ada2s2e
crw-r- 1 root operator 0, 106 Mar 1 19:39 ada0
crw-r- 1 root operator 0, 108 Mar 1 19:39 ada1
crw-r- 1 root operator 0, 114 Mar 1 19:39 ada2
crw-r- 1 root operator 0, 120 Mar 1 19:39 ada2s1
crw-r- 1 root operator 0, 130 Mar 1 19:39 ada2s1a
crw-r- 1 root operator 0, 132 Mar 1 19:39 ada2s1b
crw-r- 1 root operator 0, 134 Mar 1 19:39 ada2s1d
crw-r- 1 root operator 0, 136 Mar 1 19:39 ada2s1e
crw-r- 1 root operator 0, 138 Mar 1 19:39 ada2s1f
crw-r- 1 root operator 0, 140 Mar 1 19:39 ada2s1g
crw-r- 1 root operator 0, 122 Mar 1 19:39 ada2s2
crw-r- 1 root operator 0, 142 Mar 1 19:39 ada2s2a
crw-r- 1 root operator 0, 144 Mar 1 19:39 ada2s2b
crw-r- 1 root operator 0, 146 Mar 1 19:39 ada2s2d
crw-r- 1 root operator 0, 148 Mar 1 19:39 ada2s2e
crw-r-

Re: new jail(8) ignoring devfs_ruleset?

[Jamie Gritton]

On 03/21/13 18:20, Miroslav Lachman wrote:

Jamie Gritton wrote:

On 03/21/13 17:59, Miroslav Lachman wrote:

Jeremie Le Hen wrote:

On Mon, Feb 18, 2013 at 09:54:42AM +0100, Harald Schmalzbauer wrote:

schrieb Jamie Gritton am 16.02.2013 00:40 (localtime):

On 02/15/13 09:27, Harald Schmalzbauer wrote:

Hello,

like already posted, on 9.1-R, I highly appreciate the new jail(8)
and
jail.conf capabilities. Thanks for that extension!

Accidentally I saw that "devfs_ruleset" seems to be ignored.
If I list /dev/ I see all the hosts disk devices etc.
I set "devfs_ruleset = 4;" and "enforce_statfs = 1;" in jail.conf.
Inside the jail,
sysctl security.jail.devfs_ruleset returnes "1".
But like mentioned, I can access all devices...


[...]


I can confirm mentioned problem on my FreeBSD 9.1-RELEASE amd64 GENERIC

I am now testing new jail.conf possibilities and I am seeing all devices
in /dev in jail.

Even if I set all this in my jail.conf

exec.start = "/bin/sh /etc/rc";
exec.stop = "/bin/sh /etc/rc.shutdown";
exec.clean;
mount.devfs;
devfs_ruleset = 4;
allow.set_hostname = false;

path = "/vol0/jail/$name";
exec.consolelog = "/var/log/jail/$name.console";
mount.fstab = "/etc/fstab.$name";

## Jail bali
bali {
host.hostname = "bali.XXX.YY;
ip4.addr = xx.xx.xx.xx;
devfs_ruleset = 4;
}


[...]


Is it a problem in my understanding of manpage / configuration, or is it
a bug in jail command on 9.1-RELEASE?


It's a bug (deficiency) in the jail command.


Is there a workaround or is it impossible to use jails with devfs on
FreeBSD 9.1?
Shouldn't it be mentioned in 9.1 errata?

Is it fixed in stable/9?

Thank you for your reply and your great work on new jails!


It's not fixed anywhere yet - it sometimes works in current, and
sometimes doesn't. I've been meaning to patch it up, but it the problem
is what I think it is, the patching up is a pretty big operation.

It doesn't mean you can't use jails with devfs in 9.1, just that you
can't use them with jail.conf. The old jail rc file that's all
shell-based is still the official jail startup method, and that one
still works. So existing systems will still work as expected, hence no
errata.

- Jamie
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

Re: numeric jail name in jail.conf

[Jamie Gritton]

On 03/22/13 05:25, Nicolas de Bari Embriz Garcia Rojas wrote:

Hi, when using a numeric names for a jails something like:

10 {
 exec.start = "/bin/sh /etc/rc";
 exec.stop = "/bin/sh /etc/rc.shutdown";
 
}

20 {
 exec.start = "/bin/sh /etc/rc";
 exec.stop = "/bin/sh /etc/rc.shutdown";
 
}

the name (numeric) becomes the jail ID when, therefore when I type a jls
command i see the  10,and 20 as the jails ID's,

My question is, is this the proper way to explicitly set the id when
using jail.conf ?

For me, this as been working fine and I can match the jail ids to use
rctl on every reboot, but just want to be sure if this is the correct
way of doing it.

thanks in advance.


That's a fine and proper way to do it. You can also set it in the body
of the jail definition with e.g. "jid = 10". For that matter, these two
definitions are nearly equivalent:

10 {
name = "foo";
...
}

foo {
jid = 10;
...
}

I say "nearly" because while they'll create the same jail, you would use
10 or foo respectively on the command line if you were operating on one
of them. For example, if you wanted to start only one jail in a conf
file, "jail -c foo" would work only for the second definition.

- Jamie
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

Re: rc.d/jail and jail.conf

[Jamie Gritton]

On 03/30/13 14:59, Dirk Engling wrote:

On Sat, 30 Mar 2013, Nicolas de Bari Embriz Garcia Rojas wrote:


If I am right you can define the order of start for jails in the
jail2_list (rc.conf), something like:

jail2_list="jail1 jail2"


Thanks, I suppose it mimicks the way rc.d/jail has handled it. I just
wondered if there's a way to have this order automatically determined by
a dependency graph, it's a minor request, though.


jail(8) itself does the dependency graph. So the jail2 startup needs to
run a single jail command instead of one for each jail.

- Jamie
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

Re: rc.d/jail and jail.conf

[Jamie Gritton]

On 03/31/13 09:12, Miroslav Lachman wrote:

Jamie Gritton wrote:

On 03/30/13 14:59, Dirk Engling wrote:

On Sat, 30 Mar 2013, Nicolas de Bari Embriz Garcia Rojas wrote:


If I am right you can define the order of start for jails in the
jail2_list (rc.conf), something like:

jail2_list="jail1 jail2"


Thanks, I suppose it mimicks the way rc.d/jail has handled it. I just
wondered if there's a way to have this order automatically determined by
a dependency graph, it's a minor request, though.


jail(8) itself does the dependency graph. So the jail2 startup needs to
run a single jail command instead of one for each jail.


So it means jail2 should be fixed, because it runs
jail -c -i -J /var/run/jail_${_j}.id ${_j}
for each jail from jail2_list="jailA jailB"


Yes it does.


Is there a way to disable jail defined in jail.conf? (to avoid
jail2_list in rc.conf)


I'm not sure what you're asking. You want a jail in jail.conf that's not
started up?


And what happens if there is jail2_list="jailA jailB" in rc.conf and
jailB is defined in jail.conf as dependency of jailA? I guess rc.d/jail2
will try to start jailB again.
It will be started as dependency of jailA by first jail command starting
jailA.
Or new jail(8) doesn't start the "depend" jail automatically and just
check its existence?
I didn't try it yet.


It will start jailB as part of starting jailA, and then it will try to
start jailB again on its own. So yes, it needs work.

- Jamie
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

Re: rc.d/jail and jail.conf

[Jamie Gritton]

On 03/31/13 11:09, Miroslav Lachman wrote:

Jamie Gritton wrote:

On 03/31/13 09:12, Miroslav Lachman wrote:

>>

Is there a way to disable jail defined in jail.conf? (to avoid
jail2_list in rc.conf)


I'm not sure what you're asking. You want a jail in jail.conf that's not
started up?


Yes, I am asking if there can be some variable or parametr in jail.conf
for jail which we don't want to start by jail command, but leave its
configuration in jail.conf.
I am not saying I need it right now, but I can imagine a scenario where
it can be useful.

In the old style with rc.conf, we can have defined for example 5 jails
(jailA to jailE) and then enabled only some of them to start at boot
time by defining jail_list="jailA jailB jailC".

With syntax of new jail.conf one must delete or comment out the whole
jailD and jailE definitions to stop loading them at boot time.
Am I right?


There is a way, though not in the jail.conf file itself. When you run
"jail -c" it will start all of the jails in the file. But if you list
one or more jails on the command line, e.g. "jail -c jailA jailB", then
it will only start those jails.


So is it possible to add some keyword to jail.conf jails definition?
Something like "disabled" or "noautostart" or anything else...

foo {
disabled;
host.hostname = "foo.com";
ip4.addr = 10.1.1.1, 10.1.1.2, 10.1.1.3;
}

Then one can easily disable jail "foo" without a need to remove its
configuration.


That seems reasonable, but using a jail list in rc.conf may suffice.

- Jamie
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

Re: rc.d/jail and jail.conf

[Jamie Gritton]

On 03/31/13 12:58, Dirk Engling wrote:

On 31.03.13 20:31, Jamie Gritton wrote:


That seems reasonable, but using a jail list in rc.conf may suffice.


It is less error prone to just use 'jail_list=*' in rc.conf and disable
jails per config block, and then issue a warning like 'Skipping disabled
jail(s) foo, bar, baz'.

(Although I see this from a perspective of jails being virtual hosts
that should be configurable by automated tools, too.)


If you don't mind some slightly difficult error messages, you can always
"disable" a jail with exec.prestart="false". jail(8) requires all
commands to succeed, and in particular won't even create a jail when one
of the prestart commands fails.

- Jamie
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

Re: rc.d/jail and jail.conf

[Jamie Gritton]

On 03/31/13 20:01, Paul Schenkeveld wrote:

On Sun, Mar 31, 2013 at 09:14:23PM +0200, Dirk Engling wrote:


On Sun, 31 Mar 2013, Jamie Gritton wrote:


If you don't mind some slightly difficult error messages, you can always
"disable" a jail with exec.prestart="false". jail(8) requires all
commands to succeed, and in particular won't even create a jail when one
of the prestart commands fails.


This violates POLA, but failing with

exec.prestart="echo skipping jail; exit 1"

might work. Even though this is not a good marker from a scripting
perspective.


Will this prevent all preparations from happening, i.e. will filesystems
be mounted for jails disabled this way?


It will unroll anything that's been done, or at least try to. So
filesystems will be mounted, and then unmounted.


Although this may work, I think that this looks dirty.  I'd really prefer
a "disabled" or "noauto" keyword instead.


Oh it's definitely dirty - just something I threw out there as a hack.

- Jamie
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

Re: rc.d/jail and jail.conf

[Jamie Gritton]

On 03/31/13 14:58, Dirk Engling wrote:

On 31.03.13 22:01, Miroslav Lachman wrote:


So I guess, I am out of luck here, because users used to think of their
jails as what they saw in the hostname field on jls. If I am writing
tools that use jail_getid to map the jailname to the jid, it will never
match that hostname and I also can not copy the hostname to the jailname.


I understand what you are talking about, but jails in these days are
something different from what jails were at the begining in 4.x days and
users must accept that jailname is something different than hostname.



In these days, you can have jails with many IP addresses or without IP
address. Hostname needn't to be unique etc.

Dot (.) is not allowed in jailname because of hierarchical jails,
where dot is used as hierarchy separator.


Humm, this seems a strange thing to answer to my question. Once you see
jails as virtual servers (which I understand is not the only way to do,
but the biased way I and most jail users I talk to happen to deploy them
in huge quantities), the natural approach to name them is via their
hostname. I find it hard to grasp to tell them "don't" ;)

And still I find the choice of '.' as a separator unfortunate, '/'
springs in mind, but there might have been reasons.


It just seemed like the natural thing to do. Possibly inspired by
sysctl, but honestly I don't recall. I hadn't considered that jails
would be named after the hostname, probably because I'd never done that
with my own jails or non-jail virtualization.

- Jamie
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

Re: rc.d/jail and jail.conf

[Jamie Gritton]

On 03/31/13 21:53, Ian Smith wrote:
> On Sun, 31 Mar 2013 22:58:33 +0200, Dirk Engling wrote:
>>  Maybe meeting at a BSDcon over a beer would help ;)
>
> Unlikely to hurt, anyway :)

Perhaps I need to plan on going to BSDCan after all...

- Jamie
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

Re: jail(8) vs. rc.d/jail features - fstab, zfs, vnet

[Jamie Gritton]

On 04/11/13 12:48, Dirk Engling wrote:

Dear jail hackers,

in my ongoing quest to understand the direction jail development is
heading, I noticed that per-jail-fstabs are not (anymore?, yet?)
supported by the new jail(8)-rc.d/jail2-combo. Are there official plans
to drop the support?

A nice new jail+zfs feature is the "zfs jail" command, allowing to
attach a zfs to a jail. The way[tm] to properly use this feature is to
first create a prison, attach the zfs file system(s) to the jid and only
then run the exec.start command. So either jail(8) needs to be zfs aware
and execute the zfs jail command(s) by itself, or a exec.postprestart
command that is being passed the prison id of the new jail needs to be
run in system context. (For shutting down and unjail-ing vice versa).

The same goes with the vimage features. Most of the ways I can think of
using vnet interfaces require some configuration in the host system
after the vnet has been attached to the jid but before exec.start (and
thus the jail's rc.d/netif) is executed.

Since I speak C, posix and sh fluidly, I am willing to implement or help
implementing any of the proposals in question, so do not misunderstand
them as just demands ;)


As mentioned, fstab support is there.

I have a parameter, vnet.interface, which runs the necessary "ifconfig
... vnet" command between jail creation and exec.start. We could have a
similar parameter for zfs, or we could create another set of exec.*
parameters, which would be more flexible in the long run. But as you
hinted at with "postprestart", there doesn't seem to be a good logical
name for it.

Since the vnet.interface command exists, you could use that as a guide
for adding a "zfs" parameter. vnet.interface only does anything on jail
creation, as the interface automatically reverts to the parent on jail
removal. I don't know if the zfs stuff works the same way - if it
doesn't, then you'll need a similar "prepoststop" kind of operation.

- Jamie
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

Re: IPv4 addresses clash / jails not working after reboot…

[Jamie Gritton]

On 04/12/13 10:53, Łukasz Wąsikowski wrote:

W dniu 2013-03-08 00:22, Jamie Gritton pisze:


You're allowed to have the same address in multiple jails, but only in
the case of jails that have one address (i.e. one IPv4 address in this
case). Jails with multiple IP addresses can't share any of those
addresses with other jails. I don't know why it should work once and
then not work later though.


That's not true. You can have multiple IPs in jails. You can have
multiple jails sharing the same IP. You can have multiple jails sharing
the same multiple IPs. So:

jail1: ipv4_ip1
jail2: ipv4_ip2, ipv6_ip1, ipv6_ip2
jail3: ipv4_ip2, ipv6_ip1, ipv6_ip2

will work. But configuration like this:

jail1: ipv4_ip1
jail2: ipv4_ip2, ipv6_ip1, ipv6_ip2
jail3: ipv4_ip2, ipv6_ip1

will not, because jail2 and jail3 share only some IPs. I've tried
configuration like this on 9.1-STABLE around december 2012 and it ended
with panic. So I'm using the configuration from the first example and it
works ok.


Well ending in a panic is beyond the bounds of what's supported, and
into what apparently is broken - I was just talking about the intent as
I read it into the code. Is this panic of yours repeatable? I'd like to
get the exact configuration you were using, so I could try to repeat
(and fix) whatever the problem was.

- Jamie
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

Re: jail(8) vs. rc.d/jail features - fstab, zfs, vnet

[Jamie Gritton]

On 04/13/13 23:32, Dirk Engling wrote:

On 12.04.13 01:58, Jamie Gritton wrote:


similar parameter for zfs, or we could create another set of exec.*
parameters, which would be more flexible in the long run. But as you
hinted at with "postprestart", there doesn't seem to be a good logical
name for it.


hmm, maybe the prestart is misnamed, as it actually happens pre-create.
Then prestart would be exactly where we would put the vnet and dataset
magic - aftercreate but prestart.


Unfortunately it's too late to rename prestart. But we could sneak in a
postcreate. For limited use, you can work around this by not having any
exec.start, and then doing things in exec.poststart with the last
poststart script being a jexec command to run the jail's /etc/rc (or
whatever).


I also think, it might be helpful, if the environment could be set up
that helps the exec.* scripts finding their way back to the jail in
question. Right now I'd have to create a script for every jail, but setting

JAIL_JID=
JAIL_NAME=

before executing the scripts could make scripting nicer, especially
finding the jid is tedious in scripts. I also thought about substitution
of parameters, but maybe providing the jail name as parameter should be
the job for the jail.conf's authors.


The cleanest way is with parameter substitution. I like the environment
idea, but I think that also would fit well in a command line. My command
lines run exec in a shell if they have any shell metacharacters
(!"$&'()*;<>?[\]`{|}~). If I had thought ahead, I would have added to
that an '=' at least for the first argument, and then we could send the
environment the sh way. I should make that change in case someone wants
to try that in the future, but for the present I suppose you'd have to
use the env command.

Another possibility would be adding a new exec.env parameter, where you
can set whatever you like. I'd like to avoid having any default
environment variables that aren't somehow specified in the jail.conf.

One improvement I really need to make is to make the jid accessible.
Right now, you have to do something like `jls -j $name jid` in a command
line, which is inelegant to say the least. What I need is a $jid to be
set even when it isn't specified in the config, which is a little
complicated but doable if I just get down to it.


Since the vnet.interface command exists, you could use that as a guide
for adding a "zfs" parameter. vnet.interface only does anything on jail
creation, as the interface automatically reverts to the parent on jail
removal. I don't know if the zfs stuff works the same way - if it
doesn't, then you'll need a similar "prepoststop" kind of operation.


Looking at the zone_dataset_attach code I understand that jailed zfs
datasets are a property of the prison struct, saved as osd and vanish
when the jail dies. So it should not be necessary to undo the "zfs jail"
command. However there seems not to be a nice way to find out if any
datasets are attached to a jail, besides trying to attach them.

With that knowledge I think adding a zfs_datasets option for the
jail.conf is rather straight forward and more likely to be useful
immediately than having the user construct wrapper scripts to find out
about the jid of the jail that can then be passed to a number of "zfs
jail JID DATASET" calls.

Shall I proceed?


Perhaps it would be worthwhile to add a zfs.dataset parameter to go
along with the OSD data, to make that visible to the outside world. It
would be a nice use of the subsystem-dependent parameter stuff currently
used only by Linux emulation. The other option is to code it into
jail(8) itself, though I like the former better.

Proceed away :-).

- Jamie
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

Re: automatic garbage collection of stuff mounted (etc.) by jailed root

[Jamie Gritton]

On 04/22/13 03:17, Mateusz Guzik wrote:

Hello,

This is something that imho could be done by GSoC student.

It is possible to allow jailed root to mount various filesystems. But
once all processes are dead, mounts done by jailed root that he didn't
clean up are still hanging around.

As time passes and more stuff gets jailable we should expect problems
like this in different subsystems.

So I propose that someone(tm) implements a solution which cleans this
stuff during jail destruction.

One idea how to do it: implement a list with clean up operations. Using
mount example: you add a filesystem to be cleaned up after it is
mounted, you delete it after it is unmounted. When the jail is going to
die you just traverse the list backwareds and call cleaning functions,
in this case unmounting filesystems. Maybe this is is a bad idea in the
first place and it is better to take a look at mount tree and traverse
that, I don't know, you should investigate. :) Note that the code has to
be robust in case of errors (e.g. given fs may not be unmountable
because someone from prison0 is inside).

Again, the goal is to have jails clean up automatically after anything
jailed root was permitted to do.

Thoughts?


This already happens when jails are created using a jail.conf file. Any
mounts there are unmounted as part of the jail removal process. Just
recently I fixed it to properly do this unmounting in reverse order.

- Jamie
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

Re: automatic garbage collection of stuff mounted (etc.) by jailed root

[Jamie Gritton]

On 04/22/13 11:39, Miroslav Lachman wrote:

Jamie Gritton wrote:

On 04/22/13 03:17, Mateusz Guzik wrote:


[...]


Again, the goal is to have jails clean up automatically after anything
jailed root was permitted to do.

Thoughts?


This already happens when jails are created using a jail.conf file. Any
mounts there are unmounted as part of the jail removal process. Just
recently I fixed it to properly do this unmounting in reverse order.


Do you mean mounts defined in jail.conf or all mounts manually done by
root user in jail?

Miroslav Lachman


Ah, I see the difference. Yes, that's only for mounts in the jail.conf.
For mounts done by the jail itself, I guess we would go off the mount
record's credential. So is this something you expect to be happening
entirely in the kernel?

- Jamie
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"