Hi Arturo, On Sun, Aug 13, 2017 at 03:13:50PM +0200, Arturo Borrero Gonzalez wrote: > On 12 August 2017 at 06:15, Steve Langasek <steve.langa...@canonical.com> > wrote:
> > The conntrack-tools 1.4.4+snapshot20161117 update was blocked from reaching > > Ubuntu's 17.04 release, because it regresses its autopkgtests in Ubuntu > > compared to 1.4.3-3. > Hi Steve, > thanks for your work, comments below. > > I have so far identified two problems with the tests when running on Ubuntu: > > - the tests modprobe a bunch of nf_conntrack_* modules; if some of these > > modules don't load because they are built into the kernel (as is the case > > for some of them on Ubuntu), these tests fail. > This is good to have. I can upstream the patch. > > - several tests are marked 'isolation-container', but provide a > > configuration to conntrackd that requires host-level privileges, so > > conntrackd fails at startup when run in a container. > The scheduling & bufsize things seems a bit hackish for what we need: > a simple test run of conntrackd. > I would simplify the logic by just ignoring these configuration > options and using conntrackd/system defaults. > I mean: in this test case, we are interested in checking that > conntrackd can run with a basic configuration. The actual > configuration is less important. Ok, I wasn't sure so I went with a solution that made only minimal changes to the config when testing on a machine / VM. But if this is not part of the behavior that's important to test, I agree that it's better to drop so we get the same test behavior in VMs and containers. > Please note, that scheduling defaults have changed in conntrackd [0]. > If we don't specify a scheduling configuration, conntrackd will try to > use SCHED_RR by default. Which may require further privileges? I don't > have a full autopkgtest environment to test different setups (i.e., > LXC, virtualization, etc). I see that the upstream patch does: + if (sched_setscheduler(0, sched_type, &schedparam) < 0) + dlog(LOG_WARNING, "scheduler configuration failed: %s. " + "Likely a bug in conntrackd, please report it. " + "Continuing with system default scheduler.", + strerror(errno)); So that's fine, this will just fall back gracefully in a container. > So, please, I would ask you to: > 1) send a (separate) patch for the modules things > 2) make changes to the test suite according to the simplifications I > mentioned (separate patch) Ok, attached the split out patches. > I believe that by following this approach Ubuntu will benefit as well. > > [0] > http://git.netfilter.org/conntrack-tools/commit/?id=210f5429678dba06f361b1f37bcb946f27e2e20b Cheers, -- Steve Langasek Give me a lever long enough and a Free OS Debian Developer to set it on, and I can move the world. Ubuntu Developer http://www.debian.org/ slanga...@ubuntu.com vor...@debian.org
diff -Nru conntrack-tools-1.4.4+snapshot20161117/debian/patches/series conntrack-tools-1.4.4+snapshot20161117/debian/patches/series --- conntrack-tools-1.4.4+snapshot20161117/debian/patches/series 2016-12-05 03:52:20.000000000 -0800 +++ conntrack-tools-1.4.4+snapshot20161117/debian/patches/series 2017-08-11 14:50:43.000000000 -0700 @@ -1 +1,2 @@ missing-include.patch +skip-already-loaded-modules.patch diff -Nru conntrack-tools-1.4.4+snapshot20161117/debian/patches/skip-already-loaded-modules.patch conntrack-tools-1.4.4+snapshot20161117/debian/patches/skip-already-loaded-modules.patch --- conntrack-tools-1.4.4+snapshot20161117/debian/patches/skip-already-loaded-modules.patch 1969-12-31 16:00:00.000000000 -0800 +++ conntrack-tools-1.4.4+snapshot20161117/debian/patches/skip-already-loaded-modules.patch 2017-08-11 20:53:24.000000000 -0700 @@ -0,0 +1,56 @@ +Description: Don't fail on modprobe since the driver might be built-in + Any of these nf drivers could be built-ins instead of modules; don't cause + the testsuite to fail on modprobe, instead let it proceed and succeed/fail + later based on actual test results. + . + Ideally we would check up front if the driver is loaded rather than trying + to modprobe and ignoring failures, but there doesn't seem to be a reliable + place to check this in the kernel filesystem +Author: Steve Langasek <steve.langa...@ubuntu.com> + +Index: conntrack-tools-1.4.4+snapshot20161117/tests/conntrack/run-test.sh +=================================================================== +--- conntrack-tools-1.4.4+snapshot20161117.orig/tests/conntrack/run-test.sh ++++ conntrack-tools-1.4.4+snapshot20161117/tests/conntrack/run-test.sh +@@ -10,10 +10,12 @@ + # + # XXX: module auto-load not support by nfnetlink_cttimeout yet :-( + # +-modprobe nf_conntrack_ipv4 +-modprobe nf_conntrack_ipv6 +-modprobe nf_conntrack_proto_udplite +-modprobe nf_conntrack_proto_sctp +-modprobe nf_conntrack_proto_dccp +-modprobe nf_conntrack_proto_gre ++# any or all of these might be built-ins rather than modules, so don't error ++# out on failure from modprobe ++modprobe nf_conntrack_ipv4 || true ++modprobe nf_conntrack_ipv6 || true ++modprobe nf_conntrack_proto_udplite || true ++modprobe nf_conntrack_proto_sctp || true ++modprobe nf_conntrack_proto_dccp || true ++modprobe nf_conntrack_proto_gre || true + ./test testcases +Index: conntrack-tools-1.4.4+snapshot20161117/tests/nfct/run-test.sh +=================================================================== +--- conntrack-tools-1.4.4+snapshot20161117.orig/tests/nfct/run-test.sh ++++ conntrack-tools-1.4.4+snapshot20161117/tests/nfct/run-test.sh +@@ -11,10 +11,12 @@ + # + # XXX: module auto-load not support by nfnetlink_cttimeout yet :-( + # +-modprobe nf_conntrack_ipv4 +-modprobe nf_conntrack_ipv6 +-modprobe nf_conntrack_proto_udplite +-modprobe nf_conntrack_proto_sctp +-modprobe nf_conntrack_proto_dccp +-modprobe nf_conntrack_proto_gre ++# any or all of these might be built-ins rather than modules, so don't error ++# out on failure from modprobe ++modprobe nf_conntrack_ipv4 || true ++modprobe nf_conntrack_ipv6 || true ++modprobe nf_conntrack_proto_udplite || true ++modprobe nf_conntrack_proto_sctp || true ++modprobe nf_conntrack_proto_dccp || true ++modprobe nf_conntrack_proto_gre || true + ./test timeout diff -Nru conntrack-tools-1.4.4+snapshot20161117/debian/tests/conntrack-test.sh conntrack-tools-1.4.4+snapshot20161117/debian/tests/conntrack-test.sh --- conntrack-tools-1.4.4+snapshot20161117/debian/tests/conntrack-test.sh 2016-11-17 00:25:52.000000000 -0800 +++ conntrack-tools-1.4.4+snapshot20161117/debian/tests/conntrack-test.sh 2017-08-11 21:39:40.000000000 -0700 @@ -5,6 +5,9 @@ : INFO flushing all information conntrack -F +# Load the module in case it's not already loaded +modprobe nf_conntrack_ipv4 || true + ARGS_TIMEOUT="--timeout 100" ARGS_IP="--src 127.0.0.1 --dst 127.0.0.1" diff -Nru conntrack-tools-1.4.4+snapshot20161117/debian/tests/control conntrack-tools-1.4.4+snapshot20161117/debian/tests/control --- conntrack-tools-1.4.4+snapshot20161117/debian/tests/control 2016-12-07 01:35:34.000000000 -0800 +++ conntrack-tools-1.4.4+snapshot20161117/debian/tests/control 2017-08-11 20:55:01.000000000 -0700 @@ -24,8 +24,8 @@ Tests: conntrack-internal-testsuite.sh Depends: @, @builddeps@, kmod -Restrictions: needs-root, isolation-machine, build-needed +Restrictions: needs-root, isolation-machine, build-needed, allow-stderr Tests: nfct-internal-testsuite.sh Depends: @, @builddeps@, kmod -Restrictions: needs-root, isolation-machine, build-needed +Restrictions: needs-root, isolation-machine, build-needed, allow-stderr
diff -Nru conntrack-tools-1.4.4+snapshot20161117/debian/tests/basic-daemon-test.sh conntrack-tools-1.4.4+snapshot20161117/debian/tests/basic-daemon-test.sh
--- conntrack-tools-1.4.4+snapshot20161117/debian/tests/basic-daemon-test.sh 2016-11-17 00:25:52.000000000 -0800
+++ conntrack-tools-1.4.4+snapshot20161117/debian/tests/basic-daemon-test.sh 2017-08-14 14:27:41.000000000 -0700
@@ -55,20 +55,12 @@
Port 3780
Interface lo
Checksum on
- SndSocketBuffer 12492800
- RcvSocketBuffer 12492800
}
Options {
ExpectationSync On
}
}
General {
- Nice -20
- Scheduler {
- Type FIFO
- Priority 99
- }
-
Syslog on
LockFile $LOCKFILE
UNIX {
diff -Nru conntrack-tools-1.4.4+snapshot20161117/debian/tests/systemd-service-test.sh conntrack-tools-1.4.4+snapshot20161117/debian/tests/systemd-service-test.sh
--- conntrack-tools-1.4.4+snapshot20161117/debian/tests/systemd-service-test.sh 2016-11-17 00:25:52.000000000 -0800
+++ conntrack-tools-1.4.4+snapshot20161117/debian/tests/systemd-service-test.sh 2017-08-14 14:27:57.000000000 -0700
@@ -26,6 +26,18 @@
return 0
}
+get_ethernet_device()
+{
+ for dev in /sys/class/net/*; do
+ if [ $(cat "$dev/type") = 1 ]; then
+ echo $(basename "$dev")
+ break
+ fi
+ done
+}
+
+ETHER=$(get_ethernet_device)
+
echo "
Sync {
Mode NOTRACK {
@@ -37,22 +49,14 @@
IPv4_address 127.0.0.1
IPv4_Destination_Address 127.0.0.1
Port 3780
- Interface eth0
+ Interface $ETHER
Checksum on
- SndSocketBuffer 12492800
- RcvSocketBuffer 12492800
}
Options {
ExpectationSync On
}
}
General {
- Nice -20
- Scheduler {
- Type FIFO
- Priority 99
- }
-
Syslog on
LockFile /var/lock/conntrackd.lock
UNIX {
signature.asc
Description: PGP signature
