On 1/8/19 11:25 PM, PiBa-NL wrote:
Hi Frederic,
Hi Pieter,
Op 7-1-2019 om 10:13 schreef Frederic Lecaille:
On 12/23/18 11:38 PM, PiBa-NL wrote:
As requested hereby the regtest send for inclusion into the git
repository.
It is OK like that.
Note that you patch do not add reg-test/filters/common.pem which could
be a symlink to ../ssl/common.pem.
Also note that since 8f16148Christopher's commit, we add such a line
where possible:
${no-htx} option http-use-htx
We should also rename your test files to reg-test/filters/h00000.*
Thank you.
Fred.
Together with these changes you have supplied me already off-list, i've
also added a " --max-time 15" for the curl request, that should be
sufficient for most systems to complete the 3 second testcase, and
allows the shell command to complete without varnishtest killing it
after a timeout and not showing any of the curl output..
One last question, currently its being added to a new folder:
reg-test/filters/ , perhaps it should be in reg-test/compression/ ?
If you agree that needs changing i guess that can be done upon
committing it?
I have modified your patch to move your new files to reg-test/compression.
I have also applied this to it ;) : 's/\r$//'
Note that the test fails on my FreeBSD system when using HTX when using
'2.0-dev0-251a6b7 2019/01/08', i'm not aware it ever worked (i didn't
test it with HTX before..).
**** top 15.2 shell_out|curl: (28) Operation timed out after 15036
milliseconds with 187718 bytes received
Ok, I will take some time to have a look at this BSD specific issue.
Note that we can easily use the CLI at the end of the script to
troubleshooting anything.
Log attached.. Would it help to log it with the complete "filter trace
name BEFORE / filter compression / filter trace name AFTER" ? Or are
there other details i could try and gather?
I do not fill at ease enough on compression/filter topics to reply
to your question Pieter ;)
Nevertheless I think your test deserve to be merged.
*The patch to be merged is attached to this mail*.
Thank a lot Pieter.
Regards,
Fred.
>From a7d6b4d0c00973ab25f63491a1c860a5d14c28a9 Mon Sep 17 00:00:00 2001
From: PiBa-NL <[email protected]>
Date: Sun, 23 Dec 2018 21:21:51 +0100
Subject: [PATCH] REGTEST: filters: add compression test
This test checks that data transferred with compression is correctly received at different download speeds
---
reg-tests/compression/common.pem | 1 +
reg-tests/compression/s00000.lua | 19 +++++++++++++
reg-tests/compression/s00000.vtc | 59 ++++++++++++++++++++++++++++++++++++++++
3 files changed, 79 insertions(+)
create mode 120000 reg-tests/compression/common.pem
create mode 100644 reg-tests/compression/s00000.lua
create mode 100644 reg-tests/compression/s00000.vtc
diff --git a/reg-tests/compression/common.pem b/reg-tests/compression/common.pem
new file mode 120000
index 00000000..a4433d56
--- /dev/null
+++ b/reg-tests/compression/common.pem
@@ -0,0 +1 @@
+../ssl/common.pem
\ No newline at end of file
diff --git a/reg-tests/compression/s00000.lua b/reg-tests/compression/s00000.lua
new file mode 100644
index 00000000..2cc874b9
--- /dev/null
+++ b/reg-tests/compression/s00000.lua
@@ -0,0 +1,19 @@
+
+local data = "abcdefghijklmnopqrstuvwxyz"
+local responseblob = ""
+for i = 1,10000 do
+ responseblob = responseblob .. "\r\n" .. i .. data:sub(1, math.floor(i % 27))
+end
+
+http01applet = function(applet)
+ local response = responseblob
+ applet:set_status(200)
+ applet:add_header("Content-Type", "application/javascript")
+ applet:add_header("Content-Length", string.len(response)*10)
+ applet:start_response()
+ for i = 1,10 do
+ applet:send(response)
+ end
+end
+
+core.register_service("fileloader-http01", "http", http01applet)
diff --git a/reg-tests/compression/s00000.vtc b/reg-tests/compression/s00000.vtc
new file mode 100644
index 00000000..79e1ab56
--- /dev/null
+++ b/reg-tests/compression/s00000.vtc
@@ -0,0 +1,59 @@
+# Checks that compression doesnt cause corruption..
+
+varnishtest "Compression validation"
+#REQUIRE_VERSION=1.6
+
+feature ignore_unknown_macro
+
+haproxy h1 -conf {
+global
+# log stdout format short daemon
+ lua-load ${testdir}/s00000.lua
+
+defaults
+ mode http
+ log global
+ ${no-htx} option http-use-htx
+ option httplog
+
+frontend main-https
+ bind "fd@${fe1}" ssl crt ${testdir}/common.pem
+ compression algo gzip
+ compression type text/html text/plain application/json application/javascript
+ compression offload
+ use_backend TestBack if TRUE
+
+backend TestBack
+ server LocalSrv ${h1_fe2_addr}:${h1_fe2_port}
+
+listen fileloader
+ mode http
+ bind "fd@${fe2}"
+ http-request use-service lua.fileloader-http01
+} -start
+
+shell {
+ HOST=${h1_fe1_addr}
+ if [ "${h1_fe1_addr}" = "::1" ] ; then
+ HOST="\[::1\]"
+ fi
+
+ md5=$(which md5 || which md5sum)
+
+ if [ -z $md5 ] ; then
+ echo "MD5 checksum utility not found"
+ exit 1
+ fi
+
+ expectchecksum="4d9c62aa5370b8d5f84f17ec2e78f483"
+
+ for opt in "" "--limit-rate 300K" "--limit-rate 500K" ; do
+ checksum=$(curl --max-time 15 --compressed -k "https://$HOST:${h1_fe1_port}"; $opt | $md5 | cut -d ' ' -f1)
+ if [ "$checksum" != "$expectchecksum" ] ; then
+ echo "Expecting checksum $expectchecksum"
+ echo "Received checksum: $checksum"
+ exit 1;
+ fi
+ done
+
+} -run
--
2.11.0
