Tags: patch
You seem to have hit just exactly the wrong package combination there.
w3lib 1.19 had changes that failed this test for Scrapy 1.5.0; but was
fixed in Scrapy even before this bug report was created [2].
Essentially w3lib 1.19 is too new here.
A 1.5 point release with this fix included may be coming around
soonish(tm) [1].
In the meantime, Scrapy 1.5.0 should depend on python-w3lib<=1.18.
(Or you could ship the patch at [2] in the interim.)
[1] https://github.com/scrapy/scrapy/issues/3323
[2]
https://github.com/scrapy/scrapy/commit/ba15b63ed696dbbd6aa6082c035754a15ca8e03c
; or see attachment>From ba15b63ed696dbbd6aa6082c035754a15ca8e03c Mon Sep 17 00:00:00 2001
From: Mikhail Korobov <kmik...@gmail.com>
Date: Fri, 26 Jan 2018 02:11:49 +0500
Subject: [PATCH] TST fix tests to account for changes in w3lib 1.19
---
tests/test_http_response.py | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/tests/test_http_response.py b/tests/test_http_response.py
index a36ec3af..b228344b 100644
--- a/tests/test_http_response.py
+++ b/tests/test_http_response.py
@@ -272,7 +272,10 @@ class TextResponseTest(BaseResponseTest):
headers={"Content-type": ["text/html; charset=utf-8"]},
body=b"\xef\xbb\xbfWORD\xe3\xab")
self.assertEqual(r6.encoding, 'utf-8')
- self.assertEqual(r6.text, u'WORD\ufffd\ufffd')
+ self.assertIn(r6.text, {
+ u'WORD\ufffd\ufffd', # w3lib < 1.19.0
+ u'WORD\ufffd', # w3lib >= 1.19.0
+ })
def test_bom_is_removed_from_body(self):
# Inferring encoding from body also cache decoded body as sideeffect,
--
2.11.0
