pictuga
/
morss
1
1
Fork 0
Code Issues Pull Requests Actions Packages Releases Activity

crawler: change http header after uncompressing 

Change content-encoding to "identity"
master
pictuga 2017-02-25 18:10:43 -10:00
parent e71fc967ce
commit 026903ce73
1 changed files with 1 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
morss/crawler.py
View File

@ -37,6 +37,7 @@ class GZIPHandler(BaseHandler):
if resp.headers.get('Content-Encoding') == 'gzip': if resp.headers.get('Content-Encoding') == 'gzip':
data = resp.read() data = resp.read()
data = GzipFile(fileobj=BytesIO(data), mode='r').read() data = GzipFile(fileobj=BytesIO(data), mode='r').read()
resp.headers['Content-Encoding'] = 'identity'
fp = BytesIO(data) fp = BytesIO(data)
old_resp = resp old_resp = resp