pictuga
/
morss
1
1
Fork 0
Code Issues Pull Requests Actions Packages Releases Activity

crawler: cache 301 HTTP code, for a week

master v1.0
pictuga 2017-03-09 09:37:05 -10:00 committed by GitHub
parent e8023e4336
commit c952b85d92
1 changed files with 5 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

5
morss/crawler.py
View File

@ -308,6 +308,11 @@ class BaseCacheHandler(BaseHandler):
# still recent enough for us, use cache # still recent enough for us, use cache
pass pass
elif code == 301 and cache_age < 7*24*3600:
# "301 Moved Permanently" has to be cached...as long as we want (awesome HTTP specs), let's say a week (why not?)
# use force_min=0 if you want to bypass this (needed for a proper refresh)
pass
else: else:
# according to the www, we have to refresh when nothing is said # according to the www, we have to refresh when nothing is said
return None return None