 pictuga
|
51f1d330a4
|
Fn to access data_files & pkg files
continuous-integration/drone Build is running
Details
continuous-integration/drone/push Build is passing
Details
|
2021-12-05 12:09:01 +01:00 |
|
pictuga
|
eb47aac6f1
|
morss: respect timeout settings in all cases
continuous-integration/drone/push Build is failing
Details
Special treatment of feed fetch not justified and not documented
|
2021-11-25 22:13:38 +01:00 |
|
pictuga
|
eca546b890
|
Change HTTP error code to 404
continuous-integration/drone/push Build is failing
Details
To tell them apart from 'true' 500 errors
|
2021-11-25 21:34:46 +01:00 |
|
pictuga
|
d8cc07223e
|
readabilite: fix bug when nothing above threshold
continuous-integration/drone/push Build is failing
Details
|
2021-11-23 20:53:00 +01:00 |
|
pictuga
|
765e0ba728
|
Pass py error msg in http headers
continuous-integration/drone/push Build is passing
Details
|
2021-11-22 23:22:13 +01:00 |
|
pictuga
|
6ec3fb47d1
|
readabilite: .strip() first to save time
continuous-integration/drone/push Build is passing
Details
|
2021-11-15 21:54:07 +01:00 |
|
pictuga
|
1083f3ffbc
|
crawler: make sure to use HTTPMessage
continuous-integration/drone/push Build is passing
Details
|
2021-11-11 10:21:48 +01:00 |
|
pictuga
|
7eeb1d696c
|
crawler: clean up code
continuous-integration/drone/push Build is passing
Details
|
2021-11-10 23:25:03 +01:00 |
|
pictuga
|
e42df98f83
|
crawler: fix regression brought with 44a6b2591
continuous-integration/drone/push Build is passing
Details
|
2021-11-10 23:08:31 +01:00 |
|
pictuga
|
cb21871c35
|
crawler: clean up caching code
continuous-integration/drone/push Build is passing
Details
|
2021-11-08 22:02:23 +01:00 |
|
pictuga
|
c71cf5d5ce
|
caching: fix diskcache implementation
|
2021-11-08 21:57:43 +01:00 |
|
pictuga
|
44a6b2591d
|
crawler: cleaner http header object import
|
2021-11-07 19:44:36 +01:00 |
|
pictuga
|
a890536601
|
morss: comment code a bit
|
2021-11-07 18:26:07 +01:00 |
|
pictuga
|
8de309f2d4
|
caching: add diskcache backend
|
2021-11-07 18:15:20 +01:00 |
|
pictuga
|
cbf7b3f77b
|
caching: simplify sqlite code
|
2021-11-07 18:14:18 +01:00 |
|
pictuga
|
d023ec8d73
|
Change default port to 8000
|
2021-10-19 22:19:59 +02:00 |
|
pictuga
|
5473b77416
|
Post-clean up isort
continuous-integration/drone/push Build is passing
Details
|
2021-09-21 08:11:04 +02:00 |
|
pictuga
|
0365232a73
|
readabilite: custom xpath for article detection
continuous-integration/drone/push Build is failing
Details
|
2021-09-21 08:04:45 +02:00 |
|
pictuga
|
a523518ae8
|
cache: avoid name collision
|
2021-09-21 08:04:45 +02:00 |
|
pictuga
|
52c48b899f
|
readability: better var names
|
2021-09-21 08:04:45 +02:00 |
|
pictuga
|
9649cabb1b
|
morss: do not crash on empty pages
|
2021-09-21 08:04:45 +02:00 |
|
pictuga
|
10535a17c5
|
cache: fix isort
|
2021-09-21 08:04:45 +02:00 |
|
pictuga
|
7d86972e58
|
Add Redis cache backend
|
2021-09-21 08:04:45 +02:00 |
|
pictuga
|
5da7121a77
|
Fix Options class behaviour
|
2021-09-21 08:04:45 +02:00 |
|
pictuga
|
bb82902ad1
|
Move cache code to its own file
|
2021-09-21 08:04:45 +02:00 |
|
pictuga
|
04afa28fe7
|
crawler: cache pickle'd array
|
2021-09-21 08:04:45 +02:00 |
|
pictuga
|
75bb69f0fd
|
Make mysql optdep
|
2021-09-21 08:04:45 +02:00 |
|
pictuga
|
97d9dda547
|
crawler: support 308 redirects
|
2021-09-21 08:04:45 +02:00 |
|
pictuga
|
6c1f8da692
|
ci: added pylint (triggered upon error w/ score < 8 only)
|
2021-09-21 08:04:45 +02:00 |
|
pictuga
|
4fd730b983
|
Further isort implementation
|
2021-09-21 08:04:45 +02:00 |
|
pictuga
|
0b3e6d7749
|
Apply isort
|
2021-09-21 08:04:23 +02:00 |
|
pictuga
|
06e0ada95b
|
Allow POST requests
|
2021-09-08 20:43:21 +02:00 |
|
pictuga
|
37f5a92b05
|
wsgi: fix apache / workload
|
2021-09-06 22:01:48 +02:00 |
|
pictuga
|
24c26d3850
|
wsgi: handle url decoding
|
2021-08-31 21:51:21 +02:00 |
|
pictuga
|
8f24214915
|
crawler: better name for custom fns
|
2021-08-29 00:22:40 +02:00 |
|
pictuga
|
d5942fe5a7
|
feeds: fix issues when mode not explicited in ruleset
|
2021-08-29 00:20:29 +02:00 |
|
pictuga
|
6f50443995
|
morss: Options return None instead of False if no match
Better for default fn values
|
2021-08-29 00:19:09 +02:00 |
|
pictuga
|
5582fbef31
|
crawler: comment
|
2021-08-29 00:18:50 +02:00 |
|
pictuga
|
da5442a1dc
|
feedify: support any type (json, xml, html)
|
2021-08-29 00:17:28 +02:00 |
|
pictuga
|
f9d7794bcc
|
feedifi.ini: json time zone handling
|
2021-04-23 20:43:00 +02:00 |
|
pictuga
|
e37c8346d0
|
feeds: add fallback for time parser
|
2021-04-22 21:57:16 +02:00 |
|
pictuga
|
3a1d564992
|
feeds: fix time zone handling
|
2021-04-22 21:51:00 +02:00 |
|
pictuga
|
6880a443e0
|
crawler: improve CacheHandler code
|
2021-03-25 23:54:08 +01:00 |
|
pictuga
|
7342ab26d2
|
crawler: comment on how urllib works
|
2021-03-25 23:49:58 +01:00 |
|
pictuga
|
981da9e66a
|
crawler: SQLITE_PATH point to .db file instead of folder
|
2021-03-25 23:48:21 +01:00 |
|
pictuga
|
01a7667032
|
Fix error due to remaining log force code
|
2021-01-14 00:51:47 +01:00 |
|
pictuga
|
3e886caaab
|
crawler: drop encoding setting
|
2020-10-30 22:41:16 +01:00 |
|
pictuga
|
ad927e03a7
|
crawler: use regex instead of lxml
Less reliable but should be faster
|
2020-10-30 22:21:19 +01:00 |
|
pictuga
|
0efb096fa7
|
crawler: shift gzip & encoding-fix to intermediary handler
|
2020-10-30 22:16:51 +01:00 |
|
pictuga
|
9ab2e488ef
|
crawler: add intermediary handlers
|
2020-10-30 22:15:35 +01:00 |