 pictuga
|
d023ec8d73
|
Change default port to 8000
|
2021-10-19 22:19:59 +02:00 |
|
pictuga
|
5473b77416
|
Post-clean up isort
continuous-integration/drone/push Build is passing
Details
|
2021-09-21 08:11:04 +02:00 |
|
pictuga
|
0365232a73
|
readabilite: custom xpath for article detection
continuous-integration/drone/push Build is failing
Details
|
2021-09-21 08:04:45 +02:00 |
|
pictuga
|
a523518ae8
|
cache: avoid name collision
|
2021-09-21 08:04:45 +02:00 |
|
pictuga
|
52c48b899f
|
readability: better var names
|
2021-09-21 08:04:45 +02:00 |
|
pictuga
|
9649cabb1b
|
morss: do not crash on empty pages
|
2021-09-21 08:04:45 +02:00 |
|
pictuga
|
10535a17c5
|
cache: fix isort
|
2021-09-21 08:04:45 +02:00 |
|
pictuga
|
7d86972e58
|
Add Redis cache backend
|
2021-09-21 08:04:45 +02:00 |
|
pictuga
|
5da7121a77
|
Fix Options class behaviour
|
2021-09-21 08:04:45 +02:00 |
|
pictuga
|
bb82902ad1
|
Move cache code to its own file
|
2021-09-21 08:04:45 +02:00 |
|
pictuga
|
04afa28fe7
|
crawler: cache pickle'd array
|
2021-09-21 08:04:45 +02:00 |
|
pictuga
|
75bb69f0fd
|
Make mysql optdep
|
2021-09-21 08:04:45 +02:00 |
|
pictuga
|
97d9dda547
|
crawler: support 308 redirects
|
2021-09-21 08:04:45 +02:00 |
|
pictuga
|
6c1f8da692
|
ci: added pylint (triggered upon error w/ score < 8 only)
|
2021-09-21 08:04:45 +02:00 |
|
pictuga
|
4fd730b983
|
Further isort implementation
|
2021-09-21 08:04:45 +02:00 |
|
pictuga
|
0b3e6d7749
|
Apply isort
|
2021-09-21 08:04:23 +02:00 |
|
pictuga
|
06e0ada95b
|
Allow POST requests
|
2021-09-08 20:43:21 +02:00 |
|
pictuga
|
37f5a92b05
|
wsgi: fix apache / workload
|
2021-09-06 22:01:48 +02:00 |
|
pictuga
|
24c26d3850
|
wsgi: handle url decoding
|
2021-08-31 21:51:21 +02:00 |
|
pictuga
|
8f24214915
|
crawler: better name for custom fns
|
2021-08-29 00:22:40 +02:00 |
|
pictuga
|
d5942fe5a7
|
feeds: fix issues when mode not explicited in ruleset
|
2021-08-29 00:20:29 +02:00 |
|
pictuga
|
6f50443995
|
morss: Options return None instead of False if no match
Better for default fn values
|
2021-08-29 00:19:09 +02:00 |
|
pictuga
|
5582fbef31
|
crawler: comment
|
2021-08-29 00:18:50 +02:00 |
|
pictuga
|
da5442a1dc
|
feedify: support any type (json, xml, html)
|
2021-08-29 00:17:28 +02:00 |
|
pictuga
|
f9d7794bcc
|
feedifi.ini: json time zone handling
|
2021-04-23 20:43:00 +02:00 |
|
pictuga
|
e37c8346d0
|
feeds: add fallback for time parser
|
2021-04-22 21:57:16 +02:00 |
|
pictuga
|
3a1d564992
|
feeds: fix time zone handling
|
2021-04-22 21:51:00 +02:00 |
|
pictuga
|
6880a443e0
|
crawler: improve CacheHandler code
|
2021-03-25 23:54:08 +01:00 |
|
pictuga
|
7342ab26d2
|
crawler: comment on how urllib works
|
2021-03-25 23:49:58 +01:00 |
|
pictuga
|
981da9e66a
|
crawler: SQLITE_PATH point to .db file instead of folder
|
2021-03-25 23:48:21 +01:00 |
|
pictuga
|
01a7667032
|
Fix error due to remaining log force code
|
2021-01-14 00:51:47 +01:00 |
|
pictuga
|
3e886caaab
|
crawler: drop encoding setting
|
2020-10-30 22:41:16 +01:00 |
|
pictuga
|
ad927e03a7
|
crawler: use regex instead of lxml
Less reliable but should be faster
|
2020-10-30 22:21:19 +01:00 |
|
pictuga
|
0efb096fa7
|
crawler: shift gzip & encoding-fix to intermediary handler
|
2020-10-30 22:16:51 +01:00 |
|
pictuga
|
9ab2e488ef
|
crawler: add intermediary handlers
|
2020-10-30 22:15:35 +01:00 |
|
pictuga
|
b525ab0d26
|
crawler: fix typo
|
2020-10-30 22:12:43 +01:00 |
|
pictuga
|
bd0bca69fc
|
crawler: ignore ssl via env var
|
2020-10-03 19:57:08 +02:00 |
|
pictuga
|
8abd951d40
|
More sensible default values for cache autotrim (1k entries, 1min)
|
2020-10-03 19:55:57 +02:00 |
|
pictuga
|
2514fabd38
|
Replace memory-leak-prone Uniq with @uniq_wrapper
|
2020-10-03 19:43:55 +02:00 |
|
pictuga
|
8cb7002fe6
|
feeds: make it possible to append empty items
And return the newly appended items, to make it easy to edit them
|
2020-10-03 16:56:07 +02:00 |
|
pictuga
|
6966e03bef
|
Clean up itemClass code
To avoid globals()
|
2020-10-03 16:25:29 +02:00 |
|
pictuga
|
9ce6acba20
|
Fix gunicorn related typo
|
2020-10-01 00:07:41 +02:00 |
|
pictuga
|
056a1b143f
|
crawler: autotrim: make ctrl+c working
|
2020-10-01 00:04:36 +02:00 |
|
pictuga
|
eed949736a
|
crawler: add ability to limit cache size
|
2020-09-30 23:59:55 +02:00 |
|
pictuga
|
2fc7cd391c
|
Shift __main__'s wsgi code where it belongs
|
2020-09-30 23:24:51 +02:00 |
|
pictuga
|
d9f46b23a6
|
crawler: default value for MYSQL_HOST (localhost)
|
2020-09-30 13:17:02 +02:00 |
|
pictuga
|
bbada0436a
|
Quick guide to ignore SSL certs
|
2020-09-27 16:48:22 +02:00 |
|
pictuga
|
039a672f4e
|
wsgi: clean up url reconstruction
|
2020-09-27 16:28:26 +02:00 |
|
pictuga
|
9ecf856f10
|
Add :resolve to remove (some?) tracking links
|
2020-09-15 22:57:52 +02:00 |
|
pictuga
|
56e0c2391d
|
Missing import for served files
|
2020-08-28 20:53:03 +02:00 |