Commit Graph

1032 Commits (a613acfe7d9a56c7744c3efe02a0c36f40f8b182)
 

Author SHA1 Message Date
pictuga a613acfe7d Fix isort
continuous-integration/drone/push Build is failing Details
2022-01-31 00:44:04 +01:00
pictuga b888e068c0 pytest: first batch with test_feeds
continuous-integration/drone/push Build is failing Details
2022-01-31 00:23:09 +01:00
pictuga d6b90448f3 crawler: improve handling of non-ascii urls 2022-01-30 23:27:49 +01:00
pictuga da81edc651 log to stderr
continuous-integration/drone/push Build is failing Details
2022-01-26 07:57:57 +01:00
pictuga 4f2895f931 cli: update `--help`
continuous-integration/drone/push Build is failing Details
2022-01-25 22:36:57 +01:00
pictuga b2b04691d6 Ability to pass custom data_files location 2022-01-25 22:36:34 +01:00
pictuga bfaf7b0fac feeds: clean up default `item_link`
continuous-integration/drone/push Build is failing Details
To be supported by feeds' `_rule_parse`
2022-01-24 16:16:14 +00:00
pictuga 32d9bc9d9d feeds: proceed with conversion when rules do not match
continuous-integration/drone/push Build is failing Details
2022-01-24 09:34:57 +00:00
pictuga b138f11771 util: support more `data_files` location
continuous-integration/drone/push Build is passing Details
2022-01-23 12:40:18 +01:00
pictuga a01258700d More ordering options
continuous-integration/drone/push Build was killed Details
2022-01-23 12:27:07 +01:00
pictuga 4d6d3c9239 wsgi: limit supported mimetypes & return actual mimetype
continuous-integration/drone/push Build is passing Details
2022-01-23 11:44:07 +01:00
pictuga e81f6b173f readabilite: remove code duplicate 2022-01-23 11:41:32 +01:00
pictuga fe5dbf1ce0 wsgi: reuse mimetype table from crawler 2022-01-22 13:22:39 +01:00
pictuga fdf9acd32b helper: fix reload code
continuous-integration/drone/push Build is passing Details
2022-01-19 13:44:15 +01:00
pictuga d05706e056 crawler: fix typo
continuous-integration/drone/push Build was killed Details
2022-01-19 13:41:12 +01:00
pictuga e88a823ada feeds: better handle rulesets without a 'mode' specified
continuous-integration/drone/push Build is failing Details
2022-01-19 13:08:33 +01:00
pictuga 750850c162 crawler: avoid too many .append() 2022-01-19 13:04:33 +01:00
pictuga c8669002e4 feeds: exotic xpath in html as well
continuous-integration/drone/push Build is passing Details
2022-01-17 14:22:48 +00:00
pictuga c524e54d2d feeds: support some exotic xpath rules returning a single string
continuous-integration/drone/push Build is passing Details
2022-01-17 13:59:58 +00:00
pictuga ef14567d87 Handle `morss-helper` with `setup.py`
continuous-integration/drone/push Build is passing Details
2022-01-08 16:10:51 +01:00
pictuga fb643f5ef1 readabilite: remove unneeded reference to `features` (overriden by `builder`)
continuous-integration/drone/push Build is passing Details
2022-01-03 18:01:12 +00:00
pictuga dbdca910d8 readabilite: fix new parser code & drop PIs
continuous-integration/drone/push Build was killed Details
2022-01-03 17:51:49 +00:00
pictuga 9eb19fac04 readabilite: use custom html parser within bs4's lxml parser
continuous-integration/drone/push Build is passing Details
Solves the following obscure error:
ValueError: Invalid PI name 'b'xml''
2022-01-03 16:26:17 +00:00
pictuga d424e394d1 readabilite: use lxml bs4 parser for speed
continuous-integration/drone/push Build is passing Details
2022-01-01 14:52:48 +01:00
pictuga 3f92787b38 readabilite: limit html comments related issues
continuous-integration/drone/push Build is passing Details
2022-01-01 13:58:42 +01:00
pictuga afc31eb6e9 readabilite: avoid double parsing of html
continuous-integration/drone/push Build is passing Details
2022-01-01 12:51:30 +01:00
pictuga 87d2fe772d wsgi: fix py2 compatibility 2022-01-01 12:35:41 +01:00
pictuga 917aa0fbc5 crawler: do not re-save cached response
continuous-integration/drone/push Build is passing Details
Otherwise cache never gets invalidated!
2021-12-31 19:28:11 +01:00
pictuga 3e2b81286f xsl: add link to favicon
To limit error output when failing to fetch favicon.ico
2021-12-31 19:25:53 +01:00
pictuga 15430a2b83 helper: restore run if no param passed
continuous-integration/drone/push Build is passing Details
2021-12-29 23:35:16 +01:00
pictuga ecdb74812d Make helper & main.py executable
continuous-integration/drone/push Build is passing Details
2021-12-29 15:47:05 +01:00
pictuga 2c7844942c drone: re order deploy commands
continuous-integration/drone/push Build was killed Details
2021-12-29 15:41:29 +01:00
pictuga e12cb4567a helper: more debug options 2021-12-29 15:41:03 +01:00
pictuga b74365b121 Make helper more posix compliant 2021-12-29 15:40:43 +01:00
pictuga 2020543469 Make morss-helper executable 2021-12-29 15:37:12 +01:00
pictuga 676be4a4fe helper: work around for systems only having py3 binary
continuous-integration/drone/push Build is running Details
2021-12-29 14:07:12 +01:00
pictuga 8870400a6e Clean up morss-helper
continuous-integration/drone/push Build is failing Details
2021-12-28 16:30:20 +01:00
pictuga 8e9cc541b0 Turns out exec array is not supported in HEALTHCHECK
continuous-integration/drone/push Build is failing Details
2021-12-28 15:23:40 +01:00
pictuga 2a7a1b83ec Use alpine:edge to have up-to-date py packages
continuous-integration/drone/push Build is passing Details
2021-12-28 13:41:42 +01:00
pictuga 106f59afa1 docker: shift HEALTHCHECK to helper
continuous-integration/drone/push Build is failing Details
2021-12-27 16:08:55 +01:00
pictuga ee514e2da3 helper: remove unneeded sudo
continuous-integration/drone/push Build is passing Details
2021-12-25 22:00:22 +00:00
pictuga e7578e859a Clean up install/exec
continuous-integration/drone/push Build is passing Details
2021-12-25 18:21:55 +01:00
pictuga 3bcb8db974 Improve cloud-init (append & env var) 2021-12-25 11:02:27 +01:00
pictuga 7751792942 Shift htaccess to README 2021-12-24 18:03:55 +01:00
pictuga 6e2e5ffa00 README: cloud-init indication for env var
continuous-integration/drone/push Build is passing Details
2021-12-24 11:44:56 +01:00
pictuga f6da0e1e9b Make use of GUNICORN_CMD_ARGS 2021-12-24 11:44:24 +01:00
pictuga 2247ba13c5 drone: clean up file
continuous-integration/drone/push Build is passing Details
2021-12-23 12:03:50 +01:00
pictuga d17b9a2f27 Fix typo in DISKCACHE_DIR var name
continuous-integration/drone/push Build was killed Details
2021-12-23 12:02:24 +01:00
pictuga 5ab45e60af README: scale back on logos
continuous-integration/drone/push Build is passing Details
2021-12-16 09:03:43 +00:00
pictuga 368e4683d6 util: clean paths code
continuous-integration/drone/push Build was killed Details
2021-12-16 08:53:18 +00:00