Commit graph

276 commits

Author SHA1 Message Date
David Izquierdo
4fdef7b837 Add scrape and rewrite rules for webtoons
Although the only source I have for the rewrite rule is, in fact, https://github.com/miniflux/v2/pull/892, it does work when combined with add_dynamic_image and scraping the right element. I have not investigated further.

Works around https://github.com/miniflux/v2/issues/775 and https://github.com/miniflux/v2/issues/1871 (as in, gives us working webtoons feeds but referer spoofing would still be a nice tool to have).

Fixes https://github.com/miniflux/v2/issues/256.
2023-07-10 21:25:48 -07:00
Igor Rzegocki
9b42d0e25e feat: support for custom youtube embed URL 2023-07-07 15:59:23 -07:00
Frédéric Guillot
b13c7e328a Improve date parser to handle various broken date formats 2023-06-24 15:27:33 -07:00
Frédéric Guillot
30d4b8986a Avoid "pq: time zone displacement out of range" errors 2023-06-24 15:09:58 -07:00
fred
af74e39fa7 Add test case to parse Atom icon URL 2023-06-19 15:17:41 -07:00
fred
8646d61182 Replace copyright header with SPDX identifier 2023-06-19 15:00:45 -07:00
Ryan Stafford
1aeb1b20da
Use image included in feed as feed icon 2023-06-04 15:01:59 -07:00
Davide Masserut
5d8a8878d5 Update scraping rules for ilpost.it 2023-05-02 17:07:25 -07:00
Romain de Laage
33c4b5188c Add a rewrite rule to remove clickbait titles 2023-04-15 18:25:43 -07:00
Emiel Wiedijk
5a88e0465e Update rewrite rules for theverge.com
Articles on The Verge sometimes contain a section for related articles.
This section can be distracting in reader mode. Therefore, filter the
related article section using the scraper rules.
2023-04-07 16:12:19 -07:00
Jake Walker
8b6dd3e599 Keep other table rows and columns 2023-04-02 17:50:19 -07:00
Jake Walker
49d2596fc6 Basic table removal rule 2023-04-02 17:50:19 -07:00
rook1e
9a826bbe6f feat: support searching well-known urls in subdirectory 2023-04-02 17:44:14 -07:00
Davide Masserut
034e46700c Process older entries first
Feed entries are usually ordered from most to least recent.

Processing older entries first ensures that their creation timestamp
is lower than that of newer entries.

This is useful when we order by creation, because then we get a
consistent timeline.
2023-03-25 16:19:07 -07:00
Davide Masserut
755c9af47d Update scraping rules for ilpost.it 2023-03-01 20:04:25 -08:00
Frédéric Guillot
02e4b8eadc Update GitHub Actions to use Go 1.20 2023-03-01 19:56:06 -08:00
Frédéric Guillot
aaa1625724 Ignore empty link when discovering feeds 2023-02-26 17:19:26 -08:00
privatmamtora
8f9ccc6540
Parse <category> from Feeds (RSS, Atom and JSON) 2023-02-24 20:52:45 -08:00
Marie Ramlow
48acd1feca Add rewrite and scraper rules for blog.cloudflare.com 2023-02-05 21:01:42 -08:00
xdavidwu
08f7835f5d sanitizer: allow id in <sup>
One of blogs I read uses anchor on <sup> to link a footnote back to its
reference.
2023-01-31 17:53:45 -08:00
Davide Masserut
690d66ce0b Update scraping rules for ilpost.it 2022-12-27 13:33:41 -08:00
Davide Masserut
ef312ef770 Update scraping rule for ilpost.it 2022-12-16 15:07:10 -08:00
Davide Masserut
c0bed53b42 Add scraping rule for ilpost.it 2022-12-15 19:53:12 -08:00
Harry Cheng
d9777f1439 Skip integrations if there are no entries to push 2022-12-04 12:58:10 -08:00
Frédéric Guillot
93715b542c Revert "scraper follow the only link"
This reverts commit 10207967c4.
2022-11-14 17:45:40 -08:00
Frédéric Guillot
de1a06e3e8 Add missing check in followTheOnlyLink() that leads to a panic
Bug introduced in PR #1290. Fixes #1631.
2022-11-14 16:44:02 -08:00
jebbs
10207967c4 scraper follow the only link
* in some cases, what the scraper got is only a landing page, user can use scraper rules to extract the link of the landing page and follow it
* it also fix the  wrong scrape rule apply when the server redirects it to another host
2022-10-31 19:49:34 -07:00
Romain de Laage
550e7d0415 Add matrix bot support 2022-10-27 17:53:19 -07:00
Romain de Laage
eb86773039 Recalbox rewrite rule 2022-10-19 20:13:44 -07:00
jgbresson
7f6ce16d85 Add scraping rules for theverge.com 2022-10-16 11:58:35 -07:00
jgbresson
aa47789f55
Add add_dynamic_image rewrite rule for theverge.com 2022-10-16 11:57:01 -07:00
Frédéric Guillot
d947b0194b Handle RSS entries with only a GUID permalink 2022-10-09 16:58:25 -07:00
Frédéric Guillot
138fd926ee Do not convert anchors to absolute links 2022-09-11 22:40:52 -07:00
Adam B
4d847c6a74 Add scraping rule for royalroad.com
This is what I use for several stories I follow, and I thought it might be useful to other miniflux users.
2022-08-17 19:25:39 -07:00
Owen Valentine
f404ddde91 Add swordscomic.com 2022-08-17 19:23:29 -07:00
Owen Valentine
c8a3d953cf Add smbc-comics.com 2022-08-17 19:23:29 -07:00
Owen Valentine
f851ecac78 Sort alphabetically 2022-08-17 19:23:29 -07:00
Frédéric Guillot
cecab91298 Fix some linter issues 2022-08-08 22:06:38 -07:00
Frédéric Guillot
13fa08ad39 Handle Atom links with a text/html type defined 2022-07-31 17:43:03 -07:00
Gabe Cook
405d4febd9 Parse markdown by default for blog.laravel.com 2022-07-30 20:19:09 -07:00
Gabe Cook
36df7b36ec Add parse_markdown rewrite function 2022-07-30 20:19:09 -07:00
Gabe Cook
bd1dc3149e Add explosm.net scraper rule 2022-07-30 20:10:52 -07:00
Gabriel Augendre
6e50ce3293 Make reading speed user-configurable 2022-07-17 19:35:24 -07:00
Carsten
2659883ce5
Add rewrite rules for article URL before fetching content 2022-07-11 21:12:26 -07:00
Frédéric Guillot
c0eab5ebc5 Avoid stretched image if specified width is larger than Miniflux's layout 2022-07-04 20:10:07 -07:00
Frédéric Guillot
f0a698c6fe Add support for OPML files with several nested outlines 2022-07-04 16:02:49 -07:00
Frédéric Guillot
806a069785 sanitizer: handle image URLs in srcset attribute with comma 2022-07-04 13:50:09 -07:00
Frédéric Guillot
d85908e3de Allow width and height attributes for img tags 2022-07-03 17:44:12 -07:00
nemunaire
5a07fd8932
Add new rewrite rule to decode base64 content 2022-05-25 20:44:04 -07:00
lf94
fa8431c5c6 Try to use outermost element text when title is empty 2022-04-13 21:51:54 -07:00