Commit graph

31 commits

Author SHA1 Message Date
fred
8646d61182 Replace copyright header with SPDX identifier 2023-06-19 15:00:45 -07:00
rook1e
9a826bbe6f feat: support searching well-known urls in subdirectory 2023-04-02 17:44:14 -07:00
Frédéric Guillot
aaa1625724 Ignore empty link when discovering feeds 2023-02-26 17:19:26 -08:00
James Loh
2f6895e118 Fix finding JSON feeds with new MIME type
The 1.1 version (https://jsonfeed.org/version/1.1) for JSON feeds defines that feeds should have a MIME type of `application/feed+json` which Miniflux wasn't searching for
2021-08-21 13:01:08 -07:00
Alexandros Kosiaris
b8b16c3bdf Add /rss/ in finder's wellKnownUrls
ATCOM netvolution WCM, probably alongside others, a CMS powering several
high profile and high traffic Greek news sites, among other sites,
publishes the RSS feed under /rss/. Add it to the list. It's generic
enough to allow us to assume other software might do it to

On a select set of 627 Greek news media sites (the infamous Petsas list),
adding this rule increased discoverability of RSS feeds by a factor of
2.61% (from 498 to 511).
2021-07-22 19:46:40 -07:00
Darius
9242350f0e
Add per feed cookies option 2021-03-22 20:27:58 -07:00
Frédéric Guillot
ec3c604a83 Add option to allow self-signed or invalid certificates 2021-02-21 13:58:52 -08:00
Frédéric Guillot
246a48359c Do not follow redirects when trying known feed URLs
Some websites redirects unknown URLs to the home page.
As result, the list of known URLs is returned to the subscription list.
We don't want the user to choose between invalid feed URLs.
2020-11-06 17:46:54 -08:00
Frédéric Guillot
16b7b3bc3e http client: remove dependency on global config options 2020-09-27 14:37:46 -07:00
Kebin Liu
cf7712acea
Add HTTP proxy option for subscriptions 2020-09-09 23:28:54 -07:00
Frédéric Guillot
fc75b0cd8e Add workaround to get YouTube feed from video page 2020-08-02 12:24:46 -07:00
Frédéric Guillot
7380c64141 Add workaround to find YouTube channel feeds
YouTube doesn't expose RSS links anymore for new-style URLs.
2020-08-02 11:37:07 -07:00
Gabriel Augendre
e44b4b2540 Try known urls if no link alternate
I came across a few blogs that didn't have a link rel alternate
but offered a RSS/Atom feed.
This aims at solving this issue for "well known" feed urls, since
these urls are often the same.
2020-06-21 20:34:59 -07:00
Frédéric Guillot
3debf75eb9 Normalize URL query string before executing HTTP requests
- Make sure query strings parameters are encoded
- As opposed to the standard library, do not append equal sign
for query parameters with empty value
- Strip URL fragments like Web browsers
2019-12-26 15:56:59 -08:00
Frédéric Guillot
778346b0b0 Simplify feed fetcher
- Add browser package to handle HTTP errors
- Reduce code duplication
2018-10-14 21:43:48 -07:00
Frédéric Guillot
5870f04260 Simplify feed parser and format detection
- Avoid doing multiple buffer copies
- Move parser and format detection logic to its own package
2018-10-14 11:46:41 -07:00
Patrick
2538eea177 Add the possibility to override default user agent for each feed 2018-09-19 18:19:24 -07:00
Frédéric Guillot
dbcc5d8a97 Use canonical imports 2018-08-24 21:56:39 -07:00
Frédéric Guillot
9c0f882ba0 Add specific 404 and 401 error messages 2018-06-30 12:42:12 -07:00
Frédéric Guillot
bddca15b69 Add new fields for feed username/password 2018-06-19 22:58:29 -07:00
Frédéric Guillot
1eba1730d1 Move HTTP client to its own package 2018-04-28 10:51:07 -07:00
Frédéric Guillot
dda9114692 Improve error handling for HTTP client 2018-02-08 18:16:54 -08:00
Frédéric Guillot
0fb87eba3f Improve error handling when the response is empty 2018-02-07 18:47:47 -08:00
Frédéric Guillot
713b38e34c Handle more encoding edge cases
- Feeds with charset specified only in Content-Type header and not in XML document
- Feeds with charset specified in both places
- Feeds with charset specified only in XML document and not in HTTP header
2018-01-20 13:25:21 -08:00
Frédéric Guillot
c39f2e1a8d Rename helper packages 2018-01-02 19:15:08 -08:00
Frédéric Guillot
1d8193b892 Add logger 2017-12-15 18:55:57 -08:00
Frédéric Guillot
84d912c979 Rewrite imports 2017-12-12 21:48:13 -08:00
Frédéric Guillot
6f5350a497 Move packages http and url 2017-12-02 20:26:21 -08:00
Frédéric Guillot
fb2a73c91e Proxify image enclosures 2017-12-01 22:29:18 -08:00
Frédéric Guillot
aecda64030 Make sure XML feeds are always encoded in UTF-8 2017-11-20 17:12:37 -08:00
Frédéric Guillot
8ffb773f43 First commit 2017-11-19 22:01:46 -08:00