Commit graph

22 commits

Author SHA1 Message Date
fin444
a631bd527d options: add FETCH_NEBULA_WATCH_TIME 2024-05-02 16:30:01 -07:00
Frédéric Guillot
fb075b60b5 reader/processor: minifier is breaking HTML entry content 2024-04-23 20:31:52 -07:00
jvoisin
b205b5aad0 reader/processor: minimize the feed's entries html
Compress the html of feed entries before storing it. This should reduce the
size of the database a bit, but more importantly, reduce the amount of data
sent to clients

minify being [stupidly fast](https://github.com/tdewolff/minify/?tab=readme-ov-file#performance), the performance impact should be in the noise level.
2024-04-10 19:48:48 -07:00
Frédéric Guillot
38b80d96ea storage: change GetReadTime() function to use entries_feed_id_hash_key index 2024-04-09 20:37:30 -07:00
Frédéric Guillot
fdd1b3f18e database: entry URLs can exceeds btree index size limit 2024-04-04 20:22:23 -07:00
Jean Khawand
a78d1c79da
Add FILTER_ENTRY_MAX_AGE_DAYS config option to limit fetching all feed items 2024-03-20 02:58:53 +00:00
Frédéric Guillot
b1e73fafdf Enable go-critic linter and fix various issues detected 2024-03-17 13:52:34 -07:00
jvoisin
02a074ed26 Compile block/keep regex only once per feed
No need to compile them once for matching on the url,
once per tag, once per title, once per author, … one time is enough.
It also simplify error handling, since while regexp compilation can fail,
matching can't.
2024-03-17 12:08:03 -07:00
jvoisin
31ac62f410 Don't compute reading-time when unused
If the user doesn't display reading times, there is no need to compute them.
This should speed things up a bit, since `whatlanggo.Detect` is abysmally slow.
2024-02-29 19:14:17 -08:00
Frédéric Guillot
c493f8921e Add missing regex anchor detected by CodeQL 2024-02-28 20:50:17 -08:00
Frédéric Guillot
eae4cb1417 Add feed option to disable HTTP/2 to avoid fingerprinting 2024-02-24 22:30:26 -08:00
jvoisin
b48ad6dbfb Make use of go≥1.21 slices package instead of hand-rolled loops
This makes the code a tad smaller, moderner,
and maybe even marginally faster, yay!
2024-02-24 20:22:53 -08:00
Matt Stobo
4a50ca9122 Allow filtering feeds on entry.Author 2024-01-31 19:42:07 -08:00
Filipe de Luna
1441dc7600
Update entry processor to allow blocking/keeping entries by tags 2024-01-09 21:15:11 -08:00
Thomas J Faughnan Jr
7a03291442 Fix default User-Agent regression
The recent HTTP client refactor in 14e25ab9fe
introduced a bug in which the global default User-Agent is no longer
used for requests. Unless a per-feed User-Agent exists, the Go standard
library's default User-Agent is used, which looks something like
"Go-http-client/1.1". To fix this, make RequestBuilder.WithUserAgent
take an additional argument, the default User-Agent, which will be used
if there is no per-feed User-Agent (i.e. it is an empty string).

Fixes #2188
Fixes #2189
2023-11-18 20:57:47 +01:00
Tianfeng Wang
a1537f4b0d
Filter feed entries based on url or title 2023-10-25 19:38:08 -07:00
Frédéric Guillot
14e25ab9fe Refactor HTTP Client and LocalizedError packages 2023-10-22 13:09:30 -07:00
Frédéric Guillot
7b541af253 Replace github.com/rylans/getlang with github.com/abadojack/whatlanggo
github.com/rylans/getlang doesn't seems to be updated anymore
2023-10-06 22:04:31 -07:00
Frédéric Guillot
c0e954f19d Implement structured logging using log/slog package 2023-09-24 22:37:33 -07:00
Frédéric Guillot
48f6885f44 Add generic webhook integration 2023-09-09 13:11:42 -07:00
fuchsrot
32d33104a4 Apprise Service Urls per feed 2023-09-09 10:59:04 -07:00
Frédéric Guillot
168a870c02 Move internal packages to an internal folder
For reference: https://go.dev/doc/go1.4#internalpackages
2023-08-10 20:29:34 -07:00
Renamed from reader/processor/processor.go (Browse further)