Commit graph

329 commits

Author SHA1 Message Date
Frédéric Guillot
3db3f9884f cli: avoid misleading error message when creating an admin user 2024-03-23 14:32:55 -07:00
Frédéric Guillot
ad1d349a0c rss: use Channel tags only if there is no Item tags 2024-03-23 13:46:48 -07:00
Frédéric Guillot
c2311e316c Rename PROXY_* options to MEDIA_PROXY_* 2024-03-20 21:28:28 -07:00
jvoisin
ed20771194 Enable trusted-types
This commit adds a policy, and make use of it in the Content-Security-Policy.

I've tested it the best I could, both on a modern browser supporting
trusted-types (Chrome) and on one that doesn't (firefox).

Thanks to @lweichselbaum for giving me a hand to wrap this up!
2024-03-20 17:50:37 -07:00
jvoisin
beb8c80787 Replace a bunch of let with const
According to https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/const

> Many style guides (including MDN's) recommend using const over let whenever a
variable is not reassigned in its scope. This makes the intent clear that a
variable's type (or value, in the case of a primitive) can never change.
2024-03-20 17:36:01 -07:00
jvoisin
fc4bdf3ab0 Inline a one-liner function
No need to expose a symbol for this.
2024-03-20 17:21:30 -07:00
Frédéric Guillot
08640b27d5 Ensure enclosure URLs are always absolute 2024-03-19 21:57:46 -07:00
jvoisin
4be993e055 Minor refactoring of internal/reader/atom/atom_10_adapter.go
- Move the population of the feed's entries into a new function, to make
  `BuildFeed` easier to understand/separate concerns/implementation details
- Use `sort+compact` instead of `compact+sort` to remove duplicates
- Change `if !a { a = } if !a {a = }` constructs into `if !a { a = ; if !a {a = }}`.
  This reduce the number of comparisons, but also improves a tad the
  control-flow readability.
2024-03-19 20:41:44 -07:00
jvoisin
9df12177eb Minor idiomatic pass on internal/http/request/context.go 2024-03-19 20:21:23 -07:00
Jean Khawand
a78d1c79da
Add FILTER_ENTRY_MAX_AGE_DAYS config option to limit fetching all feed items 2024-03-20 02:58:53 +00:00
Matt Behrens
1ea3953271
Add keyboard shortcuts for scrolling to top/bottom of the item list 2024-03-19 19:30:38 -07:00
Frédéric Guillot
a15cdb1655 Fix regression in AbsoluteProxifyURL()
Regression introduced in commit 66b8483791

PR #2499
2024-03-18 20:48:20 -07:00
Frédéric Guillot
fa9697b972 Remove trailing space in SiteURL and FeedURL 2024-03-18 17:51:06 -07:00
jvoisin
8e28e41b02 Use struct embedding to reduce code duplication 2024-03-18 16:23:44 -07:00
jvoisin
e2ee74428a Minor concatenation-related simplifications in internal/storage/
Use plain strings concatenation instead of
building an array and then joining it.
2024-03-18 16:20:55 -07:00
jvoisin
863a5b3648 Simplify removeDuplicates
Use a sort+compact construct instead of doing it by hand with a hashmap. The
time complexity is now O(nlogn+n) instead of O(n), and space complexity around
O(logn) instead of O(n+uniq(n)), but it shouldn't matter anyway, since
removeDuplicates is only called to deduplicate tags.
2024-03-18 16:13:32 -07:00
jvoisin
91f5522ce0 Minor simplification of internal/reader/media/media.go
- Simplify a switch-case by moving a common condition above it.
- Remove a superfluous error-check: `strconv.ParseInt` returns `0` when passed
  an empty string.
2024-03-18 16:09:32 -07:00
Frédéric Guillot
8212f16aa2 atom: avoid debug message when the date is empty 2024-03-17 15:29:50 -07:00
Frédéric Guillot
b1e73fafdf Enable go-critic linter and fix various issues detected 2024-03-17 13:52:34 -07:00
Frédéric Guillot
f6404290ba Replace Optional{Int,Int64,Float64} with a generic function OptionalNumber() 2024-03-17 12:25:55 -07:00
jvoisin
c29ca0e313 Minor simplifications of the rewriter
- Online some one-line functions
- Transform a free-standing function into a method
- Massively simplify `removeClickbait`
- Use a proper constant instead of a magic number in `applyFuncOnTextContent`
2024-03-17 12:15:46 -07:00
jvoisin
02a074ed26 Compile block/keep regex only once per feed
No need to compile them once for matching on the url,
once per tag, once per title, once per author, … one time is enough.
It also simplify error handling, since while regexp compilation can fail,
matching can't.
2024-03-17 12:08:03 -07:00
Romain de Laage
00dabc1d3c feat: Media player: Conrol playback speed
fix  #1845
2024-03-17 11:53:30 -07:00
Frédéric Guillot
b68ada396a Rewrite API integration tests without build tags 2024-03-16 21:29:07 -07:00
Frédéric Guillot
0f17dfc7d6 Fix regressions introduced by PR #2476
'Toast' messages are broken and v hotkey opens in the same tab

Commit d25c032171
2024-03-15 20:55:32 -07:00
Frédéric Guillot
7c80d6b86d Fix download button loading label 2024-03-15 20:40:14 -07:00
Frédéric Guillot
f6f63b5282 Avoid warnings in ui package
Remove unused variables and improve JSON decoding in
saveEnclosureProgression()
2024-03-15 19:49:39 -07:00
Frédéric Guillot
309fdbb9fc Fix force refresh 2024-03-15 19:42:09 -07:00
Frédéric Guillot
e2d862f2f6 Display an error message on edit feed page when the feed URL is not unique 2024-03-15 19:07:54 -07:00
Frédéric Guillot
4834e934f2 Remove some duplicated code in RSS parser 2024-03-15 18:40:06 -07:00
Frédéric Guillot
dd4fb660c1 Refactor Atom parser to use an adapter 2024-03-15 17:27:16 -07:00
jvoisin
2ba893bc79 Bump the number of simultaneous workers
We're in 2024, I'm pretty sure we can afford to have 16 simultaneous open http
connections at the same time, instead of only 5.
2024-03-15 14:05:58 -07:00
Frédéric Guillot
7a307f8e74 Fix regression: Add to Home Screen button is unreadable
Regression introduced in commit ea58bac548
2024-03-14 17:37:50 -07:00
jvoisin
7310e13499 More trusted-types compatibility 2024-03-14 17:10:40 -07:00
Frédéric Guillot
ca919c2ff8 Fix JavaScript error on the login page 2024-03-13 21:47:23 -07:00
Frédéric Guillot
5948786b15 Add support for RSS <media:category> element 2024-03-13 21:35:39 -07:00
jvoisin
f4746a7306 Fix and simplify shaarli's integration
- The jwt token was declared as using HS256 as algorithm, but was using HS512.
- No need to base64-encode then remove the padding when we can simply encode
  without padding.
- Factorize the header+payload concatenation as data

Odds are that this integration was broken from the start (HS512 vs HS256), so
I'm not sure if it's better to add tests or to simply get rid of it.
2024-03-13 21:34:57 -07:00
Frédéric Guillot
648b9a8f6f Refactor RSS Parser to use an adapter 2024-03-13 21:25:09 -07:00
jvoisin
66b8483791 Minor simplification of internal/proxy/proxy.go
- re-use ProxifiedUrl to implement AbsoluteProxifyURL, reducing the copy-pasta
- reduce the internal indentation of ProxifiedUrl by inverting some conditions
2024-03-13 19:42:01 -07:00
jvoisin
e0ee28c013 More progress towards trusted-types
Create a new function `addIcon` and use it to add icons, instead of
operating on raw html.
2024-03-13 19:35:20 -07:00
jvoisin
d25c032171 Simplify bootstrap.js
- Don't use lambdas to return a function, use directly the function instead.
- Remove a hack for "Chrome 67 and earlier" since it was released in 2018.
2024-03-13 18:26:27 -07:00
Frédéric Guillot
8429c6b0ab Refactor JSON Feed parser to use an adapter 2024-03-12 22:37:14 -07:00
Frédéric Guillot
6bc4b35e38 Refactor RDF parser to use an adapter
Avoid tight coupling between `model.Feed` and the original XML RDF feed.
2024-03-12 20:54:05 -07:00
mcnesium
ee3486af66 align min-width with the other min-width values 2024-03-12 18:58:30 -07:00
jvoisin
45d486b919 When detecting the format, detect its version as well
There is no need to detect the format and then the version when both can be
done at the same time.

Add a benchmark as well, on large and small atom and rss files.
2024-03-12 18:56:56 -07:00
Frédéric Guillot
6d97f8b458 Parse podcast categories 2024-03-11 22:30:27 -07:00
Frédéric Guillot
f8e50947f2 Move iTunes and GooglePlay XML definitions to their own packages 2024-03-11 22:09:31 -07:00
Frédéric Guillot
9a637ce95e Refactor RSS parser to use default namespace
This change avoid some limitations of the Go XML parser regarding XML namespaces
2024-03-11 21:07:13 -07:00
Frédéric Guillot
d3a85b049b jsminifier: set JavaScript version 2024-03-11 19:02:52 -07:00
jvoisin
5bcb37901c Use crypto.GenerateRandomBytes instead of doing it by hand
This makes the code a bit shorter, and properly handle
cryptographic error conditions.
2024-03-11 16:31:43 -07:00
jvoisin
9c8a7dfffe Make use of HashFromBytes everywhere
It feels a bit silly to have a function and to not make use of it.
2024-03-11 15:22:22 -07:00
jvoisin
74e4032ffc Small refactor of app.js
- replace a lot of `let` with `const`
- inline some `querySelectorAll` calls
- reduce the scope of some variables
- use some ternaries where it makes sense
- inline one-line functions
2024-03-11 15:18:57 -07:00
jvoisin
fd1fee852c Simplify DomHelper.getVisibleElements
Use a `filter` instead of a loop with an index.
2024-03-11 15:03:00 -07:00
Frédéric Guillot
c51a3270da GitHub Actions: Add basic ESLinter checks 2024-03-10 20:57:27 -07:00
Frédéric Guillot
45fa641d26 Fix JavaScript linter path in GitHub Actions 2024-03-10 20:37:18 -07:00
jvoisin
fd8f25916b First steps towards trusted-types support
Refactor away some trival usages of `.innerHTML`. Unfortunately, there is no way to
enabled trusted-types in report-only mode via `<meta>` tags, see
https://github.com/w3c/webappsec-csp/issues/277
2024-03-10 20:14:30 -07:00
jvoisin
826e4d654f Replace DomHelper.findParent with .closest
See https://developer.mozilla.org/en-US/docs/Web/API/Element/closest
2024-03-10 20:06:54 -07:00
jvoisin
d9d17f0d69 Use a Set instead of an array in a KeyboardHandler's member
The variable `triggers` is only used to check if in contains a particular
value. Given that the number of keyboard shortcuts is starting to be
significant, let's future-proof the performances and use a `Set` instead of an
`Array` instead.
2024-03-10 19:41:13 -07:00
jvoisin
a074773e6c Use an io.ReadSeeker instead of an io.Reader to parse feeds
This will allow to make use of func (*Reader) Seek, instead of re-recreating a
new reader. It's a large commit for a small change, but anything to simply the
reader/buffer/ReadAll/… mess is a step in the right direction I think, and it
should enable more follow-up simplifications.
2024-03-06 20:13:39 -08:00
jvoisin
3d0126be0b Speed the sanitizer up a bit, again
- allow youtube urls to start with `www`
- use `strings.Builder` instead of a `bytes.Buffer`
- use a `strings.NewReader` instead of a `bytes.NewBufferString`
- sprinkles a couple of `continue` to make the code-flow more obvious
- inline calls to `inList`, and put their parameters in the right order
- simplify isPixelTracker
- simplify `isValidIframeSource`, by extracting the hostname and comparing it
  directly, instead of using the full url and checking if it starts with
  multiple variations of the same one (`//`, `http:`, `https://` multiplied by
  ``/`www.`)
- add a benchmark
2024-03-05 19:31:50 -08:00
jvoisin
111e3f2106 Reuse a Reader instead of copying to a buffer when parsing an atom feed 2024-03-04 17:36:10 -08:00
jvoisin
3339d9d3d7 Preallocate memory when exporting to OPML
This should marginally increase performance when export a large amount of feeds
to OPML.
2024-03-03 20:34:37 -08:00
jvoisin
8d80e9103f Delay call of view.New after logging the user in
There is no need to do extra work like creating a session and its associated
view until the user has been properly identified and as many possibly-failing sql request have been successfully run.
2024-03-03 20:32:15 -08:00
jvoisin
d55b410800 Use constant-time comparison for anti-csrf tokens
This is probably completely overkill, but since anti-csrf tokens are secrets,
they should be compared against untrusted inputs in constant time.
2024-03-03 20:28:13 -08:00
jvoisin
9fe99ce7fa Simplify and optimize genericProxyRewriter
- Reduce the amount of nested loops: it's preferable to search the whole page
  once and filter on it (even with filters that should always be false),
  than searching it again for every element we're looking for.
- Factorize the proxying conditions into a `shouldProxy` function to reduce the
  copy-pasta.
2024-03-03 20:25:47 -08:00
Thiago Perrotta
b8df6c31a0 sort integrations alphabetically 2024-03-03 20:19:42 -08:00
Frédéric Guillot
abdd5876a1 Move search form to a dedicated page 2024-03-01 16:56:15 -08:00
Frédéric Guillot
1b5edfc00a Add unit test to ensure each translation has the correct number of plurals 2024-02-29 20:44:08 -08:00
jvoisin
347740dce1 Speed up removeUnlikelyCandidates
`.Not` returns a brand new Selection, copied element by element.
2024-02-29 19:38:43 -08:00
jvoisin
ab85d4d678 Improve EstimateReadingTime's speed by a factor 7
- Refactorise the tests and add some
- Use 250 signs instead of the whole text
- Only check for Korean, Chinese and Japanese script
- Add a benchmark
- Use a more idiomatic control flow

```console
$ # main branch
$ go test -bench=.
goos: linux
goarch: amd64
pkg: miniflux.app/v2/internal/reader/readingtime
BenchmarkEstimateReadingTime-12              267           4821268 ns/op
PASS
ok      miniflux.app/v2/internal/reader/readingtime     1.754s
$ # speed_up_reading_time branch
$ go test -bench=.
goos: linux
goarch: amd64
pkg: miniflux.app/v2/internal/reader/readingtime
cpu: 12th Gen Intel(R) Core(TM) i7-1265U
BenchmarkEstimateReadingTime-12             1941            653312 ns/op
PASS
ok      miniflux.app/v2/internal/reader/readingtime     1.342s
$
```
2024-02-29 19:24:15 -08:00
jvoisin
31ac62f410 Don't compute reading-time when unused
If the user doesn't display reading times, there is no need to compute them.
This should speed things up a bit, since `whatlanggo.Detect` is abysmally slow.
2024-02-29 19:14:17 -08:00
Frédéric Guillot
97765b93a9 Revert "Minor internal/reader/readability/readability.go speedup"
This reverts commit 4db138d4b8.

```
panic: runtime error: index out of range [-1]

goroutine 49 [running]:
miniflux.app/v2/internal/reader/readability.getArticle.func1(0x8?, 0xc000b56570)
        /home/fred/repos/miniflux/v2/internal/reader/readability/readability.go:120 +0x2ac
github.com/PuerkitoBio/goquery.(*Selection).Each(0xc000b56510, 0xc000892fa8)
        /home/fred/go/pkg/mod/github.com/!puerkito!bio/goquery@v1.9.0/iteration.go:10 +0x62
miniflux.app/v2/internal/reader/readability.getArticle(0xc00044f1f0, 0xc000a04a50)
        /home/fred/repos/miniflux/v2/internal/reader/readability/readability.go:101 +0x15d
miniflux.app/v2/internal/reader/readability.ExtractContent({0x1005d00?, 0xc0001522d0?})
        /home/fred/repos/miniflux/v2/internal/reader/readability/readability.go:91 +0x211
miniflux.app/v2/internal/reader/scraper.ScrapeWebsite(0xc000893688?, {0xc0007ce720, 0x54}, {0x0, 0x0})
        /home/fred/repos/miniflux/v2/internal/reader/scraper/scraper.go:63 +0x859
miniflux.app/v2/internal/reader/processor.ProcessFeedEntries(0xc000133188, 0xc000502c40, 0xc0003e6360, 0x0)
        /home/fred/repos/miniflux/v2/internal/reader/processor/processor.go:77 +0x8ea
miniflux.app/v2/internal/reader/handler.RefreshFeed(0xc000133188, 0x10cf, 0x52d5c, 0x0)
        /home/fred/repos/miniflux/v2/internal/reader/handler/handler.go:301 +0x1485
miniflux.app/v2/internal/cli.refreshFeeds.func1(0x0)
        /home/fred/repos/miniflux/v2/internal/cli/refresh_feeds.go:59 +0x2d7
created by miniflux.app/v2/internal/cli.refreshFeeds in goroutine 1
        /home/fred/repos/miniflux/v2/internal/cli/refresh_feeds.go:50 +0x5d5
```
2024-02-29 19:06:03 -08:00
jvoisin
e6524f925f Simplify username generation for the tests
No need to generate random numbers 10 times, generate a single big-enough one.
A single int64 should be more than enough
2024-02-29 18:36:34 -08:00
Frédéric Guillot
c493f8921e Add missing regex anchor detected by CodeQL 2024-02-28 20:50:17 -08:00
Frédéric Guillot
b2ce98da87 Add missing plurals for some languages 2024-02-28 20:38:10 -08:00
jvoisin
4db138d4b8 Minor internal/reader/readability/readability.go speedup
- Don't use a capturing group in `divToPElementsRegexp`
- Remove a duplicate condition
- Replace a regex with a fixed-comparison and a `Contains`
2024-02-28 20:03:14 -08:00
jvoisin
f12d5131b0 Divide the sanitization time by 3
Instead of having to allocate a ~100 keys map containing possibly dynamic
values (at least to the go compiler), allocate it once in a global variable.
This significantly speeds things up, by reducing the garbage
collector/allocator involvements.

Local synthetic benchmarks have shown a improvements from 38% of wall time to only
12%.
2024-02-28 20:00:13 -08:00
jvoisin
1f5c8ce353 Don't mix up capacity and length
- `make([]a, b)` create a slice of `b` elements `a`
- `make([]a, b, c)` create a slice of `0` elements `a`, but reserve space for `c` of them

When using `append` on the former, it will result on a slice with `b` leading
elements, which is unlikely to be what we want. This commit replaces the two
instances where this happens with the latter construct.
2024-02-28 19:57:30 -08:00
jvoisin
645a817685 Use modern for loops
Go 1.22 introduced a new [for-range](https://go.dev/ref/spec#For_range)
construct that looks a tad better than the usual `for i := 0; i < N; i++`
construct. I also tool the liberty of replacing some
`for i := 0; i < len(myitemsarray); i++ { … myitemsarray[i] …}`
with  `for item := range myitemsarray` when `myitemsarray` contains only pointers.
2024-02-28 19:55:28 -08:00
jvoisin
f4f8342245 Remove a superfluous condition
No need to check if the length of `line` is positive since we're checking
afterwards that it contains the `=` sign.
2024-02-28 19:47:30 -08:00
jvoisin
543a690bfd Close resources as soon as possible, instead of using defer() in a loop
So that resources can be freed as soon as they're not used anymore, instead of
waiting for the two nested loops to finish.
2024-02-28 19:47:30 -08:00
jvoisin
c4e5dad549 Remove superfluous escaping in a regex 2024-02-28 19:47:30 -08:00
jvoisin
fa12c23d79 Use strings.ReplaceAll instead of strings.Replace(…, -1) 2024-02-28 19:47:30 -08:00
jvoisin
4fe902a5d2 Use strings.EqualFold instead of strings.ToLower(…) == 2024-02-28 19:47:30 -08:00
jvoisin
61af08a721 Use .WriteString( instead of .Write([]byte(… 2024-02-28 19:47:30 -08:00
jvoisin
b04550e2f2 Use %q instead of "%s" 2024-02-28 19:47:30 -08:00
jvoisin
5e5cb056c5 Make internal/worker/worker.go read-only
Since workers don't communicate anything back to the pool with the channel,
there is no need to have it bidirectional.
2024-02-28 19:39:03 -08:00
jvoisin
48fa64f8ec Use a switch-case construct in internal/locale/plural.go instead of an avalanche of if-if-if-if-if
Less lines or code and marginally greater readability, yay!
Oh and also preallocate a map in LoadCatalogMessages just because we can.
2024-02-28 19:36:38 -08:00
jvoisin
f274394f0e Simplify formatFileSize
No need to use a loop with divisions and multiplications when we have logarithms.
2024-02-28 19:32:38 -08:00
jvoisin
9a4a942cc4 Simplify durationImpl 2024-02-28 19:32:38 -08:00
jvoisin
6b3b8e8c9b Inline some templating functions 2024-02-28 19:32:38 -08:00
jvoisin
5a7d6f8997 Make use of printer.Print when possible 2024-02-28 19:24:41 -08:00
jvoisin
b4ed17fbac Add a printer.Print to internal/locale/printer.go
No need to use variadic functions with string format interpolation
to generate static strings.
2024-02-28 19:24:41 -08:00
jvoisin
7660910232 Use prepared statement for intervals 2024-02-27 21:25:25 -08:00
jvoisin
b054506e3a Use proper prepared statements for ArchiveEntries 2024-02-27 21:25:25 -08:00
jvoisin
c961c6db7d Use proper prepared statement for updateEnclosures 2024-02-27 21:25:25 -08:00
jvoisin
b94756bbf0 Add a warning for StripTags 2024-02-27 20:41:47 -08:00
jvoisin
db6ae707ef Add some tests for add_image_title
I'm not sure if the behaviour is expected, but I didn't manage to
get the content injection to work in my browser, so I guess it's alright?
2024-02-27 20:41:15 -08:00
Frédéric Guillot
97feec8ebf Add more URL validation in media proxy 2024-02-26 20:29:40 -08:00
jvoisin
bce21a9f91 Remove github.com/google/uuid
Replace it with a hand-rolled implementation. Heck, an UUID isn't even a
requirement, according to [omnivore](https://docs.omnivore.app/integrations/api.html#saving-a-url-with-the-api)'s
documentation, any "unique id" would do.
2024-02-26 18:31:12 -08:00