Commit graph

47 commits

Author SHA1 Message Date
Gabriel Augendre
1d80c12e18
Prevent Youtube scraping if entry already exists 2021-03-08 20:10:53 -08:00
Frédéric Guillot
9a9a271b1f Limit full-text search indexation to first 500K characters
tsvector has a size limit of 1MB. See https://www.postgresql.org/docs/13/textsearch-limitations.html

Input text is now truncated to avoid this error:

"pq: string is too long for tsvector (1057834 bytes, max 1048575 bytes)"
2021-02-06 14:10:45 -08:00
Shizun Ge
02a4c9db53
Create feed query builder 2021-01-18 13:22:09 -08:00
Frédéric Guillot
a026efce6e Use created_at instead of published_at for archiving entries 2020-12-02 21:08:55 -08:00
Frédéric Guillot
de7a613098 Calculate reading time during feed processing
The goal is to speed up the user interface.

Detecting the language based on the content is pretty slow.
2020-11-18 17:43:24 -08:00
Frédéric Guillot
c394a61a4e Add Prometheus exporter 2020-09-27 20:04:48 -07:00
Frédéric Guillot
e6c6ee441a Use a transaction to refresh and create entries
Also includes few database improvements:

- Speed up entries clean up with an index and a goroutine
- Avoid the accumulation of enclosures for some feeds
2020-09-20 23:12:23 -07:00
Frédéric Guillot
ad66250734 Archive older entries first 2020-09-14 20:21:38 -07:00
Frédéric Guillot
13c89c29c5 Add option to archive unread entries 2020-09-12 20:17:52 -07:00
Shizun Ge
e32fa059e5
Do not archive shared items 2020-07-01 20:11:36 -07:00
Frédéric Guillot
9871e4f5d0 Add page to list and remove shared entries 2020-03-22 19:53:44 -07:00
Frédéric Guillot
59c8dac851 Display Feed URL on shared page instead of user feed entries 2020-03-17 20:28:53 -07:00
Lesterpig
41a2b7e58e Add "Share article" feature
A new "shareCode" field is generated for each entry, and allows
unlogged users to access the entry through the /shared endpoint.
This feature is particularly useful to share articles from miniflux
to third-party users without having them to visit the original source.

The image proxy is disabled and special cache headers are proposed in
the shared page to avoid denial of service.
2020-03-17 20:09:46 -07:00
Peter De Wachter
2570c3410b
History: show entries in the order in which they were read
Add a changed_at timestamp to the entries table. This field is updated
whenever the entry's metadata changes.
2020-02-10 20:20:03 -08:00
Benjamin Barenblat
cb494ac416 Correct spelling of “toggle” 2020-01-29 18:36:09 -08:00
Frédéric Guillot
d3883126bf Improve storage module 2019-10-29 22:59:00 -07:00
Frédéric Guillot
d610d091fe Avoid constraint error when having duplicate entries
During feed creation, duplicated entries will generate an SQL contraint error.

This change ignore the duplicated entry to avoid showing an error.
2019-09-18 22:53:47 -07:00
Frédéric Guillot
0dff432337 Remove debug timer from most storage functions 2019-04-28 20:21:31 -07:00
Jacob Wahlgren
22b68eefd8 Allow users to disable auto-remove
Adds ARCHIVE_READ_DAYS="-1"
2019-03-10 10:51:21 -07:00
Frédéric Guillot
f3fc8b7072 Use feed ID instead of user ID to check entry URLs presence 2019-02-28 20:43:33 -08:00
Jebbs
267b706544 Sort search results by relevance 2019-02-11 22:20:07 -08:00
Jebbs
87648490fd Make configurable the number of days to archive read items 2018-12-05 20:35:30 -08:00
Frédéric Guillot
cb037910df Archive more read entries 2018-11-25 15:01:01 -08:00
Frédéric Guillot
f6028f3863 Improve Fever middleware and handle groupID=0 2018-10-26 19:49:49 -07:00
Frédéric Guillot
74c35ad000 Add more details in feed storage errors to facilitate debugging 2018-10-21 11:48:29 -07:00
Frédéric Guillot
a614f98063 Set arbitrary maximum size for tsvector column
- The max size for tsvector is 1 MiB
- We index only the first million of characters,
  it should be enough for most feed entries.
2018-10-19 21:40:59 -07:00
Frédéric Guillot
ddfe969d6c Improve Fever API performances when marking a feed or group as read 2018-10-07 12:50:59 -07:00
Frédéric Guillot
dbcc5d8a97 Use canonical imports 2018-08-24 21:56:39 -07:00
Frédéric Guillot
af15412954 Add full-text search for entries and add search parameter to the API 2018-07-04 17:40:03 -07:00
Frédéric Guillot
f19ab21b7d Archive read entries automatically after 60 days 2018-05-19 16:45:27 -07:00
Frédéric Guillot
f49b42f70f Use vanilla HTTP handlers (refactoring) 2018-04-29 16:35:04 -07:00
Ben Brooks
538d08c16c Add CommentsURL to entry 2018-04-07 13:50:45 -07:00
Frédéric Guillot
b0442e0277 Do not update entry date while refreshing a feed
We do not update the published date because some feeds do not contains any date,
in this case it default to time.Now(), which could change the order of items on
the history page.
2018-01-29 21:07:55 -08:00
Frédéric Guillot
ffabb009b8 Do not override existing entries when the crawler is enabled 2018-01-20 14:04:19 -08:00
Frédéric Guillot
3b62f904d6 Do not crawl existing entry URLs 2018-01-20 13:25:20 -08:00
Frédéric Guillot
c57cafbef2 Add link to mark everything as read 2018-01-04 18:11:15 -08:00
Frédéric Guillot
c39f2e1a8d Rename helper packages 2018-01-02 19:15:08 -08:00
Frédéric Guillot
a63105e13b Improve timezone handling 2017-12-28 19:20:14 -08:00
Frédéric Guillot
d5b8f2fb88 Improve API 2017-12-24 18:04:34 -08:00
Frédéric Guillot
9868f900e9 Add bookmarks 2017-12-22 11:33:01 -08:00
Frédéric Guillot
1d8193b892 Add logger 2017-12-15 18:55:57 -08:00
Frédéric Guillot
84d912c979 Rewrite imports 2017-12-12 21:48:13 -08:00
Frédéric Guillot
7a35c58f53 Add readability package to fetch original content 2017-12-10 19:01:38 -08:00
Frédéric Guillot
e52f0b9cd2 Cleanup entries marked as removed 2017-11-21 16:33:36 -08:00
Frédéric Guillot
9457b3e5d6 Add missing attachments while refreshing a feed 2017-11-21 16:08:43 -08:00
Frédéric Guillot
549a4277b0 Add flush history feature 2017-11-21 15:46:59 -08:00
Frédéric Guillot
8ffb773f43 First commit 2017-11-19 22:01:46 -08:00