Option to fetch read articles

[Alkarex]

With the GReader API, it is possible to fetch articles that are read and/or unread.
While fetching unread articles is the most common use-case, it can sometimes be relevant to be able to fetch read articles as well.
Indeed, some articles might have been read from the Web interface or from another device or app.

So when the interface is set to show All articles, a "pull to refresh" should also fetch read articles from the server.
Or at least when the user is in a given feed or category, to selectively retrieve read articles from that feed or category.

Related to:

• Fetch more articles for current feed or category #532

I can give more details if this is not clear enough, and do not hesitate to ask for help on back-end side (FreshRSS).

[jocmp]

@Alkarex How far back do you suggest read articles should go? Right now I'm using the ot parameter on the reader/api/0/stream/items/ids Google Reader endpoint with a limit of 3 months max. I believe this excludes read articles at the moment.

I've also read BazQux's documentation which says not to use the ot parameter but it doesn't give an alternative approach to scoping articles by a time threshold.

Let me know what you think!

[Alkarex]

I am not sure whether you already have seen FreshRSS/FreshRSS#2566 (comment) otherwise I would like to invite you to check the hints there.

ot is useful on /reader/api/0/stream/contents/user/-/state/com.google/reading-list to get new articles since the last sync (as was done by News+ in the logs provided in the link above), to avoid re-downloading again and again the same content, which is a waste of resources, because this request is very expensive. This may be supplemented by a n limit, in particular for the very first sync.

Note that I am considering adding in FreshRSS database a way to include articles, which content has been modified since the last sync. When done, this will automatically work for you if you use ot this way.

There is a need to combine different types of queries (~7) to sync correctly. In particular, one should distinguish getting the new articles' content since the last sync, from getting an update of the states (read/unread, starred/not-starred).

If you use ot on reader/api/0/stream/items/ids, I think it should be the time of the oldest entry in your local database. This way I think it would be even better than using n like News+ did.

Important note regarding timestamps: only crawlTimeMsec or timestampUsec may be used for sync purposes as they have the needed monotonous property. They correspond to when the article was discovered. Conversely, the published field is declared by the original feed and has often no relation to how fresh the article is, so it must not be used for syncing. For instance, fan fictions may have published dates far in the future or in the past, just like academic publications, which may appear with a date many months in the past.

Back to the topic on how to sync read articles: I believe they should be fetched only when refreshing from the all articles view and not when in the unread articles view, using n as a limit, and maybe supplemented by ot of the time of the last sync that included read articles.

Let me know if this is helpful already, and I can happily provide more sparing.

[Alkarex]

I hope this was not too overwhelming :-P We could also have a videoconference if that can help - feel free to ask

[jocmp]

@Alkarex nope, not overwhelming! Capy currently tracks an updated_at column separately from published at for the reasons you outlined.

Thanks again for the help. I haven't started on the feature yet so I'll have better questions to ask when I do. I'll tracking development via this ticket #564.

[Alkarex]

P.S. Another open source Android client written in Kotlin, which is also dealing with the synchronisation https://github.com/readrops/Readrops/

[nycki93]

i just installed freshrss and capyreader today and I set my retention to "forever" but I can't get it to fetch old articles, its a little confusing having my capyreader appear "empty" when I know there are articles in the freshrss web ui. +1 to this.

[jocmp]

@nycki93 this should be implemented as of 2024.11.1080-dev. so it may or may not be a bug you're seeing.

which version of the app are you on? This is under Settings > About > Version. I believe the Play Store and F-Droid are still on an older version at the moment.

[nycki93]

I installed from f-droid and I'm on 2024.11.1077. Good to know this is coming though!