What about use unified notification ?

Strongly against!
FOSS has gone from one unified google push notification service to come to another?!
What for? So that now another service knows what software I use? I do not like it. My use of Telegram, it's only mine and Telegram!
In addition, the Telegram FOSS background service does not cause any problems with device autonomy. In the absence of Google services, my Android 9 smartphone works confidently for at least 5 days on a single charge.
If you have problems with this, then it is logical for you not to use FOSS, but to pay attention to the official Telegram client, whose notifications are based on a unified push service from Google.

On the contrary, I would add a line to manifest.xml to request to ignore battery optimization "android.permission.REQUEST_IGNORE_BATTERY_OPTIMIZATIONS" so that all sorts of Chinese shells do not kill the background process.

Well, you can always self-host your notification server. My opinion is there should be at least an option for using the unified push system because as I said some people host their own instances and I don't see any problem there.

My opinion is there should be at least an option for using the unified push system because as I said some people host their own instances and I don't see any problem there.

On your part, there is an unreasoned exchange of opinion, a wish.
You do not answer the main question - "What for?".
What "FOSS client" problem do you want to solve by adding a unified push service?
Why should it be exactly https://unifiedpush.org/ and not something else like https://openpush.io ?
Who will guarantee that these services will continue to exist and work tomorrow?, and not disappear like hundreds of xmpp-servers (maintained by both individuals and groups of enthusiasts)?
If you have a clear vision for the implementation, you can develop your own "fork" individually or together with like-minded people.

@2743ab2 not having a framework for push notifications in the FOSS android ecosystem is definitely a long standing pain point. Sure if you only use a single messenger, and it implement its background sync correctly, you probably don't see a problem. But as you add more apps it start to crumble because all these background processes keep constantly waking your phone and all together they do consume quiet some power.

Now if you take a look at what unifiedphush is about you will see that it is basically an API, it doesn't mandate how the push system is to be implemented. So it neither lock you with a vendor, nor with a specific push transport technology.

I don't think anybody is asking to drop the current background sync implementation and switch to UP only, just to have optional support.

I'll bump this request - I'm trying to de-google and having a bunch of messengers syncing in the background is neither pleasant nor good for battery. I just set up my own ntfy server to handle unifiedpush notifications (as well as other push notifications from my own alerts), and it was very simple to do.

Here is some information on how the matrix client element does it: https://github.com/vector-im/element-android/blob/16ca26569f99df230888f78a8e2529c5e385ddda/docs/unifiedpush.md

Any news?

I am all for this, however, I believe in order to do this properly, you would need something on the server side to actually publish notifications to the client(s). There is a zero percent chance that Telegram themselves will do it, so that leaves someone to write a middle-layer service that would run on a server somewhere, logged into your Telegram account, that will handle pushing notifications to your other client(s) when you get a new message. It's not as easy as setting this up on something like Matrix, where they control the server implementation and have a system in place for setting up arbitrary notification systems already.

So the tldr; this would require a server component to be written in addition to client changes, which is probably out-of-scope for this project.

Telegram allows sending push notifications to arbitrary HTTPS endpoints using the web push protocol: https://core.telegram.org/api/push-updates (type=10).

I might give a go at writing a web-push=>unifiedpush proxy this weekend, which should solve this issue.

Turns out no proxy is needed, Telegram's implementation is lax enough and doesn't check the return code of the push endpoint, for example pushing to ntfy works OK even if the return code is 200 and no Link header is present.
Unifiedpush can be directly integrated in the app, no proxy required.

@2743ab2 Sorry if I sound harsh, but it seems you have not spend a dime looking into https://unifiedpush.org/, have you? This is not a "service", it's an open-source library that can be implemented in any app and it allows the user to decide themselves what push server to pick (self-hosted like ntfy or NextCloud or FCM etc.) by installing the proper push client on his/her device for it. The user could even select an open-source FCM push client (which could also be embedded, since it's not official proprietary stuff), but this makes no sense here, because people could just get the official Telegram app for that, if they really want Google to be involved.

Take a look:

The green parts are installed by the user, the app registers itself through his/her push client with the push server (own infrastructure) and passes the received endpoint to the Push Server, afaik. When the App Server wants to push the message, it just sends it to the endpoint, which points to your Push Server (or a proprietary Push Server like FCM, if the user has chosen to).

There is just a thing to be mind of: The "App-Server" would be the Telegram server in this case, which won't happen. So, there has to be something in between. Since we don't want to introduce another to-be-trusted dependency that our messages have to go through, the "App Server" needs to be a self-hosted gateway as well, so it really only makes sense to consider for self-hosting.

The benefit is huge though. For all apps that implement Unified Push you only have to have one Push Client and its notifications run through your infrastructure, your battery will thank you without relying on any other dependency.

The only things Telegram-FOSS has to do to use unified push are:

1. Get a push endpoint from the library
2. Generate ECDH key material according to RFC-8291
3. Call account.registerDevice with the generated public key and the received endpoint
4. Decrypt received notifications first with ECDH, then with MTProto
5. Handle decrypted notifications.

@danog I wasn't talking to you, I was talking to @2743ab2, but it's nice to get some more insight about what actually need to be done on Telegram-FOSS by someone who is even deeper into the specs, I scratched the surface so far.

I actually missed your last comment. Wow this is good news, but kinda unsupported. A proxy would make this stable and prevent things from breaking for long if they decide to change something. But it's great to know that this would work without a proxy.

Any update?

Not sure if this is in the Todo list.
Hope someday Telegram FOSS uses UnifiedPush.

Telegram allows sending push notifications to arbitrary HTTPS endpoints using the web push protocol: https://core.telegram.org/api/push-updates (type=10).

I might give a go at writing a web-push=>unifiedpush proxy this weekend, which should solve this issue.

hi, i want to send notification to my own telegram clone with fcm, but i can not recognize what should be the json that i send, can you help me? i can send remote message with data payload from fcm api but telegram need its own data payload

unified push is quite nice, element (f-droid) has the option to use unified push as notification system. they use https://ntfy.sh/ as default server but ntfy can easily be self hosted.
fluffy chat also uses unified push. works quite nice.

is anyone working on implementing this? or am i free to fork and make a PR?

is anyone working on implementing this? or am i free to fork and make a PR?

I have done some work to push to Telegram (https://github.com/quqkuk/Telegram-webpush) (that could be merged here too), currently it's not the best code (it uses Java's security API instead of using BoringSSL and there's not a lot of error handling), but most of the RFCs are implemented

hi,
if you want to test it I backported the patches from @quqkuk on Telegram-FOSS.

https://github.com/drizzt/Telegram-FOSS/tree/UnifiedPush

hi, if you want to test it I backported the patches from @quqkuk on Telegram-FOSS. I rebased it with the last Telegram version, I removed the unneeded FOSS patches (all the patches that "fixes" standard telegram push, since my fork is ONLY to be used with UnifiedPush) and I updated it to be built on SDK 34:

https://github.com/drizzt/Telegram-FOSS-UnifiedPush

Thank you so much for this, I'm trying it right now!
What are the recommended settings for the "keep alive service" and the "background connection" toggles? I already excluded it from battery optimizations, actually I disabled Doze altogether.

EDIT: I installed the UnifiedPush FCM Distributor (since I'm on microG but here no TG client is delivering notifications except Telegram X) and was able to register TG FOSS, let's see how it goes!

EDIT 2:
I switched from the FCM Distributor to ntfy which seems to work flawlessly so far.
Very good job, keep this beating!

hi, if you want to test it I backported the patches from @quqkuk on Telegram-FOSS. I rebased it with the last Telegram version, I removed the unneeded FOSS patches (all the patches that "fixes" standard telegram push, since my fork is ONLY to be used with UnifiedPush) and I updated it to be built on SDK 34:

https://github.com/drizzt/Telegram-FOSS-UnifiedPush

I'm testing it and everything is working right now. Do you want to merge it upstream or keep a separate fork?

hi, if you want to test it I backported the patches from @quqkuk on Telegram-FOSS. I rebased it with the last Telegram version, I removed the unneeded FOSS patches (all the patches that "fixes" standard telegram push, since my fork is ONLY to be used with UnifiedPush) and I updated it to be built on SDK 34:

https://github.com/drizzt/Telegram-FOSS-UnifiedPush

Moreover, are you willing to maintain both this fork and Mercurygram? Thanks!

hi all,
I cleaned up the patches, added a settings and proposed them as PR #715.

In the future, I'll only push stuff inside Mercurygram, that will include the features I use, and I'll open PR here with bug fixes for UnifiedPush.

hi all, I cleaned up the patches, added a settings and proposed them as PR #715.

In the future, I'll only push stuff inside Mercurygram, that will include the features I use, and I'll open PR here with bug fixes for UnifiedPush.

That's great! Thank you for all this: given my long-standing problems with Telegram notifications, it's exactly what I've been helplessly looking for since ages!

@quqkuk

I have done some work to push to Telegram (https://github.com/quqkuk/Telegram-webpush) (that could be merged here too), currently it's not the best code (it uses Java's security API instead of using BoringSSL and there's not a lot of error handling), but most of the RFCs are implemented

Hey, great work, you're going to try to have it added to Telegram itself? I didn't see a pull request there yet

I've been monitoring this issue in the last few days, thank you for the attention, but there are bad news I'll get to in a bit.

@Gabr-F said:
Hey, great work, you're going to try to have it added to Telegram itself? I didn't see a pull request there yet

Yes, that was my goal - Instead of making a patch based upon Telegram-FOSS I wanted to base it on Telegram, hoping that even if it wouldn't get merged, applying it to this codebase would still be feasible without too much hassle.

@marc0x1 said:
I'm testing it and everything is working right now.

Are you sure? I checked @drizzt's repo (thank you for the work btw) and it still has the Push Service Notification to keep the connection to Telegram's servers open, which means that notifications might be coming from that connection instead of the UnifiedPush distributor

I'm saying this because my patches never worked even for me - At some point I doubted my own RFC8188 message parser and since I had accepted that I would've had to rewrite the whole thing to follow the "guidelines" I had guessed while writing and debugging, I pulled Google's Tink which implements Message Encryption for Web Push (it's the same library that the UnifiedPush example uses) and it too threw an error about the record size field bearing an invalid value, which means that Telegram doesn't seem to return a valid encrypted WebPush payload (I'm ruling out it being a WebPush-through-UnifiedPush problem, as the UnifiedPush example treats the message received from the distributor as an encrypted WebPush payload that follows the RFC8291 spec).
Due to other complications I ended up pausing development, but I intend on fixing this in the following weeks (I assume there must be a way, since Telegram Web does receive a payload it can read).
I'm sorry I haven't communicated this well enough and I've let my repo die without mentioning anything (I intended to have some notice written in the README, but alas, I must've gotten lost while testing with Tink)

@quqkuk In my tests I disabled push notification in options and I'm using a self-hosted ntfy.
I see that Telegram app is not running and, as soon as I receive the POST from Telegram Server (sometimes it takes some minutes) to the correct ntfy topic, I receive the notification

@quqkuk In my tests I disabled push notification in options and I'm using a self-hosted ntfy. I see that Telegram app is not running and, as soon as I receive the POST from Telegram Server (sometimes it takes some minutes) to the correct ntfy topic, I receive the notification

Well, that's good news! I'll improve the code if I can, as it's clearly something that should not go in production and was there just to test things, but that will let me rest more easily

@quqkuk ugh, you were right, it seems it's not parsing the message correctly (wrong size as you told me) and I'm investigating about that.
I didn't notice that before, since I get the notifications since processRemoteMessage makes Telegram reconnects to the server, get the updates and return in standby.
For this reason, I changed my branch, and my PR, to use a simpler implementation of UnifiedPush handler that only does that (by using Simple push instead of Web Push).

Hopefully it'll possible to fix your Web Push implementation some days, but in the meanwhile the simple one works

Hopefully it'll possible to fix your Web Push implementation some days, but in the meanwhile the simple one works

JFYI Before this change it was perfectly working here, not anymore now.
I had to re-enable the keepalive service (not the background connection, but still) to receive notifications and am not even sure they come consistently yet.

EDIT: it seems that re-enabling the keepalive did the trick but I'll be testing it throughout the day...

I think I know what's going on
I suspected this, I've run a tcpdump and now I know for certain - Telegram's servers use an old draft of RFC8188, which passes the key and the salt as HTTP Headers instead of at the beginning of the payload.
I've just sent this in as a bug report but it also means that implementing UnifiedPush requires a rewrite proxy (a Gateway, in UnifiedPush lingo) to decrypt the message before it reaches the Push Provider

@quqkuk I think it'll require a proxy to be put BEFORE nfty (a proxy that directly receive Telegram messages) and that converts it to the real RFC 8291, since headers are stripped by nfty server and not forwarded

@quqkuk I think it'll require a proxy to be put BEFORE nfty (a proxy that directly receive Telegram messages) and that converts it to the real RFC 8291, since headers are stripped by nfty server and not forwarded

Yes that's what I wrote, or at least what I meant

Hopefully it'll possible to fix your Web Push implementation some days, but in the meanwhile the simple one works

JFYI Before this change it was perfectly working here, not anymore now. I had to re-enable the keepalive service (not the background connection, but still) to receive notifications and am not even sure they come consistently yet.

EDIT: it seems that re-enabling the keepalive did the trick but I'll be testing it throughout the day...

Ok, this morning's build was working reliably with the keepalive service enabled; this afternoon's build is working with all disabled, don't know whether it's coincidental or you changed something! 👍

@quqkuk ok, I created a simple proxy that appends the headers to the payload (using the same method as https://github.com/UnifiedPush/common-proxies/blob/1567aa9ea7fc6aef80162180f4e54208cd8784f9/gateway/webpush.go#L33-L42), so now we only need to change your java function in order to get the keys from the base64url headers and use it by using the old method in draft-ietf-httpbis-encryption-encoding-03.
I have my proxy on top on ntfy at https://ntfy.belloworld.it, if you want to use it to implement the remaining stuff.

Hint: if you want to install the proxy on your setup, be sure to put the hostname inside /etc/hosts (in my case I have 116.203.117.243 ntfy.belloworld.it) so you don't need to do lots of dns queries

From an user perspective: since right now the simple implementation is working like a charm, won't this new reworked one overcomplicate things?

@gcvl yes, I changed something in the UnifiedPush handler (threads, postInitApplication, etc). if you can confirm it works also after 30+ minutes from the previous notification I'll remove the draft from the PR so they could merge it

@gcvl yes, I changed something in the UnifiedPush handler (threads, postInitApplication, etc). if you can confirm it works also after 30+ minutes from the previous notification I'll remove the draft from the PR so they could merge it

Yes, I currently have it working, I tested with variable timespans, from some minutes to over 30 (screen off) and it got notifications near instantly every single time.

I'll tell you my setup though: I'm on Lineage + microG, Doze entirely disabled, Mercurygram unrestricted from battery optimizations with keepalive and TG background connection both disabled, ntfy connected to ntfy.sh and registered to FCM through microG (surprisingly I never ever managed to make the official TG receive notifications through microG, on all my degoogled phones it's the only app that just wouldn't reliably alert me no matter what I tried to do and all its other forks were no better).

I could do some tests with Doze fully enabled and battery optimization ON for MG or if you wish to remain on the safe side you can toggle ON the keepalive service as default so that it places just the persistent notification to stay active and running.

Doze ON and battery optimization ON: the app fell asleep and didn't notify.
Making other tests but I guess that we need either battery optimization OFF or the persistent notification (keepalive ON).

Doze ON and battery optimization ON: the app fell asleep and didn't notify. Making other tests but I guess that we need either battery optimization OFF or the persistent notification (keepalive ON).

https://github.com/drizzt/Mercurygram#notes

Thank you for reporting, I hope it's clear enough and it's what you said

Doze ON and battery optimization ON: the app fell asleep and didn't notify. Making other tests but I guess that we need either battery optimization OFF or the persistent notification (keepalive ON).

https://github.com/drizzt/Mercurygram#notes

Thank you for reporting, I hope it's clear enough and it's what you said

Yes, they're needed and very clear hints!
Meanwhile Doze ON and battery optimization OFF: OK. Doing some more tests anyway.

Doze ON and battery optimization ON: the app fell asleep and didn't notify. Making other tests but I guess that we need either battery optimization OFF or the persistent notification (keepalive ON).

https://github.com/drizzt/Mercurygram#notes
Thank you for reporting, I hope it's clear enough and it's what you said

Yes, they're needed and very clear hints! Meanwhile Doze ON and battery optimization OFF: OK. Doing some more tests anyway.

if you can try the last apk I pushed some minutes ago. it should not change anything, but who knows 😵‍💫

Doze ON and battery optimization ON: the app fell asleep and didn't notify. Making other tests but I guess that we need either battery optimization OFF or the persistent notification (keepalive ON).

https://github.com/drizzt/Mercurygram#notes
Thank you for reporting, I hope it's clear enough and it's what you said

Yes, they're needed and very clear hints! Meanwhile Doze ON and battery optimization OFF: OK. Doing some more tests anyway.

if you can try the last apk I pushed some minutes ago. it should not change anything, but who knows 😵‍💫

Sure! Updated, I'm doing some tests and letting you know.

Doze ON and battery optimization ON: the app fell asleep and didn't notify. Making other tests but I guess that we need either battery optimization OFF or the persistent notification (keepalive ON).

https://github.com/drizzt/Mercurygram#notes
Thank you for reporting, I hope it's clear enough and it's what you said

Yes, they're needed and very clear hints! Meanwhile Doze ON and battery optimization OFF: OK. Doing some more tests anyway.

if you can try the last apk I pushed some minutes ago. it should not change anything, but who knows 😵‍💫

It's great! It works just like the earlier one (with battery optimizations OFF) 😊🫡

Ok, I happily switched to Mercurygram from Telegram X, I'm sold 😊
It's your project, but as long as it works like this I'd recommend to just keep it simple and usable for everyone; ottimo lavoro, continua così! 👍

I'm waiting for a reply on UnifiedPush/common-proxies#64.
That will hopefully give us a straightforward way to setup a proxy that bridges between WebPush Draft 4 "aesgcm" format payloads and UnifiedPush

I'll preface this by saying that I love Mercurygram and @quqkuk I love your PR. But on the UP side we have previously discussed the merits of sync on push vs a webpush proxy: UnifiedPush/wishlist#15 (comment)

We're looking at this from the perspective of simple self hosters, not just UP enthusiasts, i.e. people that might be using their own NextPush or Prosody UP plugins.

Sync on Push: they don't need to host common proxies (or a similar gateway doesn't have to be built into all the distributors), and they don't need to use a public gateway (privacy).

With a WebPush proxy: Maybe one or two fewer requests to the server? The phone still has to wake up and talk to the server. Additionally, we endorse and support an outdated specification which is (imho) a bad thing.

@quqkuk I'm not totally familiar with the Telegram API, are there any advantages here in particular of decryption instead of sync on push?

@p1gp1g what do you think?

We had this discussion mainly for Mastodon, which is also self-hostable. In this case this is particularly inefficient to push from one mastodon server to the centralized push-gateway (common-proxy to convert old-webpush->webpush) then to the push provider, which may be on the same server than Mastodon server.
Something must be done to self-host that push gateway, to discover it, etc. which lead to many downsides (user not correctly setting this up, new things to host, etc).
All this compare to sync-on-push, which is a good way to notify, the choice is pretty straightforward.

But, in the case of Telegram, this is a bit different: its server is centralized. So, if the notif goes throw a centralized server then to the push provider, it is nearly the same than from the centralized server (telegram) to the push provider. You still have to check if this is more efficient than a sync-on-push and you have to consider the accessibility of the gateway : if it goes down no one will receive their notif.

Finally, in my opinion: you should check with sync-on-push and the push gateway. If you don't see any difference, then the sync-on-push is probably preferable

With a WebPush proxy: Maybe one or two fewer requests to the server? The phone still has to wake up and talk to the server.

The phone does not have to wake up, per Android's Documentation on Broadcasts, and the application does not have to ping the server (even though Telegram resumes the connection, it does so only after processing the notification).
I think that 2/3 fewer requests is still a lot when the requests required to get an update are 2 to 4 - that's a 50% to 100% decrease that translates in better efficiency on the wire and may translate also in less battery consumption (again, Telegram usually keeps running if Android allows it to, so it is not an easy theory to test).
It also translates in better user experience if the application has to cold start (meaning that there's no process already running to receive the notification), as it can show a notification without waiting for the TCP Handshake and subsequent Update fetching.

Additionally, we endorse and support an outdated specification which is (imho) a bad thing.

I agree, that's why I filed their implementation as a bug on the Telegram bug tracker.
Unfortunately, as was reported on the wishlist issue, most applications a couple of years ago still used the old aesgcm format. Moreover, there's too few people that are actually using WebPush for Telegram notifications, and even less want to do so from a platform that doesn't already support the old draft 4 format, which means that Telegram probably won't update their implementation until it breaks, so currently the only way to have UP support is to bridge the two protocols (or use sync on push)

they don't need to use a public gateway (privacy)

Something must be done to self-host that push gateway, to discover it, etc. which lead to many downsides (user not correctly setting this up, new things to host, etc).

That's why I sent in a PR that includes a transparent proxy - the gateway doesn't have to be public and we get a somewhat standard format for WebPush Draft 4 messages that is compatible with UnifiedPush (my PR isn't related to telegram - it just moves headers into the body) and keeps encryption (which would be kept even if the gateway were public). A Transparent Proxy put in front of the UP Provider also means that there's no need to find a way to make the gateway's endpoint discoverable - the URL given by the provider is enough.
Feature discoverability might be a problem - there's no way to point out whether a provider supports conversion from aesgcm to some "lossless" UP compatible format, but that can be solved by having the proxy reply to GET requests as was proposed in the wishlist issue (e.g. by replying to those requests with {"unifiedpush":{"version":1,"aesgcm_rewrite":1}}) - but it might also be a non-issue, as this would be useful only if the server knew about UnifiedPush and wanted to force an aesgcm-formatted message through it, which is kind of strange unless that server is a rewrite proxy (it might be of use to the Connector library, as its register methods have a features array in the parameters).
Yes, this approach requires hosting and configuring another server, I'm not going to argue that that isn't bad, but I don't really see it as a problem since any user who's using UP is probably hosting their own instance or has it managed by someone who I assume wouldn't be bugged by hosting a relatively lightweight rewrite proxy, but I'm not going to defend this further.
It would be better if Telegram supported WebPush's final document, or even better UnifiedPush, but alas, that's what we have now

All this compare to sync-on-push, which is a good way to notify, the choice is pretty straightforward.

I'd argue that sync on push is a good way to notify, but it's not the best - it adds an empty message to the whole update fetching flow and doesn't allow for applications to notify people without first going online (which often requires waiting for the TCP handshake and maybe other handshakes as the protocol stack reaches the application level).

Also, sync-on-push is not mutually exclusive to Draft 4 over UP - both can be achieved in multiple ways, even by just using aesgcm format's body to signal updates

Yeah that makes sense. Just looked at Telegram's API and it looks like there is sufficient information to notify in the message itself, so it's possible we don't even need to sync with the server?

In that context, decryption seems like a good idea to me as long as the app falls back to sync on push. i.e. If it receives a push message, and its not in the right format, it should sync with the server. That way, if someone doesn't have the gateway, their Telegram UP will still work.

And as you said, messages can be redirected to the gateway based on the aesgcm header, so we don't need the complexity of discovery. Overall seems like this gateway is a good idea then.

@quqkuk
I thing you have misunderstood my previous comment. The first part is related to federated servers, the seconds to centralized ones.

First, push gateways are something common in mobile development, they are host by the app developer to push to the device push provider. For instance for mastodon: toot-relay, PushToFcm and metatext-apns, for matrix: sygnal and fluffygate.

As long as the service server OR push provider is a centralized server, this has no real impact on the architecture for sending notifications with a centralized push gateway. Most of the time this is the case because the push provider is centralized. With telegram, we are in this case too: the service server is centralized.

Nevertheless, you should test with sync-on-push and and the gateway, if the results are comparable (latency, battery drain), then the effort of a push gateway is not worth it. (And the time required for a TCP handshake is invisible for a user)

When both are decentralized, sync on push is way more efficient. The UnifiedPush discovery response must not include anything related to push gateway. The provider must be application agnostic. Else it will start fragmenting everything.
We have done a discovery request for matrix applications, (which doesn't change UnifiedPush related requests) because the path of push gateways is hardcoded: A GET to /_matrix/push/v1/notify allows you to know if there is a matrix gateway.
And even a configuration as simple as that required for NextPush (due to Nextcloud's limitations) often gives some users a headache.