-
Notifications
You must be signed in to change notification settings - Fork 132
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Kronos dependency triggers a Local Network Permission alert on iOS >= 14 #647
Comments
hi @thibauddavid 👋 i can reproduce the issue only with local NTP pools, such as Clock.sync(from: "127.0.0.1", ...) { ... } otherwise i can't reproduce the issue and Kronos source code doesn't seem to try to access local network by itself. can you please make sure that your NTP pool isn't in your local network by any chance? |
Hi @buranmert , thanks for answering.
So I don't understand why we get this popup. |
could there be anything else in your project which triggers this popup? |
I'll have a look, but according to MobileNativeFoundation/Kronos#94 it really seems to be a problem with Kronos. |
hi again @thibauddavid 👋 |
I've been quite busy, I'll have a look as soon as I can ! |
I'm seeing a similar thing after adding the DataDog SDK to my app. It's not reproducible though, we've just gotten some random reports from some users asking why we need to scan the local network when the app shouldn't. Would it be possible to add a config to disable clock syncing? We don't really care that much about the logs being on an exact time. We segment them by user anyways, so they should be relative to each other. |
@sergiocampama is it possible to ask those users for more info? i still can't reproduce the issue although i tested scenarios such as: please let me know if you suspect of anything else. |
I'm asking a few users for this data, but they might not be savvy enough to know this. Still, an API to disable clock sync would be ideal for us to avoid this, since we don't care as our logs are user relative, so timing the logs to all users is not really worth it for us. Is this something that could be done? More than happy to send a patch if yes. |
that would be great if we could have more info from those users @sergiocampama 🙏 meanwhile, if that's okay for you too, you can fork our repo and patch it according to your needs. please let me know if that helps |
@buranmert we've just integrated DataDog into our app and like @sergiocampama are getting users report this pop up. We're not able to replicate this on our side in any way though. Did you get any further? |
hi @AJ9 , do you have any kind of more info about these reports by any chance? any possible patterns, a certain iOS version, device model, connection type, etc.? to be honest, unfortunately we haven't got any further yet as we still don't have data around the issue and we still can't reproduce it. |
Hey 👋, information is quite minimal this is what we do know: iPhone 12 Pro We also can't reproduce this, even whilst setting up similar conditions, it certainly isn't happening for all users. We're going to keep exploring the drop in connection problem as that seems the most likely cause... |
👋 Hey all! We're actively working on tracking this issue ☝️. Unfortunately, we couldn't manage to reproduce it locally and our efforts lead to conclusion that this might be occurring very rarely in some very specific and flaky network circumstances. To move our investigation forward, in #709 we're going to add extra monitoring to collect telemetry from Kronos internal execution in our dogfood 🐶 projects. Hopefully, this will lead to gathering enough data from production environment to nail down the problem. NTP time synchronisation takes important place in our SDK and it is there to guarantee that all collected telemetry is in sync (both on client and when propagated to backend in Datadog Distributed Tracing). Hence we're doing maximum effort to mitigate this issue. |
Hey all, have you managed to get anything from the logging at all? |
Hello @leearmstrong 👋. So far we collected ~1K samples from our dogfood projects (mainly our iOS app in beta channel). None of ~3.7K resolved IPs led to local network connection (checked with |
Hey @ncreated, I had this while on a cellular network connected to a VPN. I am still digging through the real cause myself but wondered if you had managed to capture any info either? |
@ncreated Any update on this issue? We are having users report the issue as well, and a few of our internal team members have seen it as well. Having our app inexplicably ask customers for obscure permissions erodes trust and is overall detrimental to our product's image and the user experience. |
Hey @eseay 👋. Although we haven't manage to reproduce this issue (also with VPN as pointed out by @leearmstrong), recently we got one hit in telemetry we collect in our own products. We now have a clear evidence that (in some circumstances) Kronos can try to query private IP during its NTP sync, bringing up the Local Network Permission alert. We already discussed mitigation plan for this problem and it is scheduled in our backlog 👍 with a high priority - we will add IP range filtering to prevent Kronos from connecting to private IPs. To leave more context for other folks stepping across this issue, in #709 we added monitoring to several phases of Kronos execution, including DNS resolution with We got one hit reporting One log is not much (vs dozen of thousands successful syncs), but it proves that Kronos can indeed reach a private IP and bring the Local Network Permission alert. It also means that IP range filtering can be a desired logic. In the same log we can also see that |
Closing - fixed in |
Hello @ncreated ! Coming with bad news here :( We've started to implement Datadog a few weeks ago and sent I came across this Github issue and updated our SDK to Is there anything I can do to help the investigations? I can have access to the device in question if needed |
I've also just updated to 1.11.0 and saw the prompt again. |
Hello 👋🥲 thanks for reports folks. There are two things that come to my mind for now:
|
Hello @ncreated! Yes, we are 100% sure this is caused by Datadog SDK, because the app release we did only contained the SDK implementation and network permission prompt reports started on that specific app version. This never happened before.
I can have access to the device on Monday so I can take a look. Could you tell me where to look at? (e.g. where to put a breakpoint/print, etc.) I'm not familiar with Kronos, I would appreciate help here to avoid spending time on research 🙏 |
Very clear, got it 👍👌.
The Kronos logic which resolves IPs to perform NTP queries goes through these lines: dd-sdk-ios/Sources/Datadog/Kronos/KronosInternetAddress.swift Lines 47 to 50 in dbfdfaf
We decide weather given I'm opening this issue again for more visibility. |
@ncreated thanks a lot for the detailed explanation, very much appreciated. 🙌 I had access to the internal device that could reproduce this issue on every app installation, those are the findings: I can not capture the exact
is the method that trigger the prompt, so I listed all I did the experience 4 times. (uninstall + reinstall, get the Host values, by printing
So at this point: dd-sdk-ios/Sources/Datadog/Kronos/KronosDNSResolver.swift Lines 54 to 55 in dbfdfaf
Does that help? I can generate more tries if needed, or please feel free to guide me if I need to look for something else 🙏 |
@victor-yn thanks a lot! It's great data - I will analyse it tomorrow. To clairfy - do you have a device at your dosposal that reproduces this problem easily 😲? What is the iOS version and device model? Does it use wifi or 3g / LTE / some other connection kind (hotspot? VPN?) when the problem occurs? |
@ncreated Yes! We do have an internal user who manages to reproduce it every time. He's using an iPhone XS & iOS 15.4.1. He's always on 4G, no VPN or hotspot used. Let me know how it goes and if you need more information! 🙏 |
@victor-yn I looked at these IPs and I wonder if the problem is not coming from Would it be possible for you to prove this assumption with more attempts? With few more, if we notice that having Also, I tried hitting exactly the IPs you listed from my iPhone (on LTE and WIFI) and it doesn't bring the alert 🧩. |
@ncreated I'm coming with bad news again 😬 As requested, I had access to the device in question this week and... we could not reproduce the prompt anymore 😭 I confirmed with the owner of the device, nothing special was done on it. Conditions were the same: 4G, no hotspot or VPN, the device did not restart, we used the exact same version of the app. The exact same steps we did to have the local prompt at every try the day before... did not work anymore. That also surprised the device owner as well, as he could always reproduce it before and now it's just magically gone. I still got the logs out of it: Host values, by printing
None of the try led to the local network prompt, so I assume that the I'll keep watching for reports, and try to see if I can find a pattern; but the fact that we can no longer reproduce it does not reassure me because the issue is unfortunately still there |
Has the option of just not relying on this extra dependency been considered? I understand that it was obviously added for some meaningful purpose, but if it is behaving in ways that we are unable to fully understand, I'd say that it's better to do without. As I mentioned in an earlier comment, especially as privacy continues to be a greater concern among the general public, having an app inexplicably ask for access to a customer's local network erodes trust and is overall detrimental. The drawback of delivering that customer experience far outweighs any kind of benefit we may perceivably get from logging and tracing. |
Hi @eseay, this topic has been a long and dificult one and we still haven't been able to get a satisfying explanation for why this issue happens, which means we can't have a proper and clean fix. This is mostly due to Apple's hidden logic on how they categorize a request as Local or not (apparently it goes beyond the standards of private network). We reached out to them to get some help but because we can't reproduce it locally, this is taking some time (and time away from adding or improving other features in our iOS SDK). In the meantime we're working on providing a mitigation to avoid this issue: in a few words, we will add an option to let developers provide their own NTP resolution. This means you will be able to use different NTP servers than ours and/or a different NTP library than Khronos, or write your own logic to compute the exact time. I don't have yet a timeline for when this option will be available but it should arrive sooner than later. |
@ncreated coming with good news again. For some unknown reasons, the internal user who couldn't reproduce the issue anymore... could reproduce it again. Nothing was changed: still on 4G, no hotspot or VPN. No iOS update. Nothing special was done on the device. Same app version. I ran the scenario 8 times: uninstall + reinstall, get the host prints before it shows the prompt as requested. Host values, by printing the
In every of the 8 tries, the Local Network prompt was displayed as soon as Does that information help you? Is there anywhere else you want me to log or don't hesitate to guide me if I need to look for something else that the |
As a follow-up to this, it is now possible to provide your own server time (or opt-out using Kronos with all its consequences) with the new API we added in dd-sdk-ios/Sources/Datadog/DatadogConfiguration.swift Lines 402 to 411 in f7f1d5e
|
Hey - we're also struggling with this prompt and just found this thread. Any updates since a year ago apart from allowing the usage of other NTP libraries? |
Also here. Our organisation already uses Datadog for our systems and platform and we really want to integrate it on our Mobile app (1.1 million customers) but we can't because of this! |
I don’t use DataDog, but I ran into this issue while using Kronos. At the time, I was able to test the IP addresses individually. In case this helps with debugging or a workaround, none of them appeared to be private IPs, and the IPv6 addresses all triggered the local network alert on the device that was seeing the issue, while IPv4 addresses did not. |
@ncreated shared the solution - you can override We implemented this a few weeks ago and it seems to have worked for us. |
Thank you! |
@nchase I can't manage to set the no-op. There is nothing in the documentation on how to do it: |
|
If i do that, then no logs are recorded anymore |
we have that configuration and we get the logs just fine |
Here we have the same problem, and we remove Kronos in the following way while we don't develop a solution on our side.
We added it to the configuration object used to start the lib but it is based on version 1 of the Datadog SDK |
Hi,
We are experiencing an issue with your SDK.
It relies on Kronos, which has an opened issue here MobileNativeFoundation/Kronos#94 about a local network permission popup being asked.
Any plan on your side to mitigate this ?
Thanks
The text was updated successfully, but these errors were encountered: