Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: 2 VMs are getting same ip address #2932

Open
1 task done
berelton opened this issue May 9, 2024 · 11 comments
Open
1 task done

[Bug]: 2 VMs are getting same ip address #2932

berelton opened this issue May 9, 2024 · 11 comments
Assignees
Labels
bug Something isn't working

Comments

@berelton
Copy link

berelton commented May 9, 2024

What happened?

Hello netmaker team, I got the issue when 2 VMs got the same ip address.

So we have 3 VMs:
main -> has netmaker instance deployed
vm1 -> connecting to meshnet
vm2 -> connecting to meshnet

On main VM we are creating 2 subnets:

$ nmctl network create --name test1 --ipv4_addr 10.10.0.0/16
$ nmctl network create --name test2 --ipv4_addr 192.168.10.0/24
$ nmctl enrollment_key create --tags test1 --unlimited --networks test1
$ nmctl enrollment_key create --tags test2 --unlimited --networks test2
$ nmctl network list
+-----------+----------------------+----------------------+-----------------------+----------------------+
|   NETID   | ADDRESS RANGE (IPV4) | ADDRESS RANGE (IPV6) | NETWORK LAST MODIFIED | NODES LAST MODIFIED  |
+-----------+----------------------+----------------------+-----------------------+----------------------+
| test1     | 10.10.0.0/16         |                      | 2024-05-06T14:43:22Z  | 2024-05-06T14:43:22Z |
| test2     | 192.168.10.0/24      |                      | 2024-05-09T18:00:11Z  | 2024-05-09T18:00:11Z |
+-----------+----------------------+----------------------+-----------------------+----------------------+

On vm1:

$ wget -O netclient https://fileserver.netmaker.io/releases/download/v0.24.0/netclient-linux-amd64 && chmod +x ./netclient && ./netclient install
$ netclient join -t <token_for_test1>
$ netclient join -t <token_for_test2>
$ hostname -I
10.10.0.1 192.168.10.1

On vm2:

$ wget -O netclient https://fileserver.netmaker.io/releases/download/v0.24.0/netclient-linux-amd64 && chmod +x ./netclient && ./netclient install
$ netclient join -t <token_for_test1>
$ netclient join -t <token_for_test2>
$ hostname -I
10.10.0.2 192.168.10.1

So as you see both the VMs are getting 192.168.10.1 ip, but they should have different ip addresses.

Version

v0.24.0

What OS are you using?

No response

Relevant log output

No response

Contributing guidelines

  • Yes, I did.
@berelton berelton added the bug Something isn't working label May 9, 2024
@NEETweeb
Copy link
Collaborator

NEETweeb commented May 10, 2024

Hello @berelton !
Thanks for reporting this issue.
Tried to reproduce it on my end but wasn't able to verify the issue.

But this doesn't mean that this is a non-issue.
It would help if you could provide the nodes list via nmctl node list.

@berelton
Copy link
Author

berelton commented May 10, 2024

hello @NEETweeb !

Appreciate the swift reply!

Sure, here is the response:

$ nmctl node list
+--------------------------------------+-----------------+-----------+--------+-----------------------+-------+
|                  ID                  |    ADDRESSES    |  NETWORK  | EGRESS | REMOTE ACCESS GATEWAY | RELAY |
+--------------------------------------+-----------------+-----------+--------+-----------------------+-------+
| 02a36777-62b2-4ff9-8952-75ff3796a03f | 10.10.0.2/16    | test1     | false  | false                 | false |
| 74e5e2f6-f5c5-4716-82a2-5371c2bef3ea | 192.168.10.1/24 | test2     | false  | false                 | false |
| cd9a6669-8dd9-47a0-aff9-53324df69273 | 10.10.0.1/16    | test1     | false  | false                 | false |
+--------------------------------------+-----------------+-----------+--------+-----------------------+-------+

If you could share your email and ssh key, I can share share you the access to the VMs to investigate, or we can have a quick call to debug together.

@aizhvaly
Copy link

hello @NEETweeb !

I'd like adjust more context about @berelton netmaker setup.

This situation happen when deploying netmaker via helm to a Kubernetes cluster with 2 replicas as indicated in the documentation examples.
Most likely, 2 requests to connect to the network came on different replicas due to the Robin Round balancing on gateway. It seems that the problem may be related to synchronization between netmaker servers.
I hope this information will help in studying the problem.

@berelton
Copy link
Author

hello @NEETweeb , @afeiszli, any updates on that?

@abhishek9686
Copy link
Member

hello @NEETweeb , @afeiszli, any updates on that?

While using HA setup make sure caching is set to false CACHING_ENABLED=false

@berelton
Copy link
Author

berelton commented May 17, 2024

Hello @abhishek9686 , it is set to false.

$ nmctl server config
{
  "CoreDNSAddr": "SERVER_PUBLIC_IP",
  "APIConnString": "api.id0.mydomain.com:443",
  "APIHost": "api.id0.mydomain.com",
  "APIPort": "8081",
  "Broker": "",
  "ServerBrokerEndpoint": "",
  "BrokerType": "mosquitto",
  "EmqxRestEndpoint": "",
  "NetclientAutoUpdate": "enabled",
  "NetclientEndpointDetection": "",
  "MasterKey": "(hidden)",
  "DNSKey": "(hidden)",
  "AllowedOrigin": "*",
  "NodeID": "netmaker-a77-0",
  "RestBackend": "on",
  "MessageQueueBackend": "",
  "DNSMode": "on",
  "DisableRemoteIPCheck": "off",
  "Version": "v0.21.2",
  "SQLConn": "",
  "Platform": "linux",
  "Database": "postgres",
  "Verbosity": 1,
  "AuthProvider": "",
  "OIDCIssuer": "",
  "ClientID": "",
  "ClientSecret": "",
  "FrontendURL": "",
  "DisplayKeys": "on",
  "AzureTenant": "",
  "Telemetry": "on",
  "HostNetwork": "",
  "Server": "id0.mydomain.com",
  "PublicIPService": "",
  "MQPassword": "",
  "MQUserName": "",
  "MetricsExporter": "",
  "BasicAuth": "",
  "LicenseValue": "",
  "NetmakerTenantID": "",
  "IsEE": "no",
  "StunPort": 3478,
  "StunList": "",
  "TurnServer": "",
  "TurnApiServer": "",
  "TurnPort": 0,
  "TurnUserName": "",
  "TurnPassword": "",
  "UseTurn": false,
  "UsersLimit": 0,
  "NetworksLimit": 0,
  "MachinesLimit": 0,
  "IngressesLimit": 0,
  "EgressesLimit": 0,
  "DeployedByOperator": false,
  "Environment": "",
  "JwtValidityDuration": 43200000000000,
  "RacAutoDisable": true,
  "CacheEnabled": "",
  "endpoint_detection": false,
  "AllowedEmailDomains": ""
}

@berelton
Copy link
Author

hello @NEETweeb , @afeiszli, any updates on that?

@abhishek9686
Copy link
Member

@berelton what is the rate at which you are joining clients to the network? are multiple clients joining the network at same time?

@berelton
Copy link
Author

@abhishek9686 it is like 5 seconds difference, yes.

Is there a way to handle this?

@abhishek9686
Copy link
Member

Hello @abhishek9686 , it is set to false.

$ nmctl server config
{
  "CoreDNSAddr": "SERVER_PUBLIC_IP",
  "APIConnString": "api.id0.mydomain.com:443",
  "APIHost": "api.id0.mydomain.com",
  "APIPort": "8081",
  "Broker": "",
  "ServerBrokerEndpoint": "",
  "BrokerType": "mosquitto",
  "EmqxRestEndpoint": "",
  "NetclientAutoUpdate": "enabled",
  "NetclientEndpointDetection": "",
  "MasterKey": "(hidden)",
  "DNSKey": "(hidden)",
  "AllowedOrigin": "*",
  "NodeID": "netmaker-a77-0",
  "RestBackend": "on",
  "MessageQueueBackend": "",
  "DNSMode": "on",
  "DisableRemoteIPCheck": "off",
  "Version": "v0.21.2",
  "SQLConn": "",
  "Platform": "linux",
  "Database": "postgres",
  "Verbosity": 1,
  "AuthProvider": "",
  "OIDCIssuer": "",
  "ClientID": "",
  "ClientSecret": "",
  "FrontendURL": "",
  "DisplayKeys": "on",
  "AzureTenant": "",
  "Telemetry": "on",
  "HostNetwork": "",
  "Server": "id0.mydomain.com",
  "PublicIPService": "",
  "MQPassword": "",
  "MQUserName": "",
  "MetricsExporter": "",
  "BasicAuth": "",
  "LicenseValue": "",
  "NetmakerTenantID": "",
  "IsEE": "no",
  "StunPort": 3478,
  "StunList": "",
  "TurnServer": "",
  "TurnApiServer": "",
  "TurnPort": 0,
  "TurnUserName": "",
  "TurnPassword": "",
  "UseTurn": false,
  "UsersLimit": 0,
  "NetworksLimit": 0,
  "MachinesLimit": 0,
  "IngressesLimit": 0,
  "EgressesLimit": 0,
  "DeployedByOperator": false,
  "Environment": "",
  "JwtValidityDuration": 43200000000000,
  "RacAutoDisable": true,
  "CacheEnabled": "",
  "endpoint_detection": false,
  "AllowedEmailDomains": ""
}

can you check the CacheEnabled config in the configmap deployed?

@sk3pp3r
Copy link

sk3pp3r commented Jul 10, 2024

can you check if both vm's have the same mac address?

To verify if both virtual machines have identical MAC addresses, you would need to check the network configuration settings of each VM. Typically, MAC addresses should be unique to each device to avoid network conflicts.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

6 participants