Bridge network DNS resolution is slow

[kkuphal]

Summary

Bridge network using VCH in resolv.conf is slow. In particular, this was evident using maridb/mysql with phpmyadmin. phpmyadmin is severely degraded performing operations over the bridge network by hostname.

Environment information

vSphere and vCenter Server version

vCenter 6.5.0.23000
vSphere 6.5.0, 11925212

VIC Appliance version

vic-v1.5.2-7206-92ebfaf5

Configuration

• Embedded or external PSC: Embedded
• How was the OVA deployed? (Flex client, HTML5 client, ovftool): Flex client
• Does the VIC appliance recieve configuration by DHCP? No. VIC is statically assigned
• What stage of the Appliance Lifecycle is the VIC appliance in? Running

• IP address of VIC appliance: 192.168.5.103
• Hostname of VIC appliance: vic.fhpaschen.com
• IP address of vCenter Server: 192.168.5.114
• Hostname of vCenter Server: fhp-vcbe.fhpaschen.com

Details

Inside the phpymadmin container using ash, ping by hostname can take as long as 16 seconds to resolve.

Steps to reproduce

1. Launch a mariadb/mysql container with a "mysql" bridge network
   docker run -d --name fhp-mysql --network mysql -e MYSQL_ROOT_PASSWORD="password" -e MYSQL_USER="cms" -e MYSQL_PASSWORD="cms" -e MYSQL_DATABASE="xibo" -v mysql-data:/var/lib/mysql/data -p 3306:3306 mariadb
2. Launch a phpmyadmin container connected to the "mysql" bridge network
   docker run -d --name fhp-phpmyadmin --network mysql -e PMA_HOST=fhp-mysql -p 8080:80 phpmyadmin/phpmyadmin
3. Connect to the phpmyadmin web interface and basic operations such as showing existing tables take considerable time to perform.
4. Running phpmyadmin connected by IP exhibits standard performance
   docker run -d --name fhp-phpmyadmin --network mysql -e PMA_HOST=172.19.0.2 -p 8080:80 phpmyadmin/phpmyadmin
5. Troubleshoot by connecting to the phpmyadmin container
   docker exec -it fhp-phpmyadmin busybox ash
6. Run "ping fhp-mysql". Can take up to 16 seconds to resolve.

Actual behavior

DNS resolution is extremely slow resulting in poor performance for certain applications which perform many DNS lookups

Expected behavior

Low latency DNS lookups on bridge networks

Support information

Logs

See also

Troubleshooting attempted

• Searched GitHub for existing issues. (Mention any similar issues under "See also", above.)
• Searched the documentation for relevant troubleshooting guidance.
• Searched for a relevant VMware KB article.

[Shogun8]

I have a similar issue. Using the same setup, my containers cannot resolve each others hostnames over the user-defined bridge network; it attempts to resolve the hostnames using the DNS server that was configured during the creation of the VCH. My guess would be that all the containers on the bridge network are attempting to resolve on an external DNS server first, then falling back to the internal DNS.

The behavior that I expect to see is that on the bridge network, the containers do not attempt to use an external DNS server at all. All the DNS resolution should be up to the VCH acting as a forwarder to the configured external DNS servers.