xdsclient: stop caching xdsChannels for potential reuse, after all re… #13210

Job	Run time
vet-proto	22s
tests (vet, 1.22)	1m 28s
tests (extras, 1.23)	2m 47s
tests (tests, 1.23)	4m 13s
tests (tests, 1.23, -race)	5m 36s
tests (tests, 1.23, 386)	4m 47s
tests (tests, 1.23, arm64)	6m 20s
tests (tests, 1.22)	3m 33s
tests (tests, 1.23, -race, GRPC_EXPERIMENTAL_ENABLE_NEW_PICK_FIRST=true)	5m 41s
	34m 47s

Provide feedback