Improve host_source locking and ring refresh concurrency

[nanassito]

We discovered during a load test of one of our service that there is a lot of locking going on in host_source.go. It turns out that a few can be optimized.

Here are the set of changes in this PR:

1. ringDescriber / GetHosts

The mutex surrounded all of getLocalHostInfo and getClusterPeerInfo (control-connection I/O). prevHosts / prevPartitioner were never written anywhere, so error returns were always nil / "". The mutex only serialized unrelated ring refreshes for no benefit. Removed mu, prevHosts, and prevPartitioner, and return nil, "", err on failure so concurrent GetHosts no longer block each other on network work.

2. HostInfo.ConnectAddressAndPort

It only reads fields; it used Lock and blocked writers. Switched to RLock so it matches other read-only accessors.

This is the most costly one exposed by our load test.

3. HostInfo.HostnameAndPort

Uses a read-optimized path: RLock when hostname is already set (common case); Lock only when lazily filling hostname.

4. isValidPeer

Previously mixed RPCAddress() (under lock) with direct reads of hostId, dataCenter, etc. (racy). One RLock covers all fields and avoids extra lock cycles from a separate RPCAddress call.

5. errorBroadcaster.broadcast

Stopped holding the mutex while sending on listener channels (could stall other newListener / broadcast callers). Snapshot and clear listeners under the lock, then unlock and notify.

[jon-whit]

We also have this same issue in our organization. Our application is being quite limited in its concurrency burstability because of the lock contention. Here's a sample mutex blocking profile taken from a real production runtime under high load.

[worryg0d]

Hey, I reviewed this patch - jfyi, I'm not an apache committer, so I'm just writing comments.

HostInfo methods might be on a hot path, so it is fine to narrow write-lock to read-lock where it is possible.

Having a benchmark would be useful to see the performance impact.

[worryg0d]

I just realized they are never used