resolver/dns: add RefreshInterval parameter to enable periodic DNS record refresh

[r9deyes]

This pull request introduces a new feature to the DNS resolver by adding a RefreshInterval variable, which defines the rate at which the resolver refreshes its DNS records. The default value is set to 0, disabling the refresh unless configured otherwise.

Key changes include:

• New RefreshInterval Variable: Allows for periodic DNS record refreshes, with a requirement that the value must be at least MinResolutionInterval.
• Updated watcher Function: Incorporates a new channel (refreshCh) to handle refresh signals, enabling automatic updates of DNS records alongside resolution requests.
• Setter Function: A new SetRefreshInterval function is added to allow users to configure the refresh interval while ensuring it meets the minimum requirement.

These enhancements improve the DNS resolver's responsiveness and accuracy in dynamic environments. Notably, similar functionality is available in the gRPC implementation for ASP.NET Core, which also features DNS address caching and refresh mechanisms (see ASP.NET Core gRPC Load Balancing Documentation). This alignment with existing practices enhances consistency across platforms.

RELEASE NOTES:

• resolver/dns: add RefreshInterval parameter to enable periodic DNS record

[linux-foundation-easycla]

• ✅login: r9deyes / (67f32d3)

The committers listed above are authorized under a signed CLA.

[codecov]

Codecov Report

Attention: Patch coverage is 14.28571% with 6 lines in your changes missing coverage. Please review.

Project coverage is 82.02%. Comparing base (4b5505d) to head (67f32d3).
Report is 7 commits behind head on master.

Files with missing lines	Patch %	Lines
internal/resolver/dns/dns_resolver.go	20.00%	3 Missing and 1 partial ⚠️
resolver/dns/dns_resolver.go	0.00%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #8230      +/-   ##
==========================================
+ Coverage   81.98%   82.02%   +0.04%     
==========================================
  Files         410      410              
  Lines       40382    40387       +5     
==========================================
+ Hits        33107    33128      +21     
+ Misses       5895     5889       -6     
+ Partials     1380     1370      -10     

Files with missing lines	Coverage Δ
resolver/dns/dns_resolver.go	50.00% <0.00%> (-16.67%)	⬇️
internal/resolver/dns/dns_resolver.go	86.98% <20.00%> (-1.36%)	⬇️

... and 23 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[arjan-bal]

The scenario when new k8s pods are added and the LB policy uses all the addresses (e.g round robin), refreshing the resolved addresses could be useful. I checked the DNS resolver in gRPC Java, but didn't find the ability to do a periodic refresh. Checking with maintainers of other languages if we need consistent behaviour for this.

[arjan-bal]

Hi @r9deyes, after discussing this with other maintainers, I found that not configuring a client side refresh interval intentional. We don't want servers to rely on hard-coded client behavior for their load balancing stability. Updating clients is very slow. There is a discussion about a similar request here: grpc/grpc-java#2514

The general suggestion is to use MAX_CONNECTION_AGE on server-side to force the client to re-resolve. This is the approach of internet-scale load balancing, though it may be less efficient.

serverOpts = append(serverOpts, grpc.KeepaliveParams(keepalive.ServerParameters{
	MaxConnectionAge: 5 * time.Minute,
}))

I'll be closing this PR, but will re-open if there are any questions.