feat: refactor peerManager for lightProtocols

[chaitanyaprem]

Background

Some tasks i can think of. Need to analyze more on this.

• No need to take relayPeerConfig if relay is not enabled. feat: modify peer-manager to consider relay target peers  #1135
• Report peer or connectivity health on topictopicHealthStatusChan which is already done in case of relay based on logic explained here.
• Need to maintain connectivity and available peersList for Filter and LightPush based on subscriptions.
  - [ ] Peer Scoring and bad peers. Options for upper layers to indicate the scoring based on which peers are selected/maintained ?

TBD

[chaitanyaprem]

For topicHealth of lightClients following should be the approach:

1. For each pubsubTopic/shard health should be reported separately.
2. Health should be a combination of active Filter-subscriptions and lightpush peers connected. If every Filter subscription (from application) has atleast 1 corresponding subscription to a FilterServiceNode/peer - we can consider that sufficiently healthy based on number of filters subscribed by user/application and
3. Here is how health status can be identified/reported:

• UnHealthy

• If there are no active filter subscriptions towards peers at all or no lightpush peers connected/available

• Partially healthy

• If there is no redundancy for filter subscriptions (i.e any of the filters have less than 2 subscriptions) are active out of all application subscriptions or only 1 lightpush peer is available

• Sufficiently Healthy

• all filters have 2 active subscriptions 2 or more lightpush peer is available/connected

Note: As of now, i am not considering case of multiple peer subscriptions since we have regular store queries in status-go that take care of any missed messages. But ideally that should also be taken into consideration while determining health status of Filter.

cc @weboko @richard-ramos @jm-clius @fryorcraken @danisharora099 Please provide comments for above thoughts regarding topic/shard health for lightClients.

[jm-clius]

Thanks for this - from Waku's perspective I see health more as a binary healthy/unhealthy. A single subscription is the combination of all content topics subscribed to by the filter client. If any part of the filter criteria within this subscription (i.e. one or more content topics) fail to receive messages, the subscription is a failure for Waku - for all we know critical application logic is spread across multiple content topics. Of course, an application may happily continue functioning with some missing messages, but in terms of defensive Waku-level logic I see the subscription simply as unhealthy.

[chaitanyaprem]

Thanks for this - from Waku's perspective I see health more as a binary healthy/unhealthy. A single subscription is the combination of all content topics subscribed to by the filter client. If any part of the filter criteria within this subscription (i.e. one or more content topics) fail to receive messages, the subscription is a failure for Waku - for all we know critical application logic is spread across multiple content topics. Of course, an application may happily continue functioning with some missing messages, but in terms of defensive Waku-level logic I see the subscription simply as unhealthy.

OK. But in case of status we normally end up with more than 1 subscription per client since there are a lot of content-topics as of now in use. So, in that case if we at Waku layer identify failed subscription we can mark that pubsubTopic as unhealthy eventhough other subscriptions are still active for it?

Note that there is a possibility that different Filter peers are used for different contentTopics as of now due to the fact that close to 200 content-topics are being subscribed to. They get batched into different subscriptions and subscribed to different peers (based on random selection)

[jm-clius]

Right! I apologise - I read this as a summary of the health of a subscription and not as a report on pubsub topic health. Indeed, I think partially healthy makes sense if speaking about the health of a pubsub topic if only subset of subscriptions on that pubsub topic is healthy. Disregard my previous comment. :)

[fryorcraken]

How is the app supposed to behave when it's "partially healthy"? I think that would help drive the definition of this state.

The fact that some subscription are failing, is really not ideal, borderline unhealthy.

I would expect partially healthy to be "we don't have redundancy". rather than explicitely knowing that messages for some topic won't be received.

[chaitanyaprem]

I would expect partially healthy to be "we don't have redundancy". rather than explicitely knowing that messages for some topic won't be received.

Agreed that this sounds better in considering health of a topic.
So, if there is no active subscription for any filter then topic can be considered as unhealthy.
If there is atleast 1 active subscription for all filters, then we can consider it partially healthy.

[chaitanyaprem]

How is the app supposed to behave when it's "partially healthy"? I think that would help drive the definition of this state.

In case of status the app can just indicate the health status in case of partially healthy/unhealthy so that user is aware there would be possible issues in reliability.