[links] stabilize latency calculation when nodes are not responsive
The following scenario is more of a corner case than normal, but
this change allows to better deal with this situation:
- 2 nodes cluster (corosync) (node A and node B)
- kill -stop $(pidof corosync) on node A
- node B will continue to send ping packets to node A
- node A is accumulating those ping packets in the kernel network socket
- wait some seconds and unpause node A
- node A will start processing the ping packets in the queue and send pong replies to node B
- node B will see an extreme increase of latency due those "obsoleted" ping/pong packets
- node B, as latency increases, will take longer and longer to notice that node A is down due to the pong_timeout adjustment for latency (required for initial cluster spike).
the solution:
- Use average latency to calculate pong_timeout_adj vs latency_max. Averate latency will go down again in time, while latency_max is never reset.
- RX thread will filter out all pong packets that have higher latency than currently configure pong_timeout. This barrier should have been in place even before.
this solution reduces the latency spike on node B to a perfectly
reasonable level and it will all eventually stabilize over time
as latency samples increase and latency will reduce.
Please be aware that using a pong_timeout smaller than latency will
simply mark the link down now.
Signed-off-by: Fabio M. Di Nitto <fdinitto@redhat.com>