Context
Extended metrics support provided in Aerospike C library 6.6.0. The latency bucket is showing cumulative value instead of instantaneous value within a configured interval.Method
First, the default interval is 20 seconds,latency_columns is default 7 and latency_shift is default 1.
// latency_columns=7 latency_shift=1 <=1ms >1ms >2ms >4ms >8ms >16ms >32ms
Here is an example of the Client extended metrics log from one of the nodes within one configured interval :
2024-04-02 11:32:35 header(1) cluster[name,cpu,mem,invalidNodeCount,tranCount,retryCount,delayQueueTimeoutCount,eventloop[],node[]] eventloop[processSize,queueSize] node[name,address,port,syncConn,asyncConn,errors,timeouts,latency[]] conn[inUse,inPool,opened,closed] latency(7,1)[type[l1,l2,l3...]] 2024-04-02 11:41:54 cluster[,118,1733540,0,28036091,0,0,[],[XXXXXXXXX,2405:200:1413:100::5:10,4000,6,11,22,4,0,0,0,0,0,0,[conn[21,0,0,0,0,0,0],write[2635632,143,100,3,0,3,16367],read[6660438,72,19,1,0,0,0],batch[0,0,0,0,0,0,0],query[0,0,0,0,0,0,0]]]]] 2024-04-02 11:42:14 cluster[,118,1733540,0,29037846,0,0,[],[XXXXXXXXX,2405:200:1413:100::5:10,4000,8,10,23,4,0,0,0,0,0,0,[conn[22,0,0,0,0,0,0],write[2729831,150,109,3,0,3,16939],read[6898430,74,19,1,0,0,0],batch[0,0,0,0,0,0,0],query[0,0,0,0,0,0,0]]]]]
The first line of client metrics explains what the fields are in the metric. For example, latency(7,1) [type[l1,l2,l3...]].
The client metrics log captured the client-side latency. If we calculate the ops/sec and latencies, the calculation should be as below. We will take client write as an example.
client: ((2729831+150+109+3+0+3+16939) - (2635632+143+100+3+0+3+16367))/20 = 4739.35 ops/sec l1 bucket write operations completed within 1ms (2729831 - 2635632)/20 = 4709.95 l2 bucket write operations completed within 1 ~ 2ms (150-143)/20 = 0.35 l7 bucket write operations completed within 16 ~ 32ms (16939 - 16367)/20 = 28.6
Because the client log captures the client-side latency, it is expected behavior that it would always be a little bit higher than the server latency.