Sporadic latency spikes on Keycloak API calls

Hi,

We have a Geo-redundant, high available Keycloak setup across two data centres in k8s clusters with MariaDB as backend. We are observing a latency spikes (> 10 seconds) 2-3 times a day on Keycloak’s health check API. These latency spikes are sporadic, with no particular pattern and we observed no memory, CPU, network IO and disk IO spikes. There isn’t any traffic spike when we observe these latency spikes.

We took memory stats with jstat/jmap and they seems to be ok. Thread dumps didn’t show any signs of deadlocks or long blocking threads.

What could be the issue for these latency spikes? What other directions I could look into to investigate to root cause the issue.

Keycloak version: 19.0.3

Thanks,