Keycloak Deployment Failing After Small Changes - Readiness Probe Failing

Getting advice
1

Hi everyone,

I’m encountering an issue with my Keycloak deployment in production. It was running perfectly fine until I made a small change to the deployment file, and now it’s no longer working.

Here’s my deployment file:

apiVersion: v1
kind: Service
metadata:
  name: keycloak
spec:
  ports:
    - protocol: TCP
      port: 80
      targetPort: 8080
  selector:
    app: keycloak
  type: ClusterIP
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: keycloak
  labels:
    app: keycloak
spec:
  replicas: 1
  selector:
    matchLabels:
      app: keycloak
  template:
    metadata:
      labels:
        app: keycloak
    spec:
      containers:
        - name: keycloak
          image: custom-keycloak:1.0.0
          imagePullPolicy: IfNotPresent
          args: ["start"]
          env:
            - name: KEYCLOAK_ADMIN
              valueFrom:
                secretKeyRef:
                  name: custom-keycloak-secret
                  key: KEYCLOAK_ADMIN
            - name: KEYCLOAK_ADMIN_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: custom-keycloak-secret
                  key: KEYCLOAK_ADMIN_PASSWORD
            - name: KC_DB
              valueFrom:
                secretKeyRef:
                  name: custom-keycloak-secret
                  key: KC_DB
            - name: KC_DB_URL_HOST
              valueFrom:
                secretKeyRef:
                  name: custom-keycloak-secret
                  key: KC_DB_URL_HOST
            - name: KC_DB_USERNAME
              valueFrom:
                secretKeyRef:
                  name: postgres-secret
                  key: POSTGRES_USER
            - name: KC_DB_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: postgres-secret
                  key: POSTGRES_PASSWORD
            - name: KC_HOSTNAME
              valueFrom:
                secretKeyRef:
                  name: custom-keycloak-secret
                  key: KC_HOSTNAME
            - name: KC_PROXY
              value: edge
            - name: KC_DB_SCHEMA
              valueFrom:
                secretKeyRef:
                  name: custom-keycloak-secret
                  key: KC_DB_SCHEMA
          ports:
            - protocol: TCP
              containerPort: 8080
          readinessProbe:
            httpGet:
              path: /realms/master
              port: 8080

Even though this is a custom Keycloak image, it was working fine before. I don’t believe the custom image is the issue.

However, now I am seeing this in the logs:

SLF4J: The requested version 2.0.6 by your slf4j binding is not compatible with [1.6, 1.7]
SLF4J: See http://www.slf4j.org/codes.html#version_mismatch for further details.
2024-09-11 11:50:22,133 INFO  [org.keycloak.quarkus.runtime.hostname.DefaultHostnameProvider] (main) Hostname settings: Base URL: <unset>, Hostname: my-domain.com, Strict HTTPS: true
2024-09-11 11:50:23,817 WARN  [io.quarkus.agroal.runtime.DataSources] (main) Datasource <default> enables XA but transaction recovery is not enabled. Please enable transaction recovery by setting quarkus.transaction-manager.enable-recovery=true, otherwise data may be lost if the application is terminated abruptly
2024-09-11 11:50:24,228 INFO  [org.infinispan.CONTAINER] (keycloak-cache-init) ISPN000556: Starting user marshaller 'org.infinispan.jboss.marshalling.core.JBossUserMarshaller'
2024-09-11 11:50:24,435 WARN  [org.jgroups.protocols.UDP] (keycloak-cache-init) JGRP000015: the receive buffer of socket MulticastSocket was set to 25MB, but the OS only allocated 12.58MB
2024-09-11 11:50:25,140 INFO  [org.keycloak.broker.provider.AbstractIdentityProviderMapper] (main) Registering class org.keycloak.broker.provider.mappersync.ConfigSyncEventListener
2024-09-11 11:50:26,447 INFO  [org.jgroups.protocols.pbcast.GMS] (keycloak-cache-init) keycloak-6f9fc99756-5vf4h-52730: no members discovered after 2002 ms: creating cluster as coordinator
2024-09-11 11:50:26,472 INFO  [org.infinispan.CLUSTER] (keycloak-cache-init) ISPN000079: Channel `ISPN` local address is `keycloak-6f9fc99756-5vf4h-52730`, physical addresses are `[xx.xxx.xxx.x:xxxxx]`

(I know that I still have some error I have to corrige in order to make it following the good practices).
And when I describe the pod, I get the following error:

Warning  Unhealthy  4s (x37 over 5m13s)  kubelet            Readiness probe failed: Get "http://xx.xxx.xxx.x:8080/realms/master": dial tcp xx.xxx.xxx.x:8080: connect: connection refused

Has anyone encountered something like this before? Any idea what might be causing the problem or where I should look to troubleshoot?

Thanks in advance!