GitHub Issue… Keycloak High Availability with PostgreSQL on Kubernetes · Issue #26550 · keycloak/keycloak · GitHub
Describe the bug
Keycloak (and Postgres) will deploy without issue and become ready, but when accessing the user interface, it renders incorrectly, and depending on which pod of keycloak you hit, it will render differently… I can’t determine or figure out if the issue is within infinispan or postgres or something else…
I have tried almost every combination of deployment options and configurations (Embedded Database, Postgres Database, DNG PING, KUBE PING, JDBC…) over the last 3 days and nothing will work!! In the reproduction section, I have the deployment manifests. In the actual behavior section, I have the logs of each related pod.
If I only deploy one instance keycloak, with or without an external database, there are no issues. Additionally, I realize not everything in the manifests are required, but I have been doing a lot of trial and error.
Version
Keycloak - Version 23.0.4 (Quay)
Expected behavior
Ability to deploy three instance of Keycloak and have functionality! 
Actual behavior
Screenshots
Keycloak Pod Logs
keycloak-785d6cd755-79vw9_keycloak.log
keycloak-785d6cd755-bjtlr_keycloak.log
keycloak-785d6cd755-vwxhh_keycloak.log
Postgres Pod Logs
postgresql-0_postgresql.log
postgresql-1_postgresql.log
postgresql-2_postgresql.log
How to Reproduce?
Keycloak Manifests
apiVersion: apps/v1
kind: Deployment
metadata:
name: keycloak
namespace: keycloak-system
labels:
app: keycloak
spec:
replicas: 3
selector:
matchLabels:
app: keycloak
template:
metadata:
labels:
app: keycloak
spec:
serviceAccount: keycloak-kubeping
serviceAccountName: keycloak-kubeping
containers:
- name: keycloak
image: quay.io/keycloak/keycloak
args: ["start"]
env:
- name: KC_DB
value: "postgres"
- name: KC_DB_URL
value: "jdbc:postgresql://postgresql-service:5432/keycloak"
- name: POSTGRES_DB
valueFrom:
secretKeyRef:
name: postgresql-credentials
key: POSTGRES_DB
- name: KC_DB_USERNAME
valueFrom:
secretKeyRef:
name: postgresql-credentials
key: POSTGRES_USER
- name: KC_DB_PASSWORD
valueFrom:
secretKeyRef:
name: postgresql-credentials
key: POSTGRES_PASSWORD
- name: KEYCLOAK_ADMIN
valueFrom:
secretKeyRef:
name: keycloak-credentials
key: KEYCLOAK_ADMIN
- name: KEYCLOAK_ADMIN_PASSWORD
valueFrom:
secretKeyRef:
name: keycloak-credentials
key: KEYCLOAK_ADMIN_PASSWORD
- name: KC_HOSTNAME
value: "<redacted>"
- name: KC_PRODUCTION
value: "true"
- name: KC_CACHE
value: "ispn"
- name: KC_CACHE_STACK
value: "kubernetes"
- name: jgroups.dns.query
value: "keycloak-headless-service.keycloak-system.svc.cluster.local"
- name: CACHE_OWNERS_COUNT
value: "3"
- name: CACHE_OWNERS_AUTH_SESSIONS_COUNT
value: "3"
- name: PROXY_ADDRESS_FORWARDING
value: "true"
- name: KC_PROXY
value: "edge"
- name: KC_HEALTH_ENABLED
value: "true"
- name: KC_METRICS_ENABLED
value: "true"
- name: KC_HOSTNAME_STRICT
value: "false"
- name: KC_HOSTNAME_STRICT_BACKCHANNEL
value: "false"
- name: KC_HOSTNAME_STRICT_HTTPS
value: "false"
- name: KC_HTTP_ENABLED
value: "true"
ports:
- name: http
containerPort: 8080
protocol: TCP
- name: https
containerPort: 8443
protocol: TCP
- name: infinispan
containerPort: 7800
protocol: UDP
---
apiVersion: v1
kind: Service
metadata:
name: keycloak
namespace: keycloak-system
labels:
app: keycloak
spec:
ports:
- name: http
port: 8080
targetPort: 8080
protocol: TCP
- name: infinispan
port: 7800
targetPort: 7800
protocol: UDP
selector:
app: keycloak
type: ClusterIP
---
apiVersion: v1
kind: Service
metadata:
name: keycloak-headless-service
namespace: keycloak-system
labels:
app: keycloak
spec:
selector:
app: keycloak
clusterIP: None
publishNotReadyAddresses: true
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
labels:
app: keycloak
name: keycloak
namespace: keycloak-system
spec:
rules:
- host: <redacted>
http:
paths:
- backend:
service:
name: keycloak
port:
number: 8080
path: /
pathType: ImplementationSpecific
tls:
- hosts:
- <redacted>
secretName: tls-certs
---
apiVersion: v1
kind: Secret
metadata:
name: keycloak-credentials
namespace: keycloak-system
type: Opaque
stringData:
KEYCLOAK_ADMIN: admin
KEYCLOAK_ADMIN_PASSWORD: <redacted
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: keycloak-kubeping
namespace: keycloak-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: keycloak-kubeping-role
namespace: keycloak-system
rules:
- apiGroups: ["*"]
resources: ["pods"]
verbs: ["get", "list", "watch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
annotations:
name: keycloak
name: keycloak-role-binding
namespace: keycloak-system
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: keycloak-kubeping-role
subjects:
- kind: ServiceAccount
name: keycloak-kubeping
namespace: keycloak-system
---
apiVersion: ui.cattle.io/v1
kind: NavLink
metadata:
name: keycloak
spec:
label: Keycloak
target: _blank
description: Keycloak Authenication
group: Keycloak Authenication
label: Keycloak
sideLabel: KEYCLOAK
toURL: <redacted>
Postgres Manifests
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: postgresql
namespace: keycloak-system
spec:
serviceName: postgresql-service
replicas: 3
selector:
matchLabels:
app: postgresql
template:
metadata:
labels:
app: postgresql
spec:
containers:
- name: postgresql
image: postgres:latest
env:
- name: POSTGRES_DB
valueFrom:
secretKeyRef:
name: postgresql-credentials
key: POSTGRES_DB
- name: POSTGRES_USER
valueFrom:
secretKeyRef:
name: postgresql-credentials
key: POSTGRES_USER
- name: POSTGRES_PASSWORD
valueFrom:
secretKeyRef:
name: postgresql-credentials
key: POSTGRES_PASSWORD
- name: PGDATA
value: /var/lib/postgresql/data/pgdata
ports:
- containerPort: 5432
volumeMounts:
- name: postgresql-data
mountPath: /var/lib/postgresql/data
volumeClaimTemplates:
- metadata:
name: postgresql-data
spec:
accessModes: ["ReadWriteOnce"]
resources:
requests:
storage: 10Gi
---
apiVersion: v1
kind: Service
metadata:
name: postgresql-service
namespace: keycloak-system
spec:
selector:
app: postgresql
ports:
- port: 5432
targetPort: 5432
type: ClusterIP
---
apiVersion: v1
kind: Secret
metadata:
name: postgresql-credentials
namespace: keycloak-system
type: Opaque
stringData:
POSTGRES_DB: keycloak
POSTGRES_USER: postgres
POSTGRES_PASSWORD: <redacted>
Anything else?
No response