ISPN000569: Unable to persist Infinispan internal caches as no global state enabled

My goal is to use cache ispn and external or remote cache like IP:11222

Keycloak version is: 24.0.3

Sharing snippets:

Dockerfile:

ARG KEYCLOAK_VERSION=24.0.3

FROM quay.io/keycloak/keycloak:${KEYCLOAK_VERSION} as keycloak-builder

ENV KC_METRICS_ENABLED=true
ENV KC_FEATURES=preview
ENV KC_DB=postgres
ENV KC_HTTP_RELATIVE_PATH=/auth

ENV KC_CACHE_CONFIG_FILE=cache-ispn.xml

COPY cache-ispn.xml /opt/keycloak/conf/cache-ispn.xml

RUN /opt/keycloak/bin/kc.sh build


FROM quay.io/keycloak/keycloak:${KEYCLOAK_VERSION}

WORKDIR /opt/keycloak

# Ensuring the directory exists and setting permissions
USER root
RUN mkdir -p /opt/keycloak/data && chmod -R 777 /opt/keycloak/data
USER keycloak

RUN ls -la /opt/keycloak/data

ENTRYPOINT ["/opt/keycloak/bin/kc.sh", "start-dev"]

cache-ispn.xml:

<?xml version="1.0" encoding="UTF-8"?>
<infinispan xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
            xsi:schemaLocation="urn:infinispan:config:14.0 https://www.infinispan.org/schemas/infinispan-config-14.0.xsd"
            xmlns="urn:infinispan:config:14.0">
    <cache-container name="keycloak" default-cache="default" statistics="true">

        <!-- Enabling global state -->
        <global-state enabled="true" persistent-location="/opt/keycloak/data"/>

        <transport lock-timeout="60000">
            <jgroups xmlns="http://jgroups.org/schema/jgroups-4.2.xsd">
                <stack name="tcp">
                    <TCP bind_port="7800"
                         bind_addr="localhost" />
                </stack>
            </jgroups>
        </transport>

        <remote-store xmlns="urn:infinispan:config:store:remote:14.0"
                      cache="sessions" raw-values="true" shared="true"
                      segmented="false" preload="false" purge="false"
                      marshaller="org.keycloak.cluster.infinispan.KeycloakHotRodMarshallerFactory">
            <remote-server host="113.21.xx.1xx" port="11222"/>
            <connection-pool max-active="20" exhausted-action="WAIT"/>
            <security>
                <authentication server-name="infinispan">
                    <digest username="admin" password="password" realm="default"/>
                </authentication>
            </security>
        </remote-store>

        <jdbc-string-cache-store name="myCacheStore" passivation="false" purge="false"
                                 preload="false" shared="true">
            <property name="connectionUrl">${env.KC_DB_URL}</property>
            <property name="userName">${env.KC_DB_USERNAME}</property>
            <property name="password">${env.KC_DB_PASSWORD}</property>
            <property name="driverClass">org.postgresql.Driver</property>
        </jdbc-string-cache-store>

    </cache-container>
</infinispan>

and docker-compose.yaml:

version: '3.3'
services:
  keycloak:
    build:
      context: .
      dockerfile: Dockerfile
    environment:
      - HOSTNAME_STRICT=false
      - KC_HOSTNAME=http://localhost:8080
      - KC_DB=postgres
      - KC_DB_URL=jdbc:postgresql://host.docker.internal:5435/key_test
      - KC_DB_USERNAME=postgres
      - KC_DB_PASSWORD=
      - KC_PORT=8080
      - KEYCLOAK_ADMIN=admin
      - KEYCLOAK_ADMIN_PASSWORD=admin
      - QUARKUS_TRANSACTION_MANAGER_ENABLE_RECOVERY=true
      - QUARKUS_PROFILE=prod
      - PROXY=edge
      - KC_CACHE=ispn
      - KC_CACHE_STACK=tcp
      - JGROUPS_DISCOVERY_PROTOCOL=TCPPING # Or your choice
      - JGROUPS_DISCOVERY_PROPERTIES=initial_hosts=keycloak1[7800]
      - JBOSS_TXN_NODE_IDENTIFIER=1
      - JBOSS_TXN_USER_TRANSACTION_OBJECT_STORE_DIR=/opt/keycloak/data/transaction-logs
    volumes:
      - ./data:/opt/keycloak/data
    ports:
      - "8080:8080"

console logs are:

keycloak-1  | 2024-05-08 06:39:17,328 INFO  [io.qua.dep.QuarkusAugmentor] (main) Quarkus augmentation completed in 4650ms
keycloak-1  | 2024-05-08 06:39:18,146 INFO  [org.keycloak.quarkus.runtime.hostname.DefaultHostnameProvider] (main) Hostname settings: Base URL: <unset>, Hostname: http://localhost:8080, Strict HTTPS: false, Path: <request>, Strict BackChannel: false, Admin URL: <unset>, Admin: <request>, Port: -1, Proxied: false
keycloak-1  | 2024-05-08 06:39:18,398 INFO  [org.infinispan.CONTAINER] (keycloak-cache-init) ISPN000556: Starting user marshaller 'org.infinispan.jboss.marshalling.core.JBossUserMarshaller'
keycloak-1  | 2024-05-08 06:39:18,685 INFO  [org.infinispan.CLUSTER] (keycloak-cache-init) ISPN000078: Starting JGroups channel `ISPN` with stack `tcp`
keycloak-1  | 2024-05-08 06:39:18,688 INFO  [org.jgroups.JChannel] (keycloak-cache-init) local_addr: 2eb0a91c-7022-4a70-a373-dad99b761b4f, name: 71b6c783ecea-8793
keycloak-1  | 2024-05-08 06:39:18,702 INFO  [org.jgroups.protocols.FD_SOCK2] (keycloak-cache-init) server listening on *.57800
keycloak-1  | 2024-05-08 06:39:20,142 INFO  [org.keycloak.broker.provider.AbstractIdentityProviderMapper] (main) Registering class org.keycloak.broker.provider.mappersync.ConfigSyncEventListener
keycloak-1  | 2024-05-08 06:39:20,709 INFO  [org.jgroups.protocols.pbcast.GMS] (keycloak-cache-init) 71b6c783ecea-8793: no members discovered after 2003 ms: creating cluster as coordinator
keycloak-1  | 2024-05-08 06:39:20,728 INFO  [org.infinispan.CLUSTER] (keycloak-cache-init) ISPN000094: Received new cluster view for channel ISPN: [71b6c783ecea-8793|0] (1) [71b6c783ecea-8793]
keycloak-1  | 2024-05-08 06:39:20,788 INFO  [org.infinispan.CLUSTER] (keycloak-cache-init) ISPN000079: Channel `ISPN` local address is `71b6c783ecea-8793`, physical addresses are `[172.29.0.2:7800]`
keycloak-1  | 2024-05-08 06:39:20,795 WARN  [org.infinispan.CONFIG] (keycloak-cache-init) ISPN000569: Unable to persist Infinispan internal caches as no global state enabled
keycloak-1  | 2024-05-08 06:39:21,090 INFO  [org.keycloak.connections.infinispan.DefaultInfinispanConnectionProviderFactory] (main) Node name: 71b6c783ecea-8793, Site name: null
keycloak-1  | 2024-05-08 06:39:21,677 INFO  [io.quarkus] (main) Keycloak 24.0.3 on JVM (powered by Quarkus 3.8.3) started in 4.243s. Listening on: http://0.0.0.0:8080
keycloak-1  | 2024-05-08 06:39:21,678 INFO  [io.quarkus] (main) Profile dev activated. 
keycloak-1  | 2024-05-08 06:39:21,678 INFO  [io.quarkus] (main) Installed features: [agroal, cdi, hibernate-orm, jdbc-postgresql, keycloak, logging-gelf, narayana-jta, reactive-routes, resteasy-reactive, resteasy-reactive-jackson, smallrye-context-propagation, vertx]
keycloak-1  | 2024-05-08 06:39:21,721 WARN  [org.keycloak.quarkus.runtime.KeycloakMain] (main) Running the server in development mode. DO NOT use this configuration in production.

Issue is:

  1. can not connect cache server through IP and port
  2. Unable to persist Infinispan internal caches as no global state enabled can not enable this one.