Java Heap Problem after long running task triggered by REST call

Getting advice
1

We need to run a user importer with is trigger nightly via REST endpoint. The REST endpoint in Keycloak then handles all the imports of nearly 15.000 users include there roles/groups/attributes. After around 7000 users we get Java Heap Problem.

We think the problem is the session object which is used throughout the whole task running.

This is the exact error:
ERROR [org.keycloak.services.error.KeycloakErrorHandler] (executor-thread-18) Uncaught server error: java.lang.OutOfMemoryError: Java heap space

WARN [io.agroal.pool] (executor-thread-18) Datasource ‘’: JDBC resources leaked: 17060 ResultSet(s) and 17060 Statement(s)

Can somehelp with how the session should be correctly handled in a long running task?

1 Like
2

I don’t know that this problem is with using a KeycloakSession. A similar thing is done (using a single session, in a single transaction), in the Keycloak code that imports users from a file. This will also potentially be many thousands of users. Take a look at how they do it, and see if your approach is different.

1 Like
3

Thank you for your answer!

One difference to my code that I noticed is that I have lots of method calls from the session object. Like the following is called multiple times (instead of saving e.g. realm information in a variable):

session.getContext().getRealm();
session.users()...
sessions.groups()...

Do you think that could make a problem?

4

Without seeing the code, and reviewing what Keycloak is doing behind those methods, I wouldn’t have any idea. I would recommend either using a debugger and/or looking at what Keycloak is doing.

1 Like
5

Update: We could not solve the problem for the long running task directly in keycloak.

So, we decided to pass smaller pieces of users from an external service to keycloak. In the external service we implemented a pagination logic for handling the large amount of users.

1 Like
6

Thanks for the update.

1 Like
7

Maybe too much is done inside one KeycloakSession. Probably JPA/Hibernate is then caching too many objects.
You could try to split the task into multiple KeycloakSessions (created via KeycloakSessionFactory).

1 Like
8

Thank you for sharing your thoughts. It was also our guess that some stuff is cached too much inside one session.

So, I already tried creating new sessions via KeycloakSessionFactory, but then the session does not include any information like session.users()

As we have some time pressure, I unfortunatelly will not dive deeper in this issue.