ahus1 · Feb 16, 2024 · Jan 15, 2024 · Jan 30, 2024 · Feb 8, 2024 · Feb 8, 2024
@@ -167,6 +167,13 @@ When enabling metrics for {project_name}'s embedded caches, the metrics now use
 For more details, check the
 link:{upgradingguide_link}[{upgradingguide_name}].
 
+= User attribute value length extension
+
+As of this release, {project_name} supports storing and searching by user attribute values longer than 255 characters, which was previously a limitation.
+
+For more details, check the
+link:{upgradingguide_link}[{upgradingguide_name}].
+
 = Authorization Policy
 
 In previous versions of Keycloak when the last member of a User, Group or Client policy was deleted then that policy would also be deleted. Unfortunately this could lead to an escalation of privileges if the policy was used in an aggregate policy. To avoid privilege escalation the effect policies are no longer deleted and an administrator will need to update those policies.

@@ -67,6 +67,7 @@ include::topics/threat/host.adoc[]
 include::topics/threat/admin.adoc[]
 include::topics/threat/brute-force.adoc[]
 include::topics/threat/read-only-attributes.adoc[]
+include::topics/threat/validate-user-attributes.adoc[]
 include::topics/threat/clickjacking.adoc[]
 include::topics/threat/ssl.adoc[]
 include::topics/threat/csrf.adoc[]

@@ -0,0 +1,10 @@
+[[validate_user_attributes]]
+=== Validate user attributes
+
+With the functionality in <<user-profile>>, administrators can restrict the data users enter for attributes, for example, in user registration or the account console.
+
+Administrators should not allow unmanaged attributes for users to prevent attackers adding an unlimited number of attributes.
+Attributes should have a validation that restricts the amount of data entered by attackers.
+
+When using regular expressions to validate user attributes, avoid regular expressions that use an excessive amount of memory or CPU.
+See https://owasp.org/www-community/attacks/Regular_expression_Denial_of_Service_-_ReDoS[OWASP's Regular expression Denial of Service] for details.
@@ -102,7 +102,7 @@ You can choose to completely disable or only support unmanaged attributes when m
 When unmanaged attributes are enabled (even if partially) you can manage them from the administration console at the `Attributes` tab in the User Details UI.
 If the policy is set to `Disabled` this tab is not available.
 
-As a recommendation, try to adhere to the most strict policy as much as possible (e.g.: `Disabled` or `Admin can edit`) to prevent unexpected
+As a security recommendation, try to adhere to the most strict policy as much as possible (e.g.: `Disabled` or `Admin can edit`) to prevent unexpected
 attributes (and values) set to your users when they are managing their profile through end-user contexts.
 Avoid setting the `Enabled` policy and prefer defining all the attributes that end-users can manage in your user profile configuration, under your control.
 
@@ -114,6 +114,9 @@ behavior when using custom themes and extending the server with their own custom
 
 As you will see in the following sections, you can also restrict the audience for an attribute by choosing if it should be visible or writable by users and/or administrators.
 
+For unmanaged attributes, the maximum length is 2048 characters.
+To specify a different minimum or maximum length, change the unmanaged attribute to a managed attribute and add a `length` validator.
+
 == Managing the User Profile
 
 The user profile configuration is managed on a per-realm basis. For that, click on the
@@ -198,6 +201,9 @@ image:images/user-profile-validation.png[]
 Validation happens at any time when writing to an attribute, and they can throw errors that will be shown in UIs when the value
 fails a validation.
 
+For security reasons, every attribute that is editable by users should have a validation to restrict the size of the values users enter.
+If no `length` validator has been specified, {project_name} defaults to a maximum length of 2048 characters.
+
 === Built-in Validators
 
 {project_name} provides some built-in validators that you can choose from, and you are also able to provide

@@ -221,6 +221,47 @@ To revert the change for an installation, use a custom Infinispan XML configurat
 <metrics names-as-tags="false" />
 ----
 
+= User attribute value length extension
+
+As of this release, {project_name} supports storing and searching by user attribute values longer than 255 characters, which was previously a limitation.
+
+In setups where users are allowed to update attributes, for example, via the account console, prevent denial of service attacks by adding validations.
+Ensure that no unmanaged attributes are allowed and all editable attributes have a validation that limits the input length.
+
+For unmanaged attributes, the maximum length is 2048 characters.
+For managed attributes, the default maximum length is 2048 characters. Administrator can change this by adding a validator of type `length`.
+
+This change adds new indexes on the tables `USER_ATTRIBUTE` and `FED_USER_ATTRIBUTE`.
+If those tables contain more than 300000 entries, Keycloak will skip the index creation by default during the automatic schema migration and instead log the SQL statement on the console during migration to be applied manually after {project_name}'s startup.
+See the link:{upgradingguide_link}[{upgradingguide_name}] for details on how to configure a different limit.
+
+== Additional migration steps for LDAP
+
+This is for installations that match all the following criteria:
+
+* User attributes in the LDAP directory are larger than 2048 characters or binary attributes that are larger than 1500 bytes.
+* The attributes are changed by admins or users via the admin console, the APIs or the account console.
+
+To be able to enable changing those attributes via UI and REST APIs, perform the following steps:
+
+. Declare the attributes identified above as managed attributes in the user profile of the realm.
+. Define a `length` validator for each attribute added in the previous step specifying the desired minimum and maximum length of the attribute value.
+For binary values, add 33% to the expected binary length to count in the overhead for {project_name}'s internal base64 encoding of binary values.
+
+== Additional migration steps for custom user storage providers
+
+This is for installations that match all the following criteria:
+
+* Running MariaDB or MySQL as a database for {project_name}.
+* Entries in table `FED_USER_ATTRIBUTE` with contents in the `VALUE` column that are larger than 2048 characters.
+This table is used for custom user providers which have federation enabled.
+* The long attributes are changed by admins or users via the admin console or the account console.
+
+To be able to enable changing those attributes via UI and REST APIs,  perform the following steps:
+
+. Declare the attributes identified above as managed attributes in the user profile of the realm.
+. Define a `length` validator for each attribute added in the previous step specifying the desired minimum and maximum length of the attribute value.
+
 = The Admin send-verify-email API now uses the same email verification template
 
 ----

@@ -30,6 +30,10 @@
     <description/>
 
     <properties>
+        <maven.compiler.release>17</maven.compiler.release>
+        <maven.compiler.source>17</maven.compiler.source>
+        <maven.compiler.target>17</maven.compiler.target>
+
         <keycloak.connectionsJpa.driver>org.h2.Driver</keycloak.connectionsJpa.driver>
         <keycloak.connectionsJpa.database>keycloak</keycloak.connectionsJpa.database>
         <keycloak.connectionsJpa.user>sa</keycloak.connectionsJpa.user>

@@ -0,0 +1,72 @@
+/*
+ * Copyright 2018 Red Hat, Inc. and/or its affiliates
+ * and other contributors as indicated by the @author tags.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.keycloak.connections.jpa.updater.liquibase.custom;
+
+import liquibase.database.core.MySQLDatabase;
+import liquibase.exception.CustomChangeException;
+import liquibase.statement.core.RawParameterizedSqlStatement;
+import org.keycloak.storage.jpa.JpaHashUtils;
+
+import java.sql.PreparedStatement;
+import java.sql.ResultSet;
+
+/**
+ * The MySQL database is the only database where columns longer than 255 characters are changed to a TEXT column, allowing
+ * for up to 64k characters. See {@link org.keycloak.connections.jpa.updater.liquibase.MySQL8VarcharType} for the implementation.
+ * As the new code expects all information longer than 2024 characters in the new column, this migration copies over the values.
+ *
+ * @author Alexander Schwartz
+ */
+public class FederatedUserAttributeTextColumnMigration extends CustomKeycloakTask {
+
+    @Override
+    protected void generateStatementsImpl() throws CustomChangeException {
+
+        if (database instanceof MySQLDatabase) {
+
+            try (PreparedStatement ps = connection.prepareStatement("SELECT t.ID, t.VALUE" +
+                    "  FROM " + getTableName("FED_USER_ATTRIBUTE") + " t" +
+                    "  WHERE LENGTH(t.VALUE) > 2024");
+                 ResultSet resultSet = ps.executeQuery()
+            ) {
+                while (resultSet.next()) {
+                    String id = resultSet.getString(1);
+                    String value = resultSet.getString(2);
+                    // The SQL LENGTH() will count bytes, where Java's length() will count Unicode characters.
+                    // There's also SQL CHAR_LENGTH() which is probably equivalent to Java's Character.codePointCount(),
+                    // but it is not a fit here as we're not using code points in the JPA entities
+                    if (value.length() > 2024) {
+                        statements.add(new RawParameterizedSqlStatement("UPDATE " + getTableName("FED_USER_ATTRIBUTE") + " SET VALUE = null, LONG_VALUE_HASH = ?, LONG_VALUE_HASH_LOWER_CASE = ?, LONG_VALUE = ? WHERE ID = ?",
+                                JpaHashUtils.hashForAttributeValue(value),
+                                JpaHashUtils.hashForAttributeValueLowerCase(value),
+                                value,
+                                id));
+                    }
+                }
+            } catch (Exception e) {
+                throw new CustomChangeException(getTaskId() + ": Exception when updating data from previous version", e);
+            }
+        }
+
+    }
+
+    @Override
+    protected String getTaskId() {
+        return "Leave only single offline session per user and client";
+    }
+
+}
@@ -61,18 +61,22 @@
 import jakarta.persistence.criteria.Predicate;
 import jakarta.persistence.criteria.Root;
 import jakarta.persistence.criteria.Subquery;
+import org.keycloak.storage.jpa.JpaHashUtils;
+
 import java.util.ArrayList;
 import java.util.Collection;
 import java.util.Comparator;
 import java.util.HashMap;
 import java.util.HashSet;
 import java.util.List;
+import java.util.Locale;
 import java.util.Map;
 import java.util.Objects;
 import java.util.Set;
 import java.util.stream.Stream;
 
 import static org.keycloak.models.jpa.PaginationUtils.paginateQuery;
+import static org.keycloak.storage.jpa.JpaHashUtils.predicateForFilteringUsersByAttributes;
 import static org.keycloak.utils.StreamsUtil.closing;
 
 
@@ -668,7 +672,7 @@ public int getUsersCount(RealmModel realm, Map<String, String> params) {
         Expression<Long> count = qb.count(from);
 
         userQuery = userQuery.select(count);
-        List<Predicate> restrictions = predicates(params, from);
+        List<Predicate> restrictions = predicates(params, from, Map.of());
         restrictions.add(qb.equal(from.get("realmId"), realm.getId()));
 
         userQuery = userQuery.where(restrictions.toArray(new Predicate[0]));
@@ -691,7 +695,7 @@ public int getUsersCount(RealmModel realm, Map<String, String> params, Set<Strin
         Root<UserEntity> root = countQuery.from(UserEntity.class);
         countQuery.select(cb.count(root));
 
-        List<Predicate> restrictions = predicates(params, root);
+        List<Predicate> restrictions = predicates(params, root, Map.of());
         restrictions.add(cb.equal(root.get("realmId"), realm.getId()));
 
         groupsWithPermissionsSubquery(countQuery, groupIds, root, restrictions);
@@ -735,7 +739,8 @@ public Stream<UserModel> searchForUserStream(RealmModel realm, Map<String, Strin
         CriteriaQuery<UserEntity> queryBuilder = builder.createQuery(UserEntity.class);
         Root<UserEntity> root = queryBuilder.from(UserEntity.class);
 
-        List<Predicate> predicates = predicates(attributes, root);
+        Map<String, String> customLongValueSearchAttributes = new HashMap<>();
+        List<Predicate> predicates = predicates(attributes, root, customLongValueSearchAttributes);
 
         predicates.add(builder.equal(root.get("realmId"), realm.getId()));
 
@@ -745,24 +750,35 @@ public Stream<UserModel> searchForUserStream(RealmModel realm, Map<String, Strin
             groupsWithPermissionsSubquery(queryBuilder, userGroups, root, predicates);
         }
 
-        queryBuilder.where(predicates.toArray(new Predicate[0])).orderBy(builder.asc(root.get(UserModel.USERNAME)));
+        queryBuilder.where(predicates.toArray(Predicate[]::new)).orderBy(builder.asc(root.get(UserModel.USERNAME)));
 
         TypedQuery<UserEntity> query = em.createQuery(queryBuilder);
 
         UserProvider users = session.users();
         return closing(paginateQuery(query, firstResult, maxResults).getResultStream())
+                // following check verifies that there are no collisions with hashes
+                .filter(predicateForFilteringUsersByAttributes(customLongValueSearchAttributes, JpaHashUtils::compareSourceValueLowerCase))
                 .map(userEntity -> users.getUserById(realm, userEntity.getId()))
                 .filter(Objects::nonNull);
     }
 
     @Override
     public Stream<UserModel> searchForUserByUserAttributeStream(RealmModel realm, String attrName, String attrValue) {
-        TypedQuery<UserEntity> query = em.createNamedQuery("getRealmUsersByAttributeNameAndValue", UserEntity.class);
-        query.setParameter("name", attrName);
-        query.setParameter("value", attrValue);
-        query.setParameter("realmId", realm.getId());
-
-        return closing(query.getResultStream().map(userEntity -> new UserAdapter(session, realm, em, userEntity)));
+        boolean longAttribute = attrValue != null && attrValue.length() > 255;
+        TypedQuery<UserEntity> query = longAttribute ?
+                em.createNamedQuery("getRealmUsersByAttributeNameAndLongValue", UserEntity.class)
+                        .setParameter("realmId", realm.getId())
+                        .setParameter("name", attrName)
+                        .setParameter("longValueHash", JpaHashUtils.hashForAttributeValue(attrValue)):
+                em.createNamedQuery("getRealmUsersByAttributeNameAndValue", UserEntity.class)
+                        .setParameter("realmId", realm.getId())
+                        .setParameter("name", attrName)
+                        .setParameter("value", attrValue);
+
+        return closing(query.getResultStream()
+                // following check verifies that there are no collisions with hashes
+                .filter(longAttribute ? predicateForFilteringUsersByAttributes(Map.of(attrName, attrValue), JpaHashUtils::compareSourceValue) : u -> true)
+                .map(userEntity -> new UserAdapter(session, realm, em, userEntity)));
     }
 
     private FederatedIdentityEntity findFederatedIdentity(UserModel user, String identityProvider, LockModeType lockMode) {
@@ -929,7 +945,7 @@ private UserEntity userInEntityManagerContext(String id) {
         return em.contains(user) ? user : null;
     }
 
-    private List<Predicate> predicates(Map<String, String> attributes, Root<UserEntity> root) {
+    private List<Predicate> predicates(Map<String, String> attributes, Root<UserEntity> root, Map<String, String> customLongValueSearchAttributes) {
         CriteriaBuilder builder = em.getCriteriaBuilder();
 
         List<Predicate> predicates = new ArrayList<>();
@@ -991,10 +1007,16 @@ private List<Predicate> predicates(Map<String, String> attributes, Root<UserEnti
                 default:
                     Join<UserEntity, UserAttributeEntity> attributesJoin = root.join("attributes", JoinType.LEFT);
 
-                    attributePredicates.add(builder.and(
-                            builder.equal(builder.lower(attributesJoin.get("name")), key.toLowerCase()),
-                            builder.equal(builder.lower(attributesJoin.get("value")), value.toLowerCase())));
-
+                    if (value.length() > 255) {
+                        customLongValueSearchAttributes.put(key, value);
+                        attributePredicates.add(builder.and(
+                                builder.equal(builder.lower(attributesJoin.get("name")), key.toLowerCase()),
+                                builder.equal(attributesJoin.get("longValueHashLowerCase"), JpaHashUtils.hashForAttributeValueLowerCase(value))));
+                    } else {
+                        attributePredicates.add(builder.and(
+                                builder.equal(builder.lower(attributesJoin.get("name")), key.toLowerCase()),
+                                builder.equal(builder.lower(attributesJoin.get("value")), value.toLowerCase())));
+                    }
                     break;
                 case UserModel.INCLUDE_SERVICE_ACCOUNT: {
                     if (!attributes.containsKey(UserModel.INCLUDE_SERVICE_ACCOUNT)

@@ -30,6 +30,7 @@
 import jakarta.persistence.NamedQueries;
 import jakarta.persistence.NamedQuery;
 import jakarta.persistence.Table;
+import org.keycloak.storage.jpa.JpaHashUtils;
 
 /**
  * @author <a href="mailto:bill@burkecentral.com">Bill Burke</a>
@@ -60,6 +61,14 @@ public class UserAttributeEntity {
     @Column(name = "VALUE")
     protected String value;
 
+    @Column(name = "LONG_VALUE_HASH")
+    private byte[] longValueHash;
+    @Column(name = "LONG_VALUE_HASH_LOWER_CASE")
+    private byte[] longValueHashLowerCase;
+    @Nationalized
+    @Column(name = "LONG_VALUE")
+    private String longValue;
+
     public String getId() {
         return id;
     }
@@ -77,11 +86,29 @@ public void setName(String name) {
     }
 
     public String getValue() {
-        return value;
+        if (value != null && longValue != null) {
+            throw new IllegalStateException(String.format("User with id %s should not have set both `value` and `longValue` for attribute %s.", user.getId(), name));
+        }
+        return value != null ? value : longValue;
     }
 
     public void setValue(String value) {
-        this.value = value;
+        if (value == null) {
+            this.value = null;
+            this.longValue = null;
+            this.longValueHash = null;
+            this.longValueHashLowerCase = null;
+        } else if (value.length() > 255) {
+            this.value = null;
+            this.longValue = value;
+            this.longValueHash = JpaHashUtils.hashForAttributeValue(value);
+            this.longValueHashLowerCase = JpaHashUtils.hashForAttributeValueLowerCase(value);
+        } else {
+            this.value = value;
+            this.longValue = null;
+            this.longValueHash = null;
+            this.longValueHashLowerCase = null;
+        }
     }
 
     public UserEntity getUser() {