8000 Support asynchronous logging by mabartos · Pull Request #38094 · keycloak/keycloak · GitHub
[go: up one dir, main page]

Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
5 changes: 5 additions & 0 deletions docs/documentation/release_notes/topics/26_3_0.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -4,4 +4,9 @@ Previously, usage of remote stores was discouraged and the experimental feature
In this release, the experimental feature was removed and usage of remote stores was restricted.
If you previously used the `cache-embedded-remote-store` feature, see the link:{upgradingguide_link}[{upgradingguide_name}] for migration steps.

= Asynchronous logging

All available log handlers now support asynchronous logging capabilities.
Asynchronous logging might be useful for deployments requiring **high throughput** and **low latency** to move log operations in a separate thread.

For more details, see the https://www.keycloak.org/server/logging[Logging guide].
51 changes: 51 additions & 0 deletions docs/guides/server/logging.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -176,6 +176,57 @@ If you want to have JSON logs in *ECS* (Elastic Common Schema) format for the co
{"@timestamp":"2025-02-03T14:53:22.539484211+01:00","event.sequence":9608,"log.logger":"io.quarkus","log.level":"INFO","message":"Keycloak 999.0.0-SNAPSHOT on JVM (powered by Quarkus 3.17.8) started in 4.615s. Listening on: http://0.0.0.0:8080","process.thread.name":"main","process.thread.id":1,"mdc":{},"ndc":"","host.hostname":"host-name","process.name":"/usr/lib/jvm/jdk-21.0.3+9/bin/java","process.pid":77561,"data_stream.type":"logs","ecs.version":"1.12.2","service.environment":"prod","service.name":"Keycloak","service.version":"999.0.0-SNAPSHOT"}
----

=== Asynchronous logging
{project_name} supports asynchronous logging, which might be useful for deployments requiring **high throughput** and **low latency**.
Asynchronous logging uses a separate thread to take care of processing all log records.
The logging handlers are invoked in exactly the same way as with synchronous logging, only done in separate threads.
You can enable asynchronous logging for all {project_name} log handlers.
A dedicated thread will be created for every log handler with enabled asynchronous logging.

The underlying mechanism for asynchronous logging uses a queue for processing log records.
Every new log record is added to the queue and then published to the particular log handler with enabled asynchronous logging.
Every log handler has a different queue.

If the queue is already full, it blocks the main thread and waits for free space in the queue.

==== When to use asynchronous logging

* You need **lower latencies** for incoming requests
* You need **higher throughput**
* You have **small worker thread pool** and want to offload logging to separate threads
* You want to reduce the impact of **I/O-heavy log handlers**
* You are logging to **remote destinations** (e.g., network syslog servers) and want to avoid blocking worker threads

WARNING: Be aware that enabling asynchronous logging might bring some **additional memory overhead** due to the additional separate thread and the inner queue.
In that c 10BC8 ase, it is not recommended to use it for resource-constrained environments.
Additionally, unexpected server shutdowns create a risk of **losing log records**.

==== Enable asynchronous logging
You can enable asynchronous logging globally for all log handlers by using `log-async` property as follows:

<@kc.start parameters="--log-async=true"/>

Or you can enable the asynchronous logging for every specific handler by using properties in the format `log-<handler>-async` (where `<handler>` is a log handler).
If the property for a specific handler is not set, the value from the parent `log-async` property is used.

You can use these properties as follows:

<@kc.start parameters="--log-console-async=true --log-file-async=true --log-syslog-async=true"/>

* `log-console-async` - Console log handler
* `log-file-async` - File log handler
* `log-syslog-async` - Syslog log handler

==== Change queue length
You can change the size of the queue used for the asynchronous logging.
The default size is **512** log records in the queue.

You can change the queue length as follows:

<@kc.start parameters="--log-console-async-queue-length=512 --log-file-async-queue-length=512 --log-syslog-async-queue-length=512"/>

These properties are available only when asynchronous logging is enabled for these specific log handlers.

== Console log handler
The console log handler is enabled by default, providing unstructured log messages for the console.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,12 @@ public String toString() {
.caseInsensitiveExpectedValues(true)
.build();

public static final Option<Boolean> LOG_ASYNC = new OptionBuilder<>("log-async", Boolean.class)
.category(OptionCategory.LOGGING)
.defaultValue(false)
.description("Indicates whether to log asynchronously to all handlers.")
.build();

public enum Output {
DEFAULT,
JSON;
Expand Down Expand Up @@ -130,6 +136,19 @@ public String toString() {
.hidden()
.build();

// Console Async
public static final Option<Boolean> LOG_CONSOLE_ASYNC = new OptionBuilder<>("log-console-async", Boolean.class)
.category(OptionCategory.LOGGING)
.defaultValue(false)
.description("Indicates whether to log asynchronously to console. If not set, value from the parent property '%s' is used.".formatted(LOG_ASYNC.getKey()))
.build();

public static final Option<Integer> LOG_CONSOLE_ASYNC_QUEUE_LENGTH = new OptionBuilder<>("log-console-async-queue-length", Integer.class)
.category(OptionCategory.LOGGING)
.defaultValue(512)
.description("The queue length to use before flushing writing when logging to console.")
.build();

// File
public static final Option<Boolean> LOG_FILE_ENABLED = new OptionBuilder<>("log-file-enabled", Boolean.class)
.category(OptionCategory.LOGGING)
Expand Down Expand Up @@ -175,6 +194,19 @@ public String toString() {
.description("Set the log output to JSON or default (plain) unstructured logging.")
.build();

// File async
public static final Option<Boolean> LOG_FILE_ASYNC = new OptionBuilder<>("log-file-async", Boolean.class)
.category(OptionCategory.LOGGING)
.defaultValue(false)
.description("Indicates whether to log asynchronously to file log. If not set, value from the parent property '%s' is used.".formatted(LOG_ASYNC.getKey()))
.build();

public static final Option<Integer> LOG_FILE_ASYNC_QUEUE_LENGTH = new OptionBuilder<>("log-file-async-queue-length", Integer.class)
.category(OptionCategory.LOGGING)
.defaultValue(512)
.description("The queue length to use before flushing writing when logging to file log.")
.build();

// Syslog
public static final Option<Boolean> LOG_SYSLOG_ENABLED = new OptionBuilder<>("log-syslog-enabled", Boolean.class)
.category(OptionCategory.LOGGING)
Expand Down Expand Up @@ -247,4 +279,16 @@ public String toString() {
.description("Set the Syslog output to JSON or default (plain) unstructured logging.")
.build();

// Syslog async
public static final Option<Boolean> LOG_SYSLOG_ASYNC = new OptionBuilder<>("log-syslog-async", Boolean.class)
.category(OptionCategory.LOGGING)
.defaultValue(false)
.description("Indicates whether to log asynchronously to Syslog. If not set, value from the parent property '%s' is used.".formatted(LOG_ASYNC.getKey()))
.build();

public static final Option<Integer> LOG_SYSLOG_ASYNC_QUEUE_LENGTH = new OptionBuilder<>("log-syslog-async-queue-length", Integer.class)
.category(OptionCategory.LOGGING)
.defaultValue(512)
.description("The queue length to use before flushing writing when logging to Syslog.")
.build();
}
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@
import static org.keycloak.quarkus.runtime.cli.OptionRenderer.decorateDuplicitOptionName;
import static org.keycloak.quarkus.runtime.cli.command.AbstractStartCommand.OPTIMIZED_BUILD_OPTION_LONG;
import static org.keycloak.quarkus.runtime.configuration.ConfigArgsConfigSource.parseConfigArgs;
import static org.keycloak.quarkus.runtime.configuration.Configuration.isUserModifiable;
import static org.keycloak.quarkus.runtime.configuration.MicroProfileConfigProvider.NS_KEYCLOAK_PREFIX;
import static org.keycloak.quarkus.runtime.configuration.mappers.PropertyMappers.maskValue;
import static picocli.CommandLine.Model.UsageMessageSpec.SECTION_KEY_COMMAND_LIST;
Expand Down Expand Up @@ -507,12 +508,6 @@ private void validateProperty(AbstractCommand abstractCommand, IncludeOptions op
mapper.getDeprecatedMetadata().ifPresent(metadata -> handleDeprecated(deprecatedInUse, mapper, configValueStr, metadata));
}

private static boolean isUserModifiable(ConfigValue configValue) {
// This could check as low as SysPropConfigSource DEFAULT_ORDINAL, which is 400
// for now we won't validate these as it's not expected for the user to specify options via system properties
return configValue.getConfigSourceOrdinal() >= KeycloakPropertiesConfigSource.PROPERTIES_FILE_ORDINAL;
}

private static void checkRuntimeSpiOptions(String key, final List<String> ignoredRunTime) {
if (!key.startsWith(PropertyMappers.KC_SPI_PREFIX)) {
return;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@
import static org.keycloak.quarkus.runtime.cli.Picocli.ARG_PREFIX;

import java.util.Map;
import java.util.Objects;
import java.util.Optional;
import java.util.Properties;

Expand All @@ -28,6 +29,7 @@
import io.smallrye.config.SmallRyeConfig;

import org.keycloak.config.Option;
import org.keycloak.quarkus.runtime.cli.Picocli;
import org.keycloak.utils.StringUtil;

import static org.keycloak.quarkus.runtime.configuration.MicroProfileConfigProvider.NS_KEYCLOAK_PREFIX;
Expand All @@ -51,6 +53,18 @@ public static boolean isTrue(Option<Boolean> option) {
return getOptionalBooleanValue(NS_KEYCLOAK_PREFIX + option.getKey()).orElse(false);
}

public static boolean isUserModifiable(ConfigValue configValue) {
// This could check as low as SysPropConfigSource DEFAULT_ORDINAL, which is 400
// for now we won't validate these as it's not expected for the user to specify options via system properties
return configValue.getConfigSourceOrdinal() >= KeycloakPropertiesConfigSource.PROPERTIES_FILE_ORDINAL;
}

public static boolean isSet(Option<?> option) {
return Optional.ofNullable(getKcConfigValue(option.getKey()))
.filter(Configuration::isUserModifiable)
.isPresent();
}

public static boolean isTrue(String propertyName) {
return getOptionalBooleanValue(propertyName).orElse(false);
}
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,10 @@
package org.keycloak.quarkus.runtime.configuration.mappers;

import static org.keycloak.config.LoggingOptions.DEFAULT_LOG_FORMAT;
import static org.keycloak.config.LoggingOptions.LOG_CONSOLE_ENABLED;
import static org.keycloak.config.LoggingOptions.LOG_FILE_ENABLED;
import static org.keycloak.config.LoggingOptions.LOG_SYSLOG_ENABLED;
import static org.keycloak.quarkus.runtime.configuration.Configuration.isSet;
import static org.keycloak.quarkus.runtime.configuration.Configuration.isTrue;
import static org.keycloak.quarkus.runtime.configuration.mappers.PropertyMapper.fromOption;

Expand Down Expand Up @@ -40,6 +44,8 @@ public static PropertyMapper<?>[] getMappers() {
fromOption(LoggingOptions.LOG)
.paramLabel("<handler>")
.build(),
fromOption(LoggingOptions.LOG_ASYNC)
.build(),
// Console
fromOption(LoggingOptions.LOG_CONSOLE_OUTPUT)
.isEnabled(LoggingPropertyMappers::isConsoleEnabled, CONSOLE_ENABLED_MSG)
Expand All @@ -60,7 +66,7 @@ public static PropertyMapper<?>[] getMappers() {
.transformer((value, ctx) -> addTracingInfo(value, LoggingOptions.LOG_CONSOLE_INCLUDE_TRACE))
.build(),
fromOption(LoggingOptions.LOG_CONSOLE_JSON_FORMAT)
.isEnabled(LoggingPropertyMappers::isConsoleJsonEnabled, CONSOLE_ENABLED_MSG + " and output is set to 'json'")
.isEnabled(LoggingPropertyMappers::isConsoleJsonEnabled, "%s and output is set 10BC8 to 'json'".formatted(CONSOLE_ENABLED_MSG))
.to("quarkus.log.console.json.log-format")
.paramLabel("format")
.build(),
Expand All @@ -76,6 +82,17 @@ public static PropertyMapper<?>[] getMappers() {
.mapFrom(LoggingOptions.LOG, LoggingPropertyMappers.resolveLogHandler(LoggingOptions.DEFAULT_LOG_HANDLER.name()))
.to("quarkus.log.console.enable")
.build(),
// Console async
fromOption(LoggingOptions.LOG_CONSOLE_ASYNC)
.mapFrom(LoggingOptions.LOG_ASYNC)
.isEnabled(LoggingPropertyMappers::isConsoleEnabled, CONSOLE_ENABLED_MSG)
.to("quarkus.log.console.async")
.build(),
fromOption(LoggingOptions.LOG_CONSOLE_ASYNC_QUEUE_LENGTH)
.isEnabled(LoggingPropertyMappers::isConsoleAsyncEnabled, "%s and asynchronous logging is enabled".formatted(CONSOLE_ENABLED_MSG))
.to("quarkus.log.console.async.queue-length")
.paramLabel("queue-length")
.build(),
// File
fromOption(LoggingOptions.LOG_FILE_ENABLED)
.mapFrom(LoggingOptions.LOG, LoggingPropertyMappers.resolveLogHandler("file"))
Expand Down Expand Up @@ -114,6 +131,17 @@ public static PropertyMapper<?>[] getMappers() {
.paramLabel("output")
.transformer(LoggingPropertyMappers::resolveLogOutput)
.build(),
// File async
fromOption(LoggingOptions.LOG_FILE_ASYNC)
.mapFrom(LoggingOptions.LOG_ASYNC)
.isEnabled(LoggingPropertyMappers::isFileEnabled, FILE_ENABLED_MSG)
.to("quarkus.log.file.async")
.build(),
fromOption(LoggingOptions.LOG_FILE_ASYNC_QUEUE_LENGTH)
.isEnabled(LoggingPropertyMappers::isFileAsyncEnabled, "%s and asynchronous logging is enabled".formatted(FILE_ENABLED_MSG))
.to("quarkus.log.file.async.queue-length")
.paramLabel("queue-length")
.build(),
// Log level
fromOption(LoggingOptions.LOG_LEVEL)
.to("quarkus.log.level")
Expand Down Expand Up @@ -187,33 +215,76 @@ public static PropertyMapper<?>[] getMappers() {
.paramLabel("output")
.transformer(LoggingPropertyMappers::resolveLogOutput)
.build(),
// Syslog async
fromOption(LoggingOptions.LOG_SYSLOG_ASYNC)
.mapFrom(LoggingOptions.LOG_ASYNC)
.isEnabled(LoggingPropertyMappers::isSyslogEnabled, SYSLOG_ENABLED_MSG)
.to("quarkus.log.syslog.async")
.build(),
fromOption(LoggingOptions.LOG_SYSLOG_ASYNC_QUEUE_LENGTH)
.isEnabled(LoggingPropertyMappers::isSyslogAsyncEnabled, "%s and asynchronous logging is enabled".formatted(SYSLOG_ENABLED_MSG))
.to("quarkus.log.syslog.async.queue-length")
.paramLabel("queue-length")
.build(),
};

return defaultMappers;
}

public static boolean isConsoleEnabled() {
return isTrue(LoggingOptions.LOG_CONSOLE_ENABLED);
return isHandlerEnabled(LoggingOptions.Handler.console);
}

public static boolean isConsoleAsyncEnabled() {
return isHandlerAsyncEnabled(LoggingOptions.Handler.console);
}

public static boolean isConsoleJsonEnabled() {
return isConsoleEnabled() && Configuration.isTrue("quarkus.log.console.json.enabled");
return isConsoleEnabled() && isTrue("quarkus.log.console.json.enabled");
}

public static boolean isFileEnabled() {
return isTrue(LoggingOptions.LOG_FILE_ENABLED);
return isHandlerEnabled(LoggingOptions.Handler.file);
}

public static boolean isFileAsyncEnabled() {
return isHandlerAsyncEnabled(LoggingOptions.Handler.file);
}

public static boolean isFileJsonEnabled() {
return isFileEnabled() && Configuration.isTrue("quarkus.log.file.json.enabled");
return isFileEnabled() && isTrue("quarkus.log.file.json.enabled");
}

public static boolean isSyslogEnabled() {
return isTrue(LoggingOptions.LOG_SYSLOG_ENABLED);
return isHandlerEnabled(LoggingOptions.Handler.syslog);
}

public static boolean isSyslogAsyncEnabled() {
return isHandlerAsyncEnabled(LoggingOptions.Handler.syslog);
}

public static boolean isSyslogJsonEnabled() {
return isSyslogEnabled() && Configuration.isTrue("quarkus.log.syslog.json.enabled");
return isSyslogEnabled() && isTrue("quarkus.log.syslog.json.enabled");
}

private static boolean isHandlerEnabled(LoggingOptions.Handler handler) {
return switch (handler) {
case console -> isTrue(LOG_CONSOLE_ENABLED);
case file -> isTrue(LOG_FILE_ENABLED);
case syslog -> isTrue(LOG_SYSLOG_ENABLED);
};
}

private static boolean isHandlerAsyncEnabled(LoggingOptions.Handler handler) {
if (!isHandlerEnabled(handler)) {
return false;
}
var property = switch (handler) {
case console -> LoggingOptions.LOG_CONSOLE_ASYNC;
case file -> LoggingOptions.LOG_FILE_ASYNC;
case syslog -> LoggingOptions.LOG_SYSLOG_ASYNC;
};
return isSet(property) ? isTrue(property) : isTrue(LoggingOptions.LOG_ASYNC);
}

private static BiFunction<String, ConfigSourceInterceptorContext, String> resolveLogHandler(String handler) {
Expand Down Expand Up @@ -318,7 +389,7 @@ private static String resolveLogOutput(String value, ConfigSourceInterceptorCont
*/
private static String addTracingInfo(String value, Option<Boolean> includeTraceOption) {
var isTracingEnabled = TracingPropertyMappers.isTracingEnabled();
var includeTrace = Configuration.isTrue(includeTraceOption);
var includeTrace = isTrue(includeTraceOption);
var isChangedLogFormat = !DEFAULT_LOG_FORMAT.equals(value);

if (!isTracingEnabled || !includeTrace || isChangedLogFormat) {
Expand Down
Loading
0