kafka/docs/upgrade.html at 3.2 · tvainika/kafka

Repository navigation

10000

<!--

Licensed to the Apache Software Foundation (ASF) under one or more

contributor license agreements. See the NOTICE file distributed with

this work for additional information regarding copyright ownership.

The ASF licenses this file to You under the Apache License, Version 2.0

(the "License"); you may not use this file except in compliance with

the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software

distributed under the License is distributed on an "AS IS" BASIS,

WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.

See the License for the specific language governing permissions and

limitations under the License.

-->

<h4><a id="upgrade_3_2_3" href="#upgrade_3_2_3">Upgrading to 3.2.3 from any version 0.8.x through 3.1.x</a></h4>

<p><b>If you are upgrading from a version prior to 2.1.x, please see the note below about the change to the schema used to store consumer offsets.

Once you have changed the inter.broker.protocol.version to the latest version, it will not be possible to downgrade to a version prior to 2.1.</b></p>

<p><b>For a rolling upgrade:</b></p>

<ol>

<li>Update server.properties on all brokers and add the following properties. CURRENT_KAFKA_VERSION refers to the version you

are upgrading from. CURRENT_MESSAGE_FORMAT_VERSION refers to the message format version currently in use. If you have previously

overridden the message format version, you should keep its current value. Alternatively, if you are upgrading from a version prior

to 0.11.0.x, then CURRENT_MESSAGE_FORMAT_VERSION should be set to match CURRENT_KAFKA_VERSION.

<ul>

<li>inter.broker.protocol.version=CURRENT_KAFKA_VERSION (e.g., <code>3.1</code>, <code>3.0</code>, etc.)</li>

<li>log.message.format.version=CURRENT_MESSAGE_FORMAT_VERSION (See <a href="#upgrade_10_performance_impact">potential performance impact

following the upgrade</a> for the details on what this configuration does.)</li>

</ul>

If you are upgrading from version 0.11.0.x or above, and you have not overridden the message format, then you only need to override

the inter-broker protocol version.

<ul>

<li>inter.broker.protocol.version=CURRENT_KAFKA_VERSION (e.g., <code>3.1</code>, <code>3.0</code>, etc.)</li>

</ul>

</li>

<li>Upgrade the brokers one at a time: shut down the broker, update the code, and restart it. Once you have done so, the

brokers will be running the latest version and you can verify that the cluster's behavior and performance meets expectations.

It is still possible to downgrade at this point if there are any problems.

</li>

<li>Once the cluster's behavior and performance has been verified, bump the protocol version by editing

<code>inter.broker.protocol.version</code> and setting it to <code>3.2</code>.

</li>

<li>Restart the brokers one by one for the new protocol version to take effect. Once the brokers begin using the latest

protocol version, it will no longer be possible to downgrade the cluster to an older version.

</li>

<li>If you have overridden the message format version as instructed above, then you need to do one more rolling restart to

upgrade it to its latest version. Once all (or most) consumers have been upgraded to 0.11.0 or later,

change log.message.format.version to 3.2 on each broker and restart them one by one. Note that the older Scala clients,

which are no longer maintained, do not support the message format introduced in 0.11, so to avoid conversion costs

(or to take advantage of <a href="#upgrade_11_exactly_once_semantics">exactly once semantics</a>),

the newer Java clients must be used.

</li>

</ol>

<h5><a id="upgrade_321_notable" href="#upgrade_321_notable">Notable changes in 3.2.1</a></h5>

<ul>

<li>A problem with OAuth token refresh was fixed. During token refresh, the Kafka client was not including

the SASL extensions which were present during the initial authentication. This leads to authentication

failures. See <a href="https://issues.apache.org/jira/browse/KAFKA-14062">KAFKA-14062</a> for details.</li>

<li>A problem in Connect with <code>errors.tolerance</code> was fixed. Starting in 3.2.0, when

a source connector uses <code>errors.tolerance=all</code> and Connect's producer fails to

successfully send a source record to the topic, the error is properly reported and the connector

continues, but Connect stops committing source offsets for that source connector and a

memory leak will eventually cause the Connect worker to fail with an OutOfMemoryError.

See <a href="https://issues.apache.org/jira/browse/KAFKA-14079">KAFKA-14079</a> for details.</li>

<li>A regression in the cooperative rebalance was fixed. Changes introduced in 3.2.0 to the offset commit

behavior prior to rebalancing could lead to the consumer getting stuck. See

<a href="https://issues.apache.org/jira/browse/KAFKA-14024">KAFKA-14024</a> for details.</li>

</ul>

<h5><a id="upgrade_320_notable" href="#upgrade_320_notable">Notable changes in 3.2.0</a></h5>

<ul>

<li>Idempotence for the producer is enabled by default if no conflicting configurations are set. When producing to brokers older than 2.8.0,

the <code>IDEMPOTENT_WRITE</code> permission is required. Check the compatibility

section of <a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-679%3A+Producer+will+enable+the+strongest+delivery+guarantee+by+default#KIP679:Producerwillenablethestrongestdeliveryguaranteebydefault-Compatibility,Deprecation,andMigrationPlan">KIP-679</>

for details. In 3.0.0 and 3.1.0, a bug prevented this default from being applied,

which meant that idempotence remained disabled unless the user had explicitly set <code>enable.idempotence</code> to true

(See <a href="https://issues.apache.org/jira/browse/KAFKA-13598">KAFKA-13598</a>for more details).

This issue was fixed and the default is properly applied in 3.0.1, 3.1.1, and 3.2.0.</li>

<li>A notable exception is Connect that by default disables idempotent behavior for all of its

producers in order to uniformly support using a wide range of Kafka broker versions.

Users can change this behavior to enable idempotence for some or all producers

via Connect worker and/or connector configuration. Connect may enable idempotent producers

by default in a future major release.</li>

<li>Kafka has replaced log4j and slf4j-log4j12 with reload4j and slf4j-reload4j due to security concerns.

This only affects modules that specify a logging backend (<code>connect-runtime</code> and <code>kafka-tools</code> are two such examples).

A number of modules, including <code>kafka-clients</code>, leave it to the application to specify the logging backend.

More information can be found at <a href"https://reload4j.qos.ch">reload4j</a>.

Projects that depend on the affected modules from the Kafka project should use

<a href="https://www.slf4j.org/manual.html#swapping">slf4j-log4j12 version 1.7.35 or above</a> or

slf4j-reload4j to avoid

<a href="https://www.slf4j.org/codes.html#no_tlm">possible compatibility issues originating from the logging framework</a>.</li>

<li>The example connectors, <code>FileStreamSourceConnector</code> and <code>FileStreamSinkConnector</code>, have been

removed from the default classpath. To use them in Kafka Connect standalone or distributed mode they need to be

explicitly added, for example <code>CLASSPATH=./lib/connect-file-3.2.0.jar ./bin/connect-distributed.sh</code>.</li>

</ul>

<h4><a id="upgrade_3_1_0" href="#upgrade_3_1_0">Upgrading to 3.1.0 from any version 0.8.x through 3.0.x</a></h4>

<p><b>If you are upgrading from a version prior to 2.1.x, please see the note below about the change to the schema used to store consumer offsets.

Once you have changed the inter.broker.protocol.version to the latest version, it will not be possible to downgrade to a version prior to 2.1.</b></p>

<p><b>For a rolling upgrade:</b></p>

<ol>

<li>Update server.properties on all brokers and add the following properties. CURRENT_KAFKA_VERSION refers to the version you

are upgrading from. CURRENT_MESSAGE_FORMAT_VERSION refers to the message format version currently in use. If you have previously

overridden the message format version, you should keep its current value. Alternatively, if you are upgrading from a version prior

to 0.11.0.x, then CURRENT_MESSAGE_FORMAT_VERSION should be set to match CURRENT_KAFKA_VERSION.

<ul>

<li>inter.broker.protocol.version=CURRENT_KAFKA_VERSION (e.g., <code>3.0</code>, <code>2.8</code>, etc.)</li>

<li>log.message.format.version=CURRENT_MESSAGE_FORMAT_VERSION (See <a href="#upgrade_10_performance_impact">potential performance impact

following the upgrade</a> for the details on what this configuration does.)</li>

</ul>

If you are upgrading from version 0.11.0.x or above, and you have not overridden the message format, then you only need to override

the inter-broker protocol version.

<ul>

<li>inter.broker.protocol.version=CURRENT_KAFKA_VERSION (e.g., <code>3.0</code>, <code>2.8</code>, etc.)</li>

</ul>

</li>

<li>Upgrade the brokers one at a time: shut down the broker, update the code, and restart it. Once you have done so, the

brokers will be running the latest version and you can verify that the cluster's behavior and performance meets expectations.

It is still possible to downgrade at this point if there are any problems.

</li>

<li>Once the cluster's behavior and performance has been verified, bump the protocol version by editing

<code>inter.broker.protocol.version</code> and setting it to <code>3.1</code>.

</li>

<li>Restart the brokers one by one for the new protocol version to take effect. Once the brokers begin using the latest

protocol version, it will no longer be possible to downgrade the cluster to an older version.

</li>

<li>If you have overridden the message format version as instructed above, then you need to do one more rolling restart to

upgrade it to its latest version. Once all (or most) consumers have been upgraded to 0.11.0 or later,

change log.message.format.version to 3.1 on each broker and restart them one by one. Note that the older Scala clients,

which are no longer maintained, do not support the message format introduced in 0.11, so to avoid conversion costs

(or to take advantage of <a href="#upgrade_11_exactly_once_semantics">exactly once semantics</a>),

the newer Java clients must be used.

</li>

</ol>

<h5><a id="upgrade_311_notable" href="#upgrade_311_notable">Notable changes in 3.1.1</a></h5>

<ul>

<li>Idempotence for the producer is enabled by default if no conflicting configurations are set. When producing to brokers older than 2.8.0,

the <code>IDEMPOTENT_WRITE</code> permission is required. Check the compatibility

for details. A bug prevented the producer idempotence default from being applied which meant that it remained disabled unless the user had

explicitly set <code>enable.idempotence</code> to true. See <a href="https://issues.apache.org/jira/browse/KAFKA-13598">KAFKA-13598</a> for

more details. This issue was fixed and the default is properly applied.</li>

<li>A notable exception is Connect that by default disables idempotent behavior for all of its

producers in order to uniformly support using a wide range of Kafka broker versions.

Users can change this behavior to enable idempotence for some or all producers

via Connect worker and/or connector configuration. Connect may enable idempotent producers

by default in a future major release.</li>

</ul>

<h5><a id="upgrade_310_notable" href="#upgrade_310_notable">Notable changes in 3.1.0</a></h5>

<ul>

<li>Apache Kafka supports Java 17.</li>

<li>The following metrics have been deprecated: <code>bufferpool-wait-time-total</code>, <code>io-waittime-total</code>,

and <code>iotime-total</code>. Please use <code>bufferpool-wait-time-ns-total</code>, <code>io-wait-time-ns-total</code>,

and <code>io-time-ns-total</code> instead. See <a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-773%3A+Differentiate+consistently+metric+latency+measured+in+millis+and+nanos">KIP-773</a>

for more details.</li>

<li>The cooperative rebalancing protocol has been the default since 2.4, but we have continued to support the

eager rebalancing protocol to provide users an upgrade path. This support will be dropped in a future release,

so any users still on the eager protocol should prepare to finish upgrading their applications to the cooperative protocol in version 3.1.

This only affects users who are still on a version older than 2.4, and users who have upgraded already but have not yet

removed the <code>upgrade.from</code> config that they set when upgrading from a version below 2.4.

Users fitting into the latter case will simply need to unset this config when upgrading beyond 3.1,

while users in the former case will need to follow a slightly different upgrade path if they attempt to upgrade from 2.3 or below to a version above 3.1.

Those applications will need to go through a bridge release, by first upgrading to a version between 2.4 - 3.1 and setting the <code>upgrade.from</code> config,

then removing that config and upgrading to the final version above 3.1. See <a href="https://issues.apache.org/jira/browse/KAFKA-8575">KAFKA-8575</a>

for more details.</li>

<li>IBP 3.1 introduces topic IDs to FetchRequest as a part of

</ul>

<h4><a id="upgrade_3_0_0" href="#upgrade_3_0_0">Upgrading to 3.0.0 from any version 0.8.x through 2.8.x</a></h4>

<p><b>If you are upgrading from a version prior to 2.1.x, please see the note below about the change to the schema used to store consumer offsets.

Once you have changed the inter.broker.protocol.version to the latest version, it will not be possible to downgrade to a version prior to 2.1.</b></p>

<p><b>For a rolling upgrade:</b></p>

<ol>

<li>Update server.properties on all brokers and add the following properties. CURRENT_KAFKA_VERSION refers to the version you

are upgrading from. CURRENT_MESSAGE_FORMAT_VERSION refers to the message format version currently in use. If you have previously

overridden the message format version, you should keep its current value. Alternatively, if you are upgrading from a version prior

to 0.11.0.x, then CURRENT_MESSAGE_FORMAT_VERSION should be set to match CURRENT_KAFKA_VERSION.

<ul>

<li>inter.broker.protocol.version=CURRENT_KAFKA_VERSION (e.g., <code>2.8</code>, <code>2.7</code>, etc.)</li>

<li>log.message.format.version=CURRENT_MESSAGE_FORMAT_VERSION (See <a href="#upgrade_10_performance_impact">potential performance impact

following the upgrade</a> for the details on what this configuration does.)</li>

</ul>

If you are upgrading from version 0.11.0.x or above, and you have not overridden the message format, then you only need to override

the inter-broker protocol version.

<ul>

<li>inter.broker.protocol.version=CURRENT_KAFKA_VERSION (e.g., <code>2.8</code>, <code>2.7</code>, etc.)</li>

</ul>

</li>

<li>Upgrade the brokers one at a time: shut down the broker, update the code, and restart it. Once you have done so, the

brokers will be running the latest version and you can verify that the cluster's behavior and performance meets expectations.

It is still possible to downgrade at this point if there are any problems.

</li>

<li>Once the cluster's behavior and performance has been verified, bump the protocol version by editing

<code>inter.broker.protocol.version</code> and setting it to <code>3.0</code>.

</li>

<li>Restart the brokers one by one for the new protocol version to take effect. Once the brokers begin using the latest

protocol version, it will no longer be possible to downgrade the cluster to an older version.

</li>

<li>If you have overridden the message format version as instructed above, then you need to do one more rolling restart to

upgrade it to its latest version. Once all (or most) consumers have been upgraded to 0.11.0 or later,

change log.message.format.version to 3.0 on each broker and restart them one by one. Note that the older Scala clients,

which are no longer maintained, do not support the message format introduced in 0.11, so to avoid conversion costs

(or to take advantage of <a href="#upgrade_11_exactly_once_semantics">exactly once semantics</a>),

the newer Java clients must be used.

</li>

</ol>

<h5><a id="upgrade_301_notable" href="#upgrade_301_notable">Notable changes in 3.0.1</a></h5>

<ul>

<li>Idempotence for the producer is enabled by default if no conflicting configurations are set. When producing to brokers older than 2.8.0,

the <code>IDEMPOTENT_WRITE</code> permission is required. Check the compatibility

for details. A bug prevented the producer idempotence default from being applied which meant that it remained disabled unless the user had

explicitly set <code>enable.idempotence</code> to true. See <a href="https://issues.apache.org/jira/browse/KAFKA-13598">KAFKA-13598</a> for

more details. This issue was fixed and the default is properly applied.</li>

</ul>

<h5><a id="upgrade_300_notable" href="#upgrade_300_notable">Notable changes in 3.0.0</a></h5>

<ul>

<li>The producer has stronger delivery guarantees by default: <code>idempotence</code> is enabled and <code>acks</code> is set to <code>all</code> instead of <code>1</code>.

See <a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-679%3A+Producer+will+enable+the+strongest+delivery+guarantee+by+default">KIP-679</a> for details.

In 3.0.0 and 3.1.0, a bug prevented the idempotence default from being applied which meant that it remained disabled unless the user had explicitly set

<code>enable.idempotence</code> to true. Note that the bug did not affect the <code>acks=all</code> change. See <a href="https://issues.apache.org/jira/browse/KAFKA-13598">KAFKA-13598</a> for more details.

This issue was fixed and the default is properly applied in 3.0.1, 3.1.1, and 3.2.0.</li>

<li>Java 8 and Scala 2.12 support have been deprecated since Apache Kafka 3.0 and will be removed in Apache Kafka 4.0.

See <a href="https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=181308223">KIP-750</a>

and <a href="https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=181308218">KIP-751</a> for more details.</li>

<li>ZooKeeper has been upgraded to version 3.6.3.</li>

<li>A preview of KRaft mode is available, though upgrading to it from the 2.8 Early Access release is not possible. See

the <code>config/kraft/README.md</code> file for details.</li>

<li>The release tarball no longer includes test, sources, javadoc and test sources jars. These are still published to the Maven Central repository. </li>

<li>A number of implementation dependency jars are <a href="https://github.com/apache/kafka/pull/10203">now available in the runtime classpath

instead of compile and runtime classpaths</a>. Compilation errors after the upgrade can be fixed by adding the missing dependency jar(s) explicitly

or updating the application not to use internal classes.</li>

<li>The default value for the consumer configuration <code>session.timeout.ms</code> was increased from 10s to 45s. See

<a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-735%3A+Increase+default+consumer+session+timeout">KIP-735</a> for more details.</li>

<li>The broker configuration <code>log.message.format.version</code> and topic configuration <code>message.format.version</code> have been deprecated.

The value of both configurations is always assumed to be <code>3.0</code> if <code>inter.broker.protocol.version</code> is <code>3.0</code> or higher.

If <code>log.message.format.version</code> or <<code>message.format.version</code> are set, we recommend clearing them at the same time as the

<code>inter.broker.protocol.version</code> upgrade to 3.0. This will avoid potential compatibility issues if the <code>inter.broker.protocol.version</code>

is downgraded. See <a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-724%3A+Drop+support+for+message+formats+v0+and+v1">KIP-724</a> for more details.</li>

<li>The Streams API removed all deprecated APIs that were deprecated in version 2.5.0 or earlier.

For a complete list of removed APIs compare the detailed Kafka Streams upgrade notes.</li>

<li>Kafka Streams no longer has a compile time dependency on "connect:json" module (<a href="https://issues.apache.org/jira/browse/KAFKA-5146">KAFKA-5146</a>).

Projects that were relying on this transitive dependency will have to explicitly declare it.</li>

<li>Custom principal builder implementations specified through <code>principal.builder.class</code> must now implement the

<code>KafkaPrincipalSerde</code> interface to allow for forwarding between brokers. See <a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-590%3A+Redirect+Zookeeper+Mutation+Protocols+to+The+Controller">KIP-590</a> for more details about the usage of KafkaPrincipalSerde.</li>

<li>A number of deprecated classes, methods and tools have been removed from the <code>clients</code>, <code>connect</code>, <code>core</code> and <code>tools</code> modules:</li>

<ul>

<li>The Scala <code>Authorizer</code>, <code>SimpleAclAuthorizer</code> and related classes have been removed. Please use the Java <code>Authorizer</code>

and <code>AclAuthorizer</code> instead.</li>

<li>The <code>Metric#value()</code> method was removed (<a href="https://issues.apache.org/jira/browse/KAFKA-12573">KAFKA-12573</a>).</li>

<li>The <code>Sum</code> and <code>Total</code> classes were removed (<a href="https://issues.apache.org/jira/browse/KAFKA-12584">KAFKA-12584</a>).

Please use <code>WindowedSum</code> and <code>CumulativeSum</code> instead.</li>

<li>The <code>Count</code> and <code>SampledTotal</code> classes were removed. Please use <code>WindowedCount</code> and <code>WindowedSum</code>

respectively instead.</li>

<li>The <code>PrincipalBuilder</code>, <code>DefaultPrincipalBuilder</code> and <code>ResourceFilter</code> classes were removed.

<li>Various constants and constructors were removed from <code>SslConfigs</code>, <code>SaslConfigs</code>, <code>AclBinding</code> and

<code>AclBindingFilter</code>.</li>

<li>The <code>Admin.electedPreferredLeaders()</code> methods were removed. Please use <code>Admin.electLeaders</code> instead.</li>

<li>The <code>kafka-preferred-replica-election</code> command line tool was removed. Please use <code>kafka-leader-election</code> instead.</li>

<li>The <code>--zookeeper</code> option was removed from the <code>kafka-topics</code> and <code>kafka-reassign-partitions</code> command line tools.

Please use <code>--bootstrap-server</code> instead.</li>

<li>In the <code>kafka-configs</code> command line tool, the <code>--zookeeper</code> option is only supported for updating <a href="#security_sasl_scram_credentials">SCRAM Credentials configuration</a>

and <a href="#dynamicbrokerconfigs">describing/updating dynamic broker configs when brokers are not running</a>. Please use <code>--bootstrap-server</code>

for other configuration operations.</li>

<li>The <code>ConfigEntry</code> constructor was removed (<a href="https://issues.apache.org/jira/browse/KAFKA-12577">KAFKA-12577</a>).

Please use the remaining public constructor instead.</li>

<li>The config value <code>default</code> for the client config <code>client.dns.lookup</code> has been removed. In the unlikely

event that you set this config explicitly, we recommend leaving the config unset (<code>use_all_dns_ips</code> is used by default).</li>

<li>The <code>ExtendedDeserializer</code> and <code>ExtendedSerializer</code> classes have been removed. Please use <code>Deserializer</code>

and <code>Serializer</code> instead.</li>

<li>The <code>close(long, TimeUnit)</code> method was removed from the producer, consumer and admin client. Please use

<code>close(Duration)</code>.</li>

<li>The <code>ConsumerConfig.addDeserializerToConfig</code> and <code>ProducerConfig.addSerializerToConfig</code> methods

were removed. These methods were not intended to be public API and there is no replacement.</li>

<li>The <code>NoOffsetForPartitionException.partition()</code> method was removed. Please use <code>partitions()</code>

instead.</li>

<li>The default <code>partition.assignment.strategy</code> is changed to "[RangeAssignor, CooperativeStickyAssignor]",

which will use the RangeAssignor by default, but allows upgrading to the CooperativeStickyAssignor with just a single rolling bounce that removes the RangeAssignor from the list.

Please check the client upgrade path guide <a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-429:+Kafka+Consumer+Incremental+Rebalance+Protocol#KIP429:KafkaConsumerIncrementalRebalanceProtocol-Consumer">here</a> for more detail.</li>

<li>The Scala <code>kafka.common.MessageFormatter</code> was removed. Please use the Java <code>org.apache.kafka.common.MessageFormatter</code>.</li>

<li>The <code>MessageFormatter.init(Properties)</code> method was removed. Please use <code>configure(Map)</code> instead.</li>

<li>The <code>checksum()</code> method has been removed from <code>ConsumerRecord</code> and <code>RecordMetadata</code>. The message

format v2, which has been the default since 0.11, moved the checksum from the record to the record batch. As such, these methods

don't make sense and no replacements exist.</li>

<li>The <code>ChecksumMessageFormatter</code> class was removed. It is not part of the public API, but it may have been used

with <code>kafka-console-consumer.sh</code>. It reported the checksum of each record, which has not been supported

since message format v2.</li>

<li>The <code>org.apache.kafka.clients.consumer.internals.PartitionAssignor</code> class has been removed. Please use

<code>org.apache.kafka.clients.consumer.ConsumerPartitionAssignor</code> instead.</li>

<li>The <code>quota.producer.default</code> and <code>quota.consumer.default</code> configurations were removed (<a href="https://issues.apache.org/jira/browse/KAFKA-12591">KAFKA-12591</a>).

Dynamic quota defaults must be used instead.</li>

<li>The <code>port</code> and <code>host.name</code> configurations were removed. Please use <code>listeners</code> instead.</li>

<li>The <code>advertised.port</code> and <code>advertised.host.name</code> configurations were removed. Please use <code>advertised.listeners</code> instead.</li>

<li>The deprecated worker configurations <code>rest.host.name</code> and <code>rest.port</code> were removed (<a href="https://issues.apache.org/jira/browse/KAFKA-12482">KAFKA-12482</a>) from the Kafka Connect worker configuration.

Please use <code>listeners</code> instead.</li>

</ul>

<li> The <code>Producer#sendOffsetsToTransaction(Map offsets, String consumerGroupId)</code> method has been deprecated. Please use

<code>Producer#sendOffsetsToTransaction(Map offsets, ConsumerGroupMetadata metadata)</code> instead, where the <code>ConsumerGroupMetadata</code>

can be retrieved via <code>KafkaConsumer#groupMetadata()</code> for stronger semantics. Note that the full set of consumer group metadata is only

understood by brokers or version 2.5 or higher, so you must upgrade your kafka cluster to get the stronger semantics. Otherwise, you can just pass

in <code>new ConsumerGroupMetadata(consumerGroupId)</code> to work with older brokers. See <a href="https://cwiki.apache.org/confluence/x/zJONCg">KIP-732</a> for more details.

</li>

<li>

The Connect <code>internal.key.converter</code> and <code>internal.value.converter</code> properties have been completely <a href="https://cwiki.apache.org/confluence/x/2YDOCg">removed</a>.

The use of these Connect worker properties has been deprecated since version 2.0.0.

Workers are now hardcoded to use the JSON converter with <code>schemas.enable</code> set to <code>false</code>. If your cluster has been using

a different internal key or value converter, you can follow the migration steps outlined in <a href="https://cwiki.apache.org/confluence/x/2YDOCg">KIP-738</a>

to safely upgrade your Connect cluster to 3.0.

</li>

<li> The Connect-based MirrorMaker (MM2) includes changes to support <code>IdentityReplicationPolicy</code>, enabling replication without renaming topics.

The existing <code>DefaultReplicationPolicy</code> is still used by default, but identity replication can be enabled via the

<code>replication.policy</code> configuration property. This is especially useful for users migrating from the older MirrorMaker (MM1), or for

use-cases with simple one-way replication topologies where topic renaming is undesirable. Note that <code>IdentityReplicationPolicy</code>, unlike

<code>DefaultReplicationPolicy</code>, cannot prevent replication cycles based on topic names, so take care to avoid cycles when constructing your

replication topology.

</li>

<li> The original MirrorMaker (MM1) and related classes have been deprecated. Please use the Connect-based

MirrorMaker (MM2), as described in the

<a href="/{{version}}/documentation/#georeplication">Geo-Replication section</a>.

</li>

</ul>

<h4><a id="upgrade_2_8_1" href="#upgrade_2_8_1">Upgrading to 2.8.1 from any version 0.8.x through 2.7.x</a></h4>

<p><b>If you are upgrading from a version prior to 2.1.x, please see the note below about the change to the schema used to store consumer offsets.

Once you have changed the inter.broker.protocol.version to the latest version, it will not be possible to downgrade to a version prior to 2.1.</b></p>

<p><b>For a rolling upgrade:</b></p>

<ol>

<li> Update server.properties on all brokers and add the following properties. CURRENT_KAFKA_VERSION refers to the version you

are upgrading from. CURRENT_MESSAGE_FORMAT_VERSION refers to the message format version currently in use. If you have previously

overridden the message format version, you should keep its current value. Alternatively, if you are upgrading from a version prior

to 0.11.0.x, then CURRENT_MESSAGE_FORMAT_VERSION should be set to match CURRENT_KAFKA_VERSION.

<ul>

<li>inter.broker.protocol.version=CURRENT_KAFKA_VERSION (e.g., <code>2.7</code>, <code>2.6</code>, etc.)</li>

<li>log.message.format.version=CURRENT_MESSAGE_FORMAT_VERSION (See <a href="#upgrade_10_performance_impact">potential performance impact

following the upgrade</a> for the details on what this configuration does.)</li>

</ul>

If you are upgrading from version 0.11.0.x or above, and you have not overridden the message format, then you only need to override

the inter-broker protocol version.

<ul>

<li>inter.broker.protocol.version=CURRENT_KAFKA_VERSION (e.g., <code>2.7</code>, <code>2.6</code>, etc.)</li>

</ul>

</li>

<li> Upgrade the brokers one at a time: shut down the broker, update the code, and restart it. Once you have done so, the

brokers will be running the latest version and you can verify that the cluster's behavior and performance meets expectations.

It is still possible to downgrade at this point if there are any problems.

</li>

<li> Once the cluster's behavior and performance has been verified, bump the protocol version by editing

<code>inter.broker.protocol.version</code> and setting it to <code>2.8</code>.

</li>

<li> Restart the brokers one by one for the new protocol version to take effect. Once the brokers begin using the latest

protocol version, it will no longer be possible to downgrade the cluster to an older version.

</li>

<li> If you have overridden the message format version as instructed above, then you need to do one more rolling restart to

upgrade it to its latest version. Once all (or most) consumers have been upgraded to 0.11.0 or later,

change log.message.format.version to 2.8 on each broker and restart them one by one. Note that the older Scala clients,

which are no longer maintained, do not support the message format introduced in 0.11, so to avoid conversion costs

(or to take advantage of <a href="#upgrade_11_exactly_once_semantics">exactly once semantics</a>),

the newer Java clients must be used.

</li>

</ol>

<h5><a id="upgrade_280_notable" href="#upgrade_280_notable">Notable changes in 2.8.0</a></h5>

<ul>

<li>

The 2.8.0 release added a new method to the Authorizer Interface introduced in

<a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-679%3A+Producer+will+enable+the+strongest+delivery+guarantee+by+default">KIP-679</a>.

The motivation is to unblock our future plan to enable the strongest message delivery guarantee by default.

Custom authorizer should consider providing a more efficient implementation that supports audit logging and any custom configs or access rules.

</li>

<li>

IBP 2.8 introduces topic IDs to topics as a part of

<a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers">KIP-516</a>.

When using ZooKeeper, this information is stored in the TopicZNode. If the cluster is downgraded to a previous IBP or version,

future topics will not get topic IDs and it is not guaranteed that topics will retain their topic IDs in ZooKeeper.

This means that upon upgrading again, some topics or all topics will be assigned new IDs.

</li>

<li>Kafka Streams introduce a type-safe <code>split()</code> operator as a substitution for deprecated <code>KStream#branch()</code> method

(cf. <a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-418%3A+A+method-chaining+way+to+branch+KStream">KIP-418</a>).

</li>

</ul>

<h4><a id="upgrade_2_7_0" href="#upgrade_2_7_0">Upgrading to 2.7.0 from any version 0.8.x through 2.6.x</a></h4>

<p><b>If you are upgrading from a version prior to 2.1.x, please see the note below about the change to the schema used to store consumer offsets.

Once you have changed the inter.broker.protocol.version to the latest version, it will not be possible to downgrade to a version prior to 2.1.</b></p>

<p><b>For a rolling upgrade:</b></p>

<ol>

<li> Update server.properties on all brokers and add the following properties. CURRENT_KAFKA_VERSION refers to the version you

are upgrading from. CURRENT_MESSAGE_FORMAT_VERSION refers to the message format version currently in use. If you have previously

overridden the message format version, you should keep its current value. Alternatively, if you are upgrading from a version prior

to 0.11.0.x, then CURRENT_MESSAGE_FORMAT_VERSION should be set to match CURRENT_KAFKA_VERSION.

<ul>

<li>inter.broker.protocol.version=CURRENT_KAFKA_VERSION (e.g., <code>2.6</code>, <code>2.5</code>, etc.)</li>

<li>log.message.format.version=CURRENT_MESSAGE_FORMAT_VERSION (See <a href="#upgrade_10_performance_impact">potential performance impact

following the upgrade</a> for the details on what this configuration does.)</li>

</ul>

If you are upgrading from version 0.11.0.x or above, and you have not overridden the message format, then you only need to override

the inter-broker protocol version.

<ul>

<li>inter.broker.protocol.version=CURRENT_KAFKA_VERSION (e.g., <code>2.6</code>, <code>2.5</code>, etc.)</li>

</ul>

</li>

<li> Upgrade the brokers one at a time: shut down the broker, update the code, and restart it. Once you have done so, the

brokers will be running the latest version and you can verify that the cluster's behavior and performance meets expectations.

It is still possible to downgrade at this point if there are any problems.

</li>

<li> Once the cluster's behavior and performance has been verified, bump the protocol version by editing

<code>inter.broker.protocol.version</code> and setting it to <code>2.7</code>.

</li>

<li> Restart the brokers one by one for the new protocol version to take effect. Once the brokers begin using the latest

protocol version, it will no longer be possible to downgrade the cluster to an older version.

</li>

<li> If you have overridden the message format version as instructed above, then you need to do one more rolling restart to

upgrade it to its latest version. Once all (or most) consumers have been upgraded to 0.11.0 or later,

change log.message.format.version to 2.7 on each broker and restart them one by one. Note that the older Scala clients,

which are no longer maintained, do not support the message format introduced in 0.11, so to avoid conversion costs

(or to take advantage of <a href="#upgrade_11_exactly_once_semantics">exactly once semantics</a>),

the newer Java clients must be used.

</li>

</ol>

<h5><a id="upgrade_270_notable" href="#upgrade_270_notable">Notable changes in 2.7.0</a></h5>

<ul>

<li>

The 2.7.0 release includes the core Raft implementation specified in

<a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-595%3A+A+Raft+Protocol+for+the+Metadata+Quorum">KIP-595</a>.

There is a separate "raft" module containing most of the logic. Until integration with the

controller is complete, there is a standalone server that users can use for testing the

performance of the Raft implementation. See the README.md in the raft module for details

</li>

<li>

KIP-651 <a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-651+-+Support+PEM+format+for+SSL+certificates+and+private+key">adds support</a>

for using PEM files for key and trust stores.

</li>

<li>

KIP-612 <a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-612%3A+Ability+to+Limit+Connection+Creation+Rate+on+Brokers">adds support</a>

for enforcing broker-wide and per-listener connection create rates. The 2.7.0 release contains

the first part of KIP-612 with dynamic configuration coming in the 2.8.0 release.

</li>

<li>

The ability to throttle topic and partition creations or

topics deletions to prevent a cluster from being harmed via

</li>

<li>

When new features become available in Kafka there are two main issues:

<ol>

<li>How do Kafka clients become aware of broker capabilities?</li>

<li>How does the broker decide which features to enable?</li>

</ol>

provides a flexible and operationally easy solution for client discovery, feature gating and rolling upgrades using a single restart.

</li>

<li>

The ability to print record offsets and headers with the <code>ConsoleConsumer</code> is now possible

via <a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-431%3A+Support+of+printing+additional+ConsumerRecord+fields+in+DefaultMessageFormatter">KIP-431</a>

</li>

<li>

The addition of <a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-554%3A+Add+Broker-side+SCRAM+Config+API">KIP-554</a>

continues progress towards the goal of Zookeeper removal from Kafka. The addition of KIP-554

means you don't have to connect directly to ZooKeeper anymore for managing SCRAM credentials.

</li>

<li>Altering non-reconfigurable configs of existent listeners causes <code>InvalidRequestException</code>.

By contrast, the previous (unintended) behavior would have caused the updated configuration to be persisted,

but it wouldn't

take effect until the broker was restarted. See <a href="https://github.com/apache/kafka/pull/9284">KAFKA-10479</a> for more discussion.

See <code>DynamicBrokerConfig.DynamicSecurityConfigs</code> and <code>SocketServer.ListenerReconfigurableConfigs</code>

for the supported reconfigurable configs of existent listeners.

</li>

<li>

Kafka Streams adds support for

<a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-450%3A+Sliding+Window+Aggregations+in+the+DSL">Sliding Windows Aggregations</a>

in the KStreams DSL.

</li>

<li>

Reverse iteration over state stores enabling more efficient most recent update searches with

</li>

<li>

End-to-End latency metrics in Kafka Steams see

for more details

</li>

<li>

Kafka Streams added metrics reporting default RocksDB properties with

</li>

<li>

Better Scala implicit Serdes support from

</li>

</ul>

<h4><a id="upgrade_2_6_0" href="#upgrade_2_6_0">Upgrading to 2.6.0 from any version 0.8.x through 2.5.x</a></h4>

<p><b>If you are upgrading from a version prior to 2.1.x, please see the note below about the change to the schema used to store consumer offsets.

Once you have changed the inter.broker.protocol.version to the latest version, it will not be possible to downgrade to a version prior to 2.1.</b></p>

<p><b>For a rolling upgrade:</b></p>

<ol>

<li> Update server.properties on all brokers and add the following properties. CURRENT_KAFKA_VERSION refers to the version you

are upgrading from. CURRENT_MESSAGE_FORMAT_VERSION refers to the message format version currently in use. If you have previously

overridden the message format version, you should keep its current value. Alternatively, if you are upgrading from a version prior

to 0.11.0.x, then CURRENT_MESSAGE_FORMAT_VERSION should be set to match CURRENT_KAFKA_VERSION.

<ul>

<li>inter.broker.protocol.version=CURRENT_KAFKA_VERSION (e.g., <code>2.5</code>, <code>2.4</code>, etc.)</li>

<li>log.message.format.version=CURRENT_MESSAGE_FORMAT_VERSION (See <a href="#upgrade_10_performance_impact">potential performance impact

following the upgrade</a> for the details on what this configuration does.)</li>

</ul>

If you are upgrading from version 0.11.0.x or above, and you have not overridden the message format, then you only need to override

the inter-broker protocol version.

<ul>

<li>inter.broker.protocol.version=CURRENT_KAFKA_VERSION (e.g., <code>2.5</code>, <code>2.4</code>, etc.)</li>

</ul>

</li>

<li> Upgrade the brokers one at a time: shut down the broker, update the code, and restart it. Once you have done so, the

brokers will be running the latest version and you can verify that the cluster's behavior and performance meets expectations.

It is still possible to downgrade at this point if there are any problems.

</li>

<li> Once the cluster's behavior and performance has been verified, bump the protocol version by editing

<code>inter.broker.protocol.version</code> and setting it to <code>2.6</code>.

</li>

<li> Restart the brokers one by one for the new protocol version to take effect. Once the brokers begin using the latest

protocol version, it will no longer be possible to downgrade the cluster to an older version.

</li>

<li> If you have overridden the message format version as instructed above, then you need to do one more rolling restart to

upgrade it to its latest version. Once all (or most) consumers have been upgraded to 0.11.0 or later,

change log.message.format.version to 2.6 on each broker and restart them one by one. Note that the older Scala clients,

which are no longer maintained, do not support the message format introduced in 0.11, so to avoid conversion costs

(or to take advantage of <a href="#upgrade_11_exactly_once_semantics">exactly once semantics</a>),

the newer Java clients must be used.

</li>

</ol>

<h5 class="anchor-heading"><a id="upgrade_260_notable" class="anchor-link"></a><a href="#upgrade_260_notable">Notable changes in 2.6.0</a></h5>

<ul>

<li>Kafka Streams adds a new processing mode (requires broker 2.5 or newer) that improves application

scalability using exactly-once guarantees

(cf. <a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-447%3A+Producer+scalability+for+exactly+once+semantics">KIP-447</a>)

</li>

<li>TLSv1.3 has been enabled by default for Java 11 or newer. The client and server will negotiate TLSv1.3 if

both support it and fallback to TLSv1.2 otherwise. See

<a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-573%3A+Enable+TLSv1.3+by+default">KIP-573</a> for more details.

</li>

<li>The default value for the <code>client.dns.lookup</code> configuration has been changed from <code>default</code>

to <code>use_all_dns_ips</code>. If a hostname resolves to multiple IP addresses, clients and brokers will now

attempt to connect to each IP in sequence until the connection is successfully established. See

for more details.

</li>

<li><code>NotLeaderForPartitionException</code> has been deprecated and replaced with <code>NotLeaderOrFollowerException</code>.

Fetch requests and other requests intended only for the leader or follower return NOT_LEADER_OR_FOLLOWER(6) instead of REPLICA_NOT_AVAILABLE(9)

if the broker is not a replica, ensuring that this transient error during reassignments is handled by all clients as a retriable exception.

</li>

</ul>

<h4><a id="upgrade_2_5_0" href="#upgrade_2_5_0">Upgrading to 2.5.0 from any version 0.8.x through 2.4.x</a></h4>

<p><b>If you are upgrading from a version prior to 2.1.x, please see the note below about the change to the schema used to store consumer offsets.

Once you have changed the inter.broker.protocol.version to the latest version, it will not be possible to downgrade to a version prior to 2.1.</b></p>

<p><b>For a rolling upgrade:</b></p>

<ol>

<li> Update server.properties on all brokers and add the following properties. CURRENT_KAFKA_VERSION refers to the version you

are upgrading from. CURRENT_MESSAGE_FORMAT_VERSION refers to the message format version currently in use. If you have previously

overridden the message format version, you should keep its current value. Alternatively, if you are upgrading from a version prior

to 0.11.0.x, then CURRENT_MESSAGE_FORMAT_VERSION should be set to match CURRENT_KAFKA_VERSION.

<ul>

<li>inter.broker.protocol.version=CURRENT_KAFKA_VERSION (e.g., <code>2.4</code>, <code>2.3</code>, etc.)</li>

<li>log.message.format.version=CURRENT_MESSAGE_FORMAT_VERSION (See <a href="#upgrade_10_performance_impact">potential performance impact

following the upgrade</a> for the details on what this configuration does.)</li>

</ul>

If you are upgrading from version 0.11.0.x or above, and you have not overridden the message format, then you only need to override

the inter-broker protocol version.

<ul>

<li>inter.broker.protocol.version=CURRENT_KAFKA_VERSION (e.g., <code>2.4</code>, <code>2.3</code>, etc.)</li>

</ul>

</li>

<li> Upgrade the brokers one at a time: shut down the broker, update the code, and restart it. Once you have done so, the

brokers will be running the latest version and you can verify that the cluster's behavior and performance meets expectations.

It is still possible to downgrade at this point if there are any problems.

</li>

<li> Once the cluster's behavior and performance has been verified, bump the protocol version by editing

<code>inter.broker.protocol.version</code> and setting it to <code>2.5</code>.

</li>

<li> Restart the brokers one by one for the new protocol version to take effect. Once the brokers begin using the latest

protocol version, it will no longer be possible to downgrade the cluster to an older version.

</li>

<li> If you have overridden the message format version as instructed above, then you need to do one more rolling restart to

upgrade it to its latest version. Once all (or most) consumers have been upgraded to 0.11.0 or later,

change log.message.format.version to 2.5 on each broker and restart them one by one. Note that the older Scala clients,

which are no longer maintained, do not support the message format introduced in 0.11, so to avoid conversion costs

(or to take advantage of <a href="#upgrade_11_exactly_once_semantics">exactly once semantics</a>),

the newer Java clients must be used.

</li>

<li>There are several notable changes to the reassignment tool <code>kafka-reassign-partitions.sh</code>

following the completion of

<a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-455%3A+Create+an+Administrative+API+for+Replica+Reassignment">KIP-455</a>.

This tool now requires the <code>--additional</code> flag to be provided when changing the throttle of an

active reassignment. Reassignment cancellation is now possible using the

<code>--cancel</code> command. Finally, reassignment with <code>--zookeeper</code>

has been deprecated in favor of <code>--bootstrap-server</code>. See the KIP for more detail.

</li>

</ol>

<h5 class="anchor-heading"><a id="upgrade_250_notable" class="anchor-link"></a><a href="#upgrade_250_notable">Notable changes in 2.5.0</a></h5>

<ul>

<li>When <code>RebalanceProtocol#COOPERATIVE</code> is used, <code>Consumer#poll</code> can still return data

while it is in the middle of a rebalance for those partitions still owned by the consumer; in addition

<code>Consumer#commitSync</code> now may throw a non-fatal <code>RebalanceInProgressException</code> to notify

users of such an event, in order to distinguish from the fatal <code>CommitFailedException</code> and allow

users to complete the ongoing rebalance and then reattempt committing offsets for those still-owned partitions.</li>

<li>For improved resiliency in typical network environments, the default value of

<code>zookeeper.session.timeout.ms</code> has been increased from 6s to 18s and

<code>replica.lag.time.max.ms</code> from 10s to 30s.</li>

<li>New DSL operator <code>cogroup()</code> has been added for aggregating multiple streams together at once.</li>

<li>Added a new <code>KStream.toTable()</code> API to translate an input event stream into a KTable.</li>

<li>Added a new Serde type <code>Void</code> to represent null keys or null values from input topic.</li>

<li>Deprecated <code>UsePreviousTimeOnInvalidTimestamp</code> and replaced it with <code>UsePartitionTimeOnInvalidTimeStamp</code>.</li>

<li>Improved exactly-once semantics by adding a pending offset fencing mechanism and stronger transactional commit

consistency check, which greatly simplifies the implementation of a scalable exactly-once application.

We also added a new exactly-once semantics code example under

<a href="https://github.com/apache/kafka/tree/2.5/examples">examples</a> folder. Check out

for the full details.</li>

<li>Added a new public api <code>KafkaStreams.queryMetadataForKey(String, K, Serializer) to get detailed information on the key being queried.

It provides information about the partition number where the key resides in addition to hosts containing the active and standby partitions for the key.</code></li>

<li>Provided support to query stale stores (for high availability) and the stores belonging to a specific partition by deprecating <code>KafkaStreams.store(String, QueryableStoreType)</code> and replacing it with <code>KafkaStreams.store(StoreQueryParameters)</code>.</li>

<li>Added a new public api to access lag information for stores local to an instance with <code>KafkaStreams.allLocalStorePartitionLags()</code>.</li>

<li>Scala 2.11 is no longer supported. See

for details.</li>

<li>All Scala classes from the package <code>kafka.security.auth</code> have been deprecated. See

for details of the new Java authorizer API added in 2.4.0. Note that <code>kafka.security.auth.Authorizer</code>

and <code>kafka.security.auth.SimpleAclAuthorizer</code> were deprecated in 2.4.0.

</li>

<li>TLSv1 and TLSv1.1 have been disabled by default since these have known security vulnerabilities. Only TLSv1.2 is now

enabled by default. You can continue to use TLSv1 and TLSv1.1 by explicitly enabling these in the configuration options

<code>ssl.protocol</code> and <code>ssl.enabled.protocols</code>.

</li>

<li>ZooKeeper has been upgraded to 3.5.7, and a ZooKeeper upgrade from 3.4.X to 3.5.7 can fail if there are no snapshot files in the 3.4 data directory.

This usually happens in test upgrades where ZooKeeper 3.5.7 is trying to load an existing 3.4 data dir in which no snapshot file has been created.

For more details about the issue please refer to <a href="https://issues.apache.org/jira/browse/ZOOKEEPER-3056">ZOOKEEPER-3056</a>.

A fix is given in <a href="https://issues.apache.org/jira/browse/ZOOKEEPER-3056">ZOOKEEPER-3056</a>, which is to set <code>snapshot.trust.empty=true</code>

config in <code>zookeeper.properties</code> before the upgrade.

</li>

<li>ZooKeeper version 3.5.7 supports TLS-encrypted connectivity to ZooKeeper both with or without client certificates,

and additional Kafka configurations are available to take advantage of this.

See <a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-515%3A+Enable+ZK+client+to+use+the+new+TLS+supported+authentication">KIP-515</a> for details.

</li>

</ul>

<h4><a id="upgrade_2_4_0" href="#upgrade_2_4_0">Upgrading from 0.8.x, 0.9.x, 0.10.0.x, 0.10.1.x, 0.10.2.x, 0.11.0.x, 1.0.x, 1.1.x, 2.0.x or 2.1.x or 2.2.x or 2.3.x to 2.4.0</a></h4>

<p><b>If you are upgrading from a version prior to 2.1.x, please see the note below about the change to the schema used to store consumer offsets.

Once you have changed the inter.broker.protocol.version to the latest version, it will not be possible to downgrade to a version prior to 2.1.</b></p>

<p><b>For a rolling upgrade:</b></p>

<ol>

<li> Update server.properties on all brokers and add the following properties. CURRENT_KAFKA_VERSION refers to the version you

are upgrading from. CURRENT_MESSAGE_FORMAT_VERSION refers to the message format version currently in use. If you have previously

overridden the message format version, you should keep its current value. Alternatively, if you are upgrading from a version prior

to 0.11.0.x, then CURRENT_MESSAGE_FORMAT_VERSION should be set to match CURRENT_KAFKA_VERSION.

<ul>

<li>inter.broker.protocol.version=CURRENT_KAFKA_VERSION (e.g. 0.10.0, 0.11.0, 1.0, 2.0, 2.2).</li>

<li>log.message.format.version=CURRENT_MESSAGE_FORMAT_VERSION (See <a href="#upgrade_10_performance_impact">potential performance impact

following the upgrade</a> for the details on what this configuration does.)</li>

</ul>

If you are upgrading from version 0.11.0.x or above, and you have not overridden the message format, then you only need to override

the inter-broker protocol version.

<ul>

<li>inter.broker.protocol.version=CURRENT_KAFKA_VERSION (0.11.0, 1.0, 1.1, 2.0, 2.1, 2.2, 2.3).</li>

</ul>

</li>

<li> Upgrade the brokers one at a time: shut down the broker, update the code, and restart it. Once you have done so, the

brokers will be running the latest version and you can verify that the cluster's behavior and performance meets expectations.

It is still possible to downgrade at this point if there are any problems.

</li>

<li> Once the cluster's behavior and performance has been verified, bump the protocol version by editing

<code>inter.broker.protocol.version</code> and setting it to 2.4.

</li>

<li> Restart the brokers one by one for the new protocol version to take effect. Once the brokers begin using the latest

protocol version, it will no longer be possible to downgrade the cluster to an older version.

</li>

<li> If you have overridden the message format version as instructed above, then you need to do one more rolling restart to

upgrade it to its latest version. Once all (or most) consumers have been upgraded to 0.11.0 or later,

change log.message.format.version to 2.4 on each broker and restart them one by one. Note that the older Scala clients,

which are no longer maintained, do not support the message format introduced in 0.11, so to avoid conversion costs

(or to take advantage of <a href="#upgrade_11_exactly_once_semantics">exactly once semantics</a>),

the newer Java clients must be used.

</li>

</ol>

<p><b>Additional Upgrade Notes:</b></p>

<ol>

<li>ZooKeeper has been upgraded to 3.5.6. ZooKeeper upgrade from 3.4.X to 3.5.6 can fail if there are no snapshot files in 3.4 data directory.

This usually happens in test upgrades where ZooKeeper 3.5.6 is trying to load an existing 3.4 data dir in which no snapshot file has been created.

For more details about the issue please refer to <a href="https://issues.apache.org/jira/browse/ZOOKEEPER-3056">ZOOKEEPER-3056</a>.

A fix is given in <a href="https://issues.apache.org/jira/browse/ZOOKEEPER-3056">ZOOKEEPER-3056</a>, which is to set <code>snapshot.trust.empty=true</code>

config in <code>zookeeper.properties</code> before the upgrade. But we have observed data loss in standalone cluster upgrades when using

<code>snapshot.trust.empty=true</code> config. For more details about the issue please refer to <a href="https://issues.apache.org/jira/browse/ZOOKEEPER-3644">ZOOKEEPER-3644</a>.

So we recommend the safe workaround of copying empty <a href="https://issues.apache.org/jira/secure/attachment/12928686/snapshot.0">snapshot</a> file to the 3.4 data directory,

if there are no snapshot files in 3.4 data directory. For more details about the workaround please refer to <a href="https://cwiki.apache.org/confluence/display/ZOOKEEPER/Upgrade+FAQ">ZooKeeper Upgrade FAQ</a>.

</li>

<li>

An embedded Jetty based <a href="http://zookeeper.apache.org/doc/r3.5.6/zookeeperAdmin.html#sc_adminserver">AdminServer</a> added in ZooKeeper 3.5.

AdminServer is enabled by default in ZooKeeper and is started on port 8080.

AdminServer is disabled by default in the ZooKeeper config (<code>zookeeper.properties</code>) provided by the Apache Kafka distribution.

Make sure to update your local <code>zookeeper.properties</code> file with <code>admin.enableServer=false</code> if you wish to disable the AdminServer.

Please refer <a href="http://zookeeper.apache.org/doc/r3.5.6/zookeeperAdmin.html#sc_adminserver">AdminServer config</a> to configure the AdminServer.

</li>

</ol>

<h5><a id="upgrade_240_notable" href="#upgrade_240_notable">Notable changes in 2.4.0</a></h5>

<ul>

<li>A new Admin API has been added for partition reassignments. Due to changing the way Kafka propagates reassignment information,

it is possible to lose reassignment state in failure edge cases while upgrading to the new version. It is not recommended to start reassignments while upgrading.</li>

<li>ZooKeeper has been upgraded from 3.4.14 to 3.5.6. TLS and dynamic reconfiguration are supported by the new version.</li>

<li>The <code>bin/kafka-preferred-replica-election.sh</code> command line tool has been deprecated. It has been replaced by <code>bin/kafka-leader-election.sh</code>.</li>

<li>The methods <code>electPreferredLeaders</code> in the Java <code>AdminClient</code> class have been deprecated in favor of the methods <code>electLeaders</code>.</li>

<li>Scala code leveraging the <code>NewTopic(String, int, short)</code> constructor with literal values will need to explicitly call <code>toShort</code> on the second literal.</li>

<li>The argument in the constructor <code>GroupAuthorizationException(String)</code> is now used to specify an exception message.

Previously it referred to the group that failed authorization. This was done for consistency with other exception types and to

avoid potential misuse. The constructor <code>TopicAuthorizationException(String)</code> which was previously used for a single

unauthorized topic was changed similarly.

</li>

<li>The internal <code>PartitionAssignor</code> interface has been deprecated and replaced with a new <code>ConsumerPartitionAssignor</code> in the public API. Some

methods/signatures are slightly different between the two interfaces. Users implementing a custom PartitionAssignor should migrate to the new interface as soon as possible.

</li>

<li>The <code>DefaultPartitioner</code> now uses a sticky partitioning strategy. This means that records for specific topic with null keys and no assigned partition

will be sent to the same partition until the batch is ready to be sent. When a new batch is created, a new partition is chosen. This decreases latency to produce, but

it may result in uneven distribution of records across partitions in edge cases. Generally users will not be impacted, but this difference may be noticeable in tests and

other situations producing records for a very short amount of time.

</li>

<li>The blocking <code>KafkaConsumer#committed</code> methods have been extended to allow a list of partitions as input parameters rather than a single partition.

It enables fewer request/response iterations between clients and brokers fetching for the committed offsets for the consumer group.

The old overloaded functions are deprecated and we would recommend users to make their code changes to leverage the new methods (details

can be found in <a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-520%3A+Add+overloaded+Consumer%23committed+for+batching+partitions">KIP-520</a>).

</li>

<li>We've introduced a new <code>INVALID_RECORD</code> error in the produce response to distinguish from the <code>CORRUPT_MESSAGE</code> error.

To be more concrete, previously when a batch of records were sent as part of a single request to the broker and one or more of the records failed

the validation due to various causes (mismatch magic bytes, crc checksum errors, null key for log compacted topics, etc), the whole batch would be rejected

with the same and misleading <code>CORRUPT_MESSAGE</code>, and the caller of the producer client would see the corresponding exception from either

the future object of <code>RecordMetadata</code> returned from the <code>send</code> call as well as in the <code>Callback#onCompletion(RecordMetadata metadata, Exception exception)</code>

Now with the new error code and improved error messages of the exception, producer callers would be better informed about the root cause why their sent records were failed.

</li>

<li>We are introducing incremental cooperative rebalancing to the clients' group protocol, which allows consumers to keep all of their assigned partitions during a rebalance

and at the end revoke only those which must be migrated to another consumer for overall cluster balance. The <code>ConsumerCoordinator</code> will choose the latest <code>RebalanceProtocol</code>

that is commonly supported by all of the consumer's supported assignors. You can use the new built-in <code>CooperativeStickyAssignor</code> or plug in your own custom cooperative assignor. To do

so you must implement the <code>ConsumerPartitionAssignor</code> interface and include <code>RebalanceProtocol.COOPERATIVE</code> in the list returned by <code>ConsumerPartitionAssignor#supportedProtocols</code>.

Your custom assignor can then leverage the <code>ownedPartitions</code> field in each consumer's <code>Subscription</code> to give partitions back to their previous owners whenever possible. Note that when

a partition is to be reassigned to another consumer, it <em>must</em> be removed from the new assignment until it has been revoked from its original owner. Any consumer that has to revoke a partition will trigger

a followup rebalance to allow the revoked partition to safely be assigned to its new owner. See the

<a href="https://kafka.apache.org/24/javadoc/index.html?org/apache/kafka/clients/consumer/ConsumerPartitionAssignor.RebalanceProtocol.html">ConsumerPartitionAssignor RebalanceProtocol javadocs</a> for more information.

<br>

To upgrade from the old (eager) protocol, which always revokes all partitions before rebalancing, to cooperative rebalancing, you must follow a specific upgrade path to get all clients on the same <code>ConsumerPartitionAssignor</code>

that supports the cooperative protocol. This can be done with two rolling bounces, using the <code>CooperativeStickyAssignor</code> for the example: during the first one, add "cooperative-sticky" to the list of supported assignors

for each member (without removing the previous assignor -- note that if previously using the default, you must include that explicitly as well). You then bounce and/or upgrade it.

Once the entire group is on 2.4+ and all members have the "cooperative-sticky" among their supported assignors, remove the other assignor(s) and perform a second rolling bounce so that by the end all members support only the

cooperative protocol. For further details on the cooperative rebalancing protocol and upgrade path, see <a href="https://cwiki.apache.org/confluence/x/vAclBg">KIP-429</a>.

</li>

<li>There are some behavioral changes to the <code>ConsumerRebalanceListener</code>, as well as a new API. Exceptions thrown during any of the listener's three callbacks will no longer be swallowed, and will instead be re-thrown

all the way up to the <code>Consumer.poll()</code> call. The <code>onPartitionsLost</code> method has been added to allow users to react to abnormal circumstances where a consumer may have lost ownership of its partitions

(such as a missed rebalance) and cannot commit offsets. By default, this will simply call the existing <code>onPartitionsRevoked</code> API to align with previous behavior. Note however that <code>onPartitionsLost</code> will not

be called when the set of lost partitions is empty. This means that no callback will be invoked at the beginning of the first rebalance of a new consumer joining the group.

<br>

The semantics of the <code>ConsumerRebalanceListener's</code> callbacks are further changed when following the cooperative rebalancing protocol described above. In addition to <code>onPartitionsLost</code>, <code>onPartitionsRevoked</code>

will also never be called when the set of revoked partitions is empty. The callback will generally be invoked only at the end of a rebalance, and only on the set of partitions that are being moved to another consumer. The

<code>onPartitionsAssigned</code> callback will however always be called, even with an empty set of partitions, as a way to notify users of a rebalance event (this is true for both cooperative and eager). For details on

the new callback semantics, see the <a href="https://kafka.apache.org/24/javadoc/index.html?org/apache/kafka/clients/consumer/ConsumerRebalanceListener.html">ConsumerRebalanceListener javadocs</a>.

</li>

<li>The Scala trait <code>kafka.security.auth.Authorizer</code> has been deprecated and replaced with a new Java API

<code>org.apache.kafka.server.authorizer.Authorizer</code>. The authorizer implementation class

<code>kafka.security.auth.SimpleAclAuthorizer</code> has also been deprecated and replaced with a new

implementation <code>kafka.security.authorizer.AclAuthorizer</code>. <code>AclAuthorizer</code> uses features

supported by the new API to improve authorization logging and is compatible with <code>SimpleAclAuthorizer</code>.

For more details, see <a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-504+-+Add+new+Java+Authorizer+Interface">KIP-504</a>.

</li>

</ul>

<h4><a id="upgrade_2_3_0" href="#upgrade_2_3_0">Upgrading from 0.8.x, 0.9.x, 0.10.0.x, 0.10.1.x, 0.10.2.x, 0.11.0.x, 1.0.x, 1.1.x, 2.0.x or 2.1.x or 2.2.x to 2.3.0</a></h4>

<p><b>If you are upgrading from a version prior to 2.1.x, please see the note below about the change to the schema used to store consumer offsets.

Once you have changed the inter.broker.protocol.version to the latest version, it will not be possible to downgrade to a version prior to 2.1.</b></p>

<p><b>For a rolling upgrade:</b></p>

<ol>

<li> Update server.properties on all brokers and add the following properties. CURRENT_KAFKA_VERSION refers to the version you

are upgrading from. CURRENT_MESSAGE_FORMAT_VERSION refers to the message format version currently in use. If you have previously

overridden the message format version, you should keep its current value. Alternatively, if you are upgrading from a version prior

to 0.11.0.x, then CURRENT_MESSAGE_FORMAT_VERSION should be set to match CURRENT_KAFKA_VERSION.

<ul>

<li>inter.broker.protocol.version=CURRENT_KAFKA_VERSION (e.g. 0.8.2, 0.9.0, 0.10.0, 0.10.1, 0.10.2, 0.11.0, 1.0, 1.1).</li>

<li>log.message.format.version=CURRENT_MESSAGE_FORMAT_VERSION (See <a href="#upgrade_10_performance_impact">potential performance impact

following the upgrade</a> for the details on what this configuration does.)</li>

</ul>

If you are upgrading from 0.11.0.x, 1.0.x, 1.1.x, 2.0.x, or 2.1.x, and you have not overridden the message format, then you only need to override

the inter-broker protocol version.

<ul>

<li>inter.broker.protocol.version=CURRENT_KAFKA_VERSION (0.11.0, 1.0, 1.1, 2.0, 2.1, 2.2).</li>

</ul>

</li>

<li> Upgrade the brokers one at a time: shut down the broker, update the code, and restart it. Once you have done so, the

brokers will be running the latest version and you can verify that the cluster's behavior and performance meets expectations.

It is still possible to downgrade at this point if there are any problems.

</li>

<li> Once the cluster's behavior and performance has been verified, bump the protocol version by editing

<code>inter.broker.protocol.version</code> and setting it to 2.3.

</li>

<li> Restart the brokers one by one for the new protocol version to take effect. Once the brokers begin using the latest

protocol version, it will no longer be possible to downgrade the cluster to an older version.

</li>

<li> If you have overridden the message format version as instructed above, then you need to do one more rolling restart to

upgrade it to its latest version. Once all (or most) consumers have been upgraded to 0.11.0 or later,

change log.message.format.version to 2.3 on each broker and restart them one by one. Note that the older Scala clients,

which are no longer maintained, do not support the message format introduced in 0.11, so to avoid conversion costs

(or to take advantage of <a href="#upgrade_11_exactly_once_semantics">exactly once semantics</a>),

the newer Java clients must be used.

</li>

</ol>

<h5><a id="upgrade_230_notable" href="#upgrade_230_notable">Notable changes in 2.3.0</a></h5>

<ul>

<li> We are introducing a new rebalancing protocol for Kafka Connect based on

<a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-415%3A+Incremental+Cooperative+Rebalancing+in+Kafka+Connect">incremental cooperative rebalancing</a>.

The new protocol does not require stopping all the tasks during a rebalancing phase between Connect workers. Instead, only the tasks that need to be exchanged

between workers are stopped and they are started in a follow up rebalance. The new Connect protocol is enabled by default beginning with 2.3.0.

For more details on how it works and how to enable the old behavior of eager rebalancing, checkout

<a href="/{{version}}/documentation/#connect_administration">incremental cooperative rebalancing design</a>.

</li>

<li> We are introducing static membership towards consumer user. This feature reduces unnecessary rebalances during normal application upgrades or rolling bounces.

For more details on how to use it, checkout <a href="/{{version}}/documentation/#static_membership">static membership design</a>.

</li>

<li> Kafka Streams DSL switches its used store types. While this change is mainly transparent to users, there are some corner cases that may require code changes.

See the <a href="/{{version}}/documentation/streams/upgrade-guide#streams_api_changes_230">Kafka Streams upgrade section</a> for more details.

</li>

<li>Kafka Streams 2.3.0 requires 0.11 message format or higher and does not work with older message format.</li>

</ul>

<h4><a id="upgrade_2_2_0" href="#upgrade_2_2_0">Upgrading from 0.8.x, 0.9.x, 0.10.0.x, 0.10.1.x, 0.10.2.x, 0.11.0.x, 1.0.x, 1.1.x, 2.0.x or 2.1.x to 2.2.0</a></h4>

<p><b>If you are upgrading from a version prior to 2.1.x, please see the note below about the change to the schema used to stor C8D5 e consumer offsets.

Once you have changed the inter.broker.protocol.version to the latest version, it will not be possible to downgrade to a version prior to 2.1.</b></p>

<p><b>For a rolling upgrade:</b></p>

<ol>

<li> Update server.properties on all brokers and add the following properties. CURRENT_KAFKA_VERSION refers to the version you

are upgrading from. CURRENT_MESSAGE_FORMAT_VERSION refers to the message format version currently in use. If you have previously

overridden the message format version, you should keep its current value. Alternatively, if you are upgrading from a version prior

to 0.11.0.x, then CURRENT_MESSAGE_FORMAT_VERSION should be set to match CURRENT_KAFKA_VERSION.

<ul>

<li>inter.broker.protocol.version=CURRENT_KAFKA_VERSION (e.g. 0.8.2, 0.9.0, 0.10.0, 0.10.1, 0.10.2, 0.11.0, 1.0, 1.1).</li>

<li>log.message.format.version=CURRENT_MESSAGE_FORMAT_VERSION (See <a href="#upgrade_10_performance_impact">potential performance impact

following the upgrade</a> for the details on what this configuration does.)</li>

</ul>

If you are upgrading from 0.11.0.x, 1.0.x, 1.1.x, or 2.0.x and you have not overridden the message format, then you only need to override

the inter-broker protocol version.

<ul>

<li>inter.broker.protocol.version=CURRENT_KAFKA_VERSION (0.11.0, 1.0, 1.1, 2.0).</li>

</ul>

</li>

<li> Upgrade the brokers one at a time: shut down the broker, update the code, and restart it. Once you have done so, the

brokers will be running the latest version and you can verify that the cluster's behavior and performance meets expectations.

It is still possible to downgrade at this point if there are any problems.

</li>

<li> Once the cluster's behavior and performance has been verified, bump the protocol version by editing

<code>inter.broker.protocol.version</code> and setting it to 2.2.

</li>

<li> Restart the brokers one by one for the new protocol version to take effect. Once the brokers begin using the latest

protocol version, it will no longer be possible to downgrade the cluster to an older version.

</li>

<li> If you have overridden the message format version as instructed above, then you need to do one more rolling restart to

upgrade it to its latest version. Once all (or most) consumers have been upgraded to 0.11.0 or later,

change log.message.format.version to 2.2 on each broker and restart them one by one. Note that the older Scala clients,

which are no longer maintained, do not support the message format introduced in 0.11, so to avoid conversion costs

(or to take advantage of <a href="#upgrade_11_exactly_once_semantics">exactly once semantics</a>),

the newer Java clients must be used.

</li>

</ol>

<h5><a id="upgrade_221_notable" href="#upgrade_221_notable">Notable changes in 2.2.1</a></h5>

<ul>

<li>Kafka Streams 2.2.1 requires 0.11 message format or higher and does not work with older message format.</li>

</ul>

<h5><a id="upgrade_220_notable" href="#upgrade_220_notable">Notable changes in 2.2.0</a></h5>

<ul>

<li>The default consumer group id has been changed from the empty string (<code>""</code>) to <code>null</code>. Consumers who use the new default group id will not be able to subscribe to topics,

and fetch or commit offsets. The empty string as consumer group id is deprecated but will be supported until a future major release. Old clients that rely on the empty string group id will now

have to explicitly provide it as part of their consumer config. For more information see

<li>The <code>bin/kafka-topics.sh</code> command line tool is now able to connect directly to brokers with <code>--bootstrap-server</code> instead of zookeeper. The old <code>--zookeeper</code>

option is still available for now. Please read <a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-377%3A+TopicCommand+to+use+AdminClient">KIP-377</a> for more information.</li>

<li>Kafka Streams depends on a newer version of RocksDBs that requires MacOS 10.13 or higher.</li>

</ul>

<h4><a id="upgrade_2_1_0" href="#upgrade_2_1_0">Upgrading from 0.8.x, 0.9.x, 0.10.0.x, 0.10.1.x, 0.10.2.x, 0.11.0.x, 1.0.x, 1.1.x, or 2.0.0 to 2.1.0</a></h4>

<p><b>Note that 2.1.x contains a change to the internal schema used to store consumer offsets. Once the upgrade is

complete, it will not be possible to downgrade to previous versions. See the rolling upgrade notes below for more detail.</b></p>

<p><b>For a rolling upgrade:</b></p>

<ol>

<li> Update server.properties on all brokers and add the following properties. CURRENT_KAFKA_VERSION refers to the version you

are upgrading from. CURRENT_MESSAGE_FORMAT_VERSION refers to the message format version currently in use. If you have previously

overridden the message format version, you should keep its current value. Alternatively, if you are upgrading from a version prior

to 0.11.0.x, then CURRENT_MESSAGE_FORMAT_VERSION should be set to match CURRENT_KAFKA_VERSION.

<ul>

<li>inter.broker.protocol.version=CURRENT_KAFKA_VERSION (e.g. 0.8.2, 0.9.0, 0.10.0, 0.10.1, 0.10.2, 0.11.0, 1.0, 1.1).</li>

<li>log.message.format.version=CURRENT_MESSAGE_FORMAT_VERSION (See <a href="#upgrade_10_performance_impact">potential performance impact

following the upgrade</a> for the details on what this configuration does.)</li>

</ul>

If you are upgrading from 0.11.0.x, 1.0.x, 1.1.x, or 2.0.x and you have not overridden the message format, then you only need to override

the inter-broker protocol version.

<ul>

<li>inter.broker.protocol.version=CURRENT_KAFKA_VERSION (0.11.0, 1.0, 1.1, 2.0).</li>

</ul>

</li>

<li> Upgrade the brokers one at a time: shut down the broker, update the code, and restart it. Once you have done so, the

brokers will be running the latest version and you can verify that the cluster's behavior and performance meets expectations.

It is still possible to downgrade at this point if there are any problems.

</li>

<li> Once the cluster's behavior and performance has been verified, bump the protocol version by editing

<code>inter.broker.protocol.version</code> and setting it to 2.1.

</li>

<li> Restart the brokers one by one for the new protocol version to take effect. Once the brokers begin using the latest

protocol version, it will no longer be possible to downgrade the cluster to an older version.

</li>

<li> If you have overridden the message format version as instructed above, then you need to do one more rolling restart to

upgrade it to its latest version. Once all (or most) consumers have been upgraded to 0.11.0 or later,

change log.message.format.version to 2.1 on each broker and restart them one by one. Note that the older Scala clients,

which are no longer maintained, do not support the message format introduced in 0.11, so to avoid conversion costs

(or to take advantage of <a href="#upgrade_11_exactly_once_semantics">exactly once semantics</a>),

the newer Java clients must be used.

</li>

</ol>

<p><b>Additional Upgrade Notes:</b></p>

<ol>

<li>Offset expiration semantics has slightly changed in this version. According to the new semantics, offsets of partitions in a group will

not be removed while the group is subscribed to the corresponding topic and is still active (has active consumers). If group becomes

empty all its offsets will be removed after default offset retention period (or the one set by broker) has passed (unless the group becomes

active again). Offsets associated with standalone (simple) consumers, that do not use Kafka group management, will be removed after default

offset retention period (or the one set by broker) has passed since their last commit.</li>

<li>The default for console consumer's <code>enable.auto.commit</code> property when no <code>group.id</code> is provided is now set to <code>false</code>.

This is to avoid polluting the consumer coordinator cache as the auto-generated group is not likely to be used by other consumers.</li>

<li>The default value for the producer's <code>retries</code> config was changed to <code>Integer.MAX_VALUE</code>, as we introduced <code>delivery.timeout.ms</code>

in <a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-91+Provide+Intuitive+User+Timeouts+in+The+Producer">KIP-91</a>,

which sets an upper bound on the total time between sending a record and receiving acknowledgement from the broker. By default,

the delivery timeout is set to 2 minutes.</li>

<li>By default, MirrorMaker now overrides <code>delivery.timeout.ms</code> to <code>Integer.MAX_VALUE</code> when

configuring the producer. If you have overridden the value of <code>retries</code> in order to fail faster,

you will instead need to override <code>delivery.timeout.ms</code>.</li>

<li>The <code>ListGroup</code> API now expects, as a recommended alternative, <code>Describe Group</code> access to the groups a user should be able to list.

Even though the old <code>Describe Cluster</code> access is still supported for backward compatibility, using it for this API is not advised.</li>

<li><a href="https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=87298242">KIP-336</a> deprecates the ExtendedSerializer and ExtendedDeserializer interfaces and

propagates the usage of Serializer and Deserializer. ExtendedSerializer and ExtendedDeserializer were introduced with

<a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-82+-+Add+Record+Headers">KIP-82</a> to provide record headers for serializers and deserializers

in a Java 7 compatible fashion. Now we consolidated these interfaces as Java 7 support has been dropped since.</li>

</ol>

<h5><a id="upgrade_210_notable" href="#upgrade_210_notable">Notable changes in 2.1.0</a></h5>

<ul>

<li>Jetty has been upgraded to 9.4.12, which excludes TLS_RSA_* ciphers by default because they do not support forward

View remainder of file in raw view

FilesExpand file tree

upgrade.html

Latest commit

History

upgrade.html

File metadata and controls