mimaison commented on code in PR #16650:
URL: https://github.com/apache/kafka/pull/16650#discussion_r1689472940
##########
docs/ops.html:
##########
@@ -1214,9 +1214,9 @@ <h4 class="anchor-heading"><a id="multitenancy-more"
class="anchor-link"></a><a
</p>
- <h3 class="anchor-heading"><a id="config" class="anchor-link"></a><a
href="#config">6.5 Kafka Configuration</a></h3>
Review Comment:
Should we undo the changes in this file as the new section is in
`configuration.html`? If so let's undo the changes in `toc.html` too.
##########
docs/toc.html:
##########
@@ -50,13 +50,23 @@
<li><a href="#sourceconnectconfigs">Source Connector
Configs</a>
<li><a href="#sinkconnectconfigs">Sink Connector
Configs</a>
</ul>
+
<li><a href="#streamsconfigs">3.6 Kafka Streams Configs</a>
<li><a href="#adminclientconfigs">3.7 AdminClient Configs</a>
<li><a href="#mirrormakerconfigs">3.8 MirrorMaker Configs</a>
<li><a href="#systemproperties">3.9 System Properties</a>
- <li><a href="#tieredstorageconfigs">3.10 Tiered Storage
Configs</a></li>
+ <li><a href="#tieredstorageconfigs">3.10 Tiered Storage
Configs</a>
+ <li><a href="#config_providers">3.11 Configuration
Providers</a>
+ <ul>
+ <li><a href="#using_providers">Using</a>
Review Comment:
Should this be `Using Configuration Providers` to match the section title?
##########
docs/configuration.html:
##########
@@ -308,6 +308,157 @@ <h3 class="anchor-heading"><a id="tieredstorageconfigs"
class="anchor-link"></a>
Below are the configuration properties for Tiered Storage.
<!--#include virtual="generated/remote_log_manager_config.html" -->
<!--#include virtual="generated/remote_log_metadata_manager_config.html" -->
-</script>
+ <h3 class="anchor-heading">
+ <a id="config_providers" class="anchor-link"></a>
+ <a href="#config_providers">3.11 Configuration Providers</a>
+ </h3>
+
+ <p>
+ Use configuration providers to load configuration data from external
sources. This might include sensitive information, such as passwords, API keys,
or other credentials.
+ </p>
+
+ <p>You have the following options:</p>
+ <ul>
+ <li>
+ Use a custom provider by creating a class implementing the
+ <a
href="/{{version}}/javadoc/org/apache/kafka/common/config/provider/ConfigProvider.html"><code>ConfigProvider</code></a>
interface and packaging it into a JAR file.
+ </li>
+ <li>Use a built-in provider:</li>
+ <ul>
+ <li><a
href="/{{version}}/javadoc/org/apache/kafka/common/config/provider/DirectoryConfigProvider.html"><code>DirectoryConfigProvider</code></a></li>
+ <li><a
href="/{{version}}/javadoc/org/apache/kafka/common/config/provider/EnvVarConfigProvider.html"><code>EnvVarConfigProvider</code></a></li>
+ <li><a
href="/{{version}}/javadoc/org/apache/kafka/common/config/provider/FileConfigProvider.html"><code>FileConfigProvider</code></a></li>
+ </ul>
+ </ul>
+
+ <p>
+ To use a configuration provider, specify it in your configuration using
the <code>config.providers</code> property.
+ </p>
+
+ <h4 class="anchor-heading">
+ <a id="using_providers" class="anchor-link"></a>
+ <a href="#using_providers">Using Configuration Providers</a>
+ </h4>
+
+ <p>All configuration providers share a common way of passing parameters.</p>
+ <p>
+ You specify a comma-separated list of aliases and the fully-specified
class that implements the configuration provider:
+ </p>
+ <pre><code>config.providers=provider1,provider2
+config.providers.provider1.class=com.example.Provider1
+config.providers.provider2.class=com.example.Provider2</code></pre>
+ <p>
+ Each provider can have its own set of parameters, which are passed in a
specific format.
+ </p>
+ <p>
+ Pass additional parameters to each provider as
<code>config.providers.<provider_alias>.param.<name>=<value></code>:
+ </p>
+ <pre><code>config.providers.provider1.param.param1=value1
+config.providers.provider1.param.param2=value2</code></pre>
+
+ <h4 class="anchor-heading">
+ <a id="config_provider" class="anchor-link"></a>
+ <a href="#config_provider">ConfigProvider</a>
Review Comment:
This section only repeats things already mentioned above. Do we really need
it?
##########
docs/configuration.html:
##########
@@ -308,6 +308,157 @@ <h3 class="anchor-heading"><a id="tieredstorageconfigs"
class="anchor-link"></a>
Below are the configuration properties for Tiered Storage.
<!--#include virtual="generated/remote_log_manager_config.html" -->
<!--#include virtual="generated/remote_log_metadata_manager_config.html" -->
-</script>
+ <h3 class="anchor-heading">
+ <a id="config_providers" class="anchor-link"></a>
+ <a href="#config_providers">3.11 Configuration Providers</a>
+ </h3>
+
+ <p>
+ Use configuration providers to load configuration data from external
sources. This might include sensitive information, such as passwords, API keys,
or other credentials.
+ </p>
+
+ <p>You have the following options:</p>
+ <ul>
+ <li>
+ Use a custom provider by creating a class implementing the
+ <a
href="/{{version}}/javadoc/org/apache/kafka/common/config/provider/ConfigProvider.html"><code>ConfigProvider</code></a>
interface and packaging it into a JAR file.
+ </li>
+ <li>Use a built-in provider:</li>
+ <ul>
+ <li><a
href="/{{version}}/javadoc/org/apache/kafka/common/config/provider/DirectoryConfigProvider.html"><code>DirectoryConfigProvider</code></a></li>
+ <li><a
href="/{{version}}/javadoc/org/apache/kafka/common/config/provider/EnvVarConfigProvider.html"><code>EnvVarConfigProvider</code></a></li>
+ <li><a
href="/{{version}}/javadoc/org/apache/kafka/common/config/provider/FileConfigProvider.html"><code>FileConfigProvider</code></a></li>
+ </ul>
+ </ul>
+
+ <p>
+ To use a configuration provider, specify it in your configuration using
the <code>config.providers</code> property.
+ </p>
+
+ <h4 class="anchor-heading">
+ <a id="using_providers" class="anchor-link"></a>
+ <a href="#using_providers">Using Configuration Providers</a>
+ </h4>
+
+ <p>All configuration providers share a common way of passing parameters.</p>
+ <p>
+ You specify a comma-separated list of aliases and the fully-specified
class that implements the configuration provider:
+ </p>
+ <pre><code>config.providers=provider1,provider2
+config.providers.provider1.class=com.example.Provider1
+config.providers.provider2.class=com.example.Provider2</code></pre>
+ <p>
+ Each provider can have its own set of parameters, which are passed in a
specific format.
+ </p>
+ <p>
+ Pass additional parameters to each provider as
<code>config.providers.<provider_alias>.param.<name>=<value></code>:
+ </p>
+ <pre><code>config.providers.provider1.param.param1=value1
+config.providers.provider1.param.param2=value2</code></pre>
+
+ <h4 class="anchor-heading">
+ <a id="config_provider" class="anchor-link"></a>
+ <a href="#config_provider">ConfigProvider</a>
+ </h4>
+
+ <p>
+ The <code>ConfigProvider</code> is a base interface for all configuration
providers. Custom implementations of this interface can retrieve configuration
data from various sources.
+ You can then package the implementation as a JAR file, add the JAR to your
classpath, and reference the provider's class in your configuration.
+ </p>
+
+ <p><b>Example custom provider configuration</b></p>
+ <pre><code>config.providers=customProvider
+config.providers.customProvider.class=com.example.customProvider
+config.providers.customProvider.param.param1=value1
+config.providers.customProvider.param.param2=value2</code></pre>
+
+ <h4 class="anchor-heading">
+ <a id="directory_config_provider" class="anchor-link"></a>
+ <a href="#directory_config_provider">DirectoryConfigProvider</a>
+ </h4>
+
+ <p>
+ The <code>DirectoryConfigProvider</code> retrieves configuration data from
files stored in a specified directory.
+ </p>
+ <p>
+ Each file represents a key, and its content is the value. You specify the
path to the directory where the configuration files are stored using the
<code>dir</code> parameter. This provider is useful for loading multiple
configuration files and for organizing configuration data into separate files.
+ </p>
+
+ <p><b>Example <code>DirectoryConfigProvider</code> configuration</b></p>
+ <pre><code>config.providers=dirProvider
+config.providers.dirProvider.class=org.apache.kafka.common.config.provider.DirectoryConfigProvider
+config.providers.dirProvider.param.dir=/path/to/config/directory</code></pre>
+
+ <h4 class="anchor-heading">
+ <a id="env_var_config_provider" class="anchor-link"></a>
+ <a href="#env_var_config_provider">EnvVarConfigProvider</a>
+ </h4>
+
+ <p>The <code>EnvVarConfigProvider</code> retrieves configuration data from
environment variables.</p>
+ <p>No specific parameters are required, as it reads directly from the
specified environment variables.</p>
+ <p>
+ This provider is useful for configuring applications running in
containers, for example, to load certificates or JAAS configuration from
environment variables mapped from secrets.
+ </p>
+
+ <p><b>Example <code>EnvVarConfigProvider</code> configuration</b></p>
+ <pre><code>config.providers=envVarProvider
+config.providers.envVarProvider.class=org.apache.kafka.common.config.provider.EnvVarConfigProvider</code></pre>
+
+ <h4 class="anchor-heading">
+ <a id="file_config_provider" class="anchor-link"></a>
+ <a href="#file_config_provider">FileConfigProvider</a>
+ </h4>
+
+ <p>The <code>FileConfigProvider</code> retrieves configuration data from a
single properties file.</p>
+ <p>This provider is useful for loading configuration data from mounted
files.</p>
+
+ <p><b>Example <code>FileConfigProvider</code> configuration</b></p>
+ <pre><code>config.providers=fileProvider
+config.providers.fileProvider.class=org.apache.kafka.common.config.provider.FileConfigProvider</code></pre>
+
+ <h4 class="anchor-heading">
+ <a id="ref_config_provider" class="anchor-link"></a>
+ <a href="#ref_config_provider">Referencing configuration providers</a>
+ </h4>
+ <p>
+ To reference a property value supplied by a configuration provider, use
the correct placeholder syntax:
+ </p>
+ <dl>
+ <dt><b><code>DirectoryConfigProvider</code></b></dt>
+ <dd>
+ <p><code>${dirProvider:<path_to_file>:<file_name>}</code></p>
+ </dd>
+ <dt><code><b>EnvVarConfigProvider</code></b></dt>
+ <dd>
+ <p><code>${envVarProvider:<enVar_name>}</code></p>
+ </dd>
+
+ <dt><b><code>FileConfigProvider</code></b></dt>
+ <dd>
+
<p><code>${fileProvider:<path_and_filename>:<property>}</code></p>
+ </dd>
+ </dl>
+ <p>
+ Here’s an example that uses a file configuration provider with Kafka
Connect to provide authentication credentials to a database for a connector.
+ </p>
+ <p>
+ First, create a <code>connector-credentials.properties</code>
configuration file with the following credentials:
+ </p>
+ <pre><code>dbUsername=my-username
+dbPassword=my-password</code></pre>
+ <p>
+ Specify a <code>FileConfigProvider</code> in the Kafka Connect
configuration:
+ </p>
+
+ <p><b>Example Kafka Connect configuration with a
<code>FileConfigProvider</code></b></p>
+ <pre><code>config.providers=fileProvider
+config.providers.fileProvider.class=org.apache.kafka.common.config.provider.FileConfigProvider</code></pre>
+
+ <p>Next, reference the properties from the file in the connector
configuration.</p>
+ <p><b>Example connector configuration referencing file properties</b></p>
+
<pre><code>database.user=${fileProvider:/path/to/connector-credentials.properties:dbUsername}
+database.password=${fileProvider:/path/to/connector-credentials.properties:dbPassword}</code></pre>
+ <p>The configuration provider reads and extracts the values from the
properties file.</p>
Review Comment:
I wonder if we add `At runtime,` at the beginning of the sentence.
##########
docs/configuration.html:
##########
@@ -308,6 +308,157 @@ <h3 class="anchor-heading"><a id="tieredstorageconfigs"
class="anchor-link"></a>
Below are the configuration properties for Tiered Storage.
<!--#include virtual="generated/remote_log_manager_config.html" -->
<!--#include virtual="generated/remote_log_metadata_manager_config.html" -->
-</script>
+ <h3 class="anchor-heading">
+ <a id="config_providers" class="anchor-link"></a>
+ <a href="#config_providers">3.11 Configuration Providers</a>
+ </h3>
+
+ <p>
+ Use configuration providers to load configuration data from external
sources. This might include sensitive information, such as passwords, API keys,
or other credentials.
+ </p>
+
+ <p>You have the following options:</p>
+ <ul>
+ <li>
+ Use a custom provider by creating a class implementing the
+ <a
href="/{{version}}/javadoc/org/apache/kafka/common/config/provider/ConfigProvider.html"><code>ConfigProvider</code></a>
interface and packaging it into a JAR file.
+ </li>
+ <li>Use a built-in provider:</li>
+ <ul>
+ <li><a
href="/{{version}}/javadoc/org/apache/kafka/common/config/provider/DirectoryConfigProvider.html"><code>DirectoryConfigProvider</code></a></li>
+ <li><a
href="/{{version}}/javadoc/org/apache/kafka/common/config/provider/EnvVarConfigProvider.html"><code>EnvVarConfigProvider</code></a></li>
+ <li><a
href="/{{version}}/javadoc/org/apache/kafka/common/config/provider/FileConfigProvider.html"><code>FileConfigProvider</code></a></li>
+ </ul>
+ </ul>
+
+ <p>
+ To use a configuration provider, specify it in your configuration using
the <code>config.providers</code> property.
+ </p>
+
+ <h4 class="anchor-heading">
+ <a id="using_providers" class="anchor-link"></a>
+ <a href="#using_providers">Using Configuration Providers</a>
+ </h4>
+
+ <p>All configuration providers share a common way of passing parameters.</p>
+ <p>
+ You specify a comma-separated list of aliases and the fully-specified
class that implements the configuration provider:
+ </p>
+ <pre><code>config.providers=provider1,provider2
+config.providers.provider1.class=com.example.Provider1
+config.providers.provider2.class=com.example.Provider2</code></pre>
+ <p>
+ Each provider can have its own set of parameters, which are passed in a
specific format.
+ </p>
+ <p>
+ Pass additional parameters to each provider as
<code>config.providers.<provider_alias>.param.<name>=<value></code>:
+ </p>
+ <pre><code>config.providers.provider1.param.param1=value1
+config.providers.provider1.param.param2=value2</code></pre>
+
+ <h4 class="anchor-heading">
+ <a id="config_provider" class="anchor-link"></a>
+ <a href="#config_provider">ConfigProvider</a>
+ </h4>
+
+ <p>
+ The <code>ConfigProvider</code> is a base interface for all configuration
providers. Custom implementations of this interface can retrieve configuration
data from various sources.
+ You can then package the implementation as a JAR file, add the JAR to your
classpath, and reference the provider's class in your configuration.
+ </p>
+
+ <p><b>Example custom provider configuration</b></p>
+ <pre><code>config.providers=customProvider
+config.providers.customProvider.class=com.example.customProvider
+config.providers.customProvider.param.param1=value1
+config.providers.customProvider.param.param2=value2</code></pre>
+
+ <h4 class="anchor-heading">
+ <a id="directory_config_provider" class="anchor-link"></a>
+ <a href="#directory_config_provider">DirectoryConfigProvider</a>
+ </h4>
+
+ <p>
+ The <code>DirectoryConfigProvider</code> retrieves configuration data from
files stored in a specified directory.
+ </p>
+ <p>
+ Each file represents a key, and its content is the value. You specify the
path to the directory where the configuration files are stored using the
<code>dir</code> parameter. This provider is useful for loading multiple
configuration files and for organizing configuration data into separate files.
+ </p>
+
+ <p><b>Example <code>DirectoryConfigProvider</code> configuration</b></p>
+ <pre><code>config.providers=dirProvider
+config.providers.dirProvider.class=org.apache.kafka.common.config.provider.DirectoryConfigProvider
+config.providers.dirProvider.param.dir=/path/to/config/directory</code></pre>
+
+ <h4 class="anchor-heading">
+ <a id="env_var_config_provider" class="anchor-link"></a>
+ <a href="#env_var_config_provider">EnvVarConfigProvider</a>
+ </h4>
+
+ <p>The <code>EnvVarConfigProvider</code> retrieves configuration data from
environment variables.</p>
+ <p>No specific parameters are required, as it reads directly from the
specified environment variables.</p>
+ <p>
+ This provider is useful for configuring applications running in
containers, for example, to load certificates or JAAS configuration from
environment variables mapped from secrets.
+ </p>
+
+ <p><b>Example <code>EnvVarConfigProvider</code> configuration</b></p>
+ <pre><code>config.providers=envVarProvider
+config.providers.envVarProvider.class=org.apache.kafka.common.config.provider.EnvVarConfigProvider</code></pre>
+
+ <h4 class="anchor-heading">
+ <a id="file_config_provider" class="anchor-link"></a>
+ <a href="#file_config_provider">FileConfigProvider</a>
Review Comment:
This provider has a configurable parameter. I think it would be nice to
mention it here:
https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/common/config/provider/FileConfigProvider.java#L45-L47
##########
docs/configuration.html:
##########
@@ -308,6 +308,157 @@ <h3 class="anchor-heading"><a id="tieredstorageconfigs"
class="anchor-link"></a>
Below are the configuration properties for Tiered Storage.
<!--#include virtual="generated/remote_log_manager_config.html" -->
<!--#include virtual="generated/remote_log_metadata_manager_config.html" -->
-</script>
+ <h3 class="anchor-heading">
+ <a id="config_providers" class="anchor-link"></a>
+ <a href="#config_providers">3.11 Configuration Providers</a>
+ </h3>
+
+ <p>
+ Use configuration providers to load configuration data from external
sources. This might include sensitive information, such as passwords, API keys,
or other credentials.
+ </p>
+
+ <p>You have the following options:</p>
+ <ul>
+ <li>
+ Use a custom provider by creating a class implementing the
+ <a
href="/{{version}}/javadoc/org/apache/kafka/common/config/provider/ConfigProvider.html"><code>ConfigProvider</code></a>
interface and packaging it into a JAR file.
+ </li>
+ <li>Use a built-in provider:</li>
+ <ul>
+ <li><a
href="/{{version}}/javadoc/org/apache/kafka/common/config/provider/DirectoryConfigProvider.html"><code>DirectoryConfigProvider</code></a></li>
+ <li><a
href="/{{version}}/javadoc/org/apache/kafka/common/config/provider/EnvVarConfigProvider.html"><code>EnvVarConfigProvider</code></a></li>
+ <li><a
href="/{{version}}/javadoc/org/apache/kafka/common/config/provider/FileConfigProvider.html"><code>FileConfigProvider</code></a></li>
+ </ul>
+ </ul>
+
+ <p>
+ To use a configuration provider, specify it in your configuration using
the <code>config.providers</code> property.
+ </p>
+
+ <h4 class="anchor-heading">
+ <a id="using_providers" class="anchor-link"></a>
+ <a href="#using_providers">Using Configuration Providers</a>
+ </h4>
+
+ <p>All configuration providers share a common way of passing parameters.</p>
+ <p>
+ You specify a comma-separated list of aliases and the fully-specified
class that implements the configuration provider:
+ </p>
+ <pre><code>config.providers=provider1,provider2
+config.providers.provider1.class=com.example.Provider1
+config.providers.provider2.class=com.example.Provider2</code></pre>
+ <p>
+ Each provider can have its own set of parameters, which are passed in a
specific format.
+ </p>
+ <p>
+ Pass additional parameters to each provider as
<code>config.providers.<provider_alias>.param.<name>=<value></code>:
+ </p>
+ <pre><code>config.providers.provider1.param.param1=value1
+config.providers.provider1.param.param2=value2</code></pre>
+
+ <h4 class="anchor-heading">
+ <a id="config_provider" class="anchor-link"></a>
+ <a href="#config_provider">ConfigProvider</a>
+ </h4>
+
+ <p>
+ The <code>ConfigProvider</code> is a base interface for all configuration
providers. Custom implementations of this interface can retrieve configuration
data from various sources.
+ You can then package the implementation as a JAR file, add the JAR to your
classpath, and reference the provider's class in your configuration.
+ </p>
+
+ <p><b>Example custom provider configuration</b></p>
+ <pre><code>config.providers=customProvider
+config.providers.customProvider.class=com.example.customProvider
+config.providers.customProvider.param.param1=value1
+config.providers.customProvider.param.param2=value2</code></pre>
+
+ <h4 class="anchor-heading">
+ <a id="directory_config_provider" class="anchor-link"></a>
+ <a href="#directory_config_provider">DirectoryConfigProvider</a>
+ </h4>
+
+ <p>
+ The <code>DirectoryConfigProvider</code> retrieves configuration data from
files stored in a specified directory.
+ </p>
+ <p>
+ Each file represents a key, and its content is the value. You specify the
path to the directory where the configuration files are stored using the
<code>dir</code> parameter. This provider is useful for loading multiple
configuration files and for organizing configuration data into separate files.
+ </p>
+
+ <p><b>Example <code>DirectoryConfigProvider</code> configuration</b></p>
+ <pre><code>config.providers=dirProvider
+config.providers.dirProvider.class=org.apache.kafka.common.config.provider.DirectoryConfigProvider
+config.providers.dirProvider.param.dir=/path/to/config/directory</code></pre>
Review Comment:
This is not correct. The path is only provided when using the provider.
##########
docs/configuration.html:
##########
@@ -308,6 +308,157 @@ <h3 class="anchor-heading"><a id="tieredstorageconfigs"
class="anchor-link"></a>
Below are the configuration properties for Tiered Storage.
<!--#include virtual="generated/remote_log_manager_config.html" -->
<!--#include virtual="generated/remote_log_metadata_manager_config.html" -->
-</script>
+ <h3 class="anchor-heading">
+ <a id="config_providers" class="anchor-link"></a>
+ <a href="#config_providers">3.11 Configuration Providers</a>
+ </h3>
+
+ <p>
+ Use configuration providers to load configuration data from external
sources. This might include sensitive information, such as passwords, API keys,
or other credentials.
+ </p>
+
+ <p>You have the following options:</p>
+ <ul>
+ <li>
+ Use a custom provider by creating a class implementing the
+ <a
href="/{{version}}/javadoc/org/apache/kafka/common/config/provider/ConfigProvider.html"><code>ConfigProvider</code></a>
interface and packaging it into a JAR file.
+ </li>
+ <li>Use a built-in provider:</li>
+ <ul>
+ <li><a
href="/{{version}}/javadoc/org/apache/kafka/common/config/provider/DirectoryConfigProvider.html"><code>DirectoryConfigProvider</code></a></li>
+ <li><a
href="/{{version}}/javadoc/org/apache/kafka/common/config/provider/EnvVarConfigProvider.html"><code>EnvVarConfigProvider</code></a></li>
+ <li><a
href="/{{version}}/javadoc/org/apache/kafka/common/config/provider/FileConfigProvider.html"><code>FileConfigProvider</code></a></li>
+ </ul>
+ </ul>
+
+ <p>
+ To use a configuration provider, specify it in your configuration using
the <code>config.providers</code> property.
+ </p>
+
+ <h4 class="anchor-heading">
+ <a id="using_providers" class="anchor-link"></a>
+ <a href="#using_providers">Using Configuration Providers</a>
+ </h4>
+
+ <p>All configuration providers share a common way of passing parameters.</p>
+ <p>
+ You specify a comma-separated list of aliases and the fully-specified
class that implements the configuration provider:
+ </p>
+ <pre><code>config.providers=provider1,provider2
+config.providers.provider1.class=com.example.Provider1
+config.providers.provider2.class=com.example.Provider2</code></pre>
+ <p>
+ Each provider can have its own set of parameters, which are passed in a
specific format.
+ </p>
+ <p>
+ Pass additional parameters to each provider as
<code>config.providers.<provider_alias>.param.<name>=<value></code>:
+ </p>
+ <pre><code>config.providers.provider1.param.param1=value1
+config.providers.provider1.param.param2=value2</code></pre>
+
+ <h4 class="anchor-heading">
+ <a id="config_provider" class="anchor-link"></a>
+ <a href="#config_provider">ConfigProvider</a>
+ </h4>
+
+ <p>
+ The <code>ConfigProvider</code> is a base interface for all configuration
providers. Custom implementations of this interface can retrieve configuration
data from various sources.
+ You can then package the implementation as a JAR file, add the JAR to your
classpath, and reference the provider's class in your configuration.
+ </p>
+
+ <p><b>Example custom provider configuration</b></p>
+ <pre><code>config.providers=customProvider
+config.providers.customProvider.class=com.example.customProvider
+config.providers.customProvider.param.param1=value1
+config.providers.customProvider.param.param2=value2</code></pre>
+
+ <h4 class="anchor-heading">
+ <a id="directory_config_provider" class="anchor-link"></a>
+ <a href="#directory_config_provider">DirectoryConfigProvider</a>
Review Comment:
This provider has a configurable parameter. I think it would be nice to
mention it here:
https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/common/config/provider/DirectoryConfigProvider.java#L48-L50
##########
docs/configuration.html:
##########
@@ -308,6 +308,157 @@ <h3 class="anchor-heading"><a id="tieredstorageconfigs"
class="anchor-link"></a>
Below are the configuration properties for Tiered Storage.
<!--#include virtual="generated/remote_log_manager_config.html" -->
<!--#include virtual="generated/remote_log_metadata_manager_config.html" -->
-</script>
+ <h3 class="anchor-heading">
+ <a id="config_providers" class="anchor-link"></a>
+ <a href="#config_providers">3.11 Configuration Providers</a>
+ </h3>
+
+ <p>
+ Use configuration providers to load configuration data from external
sources. This might include sensitive information, such as passwords, API keys,
or other credentials.
+ </p>
+
+ <p>You have the following options:</p>
+ <ul>
+ <li>
+ Use a custom provider by creating a class implementing the
+ <a
href="/{{version}}/javadoc/org/apache/kafka/common/config/provider/ConfigProvider.html"><code>ConfigProvider</code></a>
interface and packaging it into a JAR file.
+ </li>
+ <li>Use a built-in provider:</li>
+ <ul>
+ <li><a
href="/{{version}}/javadoc/org/apache/kafka/common/config/provider/DirectoryConfigProvider.html"><code>DirectoryConfigProvider</code></a></li>
+ <li><a
href="/{{version}}/javadoc/org/apache/kafka/common/config/provider/EnvVarConfigProvider.html"><code>EnvVarConfigProvider</code></a></li>
+ <li><a
href="/{{version}}/javadoc/org/apache/kafka/common/config/provider/FileConfigProvider.html"><code>FileConfigProvider</code></a></li>
+ </ul>
+ </ul>
+
+ <p>
+ To use a configuration provider, specify it in your configuration using
the <code>config.providers</code> property.
+ </p>
+
+ <h4 class="anchor-heading">
+ <a id="using_providers" class="anchor-link"></a>
+ <a href="#using_providers">Using Configuration Providers</a>
+ </h4>
+
+ <p>All configuration providers share a common way of passing parameters.</p>
+ <p>
+ You specify a comma-separated list of aliases and the fully-specified
class that implements the configuration provider:
+ </p>
+ <pre><code>config.providers=provider1,provider2
+config.providers.provider1.class=com.example.Provider1
+config.providers.provider2.class=com.example.Provider2</code></pre>
+ <p>
+ Each provider can have its own set of parameters, which are passed in a
specific format.
+ </p>
+ <p>
+ Pass additional parameters to each provider as
<code>config.providers.<provider_alias>.param.<name>=<value></code>:
+ </p>
+ <pre><code>config.providers.provider1.param.param1=value1
+config.providers.provider1.param.param2=value2</code></pre>
+
+ <h4 class="anchor-heading">
+ <a id="config_provider" class="anchor-link"></a>
+ <a href="#config_provider">ConfigProvider</a>
+ </h4>
+
+ <p>
+ The <code>ConfigProvider</code> is a base interface for all configuration
providers. Custom implementations of this interface can retrieve configuration
data from various sources.
+ You can then package the implementation as a JAR file, add the JAR to your
classpath, and reference the provider's class in your configuration.
+ </p>
+
+ <p><b>Example custom provider configuration</b></p>
+ <pre><code>config.providers=customProvider
+config.providers.customProvider.class=com.example.customProvider
+config.providers.customProvider.param.param1=value1
+config.providers.customProvider.param.param2=value2</code></pre>
+
+ <h4 class="anchor-heading">
+ <a id="directory_config_provider" class="anchor-link"></a>
+ <a href="#directory_config_provider">DirectoryConfigProvider</a>
+ </h4>
+
+ <p>
+ The <code>DirectoryConfigProvider</code> retrieves configuration data from
files stored in a specified directory.
+ </p>
+ <p>
+ Each file represents a key, and its content is the value. You specify the
path to the directory where the configuration files are stored using the
<code>dir</code> parameter. This provider is useful for loading multiple
configuration files and for organizing configuration data into separate files.
+ </p>
+
+ <p><b>Example <code>DirectoryConfigProvider</code> configuration</b></p>
+ <pre><code>config.providers=dirProvider
+config.providers.dirProvider.class=org.apache.kafka.common.config.provider.DirectoryConfigProvider
+config.providers.dirProvider.param.dir=/path/to/config/directory</code></pre>
+
+ <h4 class="anchor-heading">
+ <a id="env_var_config_provider" class="anchor-link"></a>
+ <a href="#env_var_config_provider">EnvVarConfigProvider</a>
Review Comment:
This provider has a configurable parameter. I think it would be nice to
mention it here:
https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/common/config/provider/EnvVarConfigProvider.java#L42-L44
##########
docs/configuration.html:
##########
@@ -308,6 +308,157 @@ <h3 class="anchor-heading"><a id="tieredstorageconfigs"
class="anchor-link"></a>
Below are the configuration properties for Tiered Storage.
<!--#include virtual="generated/remote_log_manager_config.html" -->
<!--#include virtual="generated/remote_log_metadata_manager_config.html" -->
-</script>
+ <h3 class="anchor-heading">
+ <a id="config_providers" class="anchor-link"></a>
+ <a href="#config_providers">3.11 Configuration Providers</a>
+ </h3>
+
+ <p>
+ Use configuration providers to load configuration data from external
sources. This might include sensitive information, such as passwords, API keys,
or other credentials.
+ </p>
+
+ <p>You have the following options:</p>
+ <ul>
+ <li>
+ Use a custom provider by creating a class implementing the
+ <a
href="/{{version}}/javadoc/org/apache/kafka/common/config/provider/ConfigProvider.html"><code>ConfigProvider</code></a>
interface and packaging it into a JAR file.
+ </li>
+ <li>Use a built-in provider:</li>
+ <ul>
+ <li><a
href="/{{version}}/javadoc/org/apache/kafka/common/config/provider/DirectoryConfigProvider.html"><code>DirectoryConfigProvider</code></a></li>
+ <li><a
href="/{{version}}/javadoc/org/apache/kafka/common/config/provider/EnvVarConfigProvider.html"><code>EnvVarConfigProvider</code></a></li>
+ <li><a
href="/{{version}}/javadoc/org/apache/kafka/common/config/provider/FileConfigProvider.html"><code>FileConfigProvider</code></a></li>
+ </ul>
+ </ul>
+
+ <p>
+ To use a configuration provider, specify it in your configuration using
the <code>config.providers</code> property.
+ </p>
+
+ <h4 class="anchor-heading">
+ <a id="using_providers" class="anchor-link"></a>
+ <a href="#using_providers">Using Configuration Providers</a>
+ </h4>
+
+ <p>All configuration providers share a common way of passing parameters.</p>
+ <p>
+ You specify a comma-separated list of aliases and the fully-specified
class that implements the configuration provider:
+ </p>
+ <pre><code>config.providers=provider1,provider2
+config.providers.provider1.class=com.example.Provider1
+config.providers.provider2.class=com.example.Provider2</code></pre>
+ <p>
+ Each provider can have its own set of parameters, which are passed in a
specific format.
+ </p>
+ <p>
+ Pass additional parameters to each provider as
<code>config.providers.<provider_alias>.param.<name>=<value></code>:
+ </p>
+ <pre><code>config.providers.provider1.param.param1=value1
+config.providers.provider1.param.param2=value2</code></pre>
+
+ <h4 class="anchor-heading">
+ <a id="config_provider" class="anchor-link"></a>
+ <a href="#config_provider">ConfigProvider</a>
+ </h4>
+
+ <p>
+ The <code>ConfigProvider</code> is a base interface for all configuration
providers. Custom implementations of this interface can retrieve configuration
data from various sources.
+ You can then package the implementation as a JAR file, add the JAR to your
classpath, and reference the provider's class in your configuration.
+ </p>
+
+ <p><b>Example custom provider configuration</b></p>
+ <pre><code>config.providers=customProvider
+config.providers.customProvider.class=com.example.customProvider
+config.providers.customProvider.param.param1=value1
+config.providers.customProvider.param.param2=value2</code></pre>
+
+ <h4 class="anchor-heading">
+ <a id="directory_config_provider" class="anchor-link"></a>
+ <a href="#directory_config_provider">DirectoryConfigProvider</a>
+ </h4>
+
+ <p>
+ The <code>DirectoryConfigProvider</code> retrieves configuration data from
files stored in a specified directory.
+ </p>
+ <p>
+ Each file represents a key, and its content is the value. You specify the
path to the directory where the configuration files are stored using the
<code>dir</code> parameter. This provider is useful for loading multiple
configuration files and for organizing configuration data into separate files.
+ </p>
+
+ <p><b>Example <code>DirectoryConfigProvider</code> configuration</b></p>
+ <pre><code>config.providers=dirProvider
+config.providers.dirProvider.class=org.apache.kafka.common.config.provider.DirectoryConfigProvider
+config.providers.dirProvider.param.dir=/path/to/config/directory</code></pre>
+
+ <h4 class="anchor-heading">
+ <a id="env_var_config_provider" class="anchor-link"></a>
+ <a href="#env_var_config_provider">EnvVarConfigProvider</a>
+ </h4>
+
+ <p>The <code>EnvVarConfigProvider</code> retrieves configuration data from
environment variables.</p>
+ <p>No specific parameters are required, as it reads directly from the
specified environment variables.</p>
+ <p>
+ This provider is useful for configuring applications running in
containers, for example, to load certificates or JAAS configuration from
environment variables mapped from secrets.
+ </p>
+
+ <p><b>Example <code>EnvVarConfigProvider</code> configuration</b></p>
+ <pre><code>config.providers=envVarProvider
+config.providers.envVarProvider.class=org.apache.kafka.common.config.provider.EnvVarConfigProvider</code></pre>
+
+ <h4 class="anchor-heading">
+ <a id="file_config_provider" class="anchor-link"></a>
+ <a href="#file_config_provider">FileConfigProvider</a>
+ </h4>
+
+ <p>The <code>FileConfigProvider</code> retrieves configuration data from a
single properties file.</p>
+ <p>This provider is useful for loading configuration data from mounted
files.</p>
+
+ <p><b>Example <code>FileConfigProvider</code> configuration</b></p>
+ <pre><code>config.providers=fileProvider
+config.providers.fileProvider.class=org.apache.kafka.common.config.provider.FileConfigProvider</code></pre>
+
+ <h4 class="anchor-heading">
+ <a id="ref_config_provider" class="anchor-link"></a>
+ <a href="#ref_config_provider">Referencing configuration providers</a>
Review Comment:
I wonder if it would read better if the syntax to use each provider were in
their respective sections? WDYT?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
