Hi,
This can also be quite useful to make better use of existing functionality in
the Kafka API — moving replicas between log directories via
ALTER_REPLICA_LOG_DIRS. If usable space information is also available the
caller can make better decisions using the same API. It means a more consistent
way of interacting with Kafka to manage replicas locations within a broker
without having to correlate Kafka metrics with information from the Kafka API.
--
Igor
On Wed, May 25, 2022, at 8:16 PM, Jun Rao wrote:
> Hi, Mickael,
>
> Thanks for the KIP. Since this is mostly for monitoring and alerting,
> could we expose them as metrics instead of as part of the API? We already
> have a size metric per log. Perhaps we could extend that to add used/total
> metrics per disk?
>
> Thanks,
>
> Jun
>
> On Thu, May 19, 2022 at 10:21 PM Raman Verma <rve...@confluent.io.invalid>
> wrote:
>
>> Hello Mikael,
>>
>> Thanks for the KIP.
>>
>> I see that the API response contains some information about each partition.
>> ```
>> { "name": "PartitionSize", "type": "int64", "versions": "0+",
>> "about": "The size of the log segments in this partition in bytes." }
>> ```
>> Can this be summed up to provide a used space in a `log.dir`
>> This will also be specific to a `log.dir` (for the case where multiple
>> log.dir are hosted on the same underlying device)
>>
>> On Thu, May 19, 2022 at 10:21 AM Cong Ding <cd...@confluent.io.invalid>
>> wrote:
>> >
>> > Hey Mickael,
>> >
>> > Great KIP!
>> >
>> > I have one question:
>> >
>> > You mentioned "DescribeLogDirs is usually a low volume API. This change
>> > should not
>> > significantly affect the latency of this API." and "That would allow to
>> > easily validate whether disk operations (like a resize), or topic
>> deletion
>> > (log deletion only happen after a short delay) have completed." I wonder
>> if
>> > there is an existing metric/API that can allow administrators to
>> determine
>> > whether we need to resize? If administrators use this API to determine
>> > whether we need a resize, would this API become a high-volume API? I
>> > understand we don't want this API to be a high-volume one because the API
>> > is already costly by returning `"name": "Topics"`.
>> >
>> > Cong
>> >
>> > On Thu, Apr 7, 2022 at 2:17 AM Mickael Maison <mickael.mai...@gmail.com>
>> > wrote:
>> >
>> > > Hi,
>> > >
>> > > I wrote a small KIP to expose the total and usable space of logdirs
>> > > via the DescribeLogDirs API:
>> > >
>> > >
>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-827%3A+Expose+logdirs+total+and+usable+space+via+Kafka+API
>> > >
>> > > Please take a look and let me know if you have any feedback.
>> > >
>> > > Thanks,
>> > > Mickael
>> > >
>>
>>
>>
>> --
>> Best Regards,
>> Raman Verma
>>
