Hello Chia-Ping I have just included the configuration migration section in the KIP-653 <https://cwiki.apache.org/confluence/x/GCx4CQ>. Please take a look.
Best, TengYao Jörn Franke <jornfra...@gmail.com> 於 2024年11月19日 週二 上午12:36寫道： > Yaml allows to add comments compared to JSON which is not only for > examples good, but also to document certain implications when deploying > production configuration. > > > Am 18.11.2024 um 15:02 schrieb TengYao Chi <kiting...@gmail.com>: > > > > Hi everyone, > > > > From my experience, it has been quite challenging to understand and > > transform the .properties file from log4j 1 to log4j 2, as the properties > > format is fairly flat and difficult to understand. > > In comparison, YAML is more structured, easier to read, and widely used. > I > > would like to +1 for using YAML properties instead. > > If a KIP is required, I am willing to prepare one. > > > > Best, > > TengYao > > > > Mickael Maison <mickael.mai...@gmail.com> 於 2024年11月18日 週一 下午9:35寫道： > > > >> Hi Chia-Ping, > >> > >> Yes in 4.0.0 we will only provide log4j2 configuration files. We will > >> still support log4j.properties files for compatibility but that will > >> print a warning when these are used. > >> > >> However my question is about the format of our log4j2 files. Currently > >> in the PR we are using the "properties" format (log4j2.properties). > >> But as pointed by Piotr (from the Apache Logging PMC), log4j2 has a > >> hierarchical structure which does not translate well into the > >> properties format, so configuration file in this format have many > >> quirks. Instead he suggested using XML, JSON or YAML in our example > >> files (so log4j2.xml or log4j2.json or log4j2.yaml) as these are the > >> preferred formats now. > >> > >> David, Viktor and myself have expressed our preference toward adopting > >> YAML for our log4j2 configuration files. I'm wondering if that's > >> enough consensus, or if people want to discuss this further, or even > >> want a vote? > >> > >> Thanks, > >> Mickael > >> > >> > >>> On Mon, Nov 18, 2024 at 1:52 PM Chia-Ping Tsai <chia7...@gmail.com> > wrote: > >>> > >>> hi, Mickael > >>> > >>> I'm +1 for using the log4j2 format. Otherwise, it's odd for users to > see > >>> deprecation warnings about log4j.properties when using our example > files. > >>> > >>> Best, > >>> Chia-Ping > >>> > >>> > >>> > >>> Mickael Maison <mickael.mai...@gmail.com> 於 2024年11月18日 週一 下午7:33寫道： > >>> > >>>> Hi, > >>>> > >>>> The log4j2 migration PR is pretty much ready to be merged and the > >>>> first deadlines for 4.0.0 is approaching fast. > >>>> I think we should decide which format to use in our example log4j2 > >>>> files to avoid having to update the format shortly after 4.0.0. > >>>> > >>>> This point is not really covered by KIP-653: Upgrade log4j to log4j2. > >>>> Do we want another KIP and vote or is a consensus emerging? > >>>> > >>>> Thanks, > >>>> Mickael > >>>> > >>>> On Wed, Nov 6, 2024 at 10:47 AM Mickael Maison < > >> mickael.mai...@gmail.com> > >>>> wrote: > >>>>> > >>>>> Hi, > >>>>> > >>>>> I think our example log4j2 file should be as idiomatic as possible. > >> If > >>>>> YAML is now the recommended format, then it makes sense to adopt it. > >> I > >>>>> would also do this directly in 4.0.0. > >>>>> > >>>>> Like David, one concern when adding extra dependencies is CVEs. YAML > >>>>> is a widely used format and the libraries are actively maintained so > >> I > >>>>> think it's acceptable. > >>>>> > >>>>> In my opinion, the configuration format for Kafka is a completely > >>>>> orthogonal issue. If people want to adopt YAML this can be done in a > >>>>> separate discussion. I don't see issues with using YAML for > >>>>> configuring log4j2 and properties for Kafka. > >>>>> > >>>>> Thanks, > >>>>> Mickael > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> On Thu, Oct 31, 2024 at 5:47 PM Piotr P. Karwasz > >>>>> <pi...@mailing.copernik.eu> wrote: > >>>>>> > >>>>>> Hi Viktor, > >>>>>> > >>>>>> On 31.10.2024 10:19, Viktor Somogyi-Vass wrote: > >>>>>>> I could see a transition for Kafka configs too to YAML. It's > >> widely > >>>> used in > >>>>>>> other projects as well and for Kafka too it would make sense to > >>>> group them > >>>>>>> for instance by log, storage, networking, security etc.. It > >>>> definitely > >>>>>>> has an advantage that we could move to a more well defined > >> structure > >>>> and > >>>>>>> away from these long prefixes which are very cumbersome in some > >>>> places. > >>>>>>> Perhaps 4.0 would have been a good candidate but given that we > >> should > >>>>>>> provide backward compatibility anyway, later is good too. > >>>>>> > >>>>>> To migrate the configuration from Java Properties to YAML and > >> maintain > >>>>>> BC, you can use `jackson-dataformat-properties`. This definitively > >> can > >>>>>> be done in a minor release. > >>>>>> > >>>>>> [1] > >>>>>> > >>>> > >> > https://github.com/FasterXML/jackson-dataformats-text/blob/master/properties/README.md > >>>>>> > >>>>>>> 4. Data bindings and parsers are common sources of CVEs. It looks > >>>> like > >>>>>>> Snakeyaml is no exception ( > >>>>>>> https://www.cvedetails.com/version-list/0/66013/1/), though it > >>>> doesn't look > >>>>>>> much worse than Jackson. Just to point out, this will add a bit > >> of > >>>>>>> dependency overhead as we keep up with security patches. > >>>>>>> > >>>>>>> It's a good point about CVEs. I haven't seen it on the dev list > >> but > >>>> were > >>>>>>> there any conversations about enabling dependabot version > >> updates? > >>>> With > >>>>>>> automatic dependabot PRs we could get fixes in as soon as it > >> opens > >>>> the PR. > >>>>>> > >>>>>> I don't know how it works in Gradle, but with Maven we don't get > >>>>>> Dependabot PRs for transitive dependencies such as SnakeYAML. > >> Since the > >>>>>> dependency is not present in the `pom.xml`, its version can not be > >>>>>> updated. Dependabot might be able to create alerts in the > >> "Security" > >>>> tab > >>>>>> though. > >>>>>> > >>>>>> Since Log4j is a library without an executable distribution, it is > >> not > >>>>>> guaranteed that a new version of Log4j will be released each time a > >>>>>> vulnerability in SnakeYAML is discovered and that vulnerability is > >>>>>> exploitable in Log4j Core. It does not make much sense to make a > >> new > >>>>>> release just to upgrade a number in `pom.xml`. > >>>>>> > >>>>>> Until VEX-es can be created automatically, the only way I can > >> think of > >>>>>> to help the Kafka team with CVEs in transitive dependencies is a > >> manual > >>>>>> process: > >>>>>> > >>>>>> * You mark `jackson-dataformat-yaml` with a comment like "Used by > >> Log4j > >>>>>> Core". > >>>>>> > >>>>>> * If a CVE is reported against that artifact or its dependencies, > >> you > >>>>>> open an issue in Log4j[2], so we can advise on the exploitability > >> of > >>>> the > >>>>>> CVE. > >>>>>> > >>>>>> * With that information you can decide whether to release a new > >> Kafka > >>>>>> version or not. > >>>>>> > >>>>>> Piotr > >>>>>> > >>>>>> [2] https://github.com/apache/logging-log4j2/issues > >>>>>> > >>>>>> > >>>> > >> >
