[GitHub] [pulsar-helm-chart] doehyunbaek commented on issue #263: node_exporter and alert_manager values do nothing.
doehyunbaek commented on issue #263: URL: https://github.com/apache/pulsar-helm-chart/issues/263#issuecomment-1117145427 I worked on adaptation of templates for node_exporter and alert_manager. For node_exporter, it was really easy, and I could just copy template file and relevant values.yaml lines. But for alert_manager it was quite contrived and it seems there would be really few benefit as their setup would be quite different between people and good default might be hard to find. Thus I suggest copying of templates for node_exporter only. It also fills out node metrics dashboard of apache pulsar helm chart(https://github.com/apache/pulsar-helm-chart). For alert_manager, I suggest either removing monitoring.alert_manager line or adding comment that it does nothing here. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar-helm-chart] frankjkelly commented on pull request #262: Add support for Horizontal Pod Autoscaling for Broker and Proxy.
frankjkelly commented on PR #262: URL: https://github.com/apache/pulsar-helm-chart/pull/262#issuecomment-1117150201 Hello @sijie @lhotari @eolivelli please take a look at your convenience. If no issues would love to get this merged as I am excited to test it under load :-) Thanks much in advance! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: RocksDB upgrade - wait for BookKeeper 4.15.0 ?
At the end we'll have Pulsar 2.11 with BookKeeper 4.15.x (Andrey is working on the BK release and the upgrade in Pulsar) and therefore with M1 support. Pulsar 2.8, 2.9, 2.10 lines will have BookKeeper 4.14 (without the M1 support). Nicolò Boschi Il giorno mar 12 apr 2022 alle ore 15:10 Devin Bost ha scritto: > So, based on this, it looks like we'd need to upgrade Pulsar to use BK > 4.15.0 to get M1 compatibility? > What kind of changes will that introduce into Pulsar? > > Devin G. Bost > > > On Fri, Apr 1, 2022 at 8:17 AM Nicolò Boschi wrote: > > > Hi all, > > > > this is a follow-up discussion about the RocksDB upgrade that happened at > > yesterday's Apache Pulsar community meeting ( > > https://lists.apache.org/thread/tq7dsws72zf9r7qzr4l567z9w346ksbm). > > > > We recently reverted the RocksDB upgrade due to runtime compatibility > > errors (more details here https://github.com/apache/pulsar/pull/14962). > > The main reason to upgrade RocksDB is the Apple M1 compatibility. > > > > Shortly, we cannot upgrade RocksDB to a version greater than 6.17.2 > because > > BookKeeper needs to be recompiled with the newer version of RocksDB. > > > > There's an ongoing discussion on BookKeeper mailing list about including > > the RocksDB upgrade in 4.14.5. Also 4.15.0 is going to be released soon > > (with the RocksDB upgrade). > > https://lists.apache.org/thread/dsdltmoy5ggqx4oj7hrt13hqok80d4jf > > > > At the moment, Pulsar 2.8, 2.9, 2.10 and master are using 4.14.4. > > > > From the Pulsar side we are interested in upgrading to 4.14.5 because > > there's an important bugfix ( > > https://github.com/apache/bookkeeper/pull/3110) > > that we should port to all the active branches. > > So basically if 4.14.5 will contain the RocksDB upgrade, we'll need to > bump > > to the same version also in the active branches to match the runtime > > compatibility. If not, we're safe to upgrade to 4.14.5 without RocksDB. > > > > My personal opinion is that the best scenario would be: > > - Update BK to 4.14.5 on active branches without RocksDB > > - Update BK to 4.15.0 on master (and so Pulsar 2.11) with RocksDB and > Apple > > M1 compatibility. > > > > I think we could push to the BookKeeper ML to not include the RocksDB > > upgrade in 4.14.5 > > > > Nicolò Boschi > > >
[GitHub] [pulsar-test-infra] yaalsn opened a new pull request, #36: [Docbot] Add doc-added label
yaalsn opened a new pull request, #36: URL: https://github.com/apache/pulsar-test-infra/pull/36 ## Motivation Currently, apache/pulsar adds a label called `doc-added`, so we need to support the label. e.g. https://github.com/apache/pulsar/pull/15438 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar-test-infra] yaalsn commented on pull request #36: [Docbot] Add doc-added label
yaalsn commented on PR #36: URL: https://github.com/apache/pulsar-test-infra/pull/36#issuecomment-1118035940 @codelipenghui @lhotari @mattisonchao PTAL -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [ANNOUNCE] new Committer Qiang Zhao (mattisonchao)
Congrats Qiang Penghui On Wed, May 4, 2022 at 2:58 PM ZhangJian He wrote: > Congratulations Qiang Zhao. > > Thanks > ZhangJian He > > Enrico Olivelli 于2022年5月4日周三 14:28写道： > > > The Project Management Committee (PMC) for Apache Pulsar > > has invited Qiang Zhao to become a committer and we are pleased > > to announce that he has accepted. > > > > Qiang Zhao (with GH id mattisonchao) contributed lots of improvements > > and bugfixes to Pulsar. > > > > Being a committer enables easier contribution to the > > project since there is no need to go via the patch > > submission process. This should enable better productivity. > > A PMC member helps manage and guide the direction of the project. > > > > Please join me in congratulating with him ! > > > > Best regards > > Enrico Olivelli > > >
Re: [ANNOUNCE] new Committer Qiang Zhao (mattisonchao)
Congrats ! Thanks, Baozi > 2022年5月5日 08:4032，PengHui Li 写道： > > Congrats
Re: [DISCUSS] PIP-157: Bucketing topic metadata to allow more topics per namespace
+1 Best, Mattison On Tue, 3 May 2022 at 22:38, Andras Beni wrote: > Hi Enrico, > > I updated the proposal with my answers: > > [The translation from user-visible name to actual storage path] will > happen by supplying the bucket number to > TopicName.getPersistenceNamingEncoding and calculating the modified path. > Since the bottleneck is listing topics, which happens using the managed > ledgers' path, there is no need to modify schema storage. Furthermore the > structure and content of data currently stored at > /managed-ledgers/tenant/namespace/domain/topic will not be changed but will > be available at the new path. > > I have a very limited POC available at > > https://github.com/andrasbeni/pulsar/commit/a7393d0affd4f62ea64de994380d38f6938eca81 > . > Please note, it was not intended for a public audience and is full of > unnecessary log messages, has a fixed number of buckets for all namespaces > (so breaks compatibility) and has not been tested with unit tests. But I > hope it helps get my goals across. > > Best regards, > Andras > > On Mon, May 2, 2022 at 12:09 PM Enrico Olivelli > wrote: > > > How do we deal with Schema storage, Cursors and other usages of > > ManagedLedger? > > > > Can you please clarify each case I'm which we use ManagedLedger? > > > > Did you try to put up a proff or concept with this proposal? > > > > I generally agree with this approach but it looks like we are not going > > deep enough in the design > > > > > > Enrico > > > > Il Lun 25 Apr 2022, 04:53 Hang Chen ha scritto: > > > > > +1 > > > > > > Thanks, > > > Hang > > > > > > PengHui Li 于2022年4月25日周一 09:20写道： > > > > > > > > +1 > > > > > > > > Penghui > > > > > > > > On Thu, Apr 21, 2022 at 9:17 PM Andras Beni > > > > wrote: > > > > > > > > > Hi everyone, > > > > > > > > > > I've just created a proposal that will help scaling up the number > of > > > topics > > > > > per namespace. > > > > > It's available at https://github.com/apache/pulsar/issues/15254 > and > > is > > > > > copied below. > > > > > Let me know what you think. > > > > > > > > > > Thanks, > > > > > Andras > > > > > > > > > > Motivation > > > > > > > > > > Pulsar is able to manage millions of topics but the number of > topics > > > within > > > > > a single namespace is limited by metadata storage. > > > > > > > > > > For each topic within a namespace there is a ZooKeeper node. > Listing > > > topics > > > > > thus requires listing children of a node, which at around 10K > topics > > > hits > > > > > the limits of ZK. > > > > > Goal > > > > > > > > > > This feature will allow a larger number of topics within a > namespace > > by > > > > > inserting an intermediate layer (buckets) before the topic nodes > like > > > > > /managed-ledgers/tenant/namespace/domain/bucket/topic. > > > > > > > > > > By default this feature will be switched off and would only be > > enabled > > > on a > > > > > per namespace basis at the creation of namespaces by setting a > > policy. > > > This > > > > > eliminates the need for migrating existing installations to this > new > > > > > scheme. > > > > > > > > > > Buckets will not have correlation with bundles. > > > > > API Changes > > > > > > > > > > A new policy numberOfTopicBuckets will be added. The default > value, 1 > > > means > > > > > no bucketing, the current behaviour will be preserved for the > > > namespace. > > > > > Greater values mean topics will be stored at a path including > > buckets. > > > > > Users will not be able to change the number of buckets after the > > > namespace > > > > > is created. > > > > > Implementation > > > > > > > > > > The goal is to implement this feature transparently to the user. > > > Clients > > > > > will continue to refer to topics by domain://tenant/namespace/topic > > but > > > > > pulsar will internally translate to the new persistence naming > where > > > > > necessary. > > > > > > > > > > The way metadata stores work will not be affected either. > > > > > > > > > > Assigning topics to buckets will be based on the topic name's hash > > > code's > > > > > absolute value modulo the number of buckets. > > > > > > > > > > The bulk of the changes necessary for this feature is to make > > namespace > > > > > policies available wherever persistence naming is calculated. Where > > > listing > > > > > of topics within a namespace is necessary, the introduction of the > > new > > > > > layer will add some overhead in the form of multiple requests to > the > > > > > metadata store. These include checking if the limit on topic number > > per > > > > > namespace has been reached. > > > > > Example > > > > > > > > > > Let's consider the following metadata hierarchy: > > > > > > > > > > managed-ledgers > > > > > \- tenant > > > > > \- namespace > > > > > \- persistent > > > > > +- nptopic1 > > > > > +- nptopic2 > > > > > +- ptopic-partition-0 > > > > > +- ptopic-partition-1 > > > > > +- ptopic-partition-2 > > >
Re: [ANNOUNCE] new Committer Qiang Zhao (mattisonchao)
Congrats Qiang! Thanks, Hang Baozi 于2022年5月5日周四 08:59写道： > > Congrats ! > > Thanks, > Baozi > > > 2022年5月5日 08:4032，PengHui Li 写道： > > > > Congrats >
Re: [ANNOUNCE] new Committer Qiang Zhao (mattisonchao)
Congratulations! Qiang Thanks, Max Xu On Thu, May 5, 2022 at 9:19 AM Hang Chen wrote: > Congrats Qiang! > > Thanks, > Hang > > Baozi 于2022年5月5日周四 08:59写道： > > > > Congrats ! > > > > Thanks, > > Baozi > > > > > 2022年5月5日 08:4032，PengHui Li 写道： > > > > > > Congrats > > >
Re: [ANNOUNCE] new Committer Qiang Zhao (mattisonchao)
Congratulations! Qiang Thanks, Li Li > On May 5, 2022, at 9:51 AM, Max Xu wrote: > > Congratulations! Qiang > > Thanks, > Max Xu > > > > > On Thu, May 5, 2022 at 9:19 AM Hang Chen wrote: > >> Congrats Qiang! >> >> Thanks, >> Hang >> >> Baozi 于2022年5月5日周四 08:59写道： >>> >>> Congrats ! >>> >>> Thanks, >>> Baozi >>> 2022年5月5日 08:4032，PengHui Li 写道： Congrats >>> >>
Re: [ANNOUNCE] new Committer Qiang Zhao (mattisonchao)
Congratulations! BR, Haiting On 2022/05/04 06:28:24 Enrico Olivelli wrote: > The Project Management Committee (PMC) for Apache Pulsar > has invited Qiang Zhao to become a committer and we are pleased > to announce that he has accepted. > > Qiang Zhao (with GH id mattisonchao) contributed lots of improvements > and bugfixes to Pulsar. > > Being a committer enables easier contribution to the > project since there is no need to go via the patch > submission process. This should enable better productivity. > A PMC member helps manage and guide the direction of the project. > > Please join me in congratulating with him ! > > Best regards > Enrico Olivelli >
Re: Call for projects and mentors for OSPP 2022
Thanks for your guys' project ideas. I can help to submit these ideas to the OSPP website this week and send invitations to your contact email, please remember to check it. The project idea is open from 4/21, 2022 to 5/9, 2022(UTC+8), 3 seats left, so welcome more ideas pump out. Best, Dianjin Wang On Sat, Apr 30, 2022 at 10:45 AM Zike Yang wrote: > Hi, > > Thank Yu! > > Sorry for the missing information: > > --- > > Your Email: z...@apache.org > Your Apache ID: zike > > And here is the Chinese version of my application: > > --- > > Project Name: 在Go客户端中支持大消息处理 > Project Description: > Apache > Pulsar对发往broker的消息引入了大小限制。目前，Pulsar的java客户端支持了消息分块的功能，使得Pulsar在生产者中能够将消息分成多个小块并在消费者中将它们组装起来，以支持大消息的处理。但是Pulsar的Go客户端并不支持这个特性。本项目将在Go客户端中添加对大消息处理的支持。 > > Difficulty Level: > - [ ] Basic > - [x] Advanced > > Project Validation Items: > Item 1: 编写一份方案来描述你的设计 > Item 2: 在Go客户端上实现消息分块的功能并推送PR > Item 3: 添加单元测试以进行验证 > Item 4: 添加关于这个特性的文档 > … > Project Mentor: 杨子棵 > Your Name: 杨子棵 > Your Email: z...@apache.org > Your Apache ID: zike > > On Sat, Apr 30, 2022 at 9:51 AM Yu wrote: > > > > Hi Zike, looks like your email and Apache ID are missing? > > > > On Fri, Apr 29, 2022 at 10:17 PM Zike Yang wrote: > > > > > Thanks all, here is my application: > > > > > > Project Name: Support processing large messages at Go client > > > Project Description: > > > > > > Apache Pulsar imposes a size limit on each message sent to the broker. > > > Currently, the pulsar java client support message chunking which > > > enables Pulsar to process large payload messages by splitting the > > > message into chunks at the producer side and combining chunked > > > messages at the consumer side. But the Pulsar go client doesn't > > > support this feature. This project aims to add support for message > > > chunking to process large messages at the go client. > > > > > > Difficulty Level: > > > - [ ] Basic > > > - [x] Advanced > > > Project Validation Items: > > > Item 1: Write a proposal to describe your design > > > Item 2: Implement the message chunking feature at the go client and > > > push the PR out > > > Item 3: Add unit tests to verify it > > > Item 4: Add the documentation for this feature > > > … > > > Project Mentor: Zike Yang > > > Your Name: Zike Yang > > > Your Email: > > > Your Apache ID: > > > > > > On Fri, Apr 29, 2022 at 5:06 PM Yu wrote: > > > > > > > > Thanks all, here are my applications (in English and Chinese). > > > > > > > > > > > > English version > > > > > > > > > > > > # Project Name > > > > Automate Documentation Workflow to Optimize Website User Experience > > > > > > > > # Project Description > > > > > > > > ## Project Goal > > > > > > > > This project aims to improve the user experience on the new Pulsar > > > website > > > > by optimizing and automating workflow for documentation and website, > > > > including but not limited to generating documentation websites from > code > > > > automatically, standardizing pull request naming convention using > GitHub > > > > Actions, and more. You can check some related tasks on > > > > > > > > https://github.com/apache/pulsar/issues?q=is%3Aopen+is%3Aissue+label%3Awebsite > > > . > > > > > > > > > > > > ## What You Can Learn from the Project > > > > > > > > - Increase your technical knowledge. > > > > You will get familiar with codebase and learn technical skills about > > > > front-end and back-end developments of Pulsar. You will be guided by > > > > veterans who have been doing this for years and who know the tips and > > > > tricks to help you be a more excellent open-source developer. > > > > > > > > - Sharpen your software skills. > > > > Pulsar adoption engagement skyrocketed over the past years. This > success > > > > depends on its community. By participating in a project in the Pulsar > > > > community, you can learn the Apache culture, communication skills, > and > > > best > > > > practices to collaborate effectively in the open-source world. This > paves > > > > your way forward for making big contributions. > > > > > > > > - Build your network. > > > > You can interact and share ideas with knowledgeable and passionate > > > > contributors in the Pulsar community. Moreover, you can make great > > > friends > > > > from all over the world. > > > > > > > > # Difficulty Level > > > > - [ ] Basic > > > > - [x] Advanced > > > > > > > > # Project Validation Items > > > > - For the project: set achievable goals and make reasonable plans; > > > complete > > > > project before deadline and achieve expected outcomes. > > > > - For the student: acquire new skills and level up your developer > > > portfolio. > > > > > > > > # Project Mentor > > > > Yu Liu > > > > > > > > # Your Email > > > > li...@apache.org > > > > y...@streamnative.io > > > > > > > > # Your Apache ID > > > > liuyu > > > > > > > > > > > > Chinese version > > > > ~~
[GitHub] [pulsar-helm-chart] haorenfsa commented on pull request #244: Add imagePullSecrets zookeeper
haorenfsa commented on PR #244: URL: https://github.com/apache/pulsar-helm-chart/pull/244#issuecomment-1118120389 milvus-helm need to update after this patch is adopted -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [ANNOUNCE] new Committer Qiang Zhao (mattisonchao)
Congrats Qiang! -Lari ke 4. toukok. 2022 klo 9.28 Enrico Olivelli kirjoitti: > The Project Management Committee (PMC) for Apache Pulsar > has invited Qiang Zhao to become a committer and we are pleased > to announce that he has accepted. > > Qiang Zhao (with GH id mattisonchao) contributed lots of improvements > and bugfixes to Pulsar. > > Being a committer enables easier contribution to the > project since there is no need to go via the patch > submission process. This should enable better productivity. > A PMC member helps manage and guide the direction of the project. > > Please join me in congratulating with him ! > > Best regards > Enrico Olivelli >
