Re: slow ingest speeds

2023-03-15 Thread Dave Marion
When you shut down the two datanodes, did you have the same "slow sync cost" messages concentrated on two nodes? If so, is it possible that a majority of the writes are going to a small set of tablet servers? You might be able to see this on the Monitor. Is it possible that tablets you are ingestin

Re: slow ingest speeds

2023-03-15 Thread Vincent Russell
I stopped the two data nodes and it had no effect. Thanks, On Wed, Mar 15, 2023 at 6:53 PM Vincent Russell wrote: > Yes. We have the hdfs rack-aware set up to divide the blocks equally. > And according to the name node http page it doesn't look like those nodes > have a much higher number of b

Re: slow ingest speeds

2023-03-15 Thread Vincent Russell
Yes. We have the hdfs rack-aware set up to divide the blocks equally. And according to the name node http page it doesn't look like those nodes have a much higher number of blocks that nother nodes. I can try temporarily shutting down one of the data nodes to see what that does. We did already

Re: slow ingest speeds

2023-03-15 Thread Dave Marion
sounds like you have a hot-spot on those two datanode hosts. Either because the blocks that it's writing to are all (or a majority) located there, or there is some type of issue with the host. Stopping the DN processes on those two hosts should confirm this, unless the hot spot moves. Do you have t

slow ingest speeds

2023-03-15 Thread Vincent Russell
Hello, I am using accumulo 2.0.1 with hadoop 3.3.1. I have two identical clusters with 28 tservers. I have writers on both clusters which are set with 10 batch writers with a max memory of 50m. However, one server is ingesting 10x faster than the other. Is there anything I should check for? I

RE: [DISCUSS] Enable Github wiki in asf.yaml?

2023-03-15 Thread dev1
One thing to keep in mind is that we should keep the mailing list in mind. Not sure how we integrate two streams of notifications. But, leveraging the mailing list will make sure that the community at large can follow and participate. Ed Coleman -Original Message- From: Christopher S

Re: [DISCUSS] Enable Github wiki in asf.yaml?

2023-03-15 Thread Dave Marion
It looks like we could generate the GH wiki from a folder in the source code. This would allow for issues and PRs. Just a thought. Ref: https://nimblehq.co/blog/create-github-wiki-pull-request and https://stackoverflow.com/questions/10642928/how-can-i-make-a-pull-request-for-a-wiki-page-on-github

Re: [DISCUSS] Enable Github wiki in asf.yaml?

2023-03-15 Thread Christopher
It seems like there's a majority consensus of those engaged. No need for a vote, but I think the question about notifications should be addressed first. On Wed, Mar 15, 2023, 13:47 Christopher Shannon < christopher.l.shan...@gmail.com> wrote: > I'm +1 to using some kind of wiki so if we can't use

Re: [DISCUSS] Enable Github wiki in asf.yaml?

2023-03-15 Thread Christopher
All discussions in GitHub issues go to the notifications list so people can follow. We don't usually need to recap, but if somebody wanted to, they could reply to the dev list in response to a message sent to notifications. The wiki doesn't have issues, discussions, or l etc., and I don't know wha

Re: [DISCUSS] Enable Github wiki in asf.yaml?

2023-03-15 Thread Christopher Shannon
I'm +1 to using some kind of wiki so if we can't use Confluence then GH sounds fine to me. Do we need to take a formal vote for using the Github wiki or is there enough consensus already? On Wed, Mar 15, 2023 at 1:43 PM Daniel Roberts wrote: > +1 for the GH wiki with major discussions still bein

Re: [DISCUSS] Enable Github wiki in asf.yaml?

2023-03-15 Thread Daniel Roberts
+1 for the GH wiki with major discussions still being fed into, or originated on the mailing lists. As a side question, if there is a lengthy discussion on a GH issue, is it standard practice to just recap that in a mailing list message? Or is there a more "formal" inclusion process to follow? O

Re: [DISCUSS] Enable Github wiki in asf.yaml?

2023-03-15 Thread Christopher
I don't think the workflow I proposed about using PRs and discussion on tickets, etc. and the accompanying arguments about keeping things consolidated and accessible to potential contributors not participating on GitHub, were really challenged at all. However, since I seem to be the only one advoca

Re: [DISCUSS] Enable Github wiki in asf.yaml?

2023-03-15 Thread Dave Marion
> At this point, I think we should move forward with a GH wiki and then we can re-evaluate things once the Apache confluence issue is sorted out. Agreed. On Wed, Mar 15, 2023 at 11:57 AM dev1 wrote: > I just tried (Wed, 3/15) and still received the same error. I asked on > the infra slack chan

RE: [DISCUSS] Enable Github wiki in asf.yaml?

2023-03-15 Thread dev1
I just tried (Wed, 3/15) and still received the same error. I asked on the infra slack channel and they replied that they are still working to determine what the issue is - signs are pointing to something inside of confluence, but no progress. At this point, I think we should move forward with