Re: Schema collision results in multiple data directories per table

2021-10-18 Thread Tom Offermann
- 20739eb0-d92e-11e6-b42f-e7eb6f21c481 - Friday, January 13, 2017 at >1:18:01 GMT >- 8ad72660-f629-11eb-a217-e1a09d8bc60c - Thursday, August 5, 2021 at > 20:13:04 GMT > > Would that have been when you added the new nodes? Any possibility that > you "merged" two clusters together? > >> -- Tom Offermann Lead Software Engineer http://newrelic.com

Re: Schema collision results in multiple data directories per table

2021-10-15 Thread Tom Offermann
e /path/to/data/keyspace/table-(id)/ on disk > > If any of those dont match, you've got a problem waiting to bite you on > next restart. > > > > On Fri, Oct 15, 2021 at 3:48 PM Tom Offermann > wrote: > >> So, if I were to do `CONSISTENCY ALL; select *` from each

Re: Schema collision results in multiple data directories per table

2021-10-15 Thread Tom Offermann
t;> >>> You may be able to confirm/demonstrate that by looking at the timestamps >>> on the data directories across all of the hosts in the cluster? >>> >>> >>> >>> On Fri, Oct 15, 2021 at 3:02 PM Tom Offermann >>> wrote: >>

Re: Schema collision results in multiple data directories per table

2021-10-15 Thread Tom Offermann
> > > On Fri, Oct 15, 2021 at 3:02 PM Tom Offermann > wrote: > >> Jeff, >> >> Thanks for describing the race condition. >> >> I understand that performing concurrent schema changes is dangerous, and >> that running an `ALTER KEYSPACE` on one n

Re: Schema collision results in multiple data directories per table

2021-10-15 Thread Tom Offermann
On Wed, Oct 13, 2021 at 8:15 AM Stefan Miklosovic < >> stefan.mikloso...@instaclustr.com> wrote: >> >>> Hi Tom, >>> >>> while I am not completely sure what might cause your issue, I just >>> want to highlight that schema agreements were overhauled

Re: Schema collision results in multiple data directories per table

2021-10-15 Thread Tom Offermann
t; (1) https://issues.apache.org/jira/browse/CASSANDRA-15158 >> >> On Fri, 1 Oct 2021 at 18:43, Tom Offermann >> wrote: >> > >> > When adding a datacenter to a keyspace (following the Last Pickle [Data >> Center Switch][lp] playbook), I ran into a "

Re: Schema collision results in multiple data directories per table

2021-10-15 Thread Tom Offermann
ed to what that ticket was trying to > fix. > > Regards > > (1) https://issues.apache.org/jira/browse/CASSANDRA-15158 > > On Fri, 1 Oct 2021 at 18:43, Tom Offermann > wrote: > > > > When adding a datacenter to a keyspace (following the Last Pickle [Data > Ce

Schema collision results in multiple data directories per table

2021-10-01 Thread Tom Offermann
in the Datastax article with great success. ## Questions * My understanding is that running concurrent schema updates should always be avoided, since that can result in schema collisions. But, in this case, I wasn't performing multiple schema updates. I was just running a single `ALTER KEYSPACE` statement. Any idea why a single schema update would result in a schema collision and two data directories per table? * Should I have waited longer before restarting nodes? Perhaps, given enough time, the Cassandra nodes would have all converged on the correct schema version, and this would have resolved on it's own? * Any suggestions for how I can avoid this problem in the future? -- Tom Offermann Lead Software Engineer http://newrelic.com