I believe I've isolated the problem to a bug in django-evolution. Haven't figured out the exact cause just yet, but working on it!
Christian -- Christian Hammond - [email protected] Review Board - http://www.reviewboard.org Beanbag, Inc. - http://www.beanbaginc.com On Fri, Apr 3, 2015 at 7:35 PM, Christian Hammond <[email protected]> wrote: > Hi Zach, > > I'm sorry, I haven't had any time to make progress on this. Been busy with > some of our premium support customers, student program, and a few other > things. I have a couple hours I can spend right now, so I'll see if > anything stands out. > > Where did those SQL statements come from? > > FWIW, Django Evolution has been around since the 1.0 alphas. > > Christian > > -- > Christian Hammond - [email protected] > Review Board - http://www.reviewboard.org > Beanbag, Inc. - http://www.beanbaginc.com > > On Thu, Apr 2, 2015 at 8:13 AM, Zach Auclair <[email protected]> wrote: > >> Hey Christian, >> >> Have you been able to make any progress on debugging this? I just wanted >> to point out that I believe before the current version that to upgrade to >> the current version we were on long ago, our previous maintainer may have >> run some migrations by hand (before this had django_evolutions?). >> >> Some of the sql I found: >> >> ``` >> CREATE INDEX "django_session_expire_date" ON "django_session" >> ("expire_date"); >> >> ALTER TABLE "diffviewer_filediff" ALTER COLUMN "source_file" TYPE >> varchar(1024) USING CAST("source_file" as varchar(1024)); >> ALTER TABLE "diffviewer_filediff" ALTER COLUMN "dest_file" TYPE >> varchar(1024) USING CAST("dest_file" as varchar(1024)); >> ALTER TABLE "diffviewer_diffset" ADD COLUMN "basedir" varchar(256) ; >> UPDATE "diffviewer_diffset" SET "basedir" = '' WHERE "basedir" IS NULL; >> ALTER TABLE "diffviewer_diffset" ALTER COLUMN "basedir" SET NOT NULL; >> ALTER TABLE "diffviewer_filediff" ADD COLUMN "status" varchar(1) ; >> UPDATE "diffviewer_filediff" SET "status" = 'M' WHERE "status" IS NULL; >> ALTER TABLE "diffviewer_filediff" ALTER COLUMN "status" SET NOT NULL; >> ALTER TABLE "reviews_reviewrequest" ADD COLUMN "last_review_timestamp" >> timestamp with time zone NULL ; >> >> UPDATE reviews_reviewrequest >> SET last_review_timestamp = ( >> SELECT reviews_review.timestamp >> FROM reviews_review >> WHERE reviews_review.review_request_id = >> reviews_reviewrequest.id >> AND reviews_review.public >> ORDER BY reviews_review.timestamp DESC >> LIMIT 1); >> >> ALTER TABLE "reviews_reviewrequest" ADD COLUMN "shipit_count" integer ; >> UPDATE "reviews_reviewrequest" SET "shipit_count" = 0 WHERE >> "shipit_count" IS NULL; >> >> UPDATE reviews_reviewrequest >> SET shipit_count = ( >> SELECT COUNT(*) >> FROM reviews_review >> WHERE reviews_review.review_request_id = >> reviews_reviewrequest.id >> AND reviews_review.public >> AND reviews_review.ship_it >> AND reviews_review.base_reply_to_id is NULL); >> >> CREATE TABLE "reviews_defaultreviewer_repository" ( >> "id" serial NOT NULL PRIMARY KEY, >> "defaultreviewer_id" integer NOT NULL, >> "repository_id" integer NOT NULL, >> UNIQUE ("defaultreviewer_id", "repository_id") >> ) >> ; >> ALTER TABLE "reviews_defaultreviewer_repository" ADD CONSTRAINT >> "defaultreviewer_id_refs_id_51486bbd" FOREIGN KEY ("defaultreviewer_id") >> REFERENCES "reviews_defaultreviewer" ("id") DEFERRABLE INITIALLY DEFERRED; >> ALTER TABLE "reviews_defaultreviewer_repository" ADD CONSTRAINT >> "repository_id_refs_id_8161d2ae" FOREIGN KEY ("repository_id") REFERENCES >> "scmtools_repository" ("id") DEFERRABLE INITIALLY DEFERRED; >> ALTER TABLE "reviews_reviewrequest" ALTER COLUMN "repository_id" DROP NOT >> NULL; >> ALTER TABLE "scmtools_repository" ADD COLUMN "raw_file_url" varchar(128) ; >> UPDATE "scmtools_repository" SET "raw_file_url" = '' WHERE "raw_file_url" >> IS NULL; >> ALTER TABLE "scmtools_repository" ALTER COLUMN "raw_file_url" SET NOT >> NULL; >> ALTER TABLE "scmtools_repository" ADD COLUMN "visible" boolean ; >> UPDATE "scmtools_repository" SET "visible" = True WHERE "visible" IS NULL; >> ALTER TABLE "scmtools_repository" ALTER COLUMN "visible" SET NOT NULL; >> ALTER TABLE "scmtools_repository" ALTER COLUMN "path" TYPE varchar(255) >> USING CAST("path" as varchar(255)); >> ALTER TABLE "scmtools_repository" ALTER COLUMN "mirror_path" TYPE >> varchar(255) USING CAST("mirror_path" as varchar(255)); >> ALTER TABLE "scmtools_repository" ALTER COLUMN "raw_file_url" TYPE >> varchar(255) USING CAST("raw_file_url" as varchar(255)); >> >> update siteconfig_siteconfiguration set settings='[redacted]'; >> ``` >> >> -Zach >> >> On Wed, Feb 11, 2015 at 5:18 PM, Zach Auclair <[email protected]> wrote: >> >>> Hey Christian, >>> >>> I ran... >>> >>> `pg_dump --host dbhost --dbname reviewboard --username reviewboard >>> --schema-only > schema-recreate.sql` >>> `pg_dump --data-only --table django_project_version --table >>> django_evolution --host dbhost --dbname reviewboard --username reviewboard >>> > django-table-data-import.sql` >>> >>> ... and have attached the data. >>> >>> Please let me know if you have a more explicit / different way you want >>> the data. >>> >>> -Zach >>> >>> On Wed, Feb 11, 2015 at 4:22 PM, Christian Hammond < >>> [email protected]> wrote: >>> >>>> Hi Zach, >>>> >>>> I wouldn't need any actual confidential data. Just the table schema and >>>> the contents of the django_evolution and django_project_version tables >>>> (which solely contain information on the schema, and nothing confidential >>>> of any sort). >>>> >>>> Christian >>>> >>>> -- >>>> Christian Hammond - [email protected] >>>> Review Board - https://www.reviewboard.org >>>> Beanbag, Inc. - https://www.beanbaginc.com >>>> >>>> -----Original Message----- >>>> From: Zach <[email protected]> >>>> Reply: Zach <[email protected]>> >>>> Date: February 11, 2015 at 1:20:29 PM >>>> To: [email protected] <[email protected]>> >>>> Cc: [email protected] <[email protected]>> >>>> Subject: Re: Issue Migrating Database from 1.5.5 to 2.0.5 (model >>>> updates fail) >>>> >>>> > Hey Christian, >>>> > >>>> > Sorry for the long delay between responses - I've had to put this >>>> project >>>> > on the back-burner for the time being. I am extremely appreciative of >>>> the >>>> > time you've been able to spend trying to help us with this issue >>>> (after >>>> > all, we would love to use the new reviewboard 2.X with gitlab!). >>>> > >>>> > In terms of the raw dump, what tables actually need to have rows in >>>> them? I >>>> > ask this because I will have to check with we are allowed to send >>>> some of >>>> > our information (if it's anything other than the table definition, >>>> and the >>>> > django rows / reviewboard settings, etc). >>>> > >>>> > -Zach >>>> > >>>> > On Tuesday, August 26, 2014 at 5:09:04 AM UTC-4, Christian Hammond >>>> wrote: >>>> > > >>>> > > Hi Zach, >>>> > > >>>> > > I know this isn't going quick. We had a big upgrade to >>>> RBCommons.com to >>>> > > prepare for and execute, and a number of premium support tickets >>>> we've been >>>> > > dealing with, so I'm behind on the community list. >>>> > > >>>> > > I took a look at the dumps, but unfortunately they're not useful to >>>> me. >>>> > > I'll need actual raw SQL dumps that I can directly import into a >>>> Postgres >>>> > > database. Would you be able to provide an importable SQL dump? >>>> > > >>>> > > - Christian >>>> > > >>>> > > -- >>>> > > Christian Hammond - [email protected] >>>> > > Review Board - http://www.reviewboard.org >>>> > > Beanbag, Inc. - http://www.beanbaginc.com >>>> > > >>>> > > >>>> > > On Thu, Aug 21, 2014 at 3:03 PM, Zach Auclair > > > wrote: >>>> > > >>>> > >> Christian, >>>> > >> >>>> > >> I've attached the data that you requested. >>>> > >> >>>> > >> -Zach >>>> > >> >>>> > >> >>>> > >> On Thu, Aug 21, 2014 at 4:41 PM, Christian Hammond < >>>> > >> [email protected] > wrote: >>>> > >> >>>> > >>> Hi Zach, >>>> > >>> >>>> > >>> Sorry, I haven’t had a chance yet. >>>> > >>> >>>> > >>> I just glanced through and the schema does look correct, as does >>>> the >>>> > >>> list of applied evolutions. Can you get me the contents of your >>>> > >>> django_project_version and django_evolution tables? That’ll give >>>> me a >>>> > >>> better understanding of how Django Evolution thinks the world >>>> looks. >>>> > >>> >>>> > >>> Christian >>>> > >>> >>>> > >>> -- >>>> > >>> Christian Hammond - [email protected] >>>> > >>> Review Board - http://www.reviewboard.org >>>> > >>> Beanbag, Inc. - http://www.beanbaginc.com >>>> > >>> >>>> > >>> On August 21, 2014 at 1:24:01 PM, Zach Auclair ([email protected] >>>> > >>> ) wrote: >>>> > >>> >>>> > >>> Hey Christian, >>>> > >>> >>>> > >>> Have you had a chance to look into this? Is there any other >>>> information >>>> > >>> I can get you? >>>> > >>> >>>> > >>> -Zach >>>> > >>> >>>> > >>> >>>> > >>> >>>> > >>> On Fri, Aug 15, 2014 at 10:11 PM, Zach Auclair > >>> > wrote: >>>> > >>> >>>> > >>>> Hi Chris, >>>> > >>>> >>>> > >>>> The output of the "list-evolutions" command is actually included >>>> in my >>>> > >>>> last paragraph in the original post - see "*... in >>>> "list-evolutions" >>>> > >>>> form.*" >>>> > >>>> >>>> > >>>> I have attached the current schema generated by `pg_dump >>>> --schema-only >>>> > >>>> ...`. >>>> > >>>> >>>> > >>>> >>>> > >>>> On Fri, Aug 15, 2014 at 4:37 PM, Christian Hammond < >>>> > >>>> [email protected] > wrote: >>>> > >>>> >>>> > >>>>> Hmm, that shouldn’t be happening. Can you run this and show me >>>> what >>>> > >>>>> you see: >>>> > >>>>> >>>> > >>>>> $ rb-site manage /path/to/site list-evolutions >>>> > >>>>> >>>> > >>>>> Also, can you dump the schema of your database and attach it? >>>> > >>>>> >>>> > >>>>> - Christian >>>> > >>>>> >>>> > >>>>> -- >>>> > >>>>> Christian Hammond - [email protected] >>>> > >>>>> Review Board - http://www.reviewboard.org >>>> > >>>>> Beanbag, Inc. - http://www.beanbaginc.com >>>> > >>>>> >>>> > >>>>> On August 15, 2014 at 12:11:45 PM, Zach ([email protected] >>>> > >>>>> ) wrote: >>>> > >>>>> >>>> > >>>>> I'm trying to "rb-site upgrade" an old 1.5.5 database to 2.0.5. >>>> > >>>>> >>>> > >>>>> *This fails with:* >>>> > >>>>> >>>> > >>>>> Rebuilding directory structure >>>> > >>>>> Updating database. This may take a while. >>>> > >>>>> >>>> > >>>>> The log output below, including warnings and errors, >>>> > >>>>> can be ignored unless upgrade fails. >>>> > >>>>> >>>> > >>>>> ------------------ ------------------ >>>> > >>>>> Creating tables ... >>>> > >>>>> There are unapplied evolutions for auth. >>>> > >>>>> There are unapplied evolutions for contenttypes. >>>> > >>>>> There are unapplied evolutions for sessions. >>>> > >>>>> There are unapplied evolutions for accounts. >>>> > >>>>> There are unapplied evolutions for changedescs. >>>> > >>>>> There are unapplied evolutions for diffviewer. >>>> > >>>>> There are unapplied evolutions for reviews. >>>> > >>>>> There are unapplied evolutions for scmtools. >>>> > >>>>> Project signature has changed - an evolution is required >>>> > >>>>> Installing custom SQL ... >>>> > >>>>> Installing indexes ... >>>> > >>>>> Installed 0 object(s) from 0 fixture(s) >>>> > >>>>> The stored evolutions do not completely resolve all model >>>> changes. >>>> > >>>>> >>>> > >>>>> Run `./manage.py evolve --hint` to see a suggestion for the >>>> changes >>>> > >>>>> required. >>>> > >>>>> >>>> > >>>>> The following are the changes that could not be resolved: >>>> > >>>>> The application markup has been deleted >>>> > >>>>> In model reviews.DefaultReviewer: >>>> > >>>>> In field 'repository': >>>> > >>>>> Property 'null' has changed >>>> > >>>>> In model reviews.ReviewRequest: >>>> > >>>>> In field 'repository': >>>> > >>>>> Property 'null' has changed >>>> > >>>>> CommandError: Your models contain changes that Django Evolution >>>> cannot >>>> > >>>>> resolve automatically. >>>> > >>>>> >>>> > >>>>> >>>> > >>>>> *The suggested --hint's are:* >>>> > >>>>> >>>> > >>>>> #----- Evolution for auth >>>> > >>>>> from django_evolution.mutations import ChangeMeta, DeleteModel >>>> > >>>>> >>>> > >>>>> >>>> > >>>>> MUTATIONS = [ >>>> > >>>>> ChangeMeta('Permission', 'unique_together', ((u'content_type', >>>> > >>>>> u'codename'),)), >>>> > >>>>> DeleteModel('Message') >>>> > >>>>> ] >>>> > >>>>> #---------------------- >>>> > >>>>> #----- Evolution for contenttypes >>>> > >>>>> from django_evolution.mutations import ChangeMeta >>>> > >>>>> >>>> > >>>>> >>>> > >>>>> MUTATIONS = [ >>>> > >>>>> ChangeMeta('ContentType', 'unique_together', (('app_label', >>>> > >>>>> 'model'),)) >>>> > >>>>> ] >>>> > >>>>> #---------------------- >>>> > >>>>> #----- Evolution for sessions >>>> > >>>>> from django_evolution.mutations import ChangeField >>>> > >>>>> >>>> > >>>>> >>>> > >>>>> MUTATIONS = [ >>>> > >>>>> ChangeField('Session', 'expire_date', initial=None, >>>> db_index=True) >>>> > >>>>> ] >>>> > >>>>> #---------------------- >>>> > >>>>> #----- Evolution for accounts >>>> > >>>>> from django_evolution.mutations import AddField, ChangeMeta, >>>> > >>>>> DeleteField >>>> > >>>>> from django.db import models >>>> > >>>>> from djblets.db.fields import JSONField >>>> > >>>>> >>>> > >>>>> >>>> > >>>>> MUTATIONS = [ >>>> > >>>>> AddField('Profile', 'should_send_email', models.BooleanField, >>>> > >>>>> initial=True), >>>> > >>>>> AddField('Profile', 'timezone', models.CharField, >>>> initial=u'UTC', >>>> > >>>>> max_length=30), >>>> > >>>>> AddField('Profile', 'open_an_issue', models.BooleanField, >>>> > >>>>> initial=True), >>>> > >>>>> AddField('Profile', 'extra_data', JSONField, null=True), >>>> > >>>>> AddField('Profile', 'show_closed', models.BooleanField, >>>> > >>>>> initial=True), >>>> > >>>>> AddField('Profile', 'is_private', models.BooleanField, >>>> > >>>>> initial=False), >>>> > >>>>> DeleteField('Profile', 'show_submitted'), >>>> > >>>>> ChangeMeta('ReviewRequestVisit', 'unique_together', ((u'user', >>>> > >>>>> u'review_request'),)) >>>> > >>>>> ] >>>> > >>>>> #---------------------- >>>> > >>>>> #----- Evolution for changedescs >>>> > >>>>> from django_evolution.mutations import AddField >>>> > >>>>> from django.db import models >>>> > >>>>> >>>> > >>>>> >>>> > >>>>> MUTATIONS = [ >>>> > >>>>> AddField('ChangeDescription', 'rich_text', models.BooleanField, >>>> > >>>>> initial=True) >>>> > >>>>> ] >>>> > >>>>> #---------------------- >>>> > >>>>> CommandError: Cannot use hinted evolution: AddField or >>>> ChangeField >>>> > >>>>> mutation for 'FileDiff.status' in 'diffviewer' requires >>>> user-specified >>>> > >>>>> initial value. >>>> > >>>>> >>>> > >>>>> *The completed evolutions are:* >>>> > >>>>> >>>> > >>>>> reviewboard=> select * from django_evolution; >>>> > >>>>> id | version_id | app_label | label >>>> > >>>>> >>>> > >>>>> >>>> ----+------------+-------------+--------------------------------------------- >>>> > >>>>> 1 | 4 | diffviewer | add_parent_diffs >>>> > >>>>> 2 | 4 | reviews | change_descriptions >>>> > >>>>> 3 | 4 | scmtools | bugzilla_url_charfield >>>> > >>>>> 4 | 5 | attachments | file_attachment_orig_filename >>>> > >>>>> 5 | 5 | attachments | file_attachment_file_max_length_512 >>>> > >>>>> 6 | 5 | attachments | file_attachment_repo_info >>>> > >>>>> 7 | 5 | attachments | file_attachment_repo_path_no_index >>>> > >>>>> 8 | 5 | attachments | >>>> > >>>>> file_attachment_repo_revision_max_length_64 >>>> > >>>>> 9 | 5 | site | localsite_public >>>> > >>>>> 10 | 5 | hostingsvcs | account_hosting_url >>>> > >>>>> 11 | 5 | hostingsvcs | account_hosting_url_max_length_255 >>>> > >>>>> 12 | 5 | hostingsvcs | account_unique_together_baseline >>>> > >>>>> (12 rows) >>>> > >>>>> >>>> > >>>>> reviewboard=> \d django_evolution >>>> > >>>>> Table "public.django_evolution" >>>> > >>>>> Column | Type | >>>> > >>>>> Modifiers >>>> > >>>>> >>>> > >>>>> >>>> ------------+------------------------+--------------------------------------------------------------- >>>> > >>>>> id | integer | not null default >>>> > >>>>> nextval('django_evolution_id_seq'::regclass) >>>> > >>>>> version_id | integer | not null >>>> > >>>>> app_label | character varying(200) | not null >>>> > >>>>> label | character varying(100) | not null >>>> > >>>>> Indexes: >>>> > >>>>> "django_evolution_pkey" PRIMARY KEY, btree (id) >>>> > >>>>> "django_evolution_version_id" btree (version_id) >>>> > >>>>> Foreign-key constraints: >>>> > >>>>> "django_evolution_version_id_fkey" FOREIGN KEY (version_id) >>>> > >>>>> REFERENCES django_project_version(id) DEFERRABLE INITIALLY >>>> DEFERRED >>>> > >>>>> >>>> > >>>>> reviewboard=> >>>> > >>>>> >>>> > >>>>> *... in "list-evolutions" form:* >>>> > >>>>> >>>> > >>>>> Applied evolutions for 'attachments': >>>> > >>>>> file_attachment_orig_filename >>>> > >>>>> file_attachment_file_max_length_512 >>>> > >>>>> file_attachment_repo_info >>>> > >>>>> file_attachment_repo_path_no_index >>>> > >>>>> file_attachment_repo_revision_max_length_64 >>>> > >>>>> >>>> > >>>>> Applied evolutions for 'diffviewer': >>>> > >>>>> add_parent_diffs >>>> > >>>>> >>>> > >>>>> Applied evolutions for 'hostingsvcs': >>>> > >>>>> account_hosting_url >>>> > >>>>> account_hosting_url_max_length_255 >>>> > >>>>> account_unique_together_baseline >>>> > >>>>> >>>> > >>>>> Applied evolutions for 'reviews': >>>> > >>>>> change_descriptions >>>> > >>>>> >>>> > >>>>> Applied evolutions for 'scmtools': >>>> > >>>>> bugzilla_url_charfield >>>> > >>>>> >>>> > >>>>> Applied evolutions for 'site': >>>> > >>>>> localsite_public >>>> > >>>>> >>>> > >>>>> >>>> > >>>>> >>>> > >>>>> -- >>>> > >>>>> Get the Review Board Power Pack at >>>> > >>>>> http://www.reviewboard.org/powerpack/ >>>> > >>>>> --- >>>> > >>>>> Sign up for Review Board hosting at RBCommons: >>>> https://rbcommons.com/ >>>> > >>>>> --- >>>> > >>>>> Happy user? Let us know at http://www.reviewboard.org/users/ >>>> > >>>>> --- >>>> > >>>>> You received this message because you are subscribed to the >>>> Google >>>> > >>>>> Groups "reviewboard" group. >>>> > >>>>> To unsubscribe from this group and stop receiving emails from >>>> it, send >>>> > >>>>> an email to [email protected] . >>>> > >>>>> For more options, visit https://groups.google.com/d/optout. >>>> > >>>>> >>>> > >>>>> >>>> > >>>> >>>> > >>> >>>> > >> -- >>>> > >> Get the Review Board Power Pack at >>>> http://www.reviewboard.org/powerpack/ >>>> > >> --- >>>> > >> Sign up for Review Board hosting at RBCommons: >>>> https://rbcommons.com/ >>>> > >> --- >>>> > >> Happy user? Let us know at http://www.reviewboard.org/users/ >>>> > >> --- >>>> > >> You received this message because you are subscribed to the Google >>>> Groups >>>> > >> "reviewboard" group. >>>> > >> To unsubscribe from this group and stop receiving emails from it, >>>> send an >>>> > >> email to [email protected] . >>>> > >> For more options, visit https://groups.google.com/d/optout. >>>> > >> >>>> > > >>>> > > >>>> >>>> >>> >> > -- Supercharge your Review Board with Power Pack: https://www.reviewboard.org/powerpack/ Want us to host Review Board for you? Check out RBCommons: https://rbcommons.com/ Happy user? Let us know! https://www.reviewboard.org/users/ --- You received this message because you are subscribed to the Google Groups "reviewboard" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
