I have not upgraded to 15.0.4 yet but found the following jobs being in failed state on my instance:
id | status | job_class_name | table_name | column_name | job_arguments ----+--------+----------------------------------------+------------+-------------+--------------- 17 | 4 | BackfillNamespaceIdForNamespaceRoute | routes | id | [] 18 | 4 | BackfillIssueSearchData | issues | id | [] 19 | 4 | BackfillMemberNamespaceForGroupMembers | members | id | [] 23 | 4 | BackfillGroupFeatures | namespaces | id | [10000] After reading around the docs Patrick linked, I noticed that both Feature.enabled?(:execute_batched_migrations_on_schedule) and Feature.enabled?(:optimize_batched_migrations) returned false. I enabled both features as the docs suggest they should be. After that, I went to /admin/background_migrations and restarted all failed jobs. Shortly after, they finished successfully. I am unsure whether those two features being disabled had anything do to with the issue at all but AIUI it shouldn't hurt to have them enabled. I had a slight hope that this would also resolve #1014904 but it did not. Note: To get gitlab-rails-console to work again, I had to fix /var/lib/gitlab/.gem/bin/rails as that was still pointing to ruby2.5: # gitlab-rails-console /usr/bin/env: ‘ruby2.5’: No such file or directory Cheers, sur5r -- ceterum censeo microsoftem esse delendam.
pgpooIipCb64A.pgp
Description: OpenPGP digital signature