And it will insert another success is NULL to usage_job table , right ? i deleted the null record and restart usage servoce and see following in usage log :
2021-06-19 18:37:52,912 INFO [cloud.usage.UsageManagerImpl] (main:null) (logid:) Implementation Version is 4.15.0.0 2021-06-19 18:37:55,942 DEBUG [cloud.usage.UsageManagerImpl] (main:null) (logid:) Usage stats aggregation time zone: GMT 2021-06-19 18:37:55,952 DEBUG [cloud.usage.UsageManagerImpl] (main:null) (logid:) Execution Time: Sat Jun 19 00:15:00 UTC 2021 2021-06-19 18:37:55,953 DEBUG [cloud.usage.UsageManagerImpl] (main:null) (logid:) Current Time: Sat Jun 19 18:37:55 UTC 2021 It seem execution time before current time ? Is that normal ? On Sun, Jun 20, 2021 at 2:39 AM Hean Seng <[email protected]> wrote: > > So, I shall stop the Cloudstack-Usage services and delete all record where > success is NULL , and start back the usage service ? > > > On Sat, Jun 19, 2021 at 11:00 PM Daan Hoogland <[email protected]> > wrote: > >> Heanm, >> >> On Sat, Jun 19, 2021 at 5:27 AM Hean Seng <[email protected]> wrote: >> >> > i have no unprocess job , but still have this issue: >> > >> > 2021-06-19 10:09:57,657 DEBUG [cloud.usage.UsageManagerImpl] >> > (Usage-HB-1:null) (logid:) Scheduling Usage job... >> > >> > 2021-06-19 10:09:57,657 INFO [cloud.usage.UsageManagerImpl] >> > (Usage-Job-1:null) (logid:) starting usage job... >> > >> > 2021-06-19 10:09:57,661 DEBUG [cloud.usage.UsageManagerImpl] >> > (Usage-Job-1:null) (logid:) Not owner of usage job, skipping... >> > >> this line ^^ means there is a unprocessed job owned by another proccess >> ... >> >> >> > 2021-06-19 10:09:57,661 INFO [cloud.usage.UsageManagerImpl] >> > (Usage-Job-1:null) (logid:) usage job complete >> > >> > >> > mysql> select id, pid, start_date ,end_date , success, heartbeat from >> > usage_job where success=0 order by start_date desc limit 100; >> > >> you should search for `where success is null` >> >> >> > Empty set (0.00 sec) >> > >> > >> > >> > >> > On Fri, Jun 18, 2021 at 11:01 PM Hean Seng <[email protected]> wrote: >> > >> > > yes, I restarted the mgmt server, wait till can login web interface >> . >> > > then only restart usage . >> > > >> > > I did follwoig also : >> > > >> > > * stop usage server >> > > >> > > * remove the record from usage_job for the unprocessed job >> > > >> > > * start usage server again >> > > >> > > >> > > >> > > Still same thing. Not sure where the Cloudstack keep the Usage server >> > PID >> > > id , and keep getting wrong id. >> > > >> > > >> > > >> > > >> > > >> > > >> > > On Fri, Jun 18, 2021 at 4:31 PM Boris Stoyanov < >> > > [email protected]> wrote: >> > > >> > >> Hi Hean Seng, >> > >> >> > >> Have you waited the management service to come up and then restarted >> > >> usage service? It might be some race condition? >> > >> >> > >> Bobby. >> > >> >> > >> From: Hean Seng <[email protected]> >> > >> Date: Friday, 18 June 2021, 7:31 >> > >> To: [email protected] <[email protected]> >> > >> Subject: Cloudstack Usage --- not owner >> > >> Hi >> > >> >> > >> I have mysql hang, and restarted mysql, restarted cloudstack mgmt, >> and >> > >> usage. >> > >> >> > >> After that facing issue of following: >> > >> >> > >> duration is 120 minutes) >> > >> >> > >> 2021-06-18 04:25:49,656 DEBUG [cloud.usage.UsageManagerImpl] >> > >> (Usage-HB-1:null) (logid:) Scheduling Usage job... >> > >> >> > >> 2021-06-18 04:25:49,657 INFO [cloud.usage.UsageManagerImpl] >> > >> (Usage-Job-1:null) (logid:) starting usage job... >> > >> >> > >> 2021-06-18 04:25:49,669 DEBUG [cloud.usage.UsageManagerImpl] >> > >> (Usage-Job-1:null) (logid:) Not owner of usage job, skipping... >> > >> >> > >> 2021-06-18 04:25:49,669 INFO [cloud.usage.UsageManagerImpl] >> > >> (Usage-Job-1:null) (logid:) usage job complete >> > >> >> > >> >> > >> >> > >> Seems usage pid is not updated to db . >> > >> >> > >> Andybody know how to fix this ? >> > >> >> > >> -- >> > >> Regards, >> > >> Hean Seng >> > >> >> > >> >> > >> >> > >> >> > > >> > > -- >> > > Regards, >> > > Hean Seng >> > > >> > >> > >> > -- >> > Regards, >> > Hean Seng >> > >> >> >> -- >> Daan >> > > > -- > Regards, > Hean Seng > -- Regards, Hean Seng
