rohityadavcloud commented on code in PR #8737:
URL: https://github.com/apache/cloudstack/pull/8737#discussion_r1533432581
##
engine/schema/src/main/java/com/cloud/upgrade/dao/Upgrade41900to41910.java:
##
@@ -61,6 +66,25 @@ public InputStream[] getCleanupScripts() {
rohityadavcloud commented on PR #8737:
URL: https://github.com/apache/cloudstack/pull/8737#issuecomment-2011636987
Merging based on reviews, smoketests and Bobby's manual testing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHu
rohityadavcloud merged PR #8737:
URL: https://github.com/apache/cloudstack/pull/8737
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...
JoaoJandre commented on code in PR #8737:
URL: https://github.com/apache/cloudstack/pull/8737#discussion_r153193
##
engine/schema/src/main/java/com/cloud/upgrade/dao/Upgrade41900to41910.java:
##
@@ -61,6 +66,25 @@ public InputStream[] getCleanupScripts() {
throw
blueorangutan commented on PR #8737:
URL: https://github.com/apache/cloudstack/pull/8737#issuecomment-2008291380
[SF] Trillian test result (tid-9524)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 43389 seconds
Marvin logs:
https://github.com
blueorangutan commented on PR #8737:
URL: https://github.com/apache/cloudstack/pull/8737#issuecomment-2006702290
@rohityadavcloud a [SL] Trillian-Jenkins test job (centos7 mgmt +
kvm-centos7) has been kicked to run smoke tests
--
This is an automated message from the Apache Git Service.
T
rohityadavcloud commented on PR #8737:
URL: https://github.com/apache/cloudstack/pull/8737#issuecomment-2006698671
@blueorangutan test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific
blueorangutan commented on PR #8737:
URL: https://github.com/apache/cloudstack/pull/8737#issuecomment-2006406250
Packaging result [SF]: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 8969
--
This is an automated message from the Apache Git Service.
To respond to the message, please log o
blueorangutan commented on PR #8737:
URL: https://github.com/apache/cloudstack/pull/8737#issuecomment-2006204880
@vishesh92 a [SL] Jenkins job has been kicked to build packages. It will be
bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you
posted as I make progress.
vishesh92 commented on PR #8737:
URL: https://github.com/apache/cloudstack/pull/8737#issuecomment-2006202140
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific co
JoaoJandre commented on PR #8737:
URL: https://github.com/apache/cloudstack/pull/8737#issuecomment-2003875216
Sure @vishesh92 , here are the results:
No index:
```
MariaDB [teste]> ANALYZE SELECT vm_stats.id, vm_stats.vm_id,
vm_stats.mgmt_server_id, vm_stats.timestamp, vm_st
blueorangutan commented on PR #8737:
URL: https://github.com/apache/cloudstack/pull/8737#issuecomment-2000762113
[SF] Trillian test result (tid-9498)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 49936 seconds
Marvin logs:
https://github.com
blueorangutan commented on PR #8737:
URL: https://github.com/apache/cloudstack/pull/8737#issuecomment-2000720694
[SF] Trillian test result (tid-9497)
Environment: vmware-67u3 (x2), Advanced Networking with Mgmt server r8
Total time taken: 48937 seconds
Marvin logs:
https://github.co
blueorangutan commented on PR #8737:
URL: https://github.com/apache/cloudstack/pull/8737#issuecomment-2000697873
[SF] Trillian test result (tid-9496)
Environment: xenserver-71 (x2), Advanced Networking with Mgmt server 7
Total time taken: 48441 seconds
Marvin logs:
https://github.co
vishesh92 commented on PR #8737:
URL: https://github.com/apache/cloudstack/pull/8737#issuecomment-1999705378
@JoaoJandre Can you share query times for this query?
```sql
SELECT vm_stats.id, vm_stats.vm_id, vm_stats.mgmt_server_id,
vm_stats.timestamp, vm_stats.vm_stats_data FROM vm_stat
JoaoJandre commented on code in PR #8737:
URL: https://github.com/apache/cloudstack/pull/8737#discussion_r1526291432
##
engine/schema/src/main/java/com/cloud/upgrade/dao/Upgrade41900to41910.java:
##
@@ -61,6 +66,25 @@ public InputStream[] getCleanupScripts() {
throw
JoaoJandre commented on PR #8737:
URL: https://github.com/apache/cloudstack/pull/8737#issuecomment-1999668050
> @JoaoJandre, could you make some tests listing and removing data with the
proposed index?
@GutoVeronezi, @vishesh92, @rohityadavcloud, @sureshanaparti, @mlsorensen,
@kohrar
blueorangutan commented on PR #8737:
URL: https://github.com/apache/cloudstack/pull/8737#issuecomment-1999324607
@borisstoyanov a [SL] Trillian-Jenkins matrix job (centos7 mgmt +
xenserver71, rocky8 mgmt + vmware67u3, centos7 mgmt + kvmcentos7) has been
kicked to run smoke tests
--
This
borisstoyanov commented on PR #8737:
URL: https://github.com/apache/cloudstack/pull/8737#issuecomment-1999321252
@blueorangutan test matrix
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the spe
blueorangutan commented on PR #8737:
URL: https://github.com/apache/cloudstack/pull/8737#issuecomment-1998711554
[SF] Trillian test result (tid-9478)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 49903 seconds
Marvin logs:
https://github.com
rohityadavcloud commented on PR #8737:
URL: https://github.com/apache/cloudstack/pull/8737#issuecomment-1997344952
fwiw @borisstoyanov is helping to test this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL ab
GutoVeronezi commented on PR #8737:
URL: https://github.com/apache/cloudstack/pull/8737#issuecomment-1997337903
@JoaoJandre, could you make some tests listing and removing data with the
proposed index?
--
This is an automated message from the Apache Git Service.
To respond to the message,
blueorangutan commented on PR #8737:
URL: https://github.com/apache/cloudstack/pull/8737#issuecomment-1997138897
@sureshanaparti a [SL] Trillian-Jenkins test job (centos7 mgmt +
kvm-centos7) has been kicked to run smoke tests
--
This is an automated message from the Apache Git Service.
To
sureshanaparti commented on PR #8737:
URL: https://github.com/apache/cloudstack/pull/8737#issuecomment-1997135543
@blueorangutan test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific
blueorangutan commented on PR #8737:
URL: https://github.com/apache/cloudstack/pull/8737#issuecomment-1997133428
Packaging result [SF]: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 8934
--
This is an automated message from the Apache Git Service.
To respond to the message, please log o
blueorangutan commented on PR #8737:
URL: https://github.com/apache/cloudstack/pull/8737#issuecomment-1997017823
@sureshanaparti a [SL] Jenkins job has been kicked to build packages. It
will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep
you posted as I make progre
sureshanaparti commented on PR #8737:
URL: https://github.com/apache/cloudstack/pull/8737#issuecomment-1997014413
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specif
sureshanaparti commented on PR #8737:
URL: https://github.com/apache/cloudstack/pull/8737#issuecomment-199652
Thanks @vishesh92 @JoaoJandre & others involved. Good to see your insights /
results on indexing the `timestamp`. I think, more tests are required with
varied data and deletion
vishesh92 commented on PR #8737:
URL: https://github.com/apache/cloudstack/pull/8737#issuecomment-1995638271
I have removed the index on `timestamp`.
The index on `(vm_id, timestamp)` should help with faster listing of metrics
for a vm. As @rohityadavcloud said this index helped bring dow
JoaoJandre commented on PR #8737:
URL: https://github.com/apache/cloudstack/pull/8737#issuecomment-1995252882
> > I only noticed this after users complained that listing VMs when
creating a port forwarding was taking >30 seconds to load
>
> @kohrar, the way to address an issue like th
GutoVeronezi commented on PR #8737:
URL: https://github.com/apache/cloudstack/pull/8737#issuecomment-1995191529
As proved by @JoaoJandre in comment
https://github.com/apache/cloudstack/pull/8737#issuecomment-1992241242, the
indiscriminate addition of an index for `timestamp` might cause mor
mlsorensen commented on PR #8737:
URL: https://github.com/apache/cloudstack/pull/8737#issuecomment-1994732867
Yes I think we can all agree that the big one here is index on vm_id.
Missing this is absolutely crushing for default listVirtualMachines calls
(where stats are included). Maybe we
kohrar commented on PR #8737:
URL: https://github.com/apache/cloudstack/pull/8737#issuecomment-1992300251
Thanks for the detailed breakdown on your deletion performance @JoaoJandre.
Since you have so many rows in the vm_stats table, what's the performance like
when you run a list VM command
JoaoJandre commented on PR #8737:
URL: https://github.com/apache/cloudstack/pull/8737#issuecomment-1992241242
> > > @JoaoJandre We can't run explain analyze with delete commands. Adding
the index also helps reduce the load on database while cleaning up the entries.
Otherwise the DB will hav
sureshanaparti commented on PR #8737:
URL: https://github.com/apache/cloudstack/pull/8737#issuecomment-1991877845
> In case of a delete operation with some filter, before deleting the
records database needs to fetch the records from the database. If the index is
not present, this fetchi
vishesh92 commented on PR #8737:
URL: https://github.com/apache/cloudstack/pull/8737#issuecomment-1991855141
> @vishesh92 _vm_stats_ table data, timestamp column cardinality in both
cases are the same? please check/confirm results with the filter matching rows,
and same delete count.
Her
sureshanaparti commented on PR #8737:
URL: https://github.com/apache/cloudstack/pull/8737#issuecomment-1991683719
> > > @JoaoJandre We can't run explain analyze with delete commands. Adding
the index also helps reduce the load on database while cleaning up the entries.
Otherwise the DB will
vishesh92 commented on PR #8737:
URL: https://github.com/apache/cloudstack/pull/8737#issuecomment-1991036038
> > @JoaoJandre We can't run explain analyze with delete commands. Adding
the index also helps reduce the load on database while cleaning up the entries.
Otherwise the DB will have t
JoaoJandre commented on PR #8737:
URL: https://github.com/apache/cloudstack/pull/8737#issuecomment-1983608195
> @JoaoJandre We can't run explain analyze with delete commands. Adding the
index also helps reduce the load on database while cleaning up the entries.
Otherwise the DB will have to
vishesh92 commented on PR #8737:
URL: https://github.com/apache/cloudstack/pull/8737#issuecomment-1981393977
>
> @vishesh92, from the description of your PR:
>
> ```
> This PR adds the upgrade path for 4.19 to 4.19.1 and creates two missing
indexes on vm_stats table
> ...
JoaoJandre commented on PR #8737:
URL: https://github.com/apache/cloudstack/pull/8737#issuecomment-1980809847
> > @vishesh92 could you share a description of the tests that you've done
and their results?
>
> Sure. I created a table with 661683 entries for timestamp ranging from
`1970
vishesh92 commented on PR #8737:
URL: https://github.com/apache/cloudstack/pull/8737#issuecomment-1979497377
> @vishesh92 could you share a description of the tests that you've done and
their results?
Sure. I created a table with 661683 entries for timestamp ranging from
`1970-11-29`
JoaoJandre commented on PR #8737:
URL: https://github.com/apache/cloudstack/pull/8737#issuecomment-1979253335
> > Hey, @vishesh92
> > Adding the `vm_stats` -> `timestamp` index to the table might not
produce the effect that you want. When deleting very specific rows, adding an
index to t
vishesh92 commented on PR #8737:
URL: https://github.com/apache/cloudstack/pull/8737#issuecomment-1978101928
> Hey, @vishesh92
>
> Adding the `vm_stats` -> `timestamp` index to the table might not produce
the effect that you want. When deleting very specific rows, adding an index to
blueorangutan commented on PR #8737:
URL: https://github.com/apache/cloudstack/pull/8737#issuecomment-1977813109
[SF] Trillian test result (tid-9382)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 47963 seconds
Marvin logs:
https://github.com
JoaoJandre commented on code in PR #8737:
URL: https://github.com/apache/cloudstack/pull/8737#discussion_r1511158717
##
engine/schema/src/main/java/com/cloud/upgrade/dao/Upgrade41900to41910.java:
##
@@ -0,0 +1,91 @@
+// Licensed to the Apache Software Foundation (ASF) under one
JoaoJandre commented on PR #8737:
URL: https://github.com/apache/cloudstack/pull/8737#issuecomment-1976576132
Hey, @vishesh92
Adding the `vm_stats` -> `timestamp` index to the table might not produce
the effect that you want. When deleting very specific rows, adding an index to
the
blueorangutan commented on PR #8737:
URL: https://github.com/apache/cloudstack/pull/8737#issuecomment-1976456644
@vishesh92 a [SL] Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has
been kicked to run smoke tests
--
This is an automated message from the Apache Git Service.
To resp
vishesh92 commented on PR #8737:
URL: https://github.com/apache/cloudstack/pull/8737#issuecomment-1976455722
@blueorangutan test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comme
blueorangutan commented on PR #8737:
URL: https://github.com/apache/cloudstack/pull/8737#issuecomment-1976453050
Packaging result [SF]: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 8846
--
This is an automated message from the Apache Git Service.
To respond to the message, please log o
codecov[bot] commented on PR #8737:
URL: https://github.com/apache/cloudstack/pull/8737#issuecomment-1976353616
##
[Codecov](https://app.codecov.io/gh/apache/cloudstack/pull/8737?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
blueorangutan commented on PR #8737:
URL: https://github.com/apache/cloudstack/pull/8737#issuecomment-1976344743
@vishesh92 a [SL] Jenkins job has been kicked to build packages. It will be
bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you
posted as I make progress.
vishesh92 commented on PR #8737:
URL: https://github.com/apache/cloudstack/pull/8737#issuecomment-197634
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific co
blueorangutan commented on PR #8737:
URL: https://github.com/apache/cloudstack/pull/8737#issuecomment-1976334509
@vishesh92 a [SL] Jenkins job has been kicked to build packages. It will be
bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you
posted as I make progress.
vishesh92 commented on PR #8737:
URL: https://github.com/apache/cloudstack/pull/8737#issuecomment-1976332445
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific co
vishesh92 opened a new pull request, #8737:
URL: https://github.com/apache/cloudstack/pull/8737
### Description
This PR adds the upgrade path for 4.19 to 4.19.1 and creates two missing
indexes on vm_stats table:
1. `vm_stats` -> `vm_id`, `timestamp` - To speed up fetching of s
56 matches
Mail list logo
