Hello,
I already talked with Alex through IRC: Testing copy version with a
smaller scale project was completely fine.
As ianw shared:
http://lists.openstack.org/pipermail/openstack-infra/2017-January/005058.html
and
http://lists.openstack.org/pipermail/openstack-i18n/2017-January/002670.html
,
gc and database status might be suspected which contribute to too slow
Zanata.
Some additional thoughts from me are:
- Different mysql driver from Trusty:
https://review.openstack.org/#/c/406398/1/manifests/init.pp
might contribute, after seeing Alex's comment on "mysql driver".
- Current master version of openstack-manuals in translate.o.o (not
translate-dev.o.o) has 17,423,952 words, which is larger
than the number of words for versions: test-version-creation2,
test-version-creation, and stable-newton.
If it is really the bug from Zanata, then future execution of
creating a new version: "stable-ocata" after Ocata documentation release
will contribute to production server downtime.
@clarkb, then might it be better to also test with larger memory size?
Or testing openstackid-dev for a while would be a good idea now?
One correction: I thought
http://git.openstack.org/cgit/openstack-infra/system-config/tree/manifests/site.pp#n1392
enumerates the root admins in translate.openstack.org and also
translate-dev.openstack.org, but it was my complete
misinterpretation on the file. After deeper investigation, I have found
that the list is for Zanata instance admins
: https://review.openstack.org/#/c/172528/ .
With many thanks,
/Ian
Alex Eng wrote on 1/16/2017 1:11 PM:
Hi,
After looking at the stack trace (log) and checking with our team
members, we agree that the memory issue might be due to Zanata and the
way it handle large project in copy version.
There are few factors which contribute to this:
- mysql driver
- way of Zanata handling large project during copy version
Obviously we didn't use the same large scale project for testing
before, that is why the issue wasn't visible.
To keep things going, I suggest test copy version on a smaller scale
project, while we try to implement the solution in Zanata.
Let me know what you think.
---------------------------------------------
Alex Eng
Senior Software Engineer
Globalisation Tools Engineering
DID:+61 3514 8262 <callto:+61+3514+8262>
Mobile:+614 2335 3457 <callto:+614+2335+3457>
Red Hat, Asia-Pacific Pty Ltd
Level 1, 193 North Quay
Brisbane 4000
Office:+61 7 3514 8100 <callto:+61+7+3514+8100>
Fax:+61 7 3514 8199 <callto:+61+7+3514+8199>
Website:www.redhat.com <http://www.redhat.com/>
On Mon, Jan 16, 2017 at 9:25 AM, Alex Eng <a...@redhat.com
<mailto:a...@redhat.com>> wrote:
Hi,
Before increasing the memory size, there's few info we need to
find out:
- JVM memory allocation (need the Zanata 3.9.6 url to get access
to the info)
- Ian, do you have the log file (stack trace) of when the error
occur? Also, which Zanata you were using to test 3.9.6?
https://translate-dev.openstack.org
<https://translate-dev.openstack.org> is not upgraded yet.
I suspect its the JVM allocation which might need some tweak. But
its best it I can access to the 3.9.6-dev and try to replicate the
error from there.
---------------------------------------------
Alex Eng
Senior Software Engineer
Globalisation Tools Engineering
DID:+61 3514 8262 <callto:+61+3514+8262>
Mobile:+614 2335 3457 <callto:+614+2335+3457>
Red Hat, Asia-Pacific Pty Ltd
Level 1, 193 North Quay
Brisbane 4000
Office:+61 7 3514 8100 <callto:+61+7+3514+8100>
Fax:+61 7 3514 8199 <callto:+61+7+3514+8199>
Website:www.redhat.com <http://www.redhat.com/>
On Mon, Jan 16, 2017 at 8:01 AM, Alex Eng <a...@redhat.com
<mailto:a...@redhat.com>> wrote:
How much memory do we think will be necessary? The current
instance is
an 8GB RAM instance. As long as we have quota for it,
there should not
be a problem redeploying on a bigger instance.
Can we look at 10GB? and run the same test again to make sure
its running alright.
I assume mysql DB is running on separate server?
I have commented on this change. I would prefer we not
backup the dev
server and only backup the production server. We don't
treat our dev
servers as permanent or reliable installs as they may need
to be
reloaded for various reasons to aid development and
testing. Also we
should set up backups similar to how we do the other
services (I left a
comment on this change with a link to an example).
Agree. I would suggest if possible, sync DB data from prod to
dev weekly.
This would help on allowing us to debug if there's any issue
in prod.
This is something that can be discussed with the infra
team, typically
we would give access to someone assisting with
implementation to the
-dev server while keeping the production server as
infra-root only. I
will make sure fungi sees this.
Again, if we can constantly sync prod to dev instance, we will
only need root access to dev server.
---------------------------------------------
Alex Eng
Senior Software Engineer
Globalisation Tools Engineering
DID:+61 3514 8262 <callto:+61+3514+8262>
Mobile:+614 2335 3457 <callto:+614+2335+3457>
Red Hat, Asia-Pacific Pty Ltd
Level 1, 193 North Quay
Brisbane 4000
Office:+61 7 3514 8100 <callto:+61+7+3514+8100>
Fax:+61 7 3514 8199 <callto:+61+7+3514+8199>
Website:www.redhat.com <http://www.redhat.com/>
On Sat, Jan 14, 2017 at 7:31 AM, Clark Boylan
<cboy...@sapwetik.org <mailto:cboy...@sapwetik.org>> wrote:
On Thu, Jan 12, 2017, at 02:36 PM, Ian Y. Choi wrote:
> - I18n team completed tests with current Zanata (3.7.3)
with Xenial [1],
> but found one error
> mentioned in [2]. It seems that more memory size
allocation for
> Zanata with Java 8 is needed.
> Could you upgrade the memory size for translate-dev
server first to
> test again?
How much memory do we think will be necessary? The current
instance is
an 8GB RAM instance. As long as we have quota for it,
there should not
be a problem redeploying on a bigger instance.
> - I remember that newer version of openstackid needs to
be tested with
> translate-dev.
> So I have just uploaded this patch:
> https://review.openstack.org/#/c/419667/
<https://review.openstack.org/#/c/419667/> .
> I18n team needs more tests, but I think it is a good
time to change
> to openstackid-dev for such testing.
> Please ping me after openstackid-dev test with
translate-dev is
> completed with no error :)
>
> - On last I18n team meeting, I18n team recognized that
the backup would
> be so important.
> Is there more disks for such backup on translate-dev and
> translate.o.o server?
> And the approach implementing like [3] looks quite a
good idea I
> think. Thanks, Frank!
I have commented on this change. I would prefer we not
backup the dev
server and only backup the production server. We don't
treat our dev
servers as permanent or reliable installs as they may need
to be
reloaded for various reasons to aid development and
testing. Also we
should set up backups similar to how we do the other
services (I left a
comment on this change with a link to an example).
> - Can I have root access to translate-dev and translate
server?
This is something that can be discussed with the infra
team, typically
we would give access to someone assisting with
implementation to the
-dev server while keeping the production server as
infra-root only. I
will make sure fungi sees this.
Hope this helps,
Clark
_______________________________________________
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
