Hi all,
The following is some outline/brainstorm level of stuff to genereate a QA
process documentation. I think it is necessary to have you all to review
this to avoid overlapped work, misunderstanding and mistakes. And I also
left a bunch of questions and blanks to be filled inside it :)
Please help add your comments! Thanks!
-------------------------------------------------------------------------
Hi Sophi,
When I was generating a draft of the process, there's several things are
not quite clear when I reviewed our irc session:
> 13:25 < sophi> yifan: we distinguish between l10n and native language
teams,
> it's not the same members (even if it is the same language)
Would you help distinguish native language teams and l10n team? Do they
have overlapped work between each other? Do they have overlapped work
with pure function test? Thanks!
> 13:26 < yifan> sophi: Is there somewhere on wiki I can read to know them?
> 13:26 < sophi> yifan: there will be one QA session dedicated to l10n, with
> the participation of the localizer, then after that will be regular members
> of QA in native language project that will do the worek
Does this mean the native language testing has dependency on l10n testing
results?
Do you have any news from German team about the process documentation, and
from Rimas about the availability of Litmus?:)
And would you help to outline the localization part below if the structure
is divided properly :) Thank you!
-------------------------------------------------------------------------
Libreoffice QA process - draft and propose to apply from Libreoffice 3.4
The process document aims to be a practical guildline for Libreoffice QA
activities, naturally the process itself canbe evolves through our QA
practice.
[New Feature QA Process]
- New feature testing takes place from dev phase to final release
- New feature testing can be happened with the latest nightly build of
Libreoffice
- The new features list of each build canbe usually found at ????
For example:
http://wiki.documentfoundation.org/ReleaseNotes/3.4
[Release QA Process - Localization]
- I am not sure about the l10n process, but as a tip, information from
Petr shows that current l10n process might be optimized by the fact all
LO localizations are built at the same time, so most functionality just
work the same.
- Moreover if there is some overlapping between native language testing
and function testing. We may want to optimize them as well.
- TBD...
[Release QA Process - Function test]
- The latest Release build (RC) for testing should be available at:
http://www.libreoffice.org/download/pre-releases/
- For each release, a test run will be created in Litmus TCM
tool(http://tcm.documentfoundation.com). QA would need to run them in at
most 10 days after the build ready (based on Release Criteria)
(howto litmus http://wiki.documentfoundation.org/Litmus)
+ Who're going to create test run?
+ When should the test run ready?
+ Who're going to define test cases?
- QA announces RC build pass/fail based on Release Criteria
http://wiki.documentfoundation.org/Release_Criteria
+ Where do we announce the results (mailing list, wiki)?
+ Who will be responsible for announcing?
- Fixed blocker Bug review
[Release QA Process - Automation test]
- The latest Release build (RC) for testing should be available at:
http://www.libreoffice.org/download/pre-releases/
- The testtool is what we used for automation testing:
http://wiki.documentfoundation.org/QA/Testing/Using_Testtool
- Upload (TCM?) and analyze log
- Automation QA announce automation pass/fail based on Release Criteria
+ Where do we announce the results (mailing list, wiki)?
+ Who will be responsible for announcing?
- TBD ...
[Generic Libreoffice QA Process]
QA is being done at any time from develpment phase to final release. This
section aims to provide a guideline to unify those testing related actions
to be followed.
- Blockers Nomination process
http://wiki.documentfoundation.org/Release_Criteria
- Bug report/triage process
http://wiki.documentfoundation.org/BugReport
http://wiki.documentfoundation.org/BugTriage
We may want to add sections in Bug report/triage pages:
Regressions
The bugzilla has a 'regression' keyword in the 'Keywords' field.
Our attitude for a regression problem? I think for the same problem,
the regression problem should have a higher priority than a normal
one.
Bug Severity
Bug Reporter's responsibility to mark it.
Bug Priority
Bug Triager's job - We may need a priority setting guide similar to
release
criteria. Do we have it already somewhere?
--
Yifan Jiang
Libreoffice
Contact: yifan - irc.freenode.net/libreoffice
=============================================
http://www.libreoffice.org/
http://www.documentfoundation.org/
_______________________________________________
LibreOffice mailing list
LibreOffice@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/libreoffice
