On 01/16/2017 12:19 PM, Jonathan Bryce wrote:
On Jan 16, 2017, at 11:58 AM, Jay S. Bryant <jsbry...@electronicjungle.net>
wrote:
On 01/13/2017 10:29 PM, Mike Perez wrote:
The way validation works is completely up to the project team. In my research
as shown in the Summit etherpad [5] there's a clear trend in projects doing
continuous integration for validation. If we wanted to we could also have the
marketplace give the current CI results, which was also requested in the
feedback from driver maintainers.
Having the CI results reported would be an interesting experiment. I wonder if
having the results even more publicly reported would result in more stable
CI's. It is a dual edged sword however. Given the instability of many CI's it
could make OpenStack look bad to customers who don't understand what they are
looking at. Just my thoughts on that idea.
That’s very useful feedback. Having that kind of background upfront is really
helpful. As we make updates on the display side, we can take into account if
certain attributes are potentially unreliable or at a higher risk of showing
instability and have the interface better support that without it looking like
everything is failing and a river of red X’s. Are there other things that might
be similar?
Jonathan
Jonathan,
Glad to be of assistance.
I think reporting some percentage of success might be the most accurate
way to report the CI results. Not necessarily flagging it good or bed
but leave it for the consumers to see and compare. Also combine that
with Anita's idea of when the CI last successfully reported and I think
it could give a good barometer for consumers. Our systems all have their
rough times so we need to avoid a 'snapshot in time' view and provide
more of a 'activity over time' view. Third party CI is a good barometer
of community activity and attention, but not always 100% accurate.
Obviously there will need to be some information included with the
results explaining what they are and helping guide interpretations.
Jay
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
