Looking. On Fri, Feb 20, 2015 at 4:17 PM, Zameer Manji <zma...@apache.org> wrote:
> The health checker test seems flaky or incorrect. Wickman, can you please > take a look? > > On Fri, Feb 20, 2015 at 4:15 PM, Apache Jenkins Server < > jenk...@builds.apache.org> wrote: > >> See <https://builds.apache.org/job/Aurora/889/changes> >> >> Changes: >> >> [zmanji] Remove single caller methods from AuroraCommandContext >> >> [wickman] Instrument the HealthChecker to export stats. >> >> ------------------------------------------ >> [...truncated 4179 lines...] >> >> >> src/test/python/apache/aurora/executor/test_executor_base.py::TestExecutorBase::test_disconnected >> [32mPASSED [0m >> >> >> src/test/python/apache/aurora/executor/test_executor_base.py::TestExecutorBase::test_error >> [32mPASSED [0m >> >> >> src/test/python/apache/aurora/executor/test_executor_base.py::TestExecutorBase::test_frameworkMessage >> [32mPASSED [0m >> >> >> src/test/python/apache/aurora/executor/test_executor_base.py::TestExecutorBase::test_log >> [32mPASSED [0m >> >> >> src/test/python/apache/aurora/executor/test_executor_base.py::TestExecutorBase::test_registered >> [32mPASSED [0m >> >> >> src/test/python/apache/aurora/executor/test_executor_base.py::TestExecutorBase::test_reregistered >> [32mPASSED [0m >> >> >> src/test/python/apache/aurora/executor/test_executor_base.py::TestExecutorBase::test_send_update >> [32mPASSED [0m >> >> >> src/test/python/apache/aurora/executor/test_executor_base.py::TestExecutorBase::test_status_is_terminal >> [32mPASSED [0m >> >> generated xml file: < >> https://builds.apache.org/job/Aurora/ws/dist/test-results/src.test.python.apache.aurora.executor.executor_base.xml >> > >> [32m [1m============ 8 passed in 0.18 seconds >> ============ [0m >> [1m============== test session starts >> =============== [0m >> platform linux2 -- Python 2.7.6 -- py-1.4.26 -- >> pytest-2.6.4 -- < >> https://builds.apache.org/job/Aurora/ws/build-support/pants.venv/bin/python2.7 >> > >> plugins: cov, timeout >> [1m >> collecting 0 items [0m [1m >> collecting 2 items [0m [1m >> collected 2 items >> [0m >> >> src/test/python/apache/aurora/executor/test_executor_vars.py::test_init >> [32mPASSED [0m >> >> src/test/python/apache/aurora/executor/test_executor_vars.py::test_sample >> [32mPASSED [0m >> >> generated xml file: < >> https://builds.apache.org/job/Aurora/ws/dist/test-results/src.test.python.apache.aurora.executor.executor_vars.xml >> > >> [32m [1m============ 2 passed in 0.11 seconds >> ============ [0m >> [1m============== test session starts >> =============== [0m >> platform linux2 -- Python 2.7.6 -- py-1.4.26 -- >> pytest-2.6.4 -- < >> https://builds.apache.org/job/Aurora/ws/build-support/pants.venv/bin/python2.7 >> > >> plugins: cov, timeout >> [1m >> collecting 0 items [0m [1m >> collecting 1 items [0m [1m >> collecting 1 items [0m [1m >> collecting 1 items [0m [1m >> collected 1 items >> [0m >> >> >> src/test/python/apache/aurora/executor/test_status_manager.py::TestStatusManager::test_run >> [32mPASSED [0m >> >> generated xml file: < >> https://builds.apache.org/job/Aurora/ws/dist/test-results/src.test.python.apache.aurora.executor.status_manager.xml >> > >> [32m [1m============ 1 passed in 0.18 seconds >> ============ [0m >> [1m============== test session starts >> =============== [0m >> platform linux2 -- Python 2.7.6 -- py-1.4.26 -- >> pytest-2.6.4 -- < >> https://builds.apache.org/job/Aurora/ws/build-support/pants.venv/bin/python2.7 >> > >> plugins: cov, timeout >> [1m >> collecting 0 items [0m [1m >> collecting 8 items [0m [1m >> collecting 8 items [0m [1m >> collecting 8 items [0m [1m >> collected 8 items >> [0m >> >> >> src/test/python/apache/aurora/executor/test_thermos_task_runner.py::TestThermosTaskRunnerIntegration::test_integration_success >> [32mPASSED [0m >> >> >> src/test/python/apache/aurora/executor/test_thermos_task_runner.py::TestThermosTaskRunnerIntegration::test_integration_failed >> [32mPASSED [0m >> >> >> src/test/python/apache/aurora/executor/test_thermos_task_runner.py::TestThermosTaskRunnerIntegration::test_integration_stop >> [32mPASSED [0m >> >> >> src/test/python/apache/aurora/executor/test_thermos_task_runner.py::TestThermosTaskRunnerIntegration::test_integration_lose >> [32mPASSED [0m >> >> >> src/test/python/apache/aurora/executor/test_thermos_task_runner.py::TestThermosTaskRunnerIntegration::test_integration_quitquitquit >> [32mPASSED [0m >> >> >> src/test/python/apache/aurora/executor/test_thermos_task_runner.py::TestThermosTaskRunnerIntegration::test_thermos_normal_exit_status >> [32mPASSED [0m >> >> >> src/test/python/apache/aurora/executor/test_thermos_task_runner.py::TestThermosTaskRunnerIntegration::test_thermos_abnormal_exit_statuses >> [32mPASSED [0m >> >> >> src/test/python/apache/aurora/executor/test_thermos_task_runner.py::TestThermosTaskRunnerIntegration::test_thermos_runner_killed >> [32mPASSED [0m >> >> generated xml file: < >> https://builds.apache.org/job/Aurora/ws/dist/test-results/src.test.python.apache.aurora.executor.thermos_task_runner.xml >> > >> [32m [1m=========== 8 passed in 76.21 seconds >> ============ [0m >> [1m============== test session starts >> =============== [0m >> platform linux2 -- Python 2.7.6 -- py-1.4.26 -- >> pytest-2.6.4 -- < >> https://builds.apache.org/job/Aurora/ws/build-support/pants.venv/bin/python2.7 >> > >> plugins: cov, timeout >> [1m >> collecting 0 items [0m [1m >> collecting 8 items [0m [1m >> collected 8 items >> [0m >> >> >> src/test/python/apache/aurora/executor/common/test_announcer.py::test_serverset_join_thread >> [32mPASSED [0m >> >> >> src/test/python/apache/aurora/executor/common/test_announcer.py::test_announcer_under_normal_circumstances >> [32mPASSED [0m >> >> >> src/test/python/apache/aurora/executor/common/test_announcer.py::test_announcer_on_expiration >> [33mSKIPPED [0m >> >> >> src/test/python/apache/aurora/executor/common/test_announcer.py::test_announcer_under_abnormal_circumstances >> [33mSKIPPED [0m >> >> >> src/test/python/apache/aurora/executor/common/test_announcer.py::test_make_empty_endpoints >> [32mPASSED [0m >> >> >> src/test/python/apache/aurora/executor/common/test_announcer.py::test_announcer_provider_with_timeout >> [32mPASSED [0m >> >> >> src/test/python/apache/aurora/executor/common/test_announcer.py::test_default_announcer_provider >> [32mPASSED [0m >> >> >> src/test/python/apache/aurora/executor/common/test_announcer.py::test_default_announcer_provider_without_announce >> [32mPASSED [0m >> >> generated xml file: < >> https://builds.apache.org/job/Aurora/ws/dist/test-results/src.test.python.apache.aurora.executor.common.announcer.xml >> > >> [32m [1m====== 6 passed, 2 skipped in 0.67 seconds >> ======= [0m >> [1m============== test session starts >> =============== [0m >> platform linux2 -- Python 2.7.6 -- py-1.4.26 -- >> pytest-2.6.4 -- < >> https://builds.apache.org/job/Aurora/ws/build-support/pants.venv/bin/python2.7 >> > >> plugins: cov, timeout >> [1m >> collecting 0 items [0m [1m >> collecting 5 items [0m [1m >> collected 5 items >> [0m >> >> >> src/test/python/apache/aurora/executor/common/test_directory_sandbox.py::test_directory_sandbox >> [32mPASSED [0m >> >> >> src/test/python/apache/aurora/executor/common/test_directory_sandbox.py::test_create >> [32mPASSED [0m >> >> >> src/test/python/apache/aurora/executor/common/test_directory_sandbox.py::test_user_does_not_exist >> [32mPASSED [0m >> >> >> src/test/python/apache/aurora/executor/common/test_directory_sandbox.py::test_create_ioerror >> [32mPASSED [0m >> >> >> src/test/python/apache/aurora/executor/common/test_directory_sandbox.py::test_destroy_ioerror >> [32mPASSED [0m >> >> generated xml file: < >> https://builds.apache.org/job/Aurora/ws/dist/test-results/src.test.python.apache.aurora.executor.common.directory_sandbox.xml >> > >> [32m [1m============ 5 passed in 0.07 seconds >> ============ [0m >> [1m============== test session starts >> =============== [0m >> platform linux2 -- Python 2.7.6 -- py-1.4.26 -- >> pytest-2.6.4 -- < >> https://builds.apache.org/job/Aurora/ws/build-support/pants.venv/bin/python2.7 >> > >> plugins: cov, timeout >> [1m >> collecting 0 items [0m [1m >> collecting 4 items [0m [1m >> collected 4 items >> [0m >> >> >> src/test/python/apache/aurora/executor/common/test_executor_detector.py::test_find_root >> [32mPASSED [0m >> >> >> src/test/python/apache/aurora/executor/common/test_executor_detector.py::test_match_inverse >> [32mPASSED [0m >> >> >> src/test/python/apache/aurora/executor/common/test_executor_detector.py::test_bad_match >> [32mPASSED [0m >> >> >> src/test/python/apache/aurora/executor/common/test_executor_detector.py::test_integration >> [32mPASSED [0m >> >> generated xml file: < >> https://builds.apache.org/job/Aurora/ws/dist/test-results/src.test.python.apache.aurora.executor.common.executor_detector.xml >> > >> [32m [1m============ 4 passed in 0.04 seconds >> ============ [0m >> [1m============== test session starts >> =============== [0m >> platform linux2 -- Python 2.7.6 -- py-1.4.26 -- >> pytest-2.6.4 -- < >> https://builds.apache.org/job/Aurora/ws/build-support/pants.venv/bin/python2.7 >> > >> plugins: cov, timeout >> [1m >> collecting 0 items [0m [1m >> collecting 2 items [0m [1m >> collecting 2 items [0m [1m >> collecting 2 items [0m [1m >> collected 2 items >> [0m >> >> >> src/test/python/apache/aurora/executor/common/test_executor_timeout.py::TestExecutorTimeout::test_run >> [32mPASSED [0m >> >> >> src/test/python/apache/aurora/executor/common/test_executor_timeout.py::TestExecutorTimeout::test_run_timeout >> [32mPASSED [0m >> >> generated xml file: < >> https://builds.apache.org/job/Aurora/ws/dist/test-results/src.test.python.apache.aurora.executor.common.executor_timeout.xml >> > >> [32m [1m============ 2 passed in 0.07 seconds >> ============ [0m >> [1m============== test session starts >> =============== [0m >> platform linux2 -- Python 2.7.6 -- py-1.4.26 -- >> pytest-2.6.4 -- < >> https://builds.apache.org/job/Aurora/ws/build-support/pants.venv/bin/python2.7 >> > >> plugins: cov, timeout >> [1m >> collecting 0 items [0m [1m >> collecting 4 items [0m [1m >> collecting 5 items [0m [1m >> collecting 12 items [0m [1m >> collecting 12 items [0m [1m >> collected 12 items >> [0m >> >> >> src/test/python/apache/aurora/executor/common/test_health_checker.py::TestHealthChecker::test_consecutive_failures >> [31mFAILED [0m >> >> >> src/test/python/apache/aurora/executor/common/test_health_checker.py::TestHealthChecker::test_health_checker_metrics >> [32mPASSED [0m >> >> >> src/test/python/apache/aurora/executor/common/test_health_checker.py::TestHealthChecker::test_initial_interval_2x >> [32mPASSED [0m >> >> >> src/test/python/apache/aurora/executor/common/test_health_checker.py::TestHealthChecker::test_initial_interval_whatev >> [32mPASSED [0m >> >> >> src/test/python/apache/aurora/executor/common/test_health_checker.py::TestHealthCheckerProvider::test_from_assigned_task >> [32mPASSED [0m >> >> >> src/test/python/apache/aurora/executor/common/test_health_checker.py::TestThreadedHealthChecker::test_maybe_update_failure_count >> [32mPASSED [0m >> >> >> src/test/python/apache/aurora/executor/common/test_health_checker.py::TestThreadedHealthChecker::test_perform_check_if_not_disabled_no_snooze_file >> [32mPASSED [0m >> >> >> src/test/python/apache/aurora/executor/common/test_health_checker.py::TestThreadedHealthChecker::test_perform_check_if_not_disabled_snooze_file_exists >> [32mPASSED [0m >> >> >> src/test/python/apache/aurora/executor/common/test_health_checker.py::TestThreadedHealthChecker::test_perform_check_if_not_disabled_snooze_file_is_none >> [32mPASSED [0m >> >> >> src/test/python/apache/aurora/executor/common/test_health_checker.py::TestThreadedHealthChecker::test_run >> [32mPASSED [0m >> >> >> src/test/python/apache/aurora/executor/common/test_health_checker.py::TestThreadedHealthChecker::test_start >> [32mPASSED [0m >> >> >> src/test/python/apache/aurora/executor/common/test_health_checker.py::TestThreadedHealthChecker::test_stop >> [32mPASSED [0m >> >> ==================== FAILURES ==================== >> __ TestHealthChecker.test_consecutive_failures ___ >> >> self = <common.test_health_checker.TestHealthChecker >> testMethod=test_consecutive_failures> >> >> [1m def test_consecutive_failures(self): [0m >> [1m '''Verify that a task is unhealthy only >> after max_consecutive_failures is exceeded''' [0m >> [1m initial_interval_secs = 2 [0m >> [1m interval_secs = 1 [0m >> [1m self.append_health_checks(False, >> num_calls=2) [0m >> [1m self.append_health_checks(True) [0m >> [1m self.append_health_checks(False, >> num_calls=3) [0m >> [1m hct = HealthChecker( [0m >> [1m self._checker.health, [0m >> [1m interval_secs=interval_secs, [0m >> [1m >> initial_interval_secs=initial_interval_secs, [0m >> [1m max_consecutive_failures=2, [0m >> [1m clock=self._clock) [0m >> [1m hct.start() [0m >> [1m [0m >> [1m # 2 consecutive health check failures >> followed by a successful health check. [0m >> [1m self._clock.tick(initial_interval_secs) [0m >> [1m assert >> self._clock.converge(threads=[hct.threaded_health_checker], timeout=1) [0m >> [1m assert hct.status is None [0m >> [1m> assert >> hct.metrics.sample()['consecutive_failures'] == 1 [0m >> [1m [31mE AssertionError: assert 0 == 1 [0m >> >> >> src/test/python/apache/aurora/executor/common/test_health_checker.py:108: >> AssertionError >> -------------- Captured stderr call -------------- >> [<twitter.common.testing.clock.ThreadedClock object >> at 0x7fc87bd87f10>] Time now: 0.0 >> [<twitter.common.testing.clock.ThreadedClock object >> at 0x7fc87bd87f10>] Time now: 2.0 >> generated xml file: < >> https://builds.apache.org/job/Aurora/ws/dist/test-results/src.test.python.apache.aurora.executor.common.health_checker.xml >> > >> [1m [31m====== 1 failed, 11 passed in 0.77 seconds >> ======= [0m >> src.test.python.apache.aurora.admin.admin >> ..... SUCCESS >> >> src.test.python.apache.aurora.admin.host_maintenance >> ..... SUCCESS >> src.test.python.apache.aurora.admin.maintenance >> ..... SUCCESS >> src.test.python.apache.aurora.client.api.api >> ..... SUCCESS >> >> src.test.python.apache.aurora.client.api.instance_watcher >> ..... SUCCESS >> >> src.test.python.apache.aurora.client.api.job_monitor >> ..... SUCCESS >> src.test.python.apache.aurora.client.api.mux >> ..... SUCCESS >> >> src.test.python.apache.aurora.client.api.quota_check >> ..... SUCCESS >> src.test.python.apache.aurora.client.api.restarter >> ..... SUCCESS >> >> src.test.python.apache.aurora.client.api.scheduler_client >> ..... SUCCESS >> src.test.python.apache.aurora.client.api.sla >> ..... SUCCESS >> src.test.python.apache.aurora.client.api.task_util >> ..... SUCCESS >> src.test.python.apache.aurora.client.api.updater >> ..... SUCCESS >> >> src.test.python.apache.aurora.client.api.updater_util >> ..... SUCCESS >> src.test.python.apache.aurora.client.base >> ..... SUCCESS >> src.test.python.apache.aurora.client.binding_helper >> ..... SUCCESS >> src.test.python.apache.aurora.client.cli.api >> ..... SUCCESS >> src.test.python.apache.aurora.client.cli.client >> ..... SUCCESS >> >> src.test.python.apache.aurora.client.cli.command_hooks >> ..... SUCCESS >> src.test.python.apache.aurora.client.cli.config >> ..... SUCCESS >> src.test.python.apache.aurora.client.cli.context >> ..... SUCCESS >> src.test.python.apache.aurora.client.cli.cron >> ..... SUCCESS >> src.test.python.apache.aurora.client.cli.inspect >> ..... SUCCESS >> src.test.python.apache.aurora.client.cli.job >> ..... SUCCESS >> src.test.python.apache.aurora.client.cli.plugins >> ..... SUCCESS >> src.test.python.apache.aurora.client.cli.quota >> ..... SUCCESS >> src.test.python.apache.aurora.client.cli.sla >> ..... SUCCESS >> src.test.python.apache.aurora.client.cli.supdate >> ..... SUCCESS >> src.test.python.apache.aurora.client.cli.task >> ..... SUCCESS >> src.test.python.apache.aurora.client.cli.update >> ..... SUCCESS >> src.test.python.apache.aurora.client.cli.version >> ..... SUCCESS >> src.test.python.apache.aurora.client.config >> ..... SUCCESS >> src.test.python.apache.aurora.client.factory >> ..... SUCCESS >> >> src.test.python.apache.aurora.client.hooks.hooked_api >> ..... SUCCESS >> >> src.test.python.apache.aurora.client.hooks.non_hooked_api >> ..... SUCCESS >> >> src.test.python.apache.aurora.common.test_aurora_job_key >> ..... SUCCESS >> src.test.python.apache.aurora.common.test_cluster >> ..... SUCCESS >> >> src.test.python.apache.aurora.common.test_cluster_option >> ..... SUCCESS >> src.test.python.apache.aurora.common.test_clusters >> ..... SUCCESS >> >> src.test.python.apache.aurora.common.test_http_signaler >> ..... SUCCESS >> >> src.test.python.apache.aurora.common.test_pex_version >> ..... SUCCESS >> src.test.python.apache.aurora.common.test_shellify >> ..... SUCCESS >> src.test.python.apache.aurora.common.test_transport >> ..... SUCCESS >> src.test.python.apache.aurora.config.test_base >> ..... SUCCESS >> >> src.test.python.apache.aurora.config.test_constraint_parsing >> ..... SUCCESS >> src.test.python.apache.aurora.config.test_loader >> ..... SUCCESS >> src.test.python.apache.aurora.config.test_thrift >> ..... SUCCESS >> >> src.test.python.apache.aurora.executor.common.announcer >> ..... SUCCESS >> >> src.test.python.apache.aurora.executor.common.directory_sandbox >> ..... SUCCESS >> >> src.test.python.apache.aurora.executor.common.executor_detector >> ..... SUCCESS >> >> src.test.python.apache.aurora.executor.common.executor_timeout >> ..... SUCCESS >> >> src.test.python.apache.aurora.executor.common.health_checker >> ..... FAILURE >> >> src.test.python.apache.aurora.executor.common.path_detector >> ..... SUCCESS >> >> src.test.python.apache.aurora.executor.common.task_info >> ..... SUCCESS >> >> src.test.python.apache.aurora.executor.executor_base >> ..... SUCCESS >> >> src.test.python.apache.aurora.executor.executor_vars >> ..... SUCCESS >> >> src.test.python.apache.aurora.executor.status_manager >> ..... SUCCESS >> >> src.test.python.apache.aurora.executor.thermos_task_runner >> ..... SUCCESS >> src.test.python.apache.thermos.common.test_pathspec >> ..... SUCCESS >> >> src.test.python.apache.thermos.core.test_runner_integration >> ..... SUCCESS >> src.test.python.apache.thermos.monitoring.test_disk >> ..... SUCCESS >> >> FAILURE >> >> [31m >> FAILURE [0m >> Build step 'Execute shell' marked build as failure >> Recording test results >> > > > > -- > Zameer Manji >
