Okay, this is reproducible within out vagrant environment (but not on OS X 10.9..?)
https://issues.apache.org/jira/browse/AURORA-682 On Wed, Sep 3, 2014 at 4:34 PM, Apache Jenkins Server < jenk...@builds.apache.org> wrote: > See <https://builds.apache.org/job/Aurora/531/changes> > > Changes: > > [kevints] Fix aurora run in end to end tests. > > ------------------------------------------ > [...truncated 6715 lines...] > src/test/python/apache/aurora/config/test_base.py::test_all_static PASSED > src/test/python/apache/aurora/config/test_base.py::test_binding PASSED > src/test/python/apache/aurora/config/test_base.py::test_cycle PASSED > src/test/python/apache/aurora/config/test_base.py::test_empty_config PASSED > src/test/python/apache/aurora/config/test_base.py::test_simple_config > PASSED > src/test/python/apache/aurora/config/test_base.py::test_ports PASSED > src/test/python/apache/aurora/config/test_base.py::test_static_port_aliasing > PASSED > > =========================== 7 passed in 1.30 seconds > =========================== > ============================= test session starts > ============================== > platform linux2 -- Python 2.7.6 -- py-1.4.23 -- pytest-2.6.1 -- > /usr/bin/python2.7 > plugins: cov, timeout > collecting ... collected 2 items > > src/test/python/apache/thermos/core/test_helper.py::test_this_is_really_our_pid > PASSED > src/test/python/apache/thermos/core/test_helper.py::test_scan_process > WARNING:root: Error gathering information on pid 12345: > WARNING:root: Error gathering information on pid 13337: > PASSED > > =========================== 2 passed in 0.15 seconds > =========================== > ============================= test session starts > ============================== > platform linux2 -- Python 2.7.6 -- py-1.4.23 -- pytest-2.6.1 -- > /usr/bin/python2.7 > plugins: cov, timeout > collecting ... collected 1 items > > src/test/python/apache/aurora/common/test_shellify.py::test_shellify PASSED > > =========================== 1 passed in 0.01 seconds > =========================== > ============================= test session starts > ============================== > platform linux2 -- Python 2.7.6 -- py-1.4.23 -- pytest-2.6.1 -- > /usr/bin/python2.7 > plugins: cov, timeout > collecting ... collected 9 items > > src/test/python/apache/aurora/admin/test_host_maintenance.py <- < > https://builds.apache.org/job/Aurora/ws/src/test/python/apache/aurora/admin/test_host_maintenance.py>::TestHostMaintenance::test_check_status > PASSED > src/test/python/apache/aurora/admin/test_host_maintenance.py <- < > https://builds.apache.org/job/Aurora/ws/src/test/python/apache/aurora/admin/test_host_maintenance.py>::TestHostMaintenance::test_complete_maintenance > PASSED > src/test/python/apache/aurora/admin/test_host_maintenance.py <- < > https://builds.apache.org/job/Aurora/ws/src/test/python/apache/aurora/admin/test_host_maintenance.py>::TestHostMaintenance::test_drain_hosts > WARNING:root:us-west-002.example.com is currently in status SCHEDULED > WARNING:root:us-west-001.example.com is currently in status SCHEDULED > WARNING:root:us-west-003.example.com is currently in status SCHEDULED > WARNING:root:us-west-002.example.com is currently in status DRAINING > WARNING:root:us-west-001.example.com is currently in status DRAINING > WARNING:root:us-west-003.example.com is currently in status DRAINING > WARNING:root:us-west-001.example.com is currently in status DRAINING > PASSED > src/test/python/apache/aurora/admin/test_host_maintenance.py <- < > https://builds.apache.org/job/Aurora/ws/src/test/python/apache/aurora/admin/test_host_maintenance.py>::TestHostMaintenance::test_end_maintenance > PASSED > src/test/python/apache/aurora/admin/test_host_maintenance.py <- < > https://builds.apache.org/job/Aurora/ws/src/test/python/apache/aurora/admin/test_host_maintenance.py>::TestHostMaintenance::test_perform_maintenance > PASSED > src/test/python/apache/aurora/admin/test_host_maintenance.py <- < > https://builds.apache.org/job/Aurora/ws/src/test/python/apache/aurora/admin/test_host_maintenance.py>::TestHostMaintenance::test_perform_maintenance_partial_sla_failure > WARNING:root:Some hosts did not pass SLA check and will not be drained! > Skipping hosts: set(['us-west-001.example.com']) > us-west-001.example.com > PASSED > src/test/python/apache/aurora/admin/test_host_maintenance.py <- < > https://builds.apache.org/job/Aurora/ws/src/test/python/apache/aurora/admin/test_host_maintenance.py>::TestHostMaintenance::test_start_maintenance > WARNING:root:Skipping maintenance for unknown hosts: set([' > us-west-001.example.com', 'us-west-003.example.com', ' > us-west-002.example.com']) > PASSED > src/test/python/apache/aurora/admin/test_host_maintenance.py <- < > https://builds.apache.org/job/Aurora/ws/src/test/python/apache/aurora/admin/test_host_maintenance.py>::test_default_grouping > PASSED > src/test/python/apache/aurora/admin/test_host_maintenance.py <- < > https://builds.apache.org/job/Aurora/ws/src/test/python/apache/aurora/admin/test_host_maintenance.py>::test_rack_grouping > PASSED > > =========================== 9 passed in 0.56 seconds > =========================== > ============================= test session starts > ============================== > platform linux2 -- Python 2.7.6 -- py-1.4.23 -- pytest-2.6.1 -- > /usr/bin/python2.7 > plugins: cov, timeout > collecting ... collected 2 items > > src/test/python/apache/aurora/executor/common/test_executor_timeout.py::TestExecutorTimeout::test_run > PASSED > src/test/python/apache/aurora/executor/common/test_executor_timeout.py::TestExecutorTimeout::test_run_timeout > ERROR:root:Executor timing out. > PASSED > > =========================== 2 passed in 0.07 seconds > =========================== > ============================= test session starts > ============================== > platform linux2 -- Python 2.7.6 -- py-1.4.23 -- pytest-2.6.1 -- > /usr/bin/python2.7 > plugins: cov, timeout > collecting ... collected 5 items > > src/test/python/apache/aurora/executor/test_thermos_task_runner.py::TestThermosTaskRunnerIntegration::test_integration_success > Writing log files to disk in /tmp/tmpVh5dUz > Build operating on top level addresses: set([BuildFileAddress(< > https://builds.apache.org/job/Aurora/ws/src/main/python/apache/aurora/executor/bin/BUILD,> > thermos_runner)]) > Building PythonBinary PythonBinary(BuildFileAddress(< > https://builds.apache.org/job/Aurora/ws/src/main/python/apache/aurora/executor/bin/BUILD,> > thermos_runner)): > Wrote <https://builds.apache.org/job/Aurora/ws/dist/thermos_runner.pex> > Writing log files to disk in /tmp/tmpVh5dUz > PASSED > src/test/python/apache/aurora/executor/test_thermos_task_runner.py::TestThermosTaskRunnerIntegration::test_integration_failed > Writing log files to disk in /tmp/tmpVh5dUz > PASSED > src/test/python/apache/aurora/executor/test_thermos_task_runner.py::TestThermosTaskRunnerIntegration::test_integration_stop > Writing log files to disk in /tmp/tmpVh5dUz > PASSED > src/test/python/apache/aurora/executor/test_thermos_task_runner.py::TestThermosTaskRunnerIntegration::test_integration_lose > Writing log files to disk in /tmp/tmpVh5dUz > PASSED > src/test/python/apache/aurora/executor/test_thermos_task_runner.py::TestThermosTaskRunnerIntegration::test_integration_quitquitquit > Writing log files to disk in /tmp/tmpVh5dUz > ERROR] is_alive got OSError: [Errno 10] No child processes > PASSED > > ========================== 5 passed in 15.12 seconds > =========================== > ============================= test session starts > ============================== > platform linux2 -- Python 2.7.6 -- py-1.4.23 -- pytest-2.6.1 -- > /usr/bin/python2.7 > plugins: cov, timeout > collecting ... collected 15 items > > src/test/python/apache/aurora/executor/test_thermos_executor.py::TestThermosExecutor::test_basic > Writing log files to disk in /tmp/tmp45q9Xn > Build operating on top level addresses: set([BuildFileAddress(< > https://builds.apache.org/job/Aurora/ws/src/main/python/apache/aurora/executor/bin/BUILD,> > thermos_runner)]) > Building PythonBinary PythonBinary(BuildFileAddress(< > https://builds.apache.org/job/Aurora/ws/src/main/python/apache/aurora/executor/bin/BUILD,> > thermos_runner)): > Wrote <https://builds.apache.org/job/Aurora/ws/dist/thermos_runner.pex> > Writing log files to disk in /tmp/tmp45q9Xn > PASSED > src/test/python/apache/aurora/executor/test_thermos_executor.py::TestThermosExecutor::test_basic_as_job > Writing log files to disk in /tmp/tmp45q9Xn > PASSED > src/test/python/apache/aurora/executor/test_thermos_executor.py::TestThermosExecutor::test_runner_disappears > Writing log files to disk in /tmp/tmp45q9Xn > PASSED > src/test/python/apache/aurora/executor/test_thermos_executor.py::TestThermosExecutor::test_task_killed > Writing log files to disk in /tmp/tmp45q9Xn > ERROR] is_alive got OSError: [Errno 10] No child processes > PASSED > src/test/python/apache/aurora/executor/test_thermos_executor.py::TestThermosExecutor::test_killTask > Writing log files to disk in /tmp/tmp45q9Xn > PASSED > src/test/python/apache/aurora/executor/test_thermos_executor.py::TestThermosExecutor::test_shutdown > Writing log files to disk in /tmp/tmp45q9Xn > PASSED > src/test/python/apache/aurora/executor/test_thermos_executor.py::TestThermosExecutor::test_task_lost > Writing log files to disk in /tmp/tmp45q9Xn > PASSED > src/test/python/apache/aurora/executor/test_thermos_executor.py::TestThermosExecutor::test_task_health_failed > Writing log files to disk in /tmp/tmp45q9Xn > FAILED > > =================================== FAILURES > =================================== > _________________ TestThermosExecutor.test_task_health_failed > __________________ > > self = <test_thermos_executor.TestThermosExecutor object at 0x7f9b97db9690> > > def test_task_health_failed(self): > proxy_driver = ProxyDriver() > with SignalServer(UnhealthyHandler) as port: > with temporary_dir() as checkpoint_root: > health_check_config = > HealthCheckConfig(initial_interval_secs=0.1, interval_secs=0.1) > _, executor = make_executor( > proxy_driver, > checkpoint_root, > MESOS_JOB(task=SLEEP60, > health_check_config=health_check_config), > ports={'health': port}, > fast_status=True, > > status_providers=(HealthCheckerProvider(),)) > > src/test/python/apache/aurora/executor/test_thermos_executor.py:405: > _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ > _ _ _ > > proxy_driver = <test_thermos_executor.ProxyDriver object at 0x7f9b97db94d0> > checkpoint_root = '/tmp/tmpGAbNgi' > task = > MesosJob(health_check_config=HealthCheckConfig(initial_interval_secs=0.1, > ...traints=ConstraintList()), > name=does_not_matter, > instances=1) > ports = {'health': 41187}, fast_status = True > runner_class = <class > 'apache.aurora.executor.thermos_task_runner.ThermosTaskRunner'> > status_providers = > (<apache.aurora.executor.common.health_checker.HealthCheckerProvider object > at 0x7f9b97f0f590>,) > > def make_executor( > proxy_driver, > checkpoint_root, > task, > ports={}, > fast_status=False, > runner_class=ThermosTaskRunner, > status_providers=()): > > status_manager_class = FastStatusManager if fast_status else > StatusManager > runner_provider = make_provider(checkpoint_root, runner_class) > te = FastThermosExecutor( > runner_provider=runner_provider, > status_manager_class=status_manager_class, > sandbox_provider=DefaultTestSandboxProvider, > status_providers=status_providers, > ) > > ExecutorTimeout(te.launched, proxy_driver, timeout=Amount(100, > Time.MILLISECONDS)).start() > task_description = make_task(task, assigned_ports=ports, > instanceId=0) > te.launchTask(proxy_driver, task_description) > > te.status_manager_started.wait() > sampled_metrics = te.metrics.sample() > assert 'kill_manager.enabled' in sampled_metrics > for checker in te._chained_checker._status_checkers: # hacky > > assert ('%s.enabled' % checker.name()) in sampled_metrics > E TypeError: 'str' object is not callable > > src/test/python/apache/aurora/executor/test_thermos_executor.py:204: > TypeError > !!!!!!!!!!!!!!!!!!!! Interrupted: stopping after 1 failures > !!!!!!!!!!!!!!!!!!!! > ===================== 1 failed, 7 passed in 14.92 seconds > ====================== > Build operating on top level addresses: set([BuildFileAddress(< > https://builds.apache.org/job/Aurora/ws/src/test/python/BUILD,> all)]) > src.test.python.apache.aurora.admin.host_maintenance > ..... SUCCESS > src.test.python.apache.aurora.client.api.api > ..... SUCCESS > src.test.python.apache.aurora.client.api.disambiguator > ..... SUCCESS > src.test.python.apache.aurora.client.api.instance_watcher > ..... SUCCESS > src.test.python.apache.aurora.client.api.job_monitor > ..... SUCCESS > src.test.python.apache.aurora.client.api.mux > ..... SUCCESS > src.test.python.apache.aurora.client.api.restarter > ..... SUCCESS > src.test.python.apache.aurora.client.api.scheduler_client > ..... SUCCESS > src.test.python.apache.aurora.client.api.updater > ..... SUCCESS > src.test.python.apache.aurora.client.api.updater_util > ..... SUCCESS > src.test.python.apache.aurora.client.binding_helper > ..... SUCCESS > src.test.python.apache.aurora.client.cli.api > ..... SUCCESS > src.test.python.apache.aurora.client.cli.command_hooks > ..... SUCCESS > src.test.python.apache.aurora.client.cli.config > ..... SUCCESS > src.test.python.apache.aurora.client.cli.help > ..... SUCCESS > src.test.python.apache.aurora.client.cli.inspect > ..... SUCCESS > src.test.python.apache.aurora.client.cli.job > ..... SUCCESS > src.test.python.apache.aurora.client.cli.logging > ..... SUCCESS > src.test.python.apache.aurora.client.cli.plugins > ..... SUCCESS > src.test.python.apache.aurora.client.cli.quota > ..... SUCCESS > src.test.python.apache.aurora.client.cli.sla > ..... SUCCESS > src.test.python.apache.aurora.client.cli.task > ..... SUCCESS > src.test.python.apache.aurora.client.commands.admin > ..... SUCCESS > src.test.python.apache.aurora.client.commands.core > ..... SUCCESS > src.test.python.apache.aurora.client.commands.ssh > ..... SUCCESS > src.test.python.apache.aurora.client.config > ..... SUCCESS > src.test.python.apache.aurora.client.hooks.hooked_api > ..... SUCCESS > src.test.python.apache.aurora.client.hooks.non_hooked_api > ..... SUCCESS > src.test.python.apache.aurora.common.test_aurora_job_key > ..... SUCCESS > src.test.python.apache.aurora.common.test_cluster > ..... SUCCESS > src.test.python.apache.aurora.common.test_cluster_option > ..... SUCCESS > src.test.python.apache.aurora.common.test_clusters > ..... SUCCESS > src.test.python.apache.aurora.common.test_http_signaler > ..... SUCCESS > src.test.python.apache.aurora.common.test_shellify > ..... SUCCESS > src.test.python.apache.aurora.common.test_transport > ..... SUCCESS > src.test.python.apache.aurora.config.test_base > ..... SUCCESS > src.test.python.apache.aurora.config.test_loader > ..... SUCCESS > src.test.python.apache.aurora.config.test_thrift > ..... SUCCESS > src.test.python.apache.aurora.executor.common.announcer > ..... SUCCESS > src.test.python.apache.aurora.executor.common.directory_sandbox > ..... SUCCESS > src.test.python.apache.aurora.executor.common.executor_timeout > ..... SUCCESS > src.test.python.apache.aurora.executor.common.health_checker > ..... SUCCESS > src.test.python.apache.aurora.executor.common.task_info > ..... SUCCESS > src.test.python.apache.aurora.executor.executor_base > ..... SUCCESS > src.test.python.apache.aurora.executor.executor_vars > ..... SUCCESS > src.test.python.apache.aurora.executor.gc_executor > ..... SUCCESS > src.test.python.apache.aurora.executor.status_manager > ..... SUCCESS > src.test.python.apache.aurora.executor.thermos_executor > ..... FAILURE > src.test.python.apache.aurora.executor.thermos_task_runner > ..... SUCCESS > src.test.python.apache.thermos.bin.test_thermos > ..... SUCCESS > src.test.python.apache.thermos.common.test_pathspec > ..... SUCCESS > src.test.python.apache.thermos.common.test_task_planner > ..... SUCCESS > src.test.python.apache.thermos.core.test_angry > ..... SUCCESS > src.test.python.apache.thermos.core.test_failing_runner > ..... SUCCESS > src.test.python.apache.thermos.core.test_failure_limit > ..... SUCCESS > src.test.python.apache.thermos.core.test_finalization > ..... SUCCESS > src.test.python.apache.thermos.core.test_helper > ..... SUCCESS > src.test.python.apache.thermos.core.test_runner_integration > ..... SUCCESS > src.test.python.apache.thermos.monitoring.test_disk > ..... SUCCESS > Process leaked file descriptors. See > http://wiki.jenkins-ci.org/display/JENKINS/Spawning+processes+from+build > for more information > Build step 'Execute shell' marked build as failure > Recording test results > ERROR] Caught exception in self.control(): [Errno 2] No such file or > directory: '/tmp/tmpGAbNgi/checkpoints/sleep60-001/runner' > ERROR] Traceback (most recent call last): > File "< > https://builds.apache.org/job/Aurora/ws/dist/thermos_runner.pex/apache/thermos/core/runner.py";,> > line 556, in control > yield > File "< > https://builds.apache.org/job/Aurora/ws/dist/thermos_runner.pex/apache/thermos/core/runner.py";,> > line 843, in run > self._run() > File "< > https://builds.apache.org/job/Aurora/ws/dist/thermos_runner.pex/apache/thermos/core/runner.py";,> > line 867, in _run > os.utime(self._pathspec.getpath('runner_checkpoint'), None) > OSError: [Errno 2] No such file or directory: > '/tmp/tmpGAbNgi/checkpoints/sleep60-001/runner' > >
