thanks Maxim. I think that would resolve the issue I have right now.
best
Yingbo
On Fri, Dec 20, 2013 at 12:23 PM, Maxim Khutornenko <max...@twitter.com>wrote:
> Yingbo,
>
> Since your job was defined as service but was exiting right away, it was
> penalized by the scheduler for its a "flapping" behavior. The penalty is
> set by the exponential back-off with a max cut off at 5 minutes. The "No
> matching hosts" displayed in the UI is a bit misleading here and will be
> improved soon when the Throttled task state is implemented.
>
> Thanks,
> Maxim
>
>
>
> On Dec 20, 2013, at 12:01 PM, Yingbo Wang <ybw...@gmail.com> wrote:
>
> > It is a good catch! Thanks Brian.
> >
> > The task indeed restarts every 5 minutes. I guess the job just doesn't
> > can't find the running service that's why it says "Not matching hosts"?
> > I will try it to see if it works.
> >
> >
> >
> >
> > On Fri, Dec 20, 2013 at 9:40 AM, Brian Wickman <wick...@twopensource.com
> >wrote:
> >
> >> There is an executor_id in the state there, so it looks like it is
> managing
> >> to launch the tasks. Since you've listed your job as a Service(), it
> means
> >> that every task will be rescheduled after it exits, and since it looks
> like
> >> your task just immediately returns, it's probably launching dozens of
> them
> >> per minute if not more. You can change this behavior by changing
> Service
> >> to Job. (Service is just an alias for Job(service=True).)
> >>
> >> ~brian
> >>
> >>
> >>
> >> On Fri, Dec 20, 2013 at 9:20 AM, Yingbo Wang <ybw...@gmail.com> wrote:
> >>
> >>> Thanks Maxim.
> >>>
> >>> did you mean hello.aurora?
> >>>
> >>>
> >>> hello = Process(
> >>> name = 'hello',
> >>> cmdline = """
> >>> echo "ls file " ;
> >>> whoami;
> >>> hostname;
> >>> """)
> >>>
> >>> task = SequentialTask(
> >>> processes = [hello],
> >>> resources = Resources(cpu = 1.0, ram = 128*MB, disk = 128*MB))
> >>>
> >>> jobs = [Service(
> >>> task = task, cluster = 'lychee', role = 'lychee', environment =
> 'prod',
> >>> name = 'hello')]
> >>>
> >>> The slave machine has enough resource to satisfy this request.
> >>>
> >>> I did encounter an issue saying "Constraint Not Satisfied, host".
> After I
> >>> change my hostname of both master and slave machines, this problem
> seems
> >> to
> >>> be solved, and now the problem becomes "No matching hosts"
> >>>
> >>>
> >>>
> >>> Here's the offers that scheduler received:
> >>>
> >>> [
> >>>
> >>> -
> >>> {
> >>> - id: "201312200632-302033088-5050-15916-396",
> >>> - framework_id: "201312110654-302033088-5050-31688-0000",
> >>> - slave_id: "201312170824-302033088-5050-25318-0",
> >>> - hostname: "192.168.0.11",
> >>> - resources:
> >>> [
> >>> -
> >>> {
> >>> - name: "cpus",
> >>> - scalar: 7.75
> >>> },
> >>> -
> >>> {
> >>> - name: "disk",
> >>> - scalar: 917136
> >>> },
> >>> -
> >>> {
> >>> - name: "mem",
> >>> - scalar: 14814
> >>> },
> >>> -
> >>> {
> >>> - name: "ports",
> >>> - ranges:
> >>> [
> >>> - "31000-32000"
> >>> ]
> >>> }
> >>> ],
> >>> - attributes:
> >>> [
> >>> -
> >>> {
> >>> - name: "host",
> >>> - text: "192.168.0.11"
> >>> },
> >>> -
> >>> {
> >>> - name: "rack",
> >>> - text: "a"
> >>> }
> >>> ],
> >>> - executor_ids:
> >>> [
> >>> -
> >>>
> >>>
> >>
> "thermos-1387559509785-lychee-prod-hello-0-ea46f1c4-4619-49c1-94b5-7d09e1238b7d"
> >>> ]
> >>> }
> >>>
> >>> ]
> >>>
> >>>
> >>> and the /vars are:
> >>>
> >>>
> >>> async_tasks_completed 393
> >>> both_instance_ids_set 0
> >>> build_date null
> >>> build_git_branchname null
> >>> build_git_revision null
> >>> build_git_revision_number null
> >>> build_git_tag null
> >>> build_machine null
> >>> build_path null
> >>> build_time null
> >>> build_timestamp null
> >>> build_user_name null
> >>> cron_job_launch_failures 0
> >>> cron_jobs_triggered 0
> >>> cron_num_pending_runs 0
> >>> empty_slots_large 0
> >>> empty_slots_medium 1
> >>> empty_slots_small 7
> >>> empty_slots_xlarge 0
> >>> framework_registered 1
> >>> http_200_responses_events 22
> >>> http_200_responses_events_per_sec 0.0
> >>> http_200_responses_nanos_per_event 0.0
> >>> http_200_responses_nanos_total 981542679
> >>> http_200_responses_nanos_total_per_sec 0.0
> >>> instance_ids_inconsistent 0
> >>> jvm_available_processors 4
> >>> jvm_class_loaded_count 6658
> >>> jvm_class_total_loaded_count 6658
> >>> jvm_class_unloaded_count 0
> >>> jvm_gc_PS_MarkSweep_collection_count 0
> >>> jvm_gc_PS_MarkSweep_collection_time_ms 0
> >>> jvm_gc_PS_Scavenge_collection_count 7
> >>> jvm_gc_PS_Scavenge_collection_time_ms 534
> >>> jvm_gc_collection_count 7
> >>> jvm_gc_collection_time_ms 534
> >>> jvm_input_arguments [-Xmx1g, -Xms1g,
> -Djava.library.path=/usr/local/lib]
> >>> jvm_memory_free_mb 688
> >>> jvm_memory_heap_mb_committed 968
> >>> jvm_memory_heap_mb_max 968
> >>> jvm_memory_heap_mb_used 279
> >>> jvm_memory_max_mb 968
> >>> jvm_memory_mb_total 968
> >>> jvm_memory_non_heap_mb_committed 39
> >>> jvm_memory_non_heap_mb_max 130
> >>> jvm_memory_non_heap_mb_used 38
> >>> jvm_prop_awt_toolkit sun.awt.X11.XToolkit
> >>> jvm_prop_file_encoding UTF-8
> >>> jvm_prop_file_encoding_pkg sun.io
> >>> jvm_prop_file_separator /
> >>> jvm_prop_java_awt_graphicsenv sun.awt.X11GraphicsEnvironment
> >>> jvm_prop_java_awt_printerjob sun.print.PSPrinterJob
> >>> jvm_prop_java_class_path
> >>>
> >>>
> >>
> /usr/local/aurora-scheduler/lib/incubator-aurora.jar:/usr/local/aurora-scheduler/lib/aopalliance-1.0.jar:/usr/local/aurora-scheduler/lib/jsr305-1.3.9.jar:/usr/local/aurora-scheduler/lib/guava-14.0.1.jar:/usr/local/aurora-scheduler/lib/guice-3.0.jar:/usr/local/aurora-scheduler/lib/protobuf-java-2.4.1.jar:/usr/local/aurora-scheduler/lib/jersey-core-1.12.jar:/usr/local/aurora-scheduler/lib/jersey-json-1.12.jar:/usr/local/aurora-scheduler/lib/jersey-server-1.12.jar:/usr/local/aurora-scheduler/lib/jersey-servlet-1.12.jar:/usr/local/aurora-scheduler/lib/jersey-guice-1.12.jar:/usr/local/aurora-scheduler/lib/commons-lang-2.5.jar:/usr/local/aurora-scheduler/lib/javax.inject-1.jar:/usr/local/aurora-scheduler/lib/servlet-api-2.5.jar:/usr/local/aurora-scheduler/lib/log4j-1.2.17.jar:/usr/local/aurora-scheduler/lib/stringtemplate-3.2.1.jar:/usr/local/aurora-scheduler/lib/mesos-0.15.0-rc3.jar:/usr/local/aurora-scheduler/lib/libthrift-0.9.1.jar:/usr/local/aurora-scheduler/lib/zookeeper-3.3.4.jar:/usr/local/aurora-scheduler/lib/slf4j-api-1.6.1.jar:/usr/local/aurora-scheduler/lib/slf4j-jdk14-1.6.1.jar:/usr/local/aurora-scheduler/lib/log4j-0.0.3.jar:/usr/local/aurora-scheduler/lib/bootstrap-0.0.1.jar:/usr/local/aurora-scheduler/lib/jquery-0.0.1.jar:/usr/local/aurora-scheduler/lib/client-flagged-0.0.2.jar:/usr/local/aurora-scheduler/lib/client-0.0.2.jar:/usr/local/aurora-scheduler/lib/candidate-0.0.48.jar:/usr/local/aurora-scheduler/lib/client-0.0.40.jar:/usr/local/aurora-scheduler/lib/group-0.0.51.jar:/usr/local/aurora-scheduler/lib/server-set-1.0.52.jar:/usr/local/aurora-scheduler/lib/singleton-service-0.0.65.jar:/usr/local/aurora-scheduler/lib/application-http-0.0.49.jar:/usr/local/aurora-scheduler/lib/application-module-http-0.0.46.jar:/usr/local/aurora-scheduler/lib/application-module-lifecycle-0.0.39.jar:/usr/local/aurora-scheduler/lib/application-module-log-0.0.47.jar:/usr/local/aurora-scheduler/lib/application-module-stats-0.0.39.jar:/usr/local/aurora-scheduler/lib/application-0.0.65.jar:/usr/local/aurora-scheduler/lib/args-0.1.71.jar:/usr/local/aurora-scheduler/lib/base-0.0.72.jar:/usr/local/aurora-scheduler/lib/collections-0.0.59.jar:/usr/local/aurora-scheduler/lib/dynamic-host-set-0.0.35.jar:/usr/local/aurora-scheduler/lib/inject-timed-0.0.8.jar:/usr/local/aurora-scheduler/lib/inject-0.0.29.jar:/usr/local/aurora-scheduler/lib/logging-0.0.49.jar:/usr/local/aurora-scheduler/lib/net-http-filters-0.0.3.jar:/usr/local/aurora-scheduler/lib/net-util-0.0.65.jar:/usr/local/aurora-scheduler/lib/quantity-0.0.56.jar:/usr/local/aurora-scheduler/lib/stats-0.0.81.jar:/usr/local/aurora-scheduler/lib/testing-easymock-0.0.1.jar:/usr/local/aurora-scheduler/lib/thrift-0.0.61.jar:/usr/local/aurora-scheduler/lib/util-system-mocks-0.0.55.jar:/usr/local/aurora-scheduler/lib/util-templating-0.0.17.jar:/usr/local/aurora-scheduler/lib/util-testing-0.0.6.jar:/usr/local/aurora-scheduler/lib/util-0.0.83.jar:/usr/local/aurora-scheduler/lib/zookeeper-testing-0.0.37.jar:/usr/local/aurora-scheduler/lib/cglib-2.2.1-v20090111.jar:/usr/local/aurora-scheduler/lib/jettison-1.1.jar:/usr/local/aurora-scheduler/lib/jaxb-impl-2.2.3-1.jar:/usr/local/aurora-scheduler/lib/jackson-core-asl-1.9.2.jar:/usr/local/aurora-scheduler/lib/jackson-mapper-asl-1.9.2.jar:/usr/local/aurora-scheduler/lib/jackson-jaxrs-1.9.2.jar:/usr/local/aurora-scheduler/lib/jackson-xc-1.9.2.jar:/usr/local/aurora-scheduler/lib/asm-3.1.jar:/usr/local/aurora-scheduler/lib/guice-servlet-3.0.jar:/usr/local/aurora-scheduler/lib/antlr-2.7.7.jar:/usr/local/aurora-scheduler/lib/commons-lang3-3.1.jar:/usr/local/aurora-scheduler/lib/httpclient-4.2.5.jar:/usr/local/aurora-scheduler/lib/httpcore-4.2.4.jar:/usr/local/aurora-scheduler/lib/jline-0.9.94.jar:/usr/local/aurora-scheduler/lib/lock-0.0.25.jar:/usr/local/aurora-scheduler/lib/map-0.0.34.jar:/usr/local/aurora-scheduler/lib/node-0.0.34.jar:/usr/local/aurora-scheduler/lib/partitioner-0.0.34.jar:/usr/local/aurora-scheduler/lib/application-action-0.0.60.jar:/usr/local/aurora-scheduler/lib/net-http-0.0.37.jar:/usr/local/aurora-scheduler/lib/net-http-handlers-0.0.61.jar:/usr/local/aurora-scheduler/lib/guice-multibindings-3.0.jar:/usr/local/aurora-scheduler/lib/net-http-handlers-time-series-0.0.42.jar:/usr/local/aurora-scheduler/lib/net-http-handlers-pprof-0.0.18.jar:/usr/local/aurora-scheduler/lib/stats-time-series-0.0.47.jar:/usr/local/aurora-scheduler/lib/stats-jvm-0.0.41.jar:/usr/local/aurora-scheduler/lib/commons-io-2.1.jar:/usr/local/aurora-scheduler/lib/commons-codec-1.6.jar:/usr/local/aurora-scheduler/lib/easymock-3.1.jar:/usr/local/aurora-scheduler/lib/junit-dep-4.10.jar:/usr/local/aurora-scheduler/lib/hamcrest-core-1.2.jar:/usr/local/aurora-scheduler/lib/test-libraries-for-java-1.1.1.jar:/usr/local/aurora-scheduler/lib/net-pool-0.0.53.jar:/usr/local/aurora-scheduler/lib/stax-api-1.0.1.jar:/usr/local/aurora-scheduler/lib/jaxb-api-2.2.2.jar:/usr/local/aurora-scheduler/lib/junit-3.8.1.jar:/usr/local/aurora-scheduler/lib/net-http-handlers-text-0.0.27.jar:/usr/local/aurora-scheduler/lib/jetty-6.1.25.jar:/usr/local/aurora-scheduler/lib/net-http-handlers-params-0.0.10.jar:/usr/local/aurora-scheduler/lib/net-http-handlers-string-template-0.0.42.jar:/usr/local/aurora-scheduler/lib/gson-2.2.2.jar:/usr/local/aurora-scheduler/lib/util-core-6.3.8.jar:/usr/local/aurora-scheduler/lib/util-jvm-6.3.8.jar:/usr/local/aurora-scheduler/lib/cglib-nodep-2.2.2.jar:/usr/local/aurora-scheduler/lib/objenesis-1.2.jar:/usr/local/aurora-scheduler/lib/stax-api-1.0-2.jar:/usr/local/aurora-scheduler/lib/activation-1.1.jar:/usr/local/aurora-scheduler/lib/jetty-util-6.1.25.jar:/usr/local/aurora-scheduler/lib/servlet-api-2.5-20081211.jar:/usr/local/aurora-scheduler/lib/scala-library-2.9.2.jar:/usr/local/aurora-scheduler/lib/joda-time-2.1.jar:/usr/local/aurora-scheduler/lib/joda-convert-1.2.jar:/usr/local/aurora-scheduler/lib/args-apt-0.0.45.jar:/usr/local/aurora-scheduler/lib/args-core-0.0.31.jar:/usr/local/aurora-scheduler/lib/io-json-0.0.34.jar:/usr/local/aurora-scheduler/lib/io-thrift-0.0.42.jar:/usr/local/aurora-scheduler/lib/stat-registry-0.0.21.jar:/usr/local/aurora-scheduler/lib/stat-0.0.23.jar:/usr/local/aurora-scheduler/lib/stats-provider-0.0.48.jar:/usr/local/aurora-scheduler/lib/util-sampler-0.0.45.jar:/usr/local/aurora-scheduler/lib/commons-logging-1.1.1.jar:/usr/local/aurora-scheduler/lib/io-0.0.44.jar:/usr/local/aurora-scheduler/lib/service-thrift-1.0.25.jar:/usr/local/aurora-scheduler/lib/application-module-applauncher-0.0.44.jar:/usr/local/aurora-scheduler/lib/util-executor-service-shutdown-0.0.41.jar:/usr/local/aurora-scheduler/lib/jdk-logging-0.0.36.jar
> >>> jvm_prop_java_class_version 51.0
> >>> jvm_prop_java_endorsed_dirs /usr/lib/jvm/java-7-oracle/jre/lib/endorsed
> >>> jvm_prop_java_ext_dirs
> >>> /usr/lib/jvm/java-7-oracle/jre/lib/ext:/usr/java/packages/lib/ext
> >>> jvm_prop_java_home /usr/lib/jvm/java-7-oracle/jre
> >>> jvm_prop_java_io_tmpdir /tmp
> >>> jvm_prop_java_library_path /usr/local/lib
> >>> jvm_prop_java_runtime_name Java(TM) SE Runtime Environment
> >>> jvm_prop_java_runtime_version 1.7.0_45-b18
> >>> jvm_prop_java_specification_name Java Platform API Specification
> >>> jvm_prop_java_specification_vendor Oracle Corporation
> >>> jvm_prop_java_specification_version 1.7
> >>> jvm_prop_java_vendor Oracle Corporation
> >>> jvm_prop_java_vendor_url http://java.oracle.com/
> >>> jvm_prop_java_vendor_url_bug http://bugreport.sun.com/bugreport/
> >>> jvm_prop_java_version 1.7.0_45
> >>> jvm_prop_java_vm_info mixed mode
> >>> jvm_prop_java_vm_name Java HotSpot(TM) 64-Bit Server VM
> >>> jvm_prop_java_vm_specification_name Java Virtual Machine Specification
> >>> jvm_prop_java_vm_specification_vendor Oracle Corporation
> >>> jvm_prop_java_vm_specification_version 1.7
> >>> jvm_prop_java_vm_vendor Oracle Corporation
> >>> jvm_prop_java_vm_version 24.45-b08
> >>> jvm_prop_line_separator
> >>>
> >>> jvm_prop_os_arch amd64
> >>> jvm_prop_os_name Linux
> >>> jvm_prop_os_version 3.5.0-44-generic
> >>> jvm_prop_path_separator :
> >>> jvm_prop_sun_arch_data_model 64
> >>> jvm_prop_sun_boot_class_path
> >>>
> >>>
> >>
> /usr/lib/jvm/java-7-oracle/jre/lib/resources.jar:/usr/lib/jvm/java-7-oracle/jre/lib/rt.jar:/usr/lib/jvm/java-7-oracle/jre/lib/sunrsasign.jar:/usr/lib/jvm/java-7-oracle/jre/lib/jsse.jar:/usr/lib/jvm/java-7-oracle/jre/lib/jce.jar:/usr/lib/jvm/java-7-oracle/jre/lib/charsets.jar:/usr/lib/jvm/java-7-oracle/jre/lib/jfr.jar:/usr/lib/jvm/java-7-oracle/jre/classes
> >>> jvm_prop_sun_boot_library_path /usr/lib/jvm/java-7-oracle/jre/lib/amd64
> >>> jvm_prop_sun_cpu_endian little
> >>> jvm_prop_sun_cpu_isalist
> >>> jvm_prop_sun_io_unicode_encoding UnicodeLittle
> >>> jvm_prop_sun_java_command
> >>> com.twitter.aurora.scheduler.app.SchedulerMain -cluster_name=lychee
> >>> -http_port=8081 -thrift_port=8082 -native_log_quorum_size=1
> >>> -zk_endpoints=192.168.0.13:2181
> >>> -mesos_master_address=zk://192.168.0.13:2181/mesos/master
> >>> -serverset_path=/aurora/scheduler
> >>> -native_log_zk_group_path=/aurora/replicated-log
> >>> -native_log_file_path=/usr/local/aurora-scheduler/db
> >>> -backup_dir=/usr/local/aurora-scheduler/backups
> >>> -thermos_executor_path=/usr/local/bin/thermos_executor
> >>> -gc_executor_path=/usr/local/bin/gc_executor -vlog=INFO -logtostderr
> >>> jvm_prop_sun_java_launcher SUN_STANDARD
> >>> jvm_prop_sun_jnu_encoding UTF-8
> >>> jvm_prop_sun_management_compiler HotSpot 64-Bit Tiered Compilers
> >>> jvm_prop_sun_os_patch_level unknown
> >>> jvm_prop_user_country US
> >>> jvm_prop_user_dir /home/lychee
> >>> jvm_prop_user_home /root
> >>> jvm_prop_user_language en
> >>> jvm_prop_user_name root
> >>> jvm_prop_user_timezone Etc/UTC
> >>> jvm_threads_active 21
> >>> jvm_threads_daemon 12
> >>> jvm_threads_peak 25
> >>> jvm_threads_started 867
> >>> jvm_time_ms 1387559947214
> >>> jvm_uptime_secs 38781
> >>> log_compressed_entry_bytes_saved 2537635
> >>> logging_dir /root
> >>> mem_storage_consistent_read_operation_events 619
> >>> mem_storage_consistent_read_operation_events_per_sec 0.0
> >>> mem_storage_consistent_read_operation_nanos_per_event 0.0
> >>> mem_storage_consistent_read_operation_nanos_total 272565131
> >>> mem_storage_consistent_read_operation_nanos_total_per_sec 0.0
> >>> mem_storage_delete_all_tasks_events 1
> >>> mem_storage_delete_all_tasks_events_per_sec 0.0
> >>> mem_storage_delete_all_tasks_nanos_per_event 0.0
> >>> mem_storage_delete_all_tasks_nanos_total 2323293
> >>> mem_storage_delete_all_tasks_nanos_total_per_sec 0.0
> >>> mem_storage_delete_tasks_events 179
> >>> mem_storage_delete_tasks_events_per_sec 0.0
> >>> mem_storage_delete_tasks_nanos_per_event 0.0
> >>> mem_storage_delete_tasks_nanos_total 8638495
> >>> mem_storage_delete_tasks_nanos_total_per_sec 0.0
> >>> mem_storage_fetch_tasks_events 3219
> >>> mem_storage_fetch_tasks_events_per_sec 0.0
> >>> mem_storage_fetch_tasks_nanos_per_event 0.0
> >>> mem_storage_fetch_tasks_nanos_total 193613049
> >>> mem_storage_fetch_tasks_nanos_total_per_sec 0.0
> >>> mem_storage_mutate_tasks_events 677
> >>> mem_storage_mutate_tasks_events_per_sec 0.0
> >>> mem_storage_mutate_tasks_nanos_per_event 0.0
> >>> mem_storage_mutate_tasks_nanos_total 149943165
> >>> mem_storage_mutate_tasks_nanos_total_per_sec 0.0
> >>> mem_storage_save_tasks_events 306
> >>> mem_storage_save_tasks_events_per_sec 0.0
> >>> mem_storage_save_tasks_nanos_per_event 0.0
> >>> mem_storage_save_tasks_nanos_total 165988968
> >>> mem_storage_save_tasks_nanos_total_per_sec 0.0
> >>> mem_storage_weakly_consistent_read_operation_events 1538
> >>> mem_storage_weakly_consistent_read_operation_events_per_sec 0.0
> >>> mem_storage_weakly_consistent_read_operation_nanos_per_event 0.0
> >>> mem_storage_weakly_consistent_read_operation_nanos_total 138584844
> >>> mem_storage_weakly_consistent_read_operation_nanos_total_per_sec 0.0
> >>> mem_storage_write_operation_events 2932
> >>> mem_storage_write_operation_events_per_sec 0.0
> >>> mem_storage_write_operation_nanos_per_event 0.0
> >>> mem_storage_write_operation_nanos_total 28077154879
> >>> mem_storage_write_operation_nanos_total_per_sec 0.0
> >>> new_instance_id_set 0
> >>> offer_accept_races 0
> >>> old_instance_id_set 0
> >>> outstanding_offers 1
> >>> preemptor_attempts 0
> >>> preemptor_failed_preemptions 0
> >>> preemptor_no_slots_found 0
> >>> preemptor_tasks_preempted 0
> >>> process_cpu_cores_utilized 0.0
> >>> process_cpu_time_nanos 73230000000
> >>> process_max_fd_count 4096
> >>> process_open_fd_count 157
> >>> read_lock_wait_nanos 545041
> >>> resources_allocated_quota_cpu 0
> >>> resources_allocated_quota_disk_gb 0
> >>> resources_allocated_quota_ram_gb 0
> >>> resources_dedicated_consumed_cpu 0
> >>> resources_dedicated_consumed_disk_gb 0
> >>> resources_dedicated_consumed_ram_gb 0
> >>> resources_free_pool_consumed_cpu 1
> >>> resources_free_pool_consumed_disk_gb 0
> >>> resources_free_pool_consumed_ram_gb 0
> >>> resources_quota_consumed_cpu 0
> >>> resources_quota_consumed_disk_gb 0
> >>> resources_quota_consumed_ram_gb 0
> >>> resources_total_consumed_cpu 1
> >>> resources_total_consumed_disk_gb 0
> >>> resources_total_consumed_ram_gb 0
> >>> schedule_attempts_failed 0
> >>> schedule_attempts_fired 135
> >>> schedule_queue_size 1
> >>> scheduler_backup_failed 0
> >>> scheduler_backup_success 10
> >>> scheduler_driver_kill_failures 0
> >>> scheduler_failed_offers 0
> >>> scheduler_framework_disconnects 0
> >>> scheduler_framework_reregisters 0
> >>> scheduler_illegal_task_state_transitions 200
> >>> scheduler_log_bad_frames_read 0
> >>> scheduler_log_bytes_read 292813
> >>> scheduler_log_bytes_written 1080061
> >>> scheduler_log_deflated_entries_read 1
> >>> scheduler_log_entries_read 175
> >>> scheduler_log_entries_written 555
> >>> scheduler_log_native_append_events 556
> >>> scheduler_log_native_append_events_per_sec 0.0
> >>> scheduler_log_native_append_failures 0
> >>> scheduler_log_native_append_nanos_per_event 0.0
> >>> scheduler_log_native_append_nanos_total 25082628348
> >>> scheduler_log_native_append_nanos_total_per_sec 0.0
> >>> scheduler_log_native_append_timeouts 0
> >>> scheduler_log_native_native_entries_skipped 1
> >>> scheduler_log_native_read_events 176
> >>> scheduler_log_native_read_events_per_sec 0.0
> >>> scheduler_log_native_read_failures 0
> >>> scheduler_log_native_read_nanos_per_event 0.0
> >>> scheduler_log_native_read_nanos_total 73255842
> >>> scheduler_log_native_read_nanos_total_per_sec 0.0
> >>> scheduler_log_native_read_timeouts 0
> >>> scheduler_log_native_truncate_events 10
> >>> scheduler_log_native_truncate_events_per_sec 0.0
> >>> scheduler_log_native_truncate_failures 0
> >>> scheduler_log_native_truncate_nanos_per_event 0.0
> >>> scheduler_log_native_truncate_nanos_total 276600750
> >>> scheduler_log_native_truncate_nanos_total_per_sec 0.0
> >>> scheduler_log_native_truncate_timeouts 0
> >>> scheduler_log_recover_events 1
> >>> scheduler_log_recover_events_per_sec 0.0
> >>> scheduler_log_recover_nanos_per_event 0.0
> >>> scheduler_log_recover_nanos_total 565333612
> >>> scheduler_log_recover_nanos_total_per_sec 0.0
> >>> scheduler_log_save_framework_id_events 2
> >>> scheduler_log_save_framework_id_events_per_sec 0.0
> >>> scheduler_log_save_framework_id_nanos_per_event 0.0
> >>> scheduler_log_save_framework_id_nanos_total 437292
> >>> scheduler_log_save_framework_id_nanos_total_per_sec 0.0
> >>> scheduler_log_snapshot_events 10
> >>> scheduler_log_snapshot_events_per_sec 0.0
> >>> scheduler_log_snapshot_nanos_per_event 0.0
> >>> scheduler_log_snapshot_nanos_total 961465418
> >>> scheduler_log_snapshot_nanos_total_per_sec 0.0
> >>> scheduler_log_snapshot_persist_events 10
> >>> scheduler_log_snapshot_persist_events_per_sec 0.0
> >>> scheduler_log_snapshot_persist_nanos_per_event 0.0
> >>> scheduler_log_snapshot_persist_nanos_total 840444765
> >>> scheduler_log_snapshot_persist_nanos_total_per_sec 0.0
> >>> scheduler_log_snapshots 10
> >>> scheduler_log_tasks_mutate_events 677
> >>> scheduler_log_tasks_mutate_events_per_sec 0.0
> >>> scheduler_log_tasks_mutate_nanos_per_event 0.0
> >>> scheduler_log_tasks_mutate_nanos_total 182346727
> >>> scheduler_log_tasks_mutate_nanos_total_per_sec 0.0
> >>> scheduler_log_tasks_remove_events 179
> >>> scheduler_log_tasks_remove_events_per_sec 0.0
> >>> scheduler_log_tasks_remove_nanos_per_event 0.0
> >>> scheduler_log_tasks_remove_nanos_total 13588163
> >>> scheduler_log_tasks_remove_nanos_total_per_sec 0.0
> >>> scheduler_log_tasks_save_events 305
> >>> scheduler_log_tasks_save_events_per_sec 0.0
> >>> scheduler_log_tasks_save_nanos_per_event 0.0
> >>> scheduler_log_tasks_save_nanos_total 114479852
> >>> scheduler_log_tasks_save_nanos_total_per_sec 0.0
> >>> scheduler_log_un_snapshotted_transactions 36
> >>> scheduler_lost_executors 0
> >>> scheduler_max_ASSIGNED_waiting_ms 0
> >>> scheduler_max_KILLING_waiting_ms 0
> >>> scheduler_max_PREEMPTING_waiting_ms 0
> >>> scheduler_max_RESTARTING_waiting_ms 0
> >>> scheduler_resource_offers 397
> >>> scheduler_resource_offers_events 397
> >>> scheduler_resource_offers_events_per_sec 0.0
> >>> scheduler_resource_offers_nanos_per_event 0.0
> >>> scheduler_resource_offers_nanos_total 305282357
> >>> scheduler_resource_offers_nanos_total_per_sec 0.0
> >>> scheduler_save_host_attribute_events 397
> >>> scheduler_save_host_attribute_events_per_sec 0.0
> >>> scheduler_save_host_attribute_nanos_per_event 0.0
> >>> scheduler_save_host_attribute_nanos_total 12466197
> >>> scheduler_save_host_attribute_nanos_total_per_sec 0.0
> >>> scheduler_status_update_events 426
> >>> scheduler_status_update_events_per_sec 0.0
> >>> scheduler_status_update_nanos_per_event 0.0
> >>> scheduler_status_update_nanos_total 19334004116
> >>> scheduler_status_update_nanos_total_per_sec 0.0
> >>> scheduler_status_updates 0
> >>> scheduler_thrift_createJob_events 2
> >>> scheduler_thrift_createJob_events_per_sec 0.0
> >>> scheduler_thrift_createJob_nanos_per_event 0.0
> >>> scheduler_thrift_createJob_nanos_total 54102009
> >>> scheduler_thrift_createJob_nanos_total_per_sec 0.0
> >>> scheduler_thrift_getTasksStatus_events 2
> >>> scheduler_thrift_getTasksStatus_events_per_sec 0.0
> >>> scheduler_thrift_getTasksStatus_nanos_per_event 0.0
> >>> scheduler_thrift_getTasksStatus_nanos_total 5684762
> >>> scheduler_thrift_getTasksStatus_nanos_total_per_sec 0.0
> >>> scheduler_thrift_getVersion_events 4
> >>> scheduler_thrift_getVersion_events_per_sec 0.0
> >>> scheduler_thrift_getVersion_nanos_per_event 0.0
> >>> scheduler_thrift_getVersion_nanos_total 13854714
> >>> scheduler_thrift_getVersion_nanos_total_per_sec 0.0
> >>> scheduler_thrift_killTasks_events 2
> >>> scheduler_thrift_killTasks_events_per_sec 0.0
> >>> scheduler_thrift_killTasks_nanos_per_event 0.0
> >>> scheduler_thrift_killTasks_nanos_total 186407312
> >>> scheduler_thrift_killTasks_nanos_total_per_sec 0.0
> >>> shard_sanity_check_failures 0
> >>> snapshot_apply_events 1
> >>> snapshot_apply_events_per_sec 0.0
> >>> snapshot_apply_nanos_per_event 0.0
> >>> snapshot_apply_nanos_total 93051831
> >>> snapshot_apply_nanos_total_per_sec 0.0
> >>> snapshot_create_events 10
> >>> snapshot_create_events_per_sec 0.0
> >>> snapshot_create_nanos_per_event 0.0
> >>> snapshot_create_nanos_total 37066472
> >>> snapshot_create_nanos_total_per_sec 0.0
> >>> system_env_GLOG_v 0
> >>> system_env_HOME /home/lychee
> >>> system_env_JAVA_OPTS -server -Xmx1g -Xms1g
> >>> -Djava.library.path=/usr/local/lib
> >>> system_env_LANG en_US.UTF-8
> >>> system_env_LIBPROCESS_IP 192.168.0.18
> >>> system_env_LIBPROCESS_PORT 8083
> >>> system_env_LOGNAME root
> >>> system_env_MAIL /var/mail/root
> >>> system_env_NLSPATH /usr/dt/lib/nls/msg/%L/%N.cat
> >>> system_env_OLDPWD /usr/local/aurora-scheduler
> >>> system_env_PATH
> >>> /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
> >>> system_env_PWD /home/lychee
> >>> system_env_SHELL /bin/bash
> >>> system_env_SHLVL 0
> >>> system_env_SUDO_COMMAND /etc/rc.aurora_scheduler.local
> >>> system_env_SUDO_GID 1002
> >>> system_env_SUDO_UID 1002
> >>> system_env_SUDO_USER lychee
> >>> system_env_TERM xterm
> >>> system_env_USER root
> >>> system_env_USERNAME root
> >>> system_env_XFILESEARCHPATH /usr/dt/app-defaults/%L/Dt
> >>> system_free_physical_memory_mb 148
> >>> system_free_swap_mb 8063
> >>> system_load_avg 2.22
> >>> task_queries_all 45
> >>> task_queries_by_id 3668
> >>> task_queries_by_job 183
> >>> task_schedule_attempt_events 135
> >>> task_schedule_attempt_events_per_sec 0.0
> >>> task_schedule_attempt_nanos_per_event 0.0
> >>> task_schedule_attempt_nanos_total 6289005821
> >>> task_schedule_attempt_nanos_total_per_sec 0.0
> >>> task_store_ASSIGNED 0
> >>> task_store_FAILED 0
> >>> task_store_FINISHED 200
> >>> task_store_INIT 0
> >>> task_store_KILLED 0
> >>> task_store_KILLING 0
> >>> task_store_LOST 0
> >>> task_store_PENDING 1
> >>> task_store_PREEMPTING 0
> >>> task_store_RESTARTING 0
> >>> task_store_RUNNING 0
> >>> task_store_STARTING 0
> >>> task_store_THROTTLED 0
> >>> task_store_UNKNOWN 0
> >>> timed_out_tasks 0
> >>> timeout_queue_size 338
> >>> transient_states 0
> >>> uncaught_exceptions 0
> >>> variable_scrape_events 38778
> >>> variable_scrape_events_per_sec 1.0000103021061324
> >>> variable_scrape_micros_per_event 455.0001369564521
> >>> variable_scrape_micros_total 18809481
> >>> variable_scrape_micros_total_per_sec 455.00482441615327
> >>> work_queue_depth 0
> >>> write_lock_wait_nanos 13138744
> >>>
> >>>
> >>>
> >>>
> >>> On Fri, Dec 20, 2013 at 9:07 AM, Maxim Khutornenko <max...@twitter.com
> >>>> wrote:
> >>>
> >>>> Hi Yingbo,
> >>>>
> >>>> Do you mind sharing your .aurora file? Perhaps your are requesting
> >>>> Resources or Constraints that are infeasible for your local setup to
> >>>> satisfy?
> >>>>
> >>>> Also, you might want to check that your scheduler is receiving offers
> >>> from
> >>>> mesos master. You could check your scheduler log file to see if any
> >>> offers
> >>>> are coming. Alternatively, you might want to look at
> >>>> "scheduler_resource_offers" stat exposed by the scheduler /vars
> >> endpoint
> >>>> (http://<your_local_scheduler_ip:port>/vars).
> >>>>
> >>>> Thanks,
> >>>> Maxim
> >>>>
> >>>>
> >>>>
> >>>> On Dec 19, 2013, at 11:20 PM, Yingbo Wang <ybw...@gmail.com> wrote:
> >>>>
> >>>>> hi,
> >>>>> Hope you guys have a fun night at city hall.
> >>>>> While you might be drunk, can anyone help me a little bit on
> >> setting
> >>>> up
> >>>>> the local dev environment to debug aurora ?
> >>>>> My current set up.
> >>>>> Machine A: 64bit ubuntu server running mesos-master,
> >>>>> Machine B: 64bit ubuntu server running mesos-slave, thermo
> >>>>> observer server;
> >>>>> Machine C: 32bit ubuntu server running zookeeper.
> >>>>> Machine D: Mac air for developing aurora.
> >>>>>
> >>>>> I tried to run aurora-scheduler on Machine A and also created a
> >>>> hello
> >>>>> job from machine A. the job was submitted successfully. however the
> >> job
> >>>>> keeps staying at "PENDING" status. i checked thermo observer from
> >> port
> >>>>> 1336, it says "No matching hosts".
> >>>>>
> >>>>> I spent a couple of hours and still can't figure out the
> >> reason
> >>> so
> >>>>> I decided to settup the dev environment on my Mac so that I can trace
> >>> the
> >>>>> error in IntelliJ. Can someone give me some hint on that? I am not
> >>> quite
> >>>>> familiar with IntelliJ and java.
> >>>>>
> >>>>>
> >>>>>
> >>>>> best
> >>>>> Yingbo
> >>>>
> >>>>
> >>>
> >>
>
>
