You know something, I just realized it's on a VM that only has 1 gig. Let
me at least get up to the 2GB documented gitlab minimum. Will keep an eye
on it, but I bet you just nailed it.
Thanks much for the assist.
On Wed, Feb 17, 2016 at 3:09 PM, Stan Hu <sta...@gmail.com> wrote:
> It looks to me the worker abruptly terminates due to lack of memory. How
> much RAM does your machine have?
>
>
> On Wed, Feb 17, 2016 at 11:42 AM, Michael Albert <m...@askkodiak.com>
> wrote:
>
>> Not exactly sure what to look for in the sidekiq log, but the only thing
>> standing out to me are a bunch of "Cannot allocate memory - ps" warnings.
>> Hopefully helpful?
>>
>> 2016-02-16_22:14:44.75819 2016-02-16T22:14:44.757Z 11097 TID-aznpw
>> ProjectServiceWorker JID-174ef3e92c00664ffe75439a INFO: start
>> 2016-02-16_22:14:45.87580 2016-02-16T22:14:45.875Z 11097 TID-aznpw
>> ProjectServiceWorker JID-174ef3e92c00664ffe75439a INFO: fail: 1.117 sec
>> 2016-02-16_22:14:45.87607 2016-02-16T22:14:45.875Z 11097 TID-aznpw WARN:
>> {"class"=>"ProjectServiceWorker", "args"=>[33, {"object_kind"=>"issue",
>> "user"=>{"name"=>"REDACTED", "username"=>"REDACTED", "avatar_url"=>"
>> https://secure.gravatar.com/avatar/REDACTED"},
>> "repository"=>{"name"=>"REDACTED", "url"=>"git@git.REDACTED",
>> "description"=>"REDACTED", "homepage"=>"REDACTED"},
>> "object_attributes"=>{"id"=>213, "title"=>"REDACTED", "assignee_id"=>nil,
>> "author_id"=>1, "project_id"=>4, "created_at"=>"2016-02-04 11:49:03 UTC",
>> "updated_at"=>"2016-02-16 21:24:18 UTC", "position"=>0, "branch_name"=>nil,
>> "description"=>"REDACTED", "milestone_id"=>nil, "state"=>"closed",
>> "iid"=>207, "updated_by_id"=>nil, "url"=>"https://git.REDACTED";,
>> "action"=>"close"}}], "retry"=>true, "queue"=>"project_web_hook",
>> "jid"=>"174ef3e92c00664ffe75439a", "created_at"=>1455657858.4603503,
>> "enqueued_at"=>1455660884.7553408, "error_message"=>"Cannot allocate memory
>> - ps", "error_class"=>"Errno::ENOMEM", "failed_at"=>1455657859.1589346,
>> "retry_count"=>7, "retried_at"=>1455660885.8727381}
>> 2016-02-16_22:14:45.87616 2016-02-16T22:14:45.876Z 11097 TID-aznpw WARN:
>> Errno::ENOMEM: Cannot allocate memory - ps
>> 2016-02-16_22:14:45.87626 2016-02-16T22:14:45.876Z 11097 TID-aznpw WARN:
>> /opt/gitlab/embedded/lib/ruby/2.1.0/open3.rb:193:in `spawn'
>> 2016-02-16_22:14:45.87627
>> /opt/gitlab/embedded/lib/ruby/2.1.0/open3.rb:193:in `popen_run'
>> 2016-02-16_22:14:45.87627
>> /opt/gitlab/embedded/lib/ruby/2.1.0/open3.rb:93:in `popen3'
>> 2016-02-16_22:14:45.87627
>> /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/popen.rb:23:in `popen'
>> 2016-02-16_22:14:45.87628
>> /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/sidekiq_middleware/memory_killer.rb:47:in
>> `get_rss'
>> 2016-02-16_22:14:45.87628
>> /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/sidekiq_middleware/memory_killer.rb:18:in
>> `call'
>> 2016-02-16_22:14:45.87628
>> /opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/sidekiq-4.0.1/lib/sidekiq/middleware/chain.rb:129:in
>> `block in invoke'
>> 2016-02-16_22:14:45.87628
>> /opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/sidekiq-4.0.1/lib/sidekiq/middleware/server/active_record.rb:6:in
>> `call'
>> 2016-02-16_22:14:45.87629
>> /opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/sidekiq-4.0.1/lib/sidekiq/middleware/chain.rb:129:in
>> `block in invoke'
>> 2016-02-16_22:14:45.87629
>> /opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/sidekiq-4.0.1/lib/sidekiq/middleware/server/retry_jobs.rb:74:in
>> `call'
>> 2016-02-16_22:14:45.87629
>> /opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/sidekiq-4.0.1/lib/sidekiq/middleware/chain.rb:129:in
>> `block in invoke'
>> 2016-02-16_22:14:45.87630
>> /opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/sidekiq-4.0.1/lib/sidekiq/middleware/server/logging.rb:11:in
>> `block in call'
>> 2016-02-16_22:14:45.87630
>> /opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/sidekiq-4.0.1/lib/sidekiq/logging.rb:30:in
>> `with_context'
>> 2016-02-16_22:14:45.87630
>> /opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/sidekiq-4.0.1/lib/sidekiq/middleware/server/logging.rb:7:in
>> `call'
>> 2016-02-16_22:14:45.87630
>> /opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/sidekiq-4.0.1/lib/sidekiq/middleware/chain.rb:129:in
>> `block in invoke'
>> 2016-02-16_22:14:45.87631
>> /opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/sidekiq-4.0.1/lib/sidekiq/middleware/chain.rb:132:in
>> `call'
>> 2016-02-16_22:14:45.87631
>> /opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/sidekiq-4.0.1/lib/sidekiq/middleware/chain.rb:132:in
>> `invoke'
>> 2016-02-16_22:14:45.87631
>> /opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/sidekiq-4.0.1/lib/sidekiq/processor.rb:127:in
>> `block in process'
>> 2016-02-16_22:14:45.87633
>> /opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/sidekiq-4.0.1/lib/sidekiq/processor.rb:166:in
>> `stats'
>> 2016-02-16_22:14:45.87634
>> /opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/sidekiq-4.0.1/lib/sidekiq/processor.rb:126:in
>> `process'
>> 2016-02-16_22:14:45.87634
>> /opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/sidekiq-4.0.1/lib/sidekiq/processor.rb:79:in
>> `process_one'
>> 2016-02-16_22:14:45.87634
>> /opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/sidekiq-4.0.1/lib/sidekiq/processor.rb:67:in
>> `run'
>> 2016-02-16_22:14:45.87635
>> /opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/sidekiq-4.0.1/lib/sidekiq/util.rb:16:in
>> `watchdog'
>> 2016-02-16_22:14:45.87635
>> /opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/sidekiq-4.0.1/lib/sidekiq/util.rb:24:in
>> `block in safe_thread'
>>
>>
>>
>> On Wed, Feb 17, 2016 at 2:03 PM, Stan Hu <sta...@gmail.com> wrote:
>>
>>> Actually, I don't think that bug fixes this specific problem. It looks
>>> to me that the Sidekiq task keeps re-running for some reason, perhaps
>>> because it didn't get a 200 response from the endpoint in time. Can you
>>> check /var/log/gitlab/sidekiq/current to see if there are errors?
>>>
>>>
>>> On Wed, Feb 17, 2016 at 12:36 AM, Achilleas Pipinellis <
>>> markel...@gmail.com> wrote:
>>>
>>>> On 02/16/2016 11:35 PM, m...@askkodiak.com wrote:
>>>> > Rocking GitLab community edition with Slack integration for
>>>> > issues/commits/etc. Have been using since last April or so. Recently
>>>> > started seeing duplicate updates trickle out of GitLab as shown in the
>>>> > attached screenshot. Am I alone? Anyone got any tips for me?
>>>> >
>>>> > Thanks in advance.
>>>> >
>>>> > Mike
>>>> >
>>>> >
>>>>
>>>> Ti should be fixed by
>>>> https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/2763 which will
>>>> be included in 8.5.
>>>>
>>>>
>>>> --
>>>> Blog: http://axilleas.me
>>>>
>>>> --
>>>> You received this message because you are subscribed to the Google
>>>> Groups "GitLab" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>> an email to gitlabhq+unsubscr...@googlegroups.com.
>>>> To view this discussion on the web visit
>>>> https://groups.google.com/d/msgid/gitlabhq/56C43121.6040303%40gmail.com
>>>> .
>>>> For more options, visit https://groups.google.com/d/optout.
>>>>
>>>
>>> --
>>> You received this message because you are subscribed to a topic in the
>>> Google Groups "GitLab" group.
>>> To unsubscribe from this topic, visit
>>> https://groups.google.com/d/topic/gitlabhq/cBKh90iF0fk/unsubscribe.
>>> To unsubscribe from this group and all its topics, send an email to
>>> gitlabhq+unsubscr...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/gitlabhq/CAMBWrQmcueZJKBvx5X%3DhAAEAgAWLQj6XiGNYUEDemR8NeY_L9g%40mail.gmail.com
>>> <https://groups.google.com/d/msgid/gitlabhq/CAMBWrQmcueZJKBvx5X%3DhAAEAgAWLQj6XiGNYUEDemR8NeY_L9g%40mail.gmail.com?utm_medium=email&utm_source=footer>
>>> .
>>>
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "GitLab" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to gitlabhq+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/gitlabhq/CAHVQ%2BkVBYKWneWEs-MiA089-fjycRDfyuU_C_mHspEF7sW8VVw%40mail.gmail.com
>> <https://groups.google.com/d/msgid/gitlabhq/CAHVQ%2BkVBYKWneWEs-MiA089-fjycRDfyuU_C_mHspEF7sW8VVw%40mail.gmail.com?utm_medium=email&utm_source=footer>
>> .
>>
>> For more options, visit https://groups.google.com/d/optout.
>>
>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "GitLab" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/gitlabhq/cBKh90iF0fk/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> gitlabhq+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/gitlabhq/CAMBWrQk_9iST6G2UXFJdm86oR2SMDREcx-co2pYS4upWf5hyrg%40mail.gmail.com
> <https://groups.google.com/d/msgid/gitlabhq/CAMBWrQk_9iST6G2UXFJdm86oR2SMDREcx-co2pYS4upWf5hyrg%40mail.gmail.com?utm_medium=email&utm_source=footer>
> .
>
> For more options, visit https://groups.google.com/d/optout.
>
--
You received this message because you are subscribed to the Google Groups
"GitLab" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to gitlabhq+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/gitlabhq/CAHVQ%2BkXwzUKomuDEPc9M-eGwnOS0Y0whyMpqRQFRDegSzTp2jQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
