Hi, I created the following patch, infra cores must approve that: https://review.openstack.org/285641 Add ssh key of JP Maxwell to wiki.o.o
Marton On Sat, Feb 27, 2016 at 6:41 AM JP Maxwell <j...@tipit.net> wrote: > Marton has SSH access and applied a patch earlier today. It appears the > spam continues to flow: > > > https://wiki.openstack.org/wiki/40_Thoughts_Of_Using_Open_Shelves_On_A_Kitchen > > Marton let me know if you can look at it some more or Infra if you want to > give me SSH I'll do so as well in the morning (public key attached). > > > > ssh-rsa > AAAAB3NzaC1yc2EAAAABIwAAAQEA2b5I7Yff9FCrtRmSjpILUePi54Vbc8zqJTbzrIAQZGFLBi3xd2MLlhV5QVgpDBC9H3lGjbdnc81D3aFd3HwHT4dvvvyedT12PR3VDEpftdW84vw3jzdtALcayOQznjbGnScwvX5SgnRhNxuX9Rkh8qNvOsjYPUafRr9azkQoomJFkdNVI4Vb5DbLhTpt18FPeOf0UuqDt/J2tHI4SjZ3kjzr7Nbwpg8xGgANPNE0+2pJbwCA8YDt4g3bzfzvVafQs5o9Gfc9tudkR9ugQG1M+EWCgu42CleOwMTd/rYEB2fgNNPsZAWqwQfdPajVuk70EBKUEQSyoA09eEZX+xJN9Q== > jpmax...@tipit.net > > > > > J.P. Maxwell / tipit.net <http://www.tipit.net> > > > On Fri, Feb 26, 2016 at 12:09 PM, Jimmy McArthur <ji...@openstack.org> > wrote: > >> Super thankful for all the folks that have jumped in over the last couple >> of days to help with the puppetization, etc... I just feel like we're >> taking a very wrong approach here. >> >> Paul Belanger wrote: >> >> Right, and I don't have an issue with that approach. Based on the work we >> did >> yesterday, anybody can do that via our workflow. Please submit a patch to >> puppet-mediawiki[1] and ping an infra-root in #openstack-infra IRC. >> >> What I'm proposing is the workflow is really meant for software, not for >> web applications. It's tedious and time consuming when what's needed here >> is a set of tests on the server. Submitting a patch, waiting for a +1, then >> getting on IRC to find someone with access (and time) to paste the logs is >> a pretty time consuming process for what should be a series of rapid-fire >> changes/fixes on the server. Especially when we're dealign with an active >> attack. >> >> >> We can then have somebody look at the logs. I think it is more about >> scheduling >> the task since more infra-root as travling back from the mid-cycle last night >> and today. >> >> Right, this is my point. This has been going on for 3 weeks (or more). >> Tom Fifeldt was asking for help without response. And here we are through >> another week and no closer to stemming the flow. >> >> I'm fully aware what I'm proposing goes against what Infra and the >> OpenStack workflow is all about, but I'd ask you all to look at this from a >> web development perspective instead of a software development perspective. >> >> Jimmy >> >> >> Last email from me, just on a plane. Will follow up when I land. >> >> [1] https://git.openstack.org/cgit/openstack-infra/puppet-mediawiki >> >> J.P. Maxwell | tipit.net [http://tipit.net] | fibercove.com >> [http://www.fibercove.com] >> On Fri, Feb 26, 2016 at 11:25 AM, Paul Belanger <pabelan...@redhat.com> >> <pabelan...@redhat.com> >> wrote: >> On Fri, Feb 26, 2016 at 11:08:18AM -0600, Jimmy McArthur wrote: >> >> Given the state of the wiki a the moment, I think taking the quickest path >> to get it fixed would be prudent. Is there a way we can get JP root access >> to this server, even temporarily? We get 25% of our website traffic (2 >> million visitors) to the wiki. I realize we're all after the same thing, >> >> but >> >> spammers are not going to hit the dev environment, so there's really no >> >> way >> >> to tell if teh problem is fixed without actually working directly on the >> production machine. This should be a 30 minute fix. >> >> >> I am still unclear what the 30min fix is. If really 30mins, then it >> shouldn't be >> hard to get the fix into our workflow. Could somebody please elaborate. >> >> If we are talking about deploying new versions of php or mediawiki manually, >> I >> not be in-favor of this. To me, while the attack sucks, we should be working >> on >> 2 fronts. Getting the help needed to mitigate the attack, then adding the >> changes into -infra workflow in parallel. >> >> >> I realize there is a lot of risk in giving ssh access to infra machines, >> >> but >> >> I think it's worth taking a look at either putting this machine in a place >> where a different level of admin could access it without giving away the >> keys to the entire OpenStack infrastructure or figuring out a way to set >> >> up >> >> credentials with varying levels of access. >> >> >> As a note, all the work I've been doing to help with the attack hasn't >> require >> SSH access for me to wiki.o.o. I did need infra-root help to expose our >> configuration safely. I'd rather take some time to see what the fixes are, >> having infra-root apply changes, then move them into puppet. >> >> It also has been discussed to simply disable write access to the wiki if we >> really want spamming to stop, obviously that will affect normal usage. >> >> >> Jimmy >> >> Paul Belanger wrote: >> >> On Fri, Feb 26, 2016 at 10:12:12AM -0600, JP Maxwell wrote: >> >> But if you wanted to upgrade everything, remove the mobile view >> >> extension, >> >> test in a dev/staging environment then deploy to production fingers >> crossed, I think that would be a valid approach as well. >> >> >> Current review up[1]. I'll launch a node tonight / tomorrow locally to >> >> see >> how >> >> puppet reacts. I suspect there will be some issues. >> >> If infra-roots are fine with this approach, we can use that box to test >> >> against. >> >> [1] https://review.openstack.org/#/c/285405/ >> >> J.P. Maxwell | tipit.net | fibercove.com >> On Feb 26, 2016 10:08 AM, "JP Maxwell"<j...@tipit.net> <j...@tipit.net> >> wrote: >> >> >> Plus one except in this case it is much easier to know if our efforts >> >> are >> >> working on production because the spam either stops or not. >> >> J.P. Maxwell | tipit.net | fibercove.com >> On Feb 26, 2016 9:48 AM, "Paul Belanger"<pabelan...@redhat.com> >> <pabelan...@redhat.com> wrote: >> >> >> On Fri, Feb 26, 2016 at 09:18:00AM -0600, JP Maxwell wrote: >> >> I really think you might consider the option that there is a >> >> vulnerability >> >> in one of the extensions. If that is the case black listing IPs will >> >> be >> >> an >> >> ongoing wild goose chase. >> >> I think this would be easily proven or disproven by making the questy >> question impossible and see if the spam continues. >> >> >> We'll have to let an infra-root make that call. Since nobody would be >> able to >> use the wiki. Honestly, I'd rather spend the time standing up a mirror >> >> dev >> >> instance for us to work on, rather then production. >> >> >> J.P. Maxwell | tipit.net | fibercove.com >> On Feb 26, 2016 9:12 AM, "Paul Belanger"<pabelan...@redhat.com> >> <pabelan...@redhat.com> >> >> wrote: >> >> On Thu, Feb 25, 2016 at 08:10:34PM -0800, Elizabeth K. Joseph wrote: >> >> On Thu, Feb 25, 2016 at 6:35 AM, Jeremy Stanley<fu...@yuggoth.org> >> <fu...@yuggoth.org> >> >> wrote: >> >> On 2016-02-25 02:46:13 -0600 (-0600), JP Maxwell wrote: >> >> Please be aware that you can now create accounts under the mobile >> view in the wiki native user table. I just created an account for >> JpMaxMan. Not sure if this matters but wanted to make sure you >> were aware. >> >> Oh, yes I think having a random garbage question/answer was in >> >> fact >> >> previously preventing account creation under the mobile view. We >> probably need a way to disable mobile view account creation as it >> bypasses OpenID authentication entirely. >> >> So that's what it was doing! We'll have to tackle the mobile view >> >> issue. >> >> Otherwise, quick update here: >> >> The captcha didn't appear to help stem the spam tide. We'll want to >> explore and start implementing some of the other solutions. >> >> I did some database poking around today and it does seem like all >> >> the >> >> users do have launchpad accounts and email addresses. >> >> >> So, I have a few hours before jumping on my plane and checked into >> >> this. >> >> We are >> using QuestyCaptcha which according to docs, should almost be >> >> impossible >> >> for >> spammers to by pass in an automated fashion. So, either our captcha >> >> is too >> >> easy, or we didn't set it up properly. I don't have SSH on wiki.o.o >> >> so >> >> others >> will have to check logs. I did test new pages and edits, and was >> >> promoted >> >> by >> captcha. >> >> As a next step, we might need to add additional apache2 >> >> configuration >> >> to >> >> blacklist IPs. I am reading up on that now. >> >> >> -- >> Elizabeth Krumbach Joseph || Lyz || pleia2 >> >> _______________________________________________ >> OpenStack-Infra mailing listopenstack-in...@lists.openstack.org >> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra >> _______________________________________________ >> OpenStack-Infra mailing listopenstack-in...@lists.openstack.org >> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra >> >> _______________________________________________ >> OpenStack-Infra mailing >> listOpenStack-Infra@lists.openstack.orghttp://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra >> >> _______________________________________________ >> OpenStack-Infra mailing >> listOpenStack-Infra@lists.openstack.orghttp://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra >> >> _______________________________________________ >> OpenStack-Infra mailing >> listOpenStack-Infra@lists.openstack.orghttp://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra >> >> >> > _______________________________________________ > OpenStack-Infra mailing list > OpenStack-Infra@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra >
_______________________________________________ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra