Hi Luke & All, Great that there is a session at the summit to follow on form the discussions at the Paris event. I am 100% behind as much automation as we can secure to strengthen and sustain our security badge and I see these steps as part of our infra group initiatives as a great way of doing that. I think what Bryan may be sharing some concerns around is the ability of our community to adapt to emerging source code locations and activities.
This is of course a balance that we need to be able to reach between achieving our objectives of providing a secure and trustworthy reference implementation and enabling innovation. It may be worth publishing processes for inclusion and expected security reviews as part of that process in order that this is seen as a “living system” for security validation rather than a prohibition on integration. ☺ As a further step we may consider differentiating between “release criteria” and “incubation criteria” as part of this solution. These are topics I would hope to be able to bring to the table at the summit session (if I can escape the DoveTail session in the other room). / Chris From: <[email protected]> on behalf of Luke Hinds <[email protected]> Date: Thursday, 1 June 2017 at 11:10 To: "SULLIVAN, BRYAN L" <[email protected]> Cc: TECH-DISCUSS OPNFV <[email protected]>, "Ashlee Young (Ash)" <[email protected]> Subject: Re: [opnfv-tech-discuss] [infra] [security] Wiki page for Anteater Hi Bryan, Apologies I missed this email, just found it by searching for a different thread. I would recommend we meet up at the summit, where there is also a session booked (Room Pastel, Day 2, 15;00) and I will be doing a demo, but please see inline replies for now though. On Thu, May 25, 2017 at 3:45 PM, SULLIVAN, BRYAN L <[email protected]<mailto:[email protected]>> wrote: Luke, Re your call for review on the draft tool in gerrit (https://gerrit.opnfv.org/gerrit/#/c/34901/<https://urldefense.proofpoint.com/v2/url?u=https-3A__gerrit.opnfv.org_gerrit_-23_c_34901_&d=DwMFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=OrbtGCluczz9awEKz9Fv7g&m=EF5J2_nx-1ZrTGyx9OIiq2t3jbbnUg_IKjbaw2AcOnM&s=uMQjY_qc1MpICL91WyNBFcFj_IdcHS8gB-jeLH6AFjo&e=>), and “The main driver is really to clamp down on people pulling down files and repos from external sources that we don't know whether we can trust or not”: We need to discuss this fully and in F2F before any implementation. I think I understand the motivation but what you’re talking about (blocking URLs in OPNFV commits, that might relate to resources being pulled from somewhere outside LF/OPNFV “.org” domains, is an overreaching, and ultimately superficial and fragile approach, as: * Overreach: It should be up to the project to determine what is trusted, and what is needed for their specific project. * We have in the past used 3rd-party git repos due to necessity and that should not be blocked if the project determines that the repo is trusted. * In many cases, it is necessary to clone repos on github or elsewhere (e.g. OpenStack clients, to install a particular version from source, or to modify the config for some component and then install). Blocking that would likely break many current tools/scripts used in OPNFV. This is all possible with the tool and projects are able to make their own waivers by sending a patch. So for example, lets say someone wishes to clone the following repo: https://github.com/john_doe/somelibrary The following waiver is all that's needed. someproject: file_contents: "git (.*)github\\.com\\john_doe/somelibrary" They are then free to clone, branch cherry pick (all sub args) they need with no further restrictions from there on in. We also currently allow all opnfv and openstack repos to be used: "git (.*)gerrit\\.opnfv\\.org","git (.*)\\.openstack\\.org" The same stands for wget, curl etc. This is allowed: wget http://repo1\\.maven\\.org This would be blocked until a waiver is added wget https://212.167.34.5/somefile As would anything like this: curl -sSL https://212.167.34.5/install.sh | sudo bash Granted, this is in a basic form for now, and I plan to improve this with ClamAV scanning, sha256 binary hashes. I am also open to any further proposed changes (especially if some other engineers could come forward to help contribute code). · * * Superficial: what OPNFV git content refers to is only the skin in an incredibly complex and deep hierarchy of references to content (repos, images, packages, …). * It seems by this OPNFV would be saying that we trust all those external sources, if the content is retrieved by some second and beyond stage of the deployment/test process. * I am totally for, as I noted in my comments below, on us developing/using some toolset to provide advice on “level of trust” in *all* references to content obtained from anywhere, at any stage in the deployment process and by any component in that process. * What we *do* with such an assessment needs to be further discussed · * * But just blocking URLs outside LF/OPNFV space is totally inadequate for the purpose you are seeking. I am certainly open minded to this, and welcome input here. I would argue that its not totally inadequate and hopefully someone from releng can comment here as they witnessed the volume of risks it highlighted at the plugfest. The early prototype of the tool found a good number of risks (10+ security patches and a CVE). · * * Fragile: there are many ways to obtain packages (used here in the generic sense – specific files, tarballs, repos, etc) that would not be covered by the tool as designed. Without even any intent to obfuscate, a good design approach to installing git repos for example would not refer directly to the repo but use a parameter and other options (e.g. the branch to install), and the reference to the repo might be broken up into “parent” (e.g. https://github.com/openstack/) and a project (e.g. openstack-helm) references in some YAML file that is used by a script/installer to install dependencies. While as noted above it would be good to be as clever (whether good or bad clever) as the script designers so that we could develop a profile of what is actually included/used in OPNFV deploys, this would be an incredibly complex undertaking, that I am not sure we are staffed for as a project. The tool can cover any retrieval method, be that curl , wget , git (or any others) and as shown above we can go as detailed as we need and drill down to a specific parent repo: "git (.*)\\.openstack\\.org\openstack-helm" Overall I am reluctant to approve a tool that mixes the dual intent of license checks and trust, using an ultimately superficial method. I would not want the community to give any impression that we had done a good and thorough job on this, without actually doing it. I am not strongly opinionated on the license functionality, I added it as there was a strong interest at the plugfest. Also thanks for the feedback, as said I recommend coming along to the session at the summit. Thanks, Bryan Sullivan | AT&T From: Luke Hinds [mailto:[email protected]<mailto:[email protected]>] Sent: Thursday, May 18, 2017 1:17 PM To: SULLIVAN, BRYAN L <[email protected]<mailto:[email protected]>> Cc: opnfv-tech-discuss <[email protected]<mailto:[email protected]>>; degirmenci, fatih <[email protected]<mailto:[email protected]>>; Ulrich Kleber <[email protected]<mailto:[email protected]>>; Jack Morgan <[email protected]<mailto:[email protected]>>; Ashlee Young (Ash) <[email protected]<mailto:[email protected]>> Subject: Re: [opnfv-tech-discuss] [infra] [security] Wiki page for Anteater On Thu, May 18, 2017 at 6:25 PM, SULLIVAN, BRYAN L <[email protected]<mailto:[email protected]>> wrote: Luke, I assume that the edits I made to the page are acceptable. I see that further edits have been made. I added some additional extensions to the list of standard binary artifact types that are allowed (doc, docx, ppt, pptx). I suggest we scan the current repos to see how prevalent binaries are, and what types there are before we kick this off. I would suggest that it start out allowing anything that is currently there, and we can back off the list, and add project-specific exceptions as we get more experience with why/how we would want to restrict/assess binaries more. Some examples might include: * Any situations that we can detect, in which binaries might have questionable/risky content. This would imply e.g. a virus etc scan of the binaries, or a license scan. * If we want to encourage projects to include only certain types of binaries (not sure how or why). Using this tool as a hoop that projects would have to jump thru (however easy) to encourage sticking to certain content types, seems pretty useless. The goal is that as this rolls out, we do not impede any commits at start, rather we gather information about what *may* need to change in the repos, and when ready we back off the exceptions or make them project specific. We discussed implementation approaches at the hackfest and arrived a similar phased approach: Perform daily or weekly scans for projects that are there only to inform, not enforce. Implement the gate checks for E release as non voting (on new patches only). At F release it becomes voting, but only on each new patch set. I have reserved a session at the summit for discussion around the tool and what direction we should take. I would like to see some more features added to the process though. The cursory license check is OK for code contributed to OPNFV, but just as important is any reference to code that the submitted code interfaces with. So we need to be able to scan the references to ensure that the contribution and its references are compatible under OPNFV’s policy. For example: * It is acceptable for an OPFNV-hosted module to be Eclipse licensed, and import a GPL-licensed module’s interfaces (example: the VES collectd plugin in the Barometer project: https://git.opnfv.org/barometer/tree/3rd_party/collectd-ves-plugin/ves_plugin/ves_plugin.py<https://urldefense.proofpoint.com/v2/url?u=https-3A__git.opnfv.org_barometer_tree_3rd-5Fparty_collectd-2Dves-2Dplugin_ves-5Fplugin_ves-5Fplugin.py&d=DwMFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=OrbtGCluczz9awEKz9Fv7g&m=N0kl4c4VoE-w1kESTKRujNpyrBEsI2M7Zxt_QNwKtmM&s=6gmiQPu-UYuhpjHDpwDNsjLNHf7BfmkiGGnM0VTLJBA&e=>) . * It would *not* be acceptable for the VES collectd plugin to be APL 2.0 licensed and to import a GPL-licensed module’s interfaces. * If possible, license metadata inside binaries (e.g. an image, document, slide deck, …) needs to be explicit (our current practice is to have an umbrella license in the root of the repo). * Scanning of code and referenced code (which becomes part of the OPNFV platform when built) for licenses and known vulnerabilities * … other examples need to be developed to establish some policies that the tool can validate So as I understand it, the reason for just doing a cursory check, is because the LF have a lawyer who checks over these each release. So as I understand it, the tool is there just to lighten the load and encourage people to apply a licence. Aside to this though, I am happy to implement whatever consensus arrives at. The main driver is really to clamp down on people pulling down files and repos from external sources that we don't know whether we can trust or not (we had once instance of external IP address, which turned out to be someones laptop left running). This is the big concern for me and other infra. The tool is set up right now to block all git clones / wgets / curls etc from anything apart from opnfv.org<https://urldefense.proofpoint.com/v2/url?u=http-3A__opnfv.org&d=DwMFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=OrbtGCluczz9awEKz9Fv7g&m=N0kl4c4VoE-w1kESTKRujNpyrBEsI2M7Zxt_QNwKtmM&s=snBGDlRQgUZEezd0vh12p-ZUSjpEiaiYPXX6juhLZH8&e=> build servers and openstack.org<https://urldefense.proofpoint.com/v2/url?u=http-3A__openstack.org&d=DwMFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=OrbtGCluczz9awEKz9Fv7g&m=N0kl4c4VoE-w1kESTKRujNpyrBEsI2M7Zxt_QNwKtmM&s=WXDQynX6BGjqjlW9nR_Upx0hZjHRu9Z8Ph1Rg8D03Yk&e=> sites. Its then easy to allow access to sites, once we know we can trust them. So we work with a principle of least privateer and extend from there. I know of a incident in a different project, where a developers SSH key was stolen, and someone staged malicious code into a repo that was then deployed into production, and this is what gives me the shudders. We have numerous operators and NEP's deploying opnfv into various LABs / sites and all it takes is one wayward Trojan binary to get included or a script to call home and pull something nasty in, and our reputation will go spinning down the toilet. So what might seem like an impedance to working quickly, is well worth the hindrance when we consider the possible fall out of something untoward finding its way into the 30+ repos we have. BTW I am talking to all now, and not just you Bryan who I know is already savvy on the topic. This is an interesting write up on the level of people we very likely have taking an interest in our code and build systems already: https://medium.com/@chrismcnab/alexseys-ttps-1204d9050551<https://urldefense.proofpoint.com/v2/url?u=https-3A__medium.com_-40chrismcnab_alexseys-2Dttps-2D1204d9050551&d=DwMFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=OrbtGCluczz9awEKz9Fv7g&m=N0kl4c4VoE-w1kESTKRujNpyrBEsI2M7Zxt_QNwKtmM&s=I11qXkW7GfSGnHJhf_mznEDSubYx1Xt2lsfDs3xdvpM&e=> - Its a scary read and a good view of the current landscape. We may need to incorporate additional tools e.g. Fossology or proprietary toolchains (e.g. Blackduck – we should see if we can get an Open Source project use license from them). Thanks, Bryan Sullivan | AT&T From: [email protected]<mailto:[email protected]> [mailto:[email protected]<mailto:[email protected]>] On Behalf Of Luke Hinds Sent: Wednesday, May 17, 2017 11:22 AM To: opnfv-tech-discuss <[email protected]<mailto:[email protected]>>; degirmenci, fatih <[email protected]<mailto:[email protected]>>; Ulrich Kleber <[email protected]<mailto:[email protected]>>; Jack Morgan <[email protected]<mailto:[email protected]>>; Ashlee Young (Ash) <[email protected]<mailto:[email protected]>> Subject: [opnfv-tech-discuss] [infra] [security] Wiki page for Anteater I have pushed up the code for anteater (releng-anteater) and have added a lot more body to the wiki page for anyone interested in the project [1]. This will also become release worthy documentation forthcoming, once I have got a feel for what needs communicating and used feedback to gather an FAQ. It is worth PTLs / committers getting familiar with writing your own regex over the coming E release cycle, as you might find Anteater falsely reports a string / binary etc, and you need to commit a patch with a project waiver. For more details on this, see the wiki section 'Anteater Has Blocked my patch, what should I do?' Anteater is planned to be non-voting for E-release, and voting for F. If anyone knows of security folk / researchers who can help add new additional to the string blacklists, please encourage them to contribute. Same for general code contributions as well. [1] https://wiki.opnfv.org/pages/viewpage.action?pageId=10294496<https://urldefense.proofpoint.com/v2/url?u=https-3A__wiki.opnfv.org_pages_viewpage.action-3FpageId-3D10294496&d=DwMFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=OrbtGCluczz9awEKz9Fv7g&m=tvgj3rZU8k95hpdxYmC2JBDssmUX2OVG-u7sehtnywE&s=wzis0JQJeIPUvl4uhJ8hit4CUW3Wdon1F2Iy0IMeuVY&e=> -- Luke Hinds | NFV Partner Engineering | Office of Technology | Red Hat e: [email protected]<mailto:[email protected]> | irc: lhinds @freenode | m: +44 77 45 63 98 84 | t: +44 12 52 36 2483 -- Luke Hinds | NFV Partner Engineering | Office of Technology | Red Hat e: [email protected]<mailto:[email protected]> | irc: lhinds @freenode | m: +44 77 45 63 98 84 | t: +44 12 52 36 2483 -- Luke Hinds | NFV Partner Engineering | Office of Technology | Red Hat e: [email protected]<mailto:[email protected]> | irc: lhinds @freenode | m: +44 77 45 63 98 84 | t: +44 12 52 36 2483
_______________________________________________ opnfv-tech-discuss mailing list [email protected] https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss
