Re: [python-uk] Reviewing third-party packages
We do similar with a checklist for the practicalities (though I for one still have no good solution for guaranteeing the security of code beyond reviewing it line-by-line...) - we've gone slightly more general so as to apply to "technologies" as well as just libraries, but our process is roughly: Costs - ongoing/one-off License - usually a check against http://copyfree.org/ - though it gets a lot more complicated with certain licenses and architectures, eg: GPL'ed standalone services can be used without worry in much wider context than GPL'ed libraries Check for project activity - evidenced by number of contributors, recent commit activity and release schedules Maturity/age of project Number of "users" - trying to get a rough idea of how many people/companies use a library Details on paid support options if applicable Check if the team expertise and scale lines up with taking maintaining the project should the worst happen Checking team expertise for actually using the technology - and any training/etc required Checking scalability of the technology ...and a few things that are only really relevant for SaaS-type integrations, such as identifying points of failure in the provider's infrastructure. The other thing I try and push is to ensure that alternatives are considered where appropriate - which is a bit more contextual, but it's very easy to jump to "I want to use this" long before checking if there are better alternatives around. Thanks James On Fri, 28 Jul 2017 at 10:08 Patrick Morris wrote: > On 28/07/2017 05:54, Steve - Gadget Barnes wrote: > > > > > > On 28/07/2017 00:27, p...@getaroundtoit.co.uk wrote: > >> S, (Andy and Mike) > >> > >> Yes, you've hit a couple of pertinent points; and it might make for an > >> interesting project. > >> > >> However, I was looking for a check-list or similar which I can give to > >> the pertinent dev.teams to ensure that they are 'covering all the bases' > >> - whereas the question: "have you checked 'everything'?" produces a > >> rather predictable response. > >> > >> I'm thinking someone wiser than I will have written these things down - > >> just can't find such... > >> > >> > > > > As a starting point, my personal mini-checklist, for considering > > including packages: > > > > 1. Licensing: Is the project only ever going to be internal? If there is > > any chance of it's being included in a commercial deliverable then the > > licence and all of it's dependencies must be "Apache or better" and have > > a chart of the acceptable permissive licences vs. usage. Basically the > > licences that we generally have green flags for are MIT, BSD, CC-BY, > > MMS-PL & PSF. With some yellow flags on Artistic/Perl & Apache. > > 2. A quick look at the project repository for indications of activity > > such as recent check-ins, outstanding tickets, age of pull requests, > > discussion levels & tone on tickets. (Call this an abandonware check). > > 3. The reputation, on-line & off, of the authors & maintainers. > > 4. Is it hosted on a host that hasn't announced its own demise. > > 5. Test coverage &/or Coverity > > 6. Do the requirements look reasonable for the nature of the package, > > e.g. I wouldn't expect network or server type dependencies in a screen > > shot package. > > 7. Ditto the imports > > 8. If there are none-Python elements are they about what I would expect, > > i.e. things that you would expect performance issues with? > > 9. Support for Python 3 & 2 or at least a clear statement. > > > > All of the above are good > > You could also use the following to check for known vulnerabilities > > https://www.openhub.net/explore/projects > > Patrick > ___ > python-uk mailing list > python-uk@python.org > https://mail.python.org/mailman/listinfo/python-uk > ___ python-uk mailing list python-uk@python.org https://mail.python.org/mailman/listinfo/python-uk
Re: [python-uk] Tumbleweed badge
Hi Hansel, Might be missing the point, but... If you're looking to reference the individual fields, can you leverage the Postgres JsonField's key lookup functionality? As per https://docs.djangoproject.com/en/1.11/ref/contrib/postgres/fields/#key-index-and-path-lookups A quick bit of code inspection appears to show Django's ORM assembling the necessary SQL for you: https://github.com/django/django/blob/b29c6c96c738bd7250a408b079dd8a4d4657849a/django/contrib/postgres/fields/jsonb.py#L90 At very least, this might give you somewhere to start from - eg: wrapping this capability, or implementing your own get_transform for the field may allow you to build the JSON paths manually if you're finding it's not suitable. Hope that helps... James On Fri, 27 Oct 2017 at 05:43 Hansel Dunlop wrote: > Thank you Kuba! That's interesting. Good to know it's possible . If not, > perhaps, advisable. I imagine my translated fields could register the > models they're on and "as_sql" could do a quick lookup to see if we were > referencing any of them in the particular query. > > I have looked at existing alternatives. In fact we're using > django-modeltranslation currently. The API of which I think is great. But I > don't want to continue to scale it because of the number of extra columns > it's adding to the database and the size and ugliness of the queries it's > generating. If we have a model with 10 translated fields and 160 translated > languages we would have just hit postgresql's 1600 column limit and the > queries themselves would be > 10KB. Not to mention the number of extra > database migrations we would have generated to get there. > > The APIs for django-hvad and django-parler both seem to have slightly > different goals. I want to completely avoid talking about translations in > my general code. Set the language for a request once based on the headers > it provides. And return an api response where any localised fields are > provided in that language without modifying any view or serializer code. > > I have a custom TranslatedField that achieves these goals, quite simply. > And it's good to know I can make further optimisations to it should I need > to. > > > On Fri, Oct 27, 2017 at 12:03 AM, Kuba wrote: > >> Hi Hansel, >> >> >> On 26 October 2017 at 18:52, Hansel Dunlop >> wrote: >> >>> Hi all, >>> >>> So I just got a notification from Stackoverflow that this question - >>> https://stackoverflow.com/questions/46804936/custom-django-field-type-with-modified-column-look-up-in-select-part >>> - was just awarded the tumbleweed badge (no, votes, no answers, and no >>> views)... >>> >> >>> So throwing this open to the wider community here. I do suspect there is >>> no way of doing this in Django. But I wish there was. Because I'm trying to >>> create a custom translation infrastructure and it would help if I could >>> just return the actual text in the query rather than the whole blob of json. >>> >> >> >> so just regarding this part: *Does Django have any hooks that let me >> dynamically change the 'SELECT' part of the query?* >> >> Writing custom SQL backend (with only one custom part - the compiler) >> would be one relatively "easy" way to do that. As weird as it sounds it >> would take one method to override and mess up with. >> django.db.models.sql.compiler.SQLCompiler.as_sql is your friend. You'd need >> to find a clever way to figure out if given compiled sql query is the one >> you're interested in and only then alter it, to avoid performance >> penalties. I'm not sure it's a good idea, but it is possible. Of course >> it's quite deep in internals so passing any custom stuff there would be >> anyway kinda "threadlocal-magical". >> >> Regarding translations, just curious: have you considered django-hvad / >> django-parler? >> >> Cheers, >> Jakub >> >> >> >>> >>> I mean maybe I could on the fly modify the db_column value. But would >>> that be thread safe? Doubt it. >>> >>> -- >>> >>> Hansel >>> >>> ___ >>> python-uk mailing list >>> python-uk@python.org >>> https://mail.python.org/mailman/listinfo/python-uk >>> >>> >> >> ___ >> python-uk mailing list >> python-uk@python.org >> https://mail.python.org/mailman/listinfo/python-uk >> >> > > > -- > > Hansel > ___ > python-uk mailing list > python-uk@python.org > https://mail.python.org/mailman/listinfo/python-uk > ___ python-uk mailing list python-uk@python.org https://mail.python.org/mailman/listinfo/python-uk
