How to create an object in database only if the database is not there?
Hi, I am using `pony` orm to write a simple class as my model. Here is the class. ``` from pony.orm import Database from pony.orm import Required, Optional from pony.orm import db_session from pony.orm import select, commit DB_PATH = ‘db.sqlite’ db = Database() class Course(db.Entity): """ A class to represent a course """ title = Required(str, unique=True) url = Optional(str, unique=True) thumbnail = Optional(str) processed = Optional(bool, default=False) db.bind(provider='sqlite', filename=DB_PATH, create_db=True) db.generate_mapping(create_tables=True) ``` Now when I create a Course object like this: >>> Course(title=‘A new course’) An object is create in the database. I don’t want to have this behaviour, but what I want to be doing is create a Course object and then only commit in the database if the course is not already available in the database. If the course is already in the database, the orm should just return the same object. I have implemented the requirement as follows: ``` class Course(db.Entity): """ A class to represent a course """ title = Required(str, unique=True) url = Optional(str, unique=True) thumbnail = Optional(str) processed = Optional(bool, default=False) @staticmethod @db_session def new(title, url=None, thumbnail=None, processed=False): """Return a Course either new or from database""" course = select(c for c in Course if c.title == title)[:] if course: return course[0] return Course(title=title, url=url, thumbnail=thumbnail, processed=processed) db.bind(provider='sqlite', filename=DB_PATH, create_db=True) db.generate_mapping(create_tables=True) ``` Can there be a better method of doing that? Please let me know. Thanks. “ You are not born knowing everything. You go on learning” - Anubhav Yadav -- https://mail.python.org/mailman/listinfo/python-list
Re: [python-uk] URGENT: Volunteers for teacher collaboration please... :-)
I met @ntoll at an event in India. It was refreshing to see his passion for teaching kids about programming and python. I would have loved to be there, but I don't reside in the UK. Nevertheless, just replying to wish you guys all the best. On Wed, Feb 17, 2016 at 2:50 PM, Tim Golden wrote: > [Forwarding an email from python-uk in the hope that UK-based > Pythonistas might see it here who don't hang out there...] > > From: Nicholas H.Tollervey > Reply-To: UK Python Users > To: UK Python Users > > Hi Folks, > > I realise I must sound like a stuck record about this sort of thing - > please accept my sincere apologies. > > TL;DR: I need volunteers from around the country to support a twilight > meetup of teachers happening in various parts of the UK. It's not > difficult and likely to be a lot of fun and will only take a few hours > of your time in the early evening of a single day. I may be able to > cover travel expenses. Please get in touch. More detail below... > > Computing at School (see: http://www.computingatschool.org.uk/), a grass > roots movement of computing teachers in the UK would like to run a > series of training courses for "Master Teachers" in MicroPython on the > BBC micro:bit during March. These teachers would go on to act as the > seed / catalyst for other teachers who require Python training during a > series of training events over the summer. Put simply, this is an > exercise in Python evangelism for teachers. > > Master teachers are those who have demonstrated a combination of deep > subject knowledge and teaching skill. Put simply, they're the most > senior teachers you can get. They're also the leaders in the field and > what they say or do influences many hundreds of their colleagues. > > The idea is for the master teachers to get together with Python > developers (that'd be *you*) for a few hours to work through MicroPython > related educational resources. These events would happen at university > based hubs around the country. As a Python developer you'll *get a BBC > micro:bit* and be expected to offer advice, answer questions and > demonstrate Python as needed. Honestly, it's not an onerous task and > will only last a few hours in a "twilight" session (i.e. after work). > > The locations and proposed dates are as follows: > > London: 25th February > Birmingham: 9th March > Nottingham: 15th March > Lancaster: 16th March > Newcastle: 17th March > Hertfordshire: 21st March > Manchester: 23rd March > Southampton: 23rd March > > It's easy for UK Python to be very London-centric. This is an > opportunity for Pythonistas throughout the UK to step up and get involved. > > Why should you volunteer a few hours of your time to help teachers? Need > you ask? Your help and influence will ultimately contribute to the > education of the next generation of programmers - your future > colleagues. It's a way to give back to the community by fostering the > next generation of Pythonistas with the help of the CAS Master Teachers. > It's also, from a moral point of view, simply a selfless and > unambiguously good thing to do. > > If you're thinking "oh, they won't want me", then YOU ARE EXACTLY THE > PERSON WE NEED! Your experience, perspective and knowledge is invaluable > and teachers need to hear from you. Rest assured, this will not be a > difficult or high-pressure activity. In fact, it's likely to be a lot of > fun. > > Remember that awesome person who mentored you and/or gave you a step up? > Now's your chance to be that person for a group of master teachers. > > If this is of interest to you, please get in touch ASAP and I can start > to coordinate things with CAS. > > I'm going to put in a grant request to the PSF to see if we can cover > travel costs for developers. But there's no guarantee this will come about. > > Best wishes, > > N. > > > -- > https://mail.python.org/mailman/listinfo/python-list > > -- Regards, Anubhav Yadav KPIT Technologies, Pune. -- https://mail.python.org/mailman/listinfo/python-list
Need advice on writing better test cases.
Hello, I am a (self-learned) python developer and I write a lot of python code everyday. I try to do as much unit testing as possible. But I want to be better at it, I want to write more test cases, specially that rely on database insertions and reads and file IO. Here are my use-cases for testing. How to test if things are going into the database properly or not? (mysql/mongo). I want to be able to create a test database environment as simple as possible. Create and delete the test environment before each functional test case is run. Sometimes I write code that read some data from some rabbitmq queue and do certain things. How can I write end to end functional test that creates a test rabbitmq environment (exchanges and queues) -> wait for sometime -> see if the intended work has been done -> delete the test environment. I want to be able to make sure that any new commit on my self hosted gitlab server should first run all functional test cases first before accepting the merge. Since we use lot of docker here to deploy modules to productions, I want to write functional test cases that test the whole system as a whole and see if things are happening the way they are supposed to happen or not. This means firing up lot of docker containers, lot of test databases with some data, and run all the test cases from an end user point of view. Can you suggest me the right python testing frameworks that I should be using? Right now I am using unittest to write test cases and manual if/else statements to run the functional test cases. I try to create rabbitmq queues and bind them to rabbitmq exchanges using the pika module. I then run the module using python -m moduleName and then sleep for sometime. Then I kill the processs (subprocess) and then I see if the intended consequences have happened or not. It's a pain in the ass to be doing so many things for test cases. I clearly need to learn how to do things better. Any suggestion/book/article/course/video will help me immensely in writing better test cases. Thanks for reading. -- https://mail.python.org/mailman/listinfo/python-list
Re: Need advice on writing better test cases.
> On 28-Aug-2017, at 04:35, Ben Finney wrote: > > Anubhav Yadav writes: > >> I want to write more test cases, specially that rely on database >> insertions and reads and file IO. > > Thanks for taking seriously the importance of test cases for your code! > > One important thing to recognise is that a unit test is only one type of > test. It tests one unit of code, typically a function, and should assert > exactly one clearly true-or-false result of calling that code unit. > > If you have a function and you want to assert *that function's* > behaviour, you can avoid external dependencies during the test run by > providing fake resources. These can be mocks (e.g. with ‘unittest.mock’) > or other fake resources that are going to behave exactly how you want, > for the purpose of testing the code unit. > > Unit test cases: > > * Exercise a small unit of code in isolation. > * Each test exactly one obvious behavour of the code unit. > * Aim to have exactly one reason the test case can fail. > > Because they are isolated and test a small code unit, they are typically > *fast* and can be run very often, because the entire unit test suite > completes in seconds. > >> How to test if things are going into the database properly or not? > > That is *not* a unit test; it is a test that one part of your code > has the right effect on some other part of the system. This meets the > description not of a unit test but of an integration test. > > These integration tests, because they will likely be a lot slower than > your unit tests, should be in a separate suite of integration tests, to > be run when the time is available to run them. > > Integration tests: > > * Exercise many code units together. > * Typically make an assertion about the *resulting state* of many > underlying actions. > * Can have many things that can cause the test case to fail. > >> (mysql/mongo). I want to be able to create a test database environment >> as simple as possible. Create and delete the test environment before >> each functional test case is run. > > One good article discussion how to make integration tests, specifically > for database integration with your app, is this one > https://julien.danjou.info/blog/2014/db-integration-testing-strategies-python>. > > I hope that helps. > > -- > \ “The enjoyment of one's tools is an essential ingredient of | > `\ successful work.” —Donald Knuth, _The Art of Computer | > _o__) Programming_ | > Ben Finney > > -- > https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: Need advice on writing better test cases.
> If you have a function and you want to assert *that function's* > behaviour, you can avoid external dependencies during the test run by > providing fake resources. These can be mocks (e.g. with ‘unittest.mock’) > or other fake resources that are going to behave exactly how you want, > for the purpose of testing the code unit. Yes I have been learning how to mock systems using `unittest.mock` package. I learned how to patch the `requests.get` method to just return true with a fixed json so that I could run my tests cases without relying on the actual server to be running. I think it’s important to read more and more examples on how different libraries and systems are mocked and patched. > > These integration tests, because they will likely be a lot slower than > your unit tests, should be in a separate suite of integration tests, to > be run when the time is available to run them. Is there are standard or structure as to how these integration tests are written? Do we just define different classes for unit tests and different classes for integration tests? Is there any library that can help with writing integration tests? Also how are integration tests different than functional tests? I also want to have a good understanding of functional tests so that I can start writing them first before I start writing my code. This way whatever changes I make my tests will always tell me if the required responsibility of the system if fulfilled or not. Later I would like to run these tests cases automatically when I push my code to GitHub/Gitlab? > Integration tests: > > * Exercise many code units together. > * Typically make an assertion about the *resulting state* of many > underlying actions. > * Can have many things that can cause the test case to fail. I tried to insert some data in a sqlite database and then ran some code and then again ran some sql queries to see if the resulting state is what I intend it to be or not. As the code base grew I realised that it’s a pain to write so many sql queries manually. Is there a framework or library that I can use to simplify this? > >> (mysql/mongo). I want to be able to create a test database environment >> as simple as possible. Create and delete the test environment before >> each functional test case is run. > > One good article discussion how to make integration tests, specifically > for database integration with your app, is this one > https://julien.danjou.info/blog/2014/db-integration-testing-strategies-python>. Thank you for the link. Much much helpful. My problem is that I have to write lot of test cases to insert dummy data into the database and then test if the data is changed properly or not. I wanted to know the industry standards and the general practise of doing things. Thanks a lot for your reply. Much much appreciated and helpful. If you can mention some books or videos to watch it will help a lot. Cheers. -- https://mail.python.org/mailman/listinfo/python-list
Re: Need advice on writing better test cases.
> A good way to learn unit testing and regression testing is to download > the Python source code and read the test suites. It’s a fantastic idea. I will also have a good understanding of the internals of the standard library and at I can learn more about testing. Any specific module that you recommend to get started with? `collections` module maybe? > > Also consider using doc testing, for both documentation and unit tests. > See the doctest module. I have seen some modules with these kind of test cases. I thought it’s just a way to test your code like `unittest`. I will have another look at it. > > Can you explain what you mean by "functional test cases", and show an > example or two of the sorts of tests you would write? So let’s say I want to write a server which listens to client sending data at 12 seconds interval. Here are it’s requirements: 1. Listen to data from all clients. 2. Never lose any data packet. 3. If the data is garbage (not following the spec) discard it. 4. If the data is good, append a timestamp and push it to a rabbitmq queue. So what I would like to do is: 1. Send 100 packets from 1 client and see if the rabbitmq queue has 100 packets. 2. Send 90 good packets and 10 garbage packets and see if the rabbitmq queue has 90 good packets. 3. Send 10 packets from 10 different clients at the same time and see if there is correct data in the rabbitmq queues. 4. So on and so forth. This is just one example. Extension would be another module (which has it’s own suite of unittests) will take this data from queues and do some post processing and put it in some database like `mongo` or `mysql`. So the same set of functional tests should start this second module and see if the intended data is getting parsed properly and delivered to the databases. I don’t really know if this is the write way of writing tests. This is just something that I would do if I am told to make sure your Software does what it’s intended to do. - Anubhav. -- https://mail.python.org/mailman/listinfo/python-list
