On Thu, Feb 5, 2015 at 3:16 PM, Carl Meyer <c...@oddbird.net> wrote: > On 02/05/2015 11:20 AM, Larry Martell wrote: >> On Thu, Feb 5, 2015 at 10:53 AM, Carl Meyer <c...@oddbird.net> wrote: >>> TransactionTestCase has been around for quite a long time (since 1.1, >>> IIRC). It's definitely in 1.5. >> >> I thought it was not in 1.5 because I went to >> https://docs.djangoproject.com/en/1.5/topics/testing/tools/#transactiontestcase >> and got a 404. > > Ah, yeah; I think the docs just moved around. > >>>> and so on. I need the temp table to get dropped between each get call. >>>> The best way to do this is if each is in its own MySQL session. Having >>>> the table truncated doesn't really accomplish this. I think I will >>>> have to modify the code to drop the temp table after it's used. I hate >>>> to muck with working production code to accommodate a test, but I also >>>> don't want the test branch to have a different code base from the >>>> production branch. I'll probably go with the former option. >>> >>> I agree, I don't think TransactionTestCase will help with this situation. >>> >>> I find the production design a bit questionable, >> >> Why do you say that? I recently added the temp table for performance. >> I have a query that was taking over an hour. When I broke it into 2 >> queries using a temp table it then ran in 10 seconds. If I've >> introduced something bad in my app, I'd really like to know about >> that. > > I don't know if it's bad or not, given your needs, but introducing the > constraint "every request must run in its own MySQL session which is > shut down after the request" would give me pause, since reusing > connections (using the built-in Django connection-reuse feature, which > isn't in 1.5 yet I don't think, or using an external connection pooler) > is often a quick and easy performance win. And because having every > request leave the connection in an unusable state makes testing > harder/slower, too. It just feels wrong :-)
I changed the main code to drop the temp table. This made the tests run, and also is much cleaner. Thanks for the impetus to do this. > I don't know exactly why restructuring the query made it faster. The main table being selected from is largish - 38 million rows with 282 columns - and it's joined with 6 other tables. > In PostgreSQL I would usually use a CTE (WITH clause) to allow the same > type of query decomposition without needing an actual temp table; not > sure if MySQL supports those. (I recently reworked an ORM query using > joins to raw SQL with CTEs and saw a similar many-orders-of-magnitude > performance improvement, because the version with joins was internally > generating a result-set that grew with the square of the size of a > particular table.) I wasn't familiar with CTEs - I just read up on them. I don't think MySQL supports those. > But if the temp table is the only approach that > works, I would probably prefer to have my production code clean it up to > make session reuse possible. (Ideally, by encapsulating the creation and > destruction of the temp table in a database stored procedure, so it > becomes what it ought to be, a query implementation detail, not > something that application logic needs to be concerned about.) > >>> but taking it as a >>> given that it meets your needs adequately and you don't want to change >>> it, you could also address this problem by simply dropping the temp >>> table in a tearDown() method, no? >> >> A tearDown() method would be run after the test, right? I am sending >> multiple requests within one test. > > Can you restructure your tests to avoid that? The more focused a test > is, usually the better. > > If not, then I think you just need a utility method to destroy the temp > table, which you either call explicitly from your tests after each > request, or perhaps build into a subclass of the test client so it's > automated. Although a bit ugly, I think that's still better than trying > to reconnect to the database for each test; that'll kill your > test-running speed. (But I'd still prefer fixing the problem at the source.) -- You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscr...@googlegroups.com. To post to this group, send email to django-users@googlegroups.com. Visit this group at http://groups.google.com/group/django-users. To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/CACwCsY4c%2BH2WsrEWsaekofznaLomRQkk2ufQjfvUC%2BucQYB2DQ%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
