Re: OSError: [Errno 13] Permission denied:... /_dummy_thread

[Avraham Serour]

don't use sudo to pip install packages, you should create a virtualenv for
your project

On Mon, Nov 6, 2017 at 8:24 AM, Vineet Kothari 
wrote:

> try to update your python : https://stackoverflow.com/
> questions/2720014/upgrading-all-packages-with-pip
>
> On Mon, Nov 6, 2017 at 12:10 AM, Mel DeJesus 
> wrote:
>
>> Do that only leads to another permission denied error -
>>
>> Installing collected packages: argparse, configparser, konfig, pyzmq,
>> inflect, jaraco.itertools, tempora, jaraco.stream, jaraco.classes,
>> backports.functools-lru-cache, jaraco.functools, jaraco.text,
>> jaraco.collections, jaraco.logging, irc, webob, WSGIProxy2, greenlet,
>> gevent, psutil, waitress, beautifulsoup4, webtest, ws4py, ujson, loads
>> Exception:
>> Traceback (most recent call last):
>>   File "/Library/Python/2.7/site-packages/pip/basecommand.py", line 215,
>> in main
>> status = self.run(options, args)
>>   File "/Library/Python/2.7/site-packages/pip/commands/install.py", line
>> 342, in run
>> prefix=options.prefix_path,
>>   File "/Library/Python/2.7/site-packages/pip/req/req_set.py", line 784,
>> in install
>> **kwargs
>>   File "/Library/Python/2.7/site-packages/pip/req/req_install.py", line
>> 851, in install
>> self.move_wheel_files(self.source_dir, root=root, prefix=prefix)
>>   File "/Library/Python/2.7/site-packages/pip/req/req_install.py", line
>> 1064, in move_wheel_files
>> isolated=self.isolated,
>>   File "/Library/Python/2.7/site-packages/pip/wheel.py", line 377, in
>> move_wheel_files
>> clobber(source, dest, False, fixer=fixer, filter=filter)
>>   File "/Library/Python/2.7/site-packages/pip/wheel.py", line 287, in
>> clobber
>> ensure_dir(dest)  # common for the 'include' path
>>   File "/Library/Python/2.7/site-packages/pip/utils/__init__.py", line
>> 83, in ensure_dir
>> os.makedirs(path)
>>   File 
>> "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/os.py",
>> line 157, in makedirs
>> mkdir(name, mode)
>> OSError: [Errno 1] Operation not permitted: '/System/Library/Frameworks/Py
>> thon.framework/Versions/2.7/include/python2.7/greenlet'
>>
>>
>> On Sunday, November 5, 2017 at 1:36:18 PM UTC-5, Mel DeJesus wrote:
>>>
>>> as in:
>>> sudo pip install -r requirements.txt
>>> ?
>>>
>>> On Sunday, November 5, 2017 at 1:33:07 PM UTC-5, me.vineetkothari wrote:

 Use sudo

 On Nov 6, 2017 12:00 AM, "Mel DeJesus"  wrote:



 Any suggestions? -->

 After attempting pip install -r requirements.txt in a virtual
 environment, I got a number of files loaded before getting this:


 Exception:
 Traceback (most recent call last):
   File "/Library/Python/2.7/site-packages/pip/basecommand.py", line
 215, in main
 status = self.run(options, args)
   File "/Library/Python/2.7/site-packages/pip/commands/install.py",
 line 342, in run
 prefix=options.prefix_path,
   File "/Library/Python/2.7/site-packages/pip/req/req_set.py", line
 784, in install
 **kwargs
   File "/Library/Python/2.7/site-packages/pip/req/req_install.py",
 line 851, in install
 self.move_wheel_files(self.source_dir, root=root, prefix=prefix)
   File "/Library/Python/2.7/site-packages/pip/req/req_install.py",
 line 1064, in move_wheel_files
 isolated=self.isolated,
   File "/Library/Python/2.7/site-packages/pip/wheel.py", line 345, in
 move_wheel_files
 clobber(source, lib_dir, True)
   File "/Library/Python/2.7/site-packages/pip/wheel.py", line 316, in
 clobber
 ensure_dir(destdir)
   File "/Library/Python/2.7/site-packages/pip/utils/__init__.py", line
 83, in ensure_dir
 os.makedirs(path)
   File 
 "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/os.py",
 line 157, in makedirs
 mkdir(name, mode)
 OSError: [Errno 13] Permission denied: '/Library/Python/2.7/site-pack
 ages/_dummy_thread

 --
 You received this message because you are subscribed to the Google
 Groups "Django users" group.
 To unsubscribe from this group and stop receiving emails from it, send
 an email to django-users...@googlegroups.com.
 To post to this group, send email to django...@googlegroups.com.
 Visit this group at https://groups.google.com/group/django-users.
 To view this discussion on the web visit https://groups.google.com/d/ms
 gid/django-users/14fac784-dfdb-481b-b6bc-bf03f8499871%40goog
 legroups.com
 
 .
 For more options, visit https://groups.google.com/d/optout.


 --
>> You received this message because you are subscribed to the Google Groups
>> "Django users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to django-users+uns

Guidance with testing reusable app the way Django does

[Ramez Ashraf]

Hello dear developers,

I need your help in creating a test suit similar to Django's in the sense 
that tests are in separate folders (modules), each might contain models of 
their own.

I cheated from the runtests.py Django have, edited out some warning and so.
The problem is that the apps are not "migrated" and the test fail not 
finding `auth_user` , and if the migration folder exists the test fails not 
finding the database table.

What might be wrong here ?!

Thank you in advance.

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/bd6d7d5b-3270-4461-a507-b43a4b3b5bf3%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

DJANGO app loose path from batch file

[Xristos Xristoou]

I try to learn more about Django celery and rabbitmq to create some async 
tasks and I have some question.

Some programs can provide PYTHON API to can some development to use modules 
from this program in python.

one most way to take that PYTHON API from this program is to use batch file 
like this :

@echo off
SET PROGRAM_ROOT=C:\main-folder
call "%PROGRAM_ROOT%"\bin\some-batch-file.bat

SET PYCHARM="C:\Program Files\JetBrains\PyCharm 2017.1.5\bin\pycharm64.exe"

set PYTHONPATH=%PYTHONPATH%;%PROGRAM_ROOT%\python;
set PYTHONPATH=%PYTHONPATH%;%PROGRAM_ROOT%\site-packages;


start "PyCharm aware of PROGRAM" /B %PYCHARM% %*

if I run this batch file can I use all imports and modules from this 
program.

that to release if add celery in my app and broker url rabbitmq server then 
if i use some module or some import from this python api then I take error 
message : no module name some_module_from_program (for all other modules 
celery and rabbitmq app work fine).

that mean in celery and rabbitmq server loose that paths from batch file 
and I take this error.

my question how to can define rabbitmq server to start server with that 
paths like batch file before to avoid this error?

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/8205be0a-6a15-4ce1-8347-fbaed9fe7989%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: DJANGO app loose path from batch file

[Daniel Roseman]

On Monday, 6 November 2017 15:43:36 UTC, Xristos Xristoou wrote:
>
> I try to learn more about Django celery and rabbitmq to create some async 
> tasks and I have some question.
>
> Some programs can provide PYTHON API to can some development to use 
> modules from this program in python.
>
> one most way to take that PYTHON API from this program is to use batch 
> file like this :
>
> @echo off
> SET PROGRAM_ROOT=C:\main-folder
> call "%PROGRAM_ROOT%"\bin\some-batch-file.bat
>
> SET PYCHARM="C:\Program Files\JetBrains\PyCharm 2017.1.5\bin\pycharm64.exe"
>
> set PYTHONPATH=%PYTHONPATH%;%PROGRAM_ROOT%\python;
> set PYTHONPATH=%PYTHONPATH%;%PROGRAM_ROOT%\site-packages;
>
>
> start "PyCharm aware of PROGRAM" /B %PYCHARM% %*
>
> if I run this batch file can I use all imports and modules from this 
> program.
>
> that to release if add celery in my app and broker url rabbitmq server 
> then if i use some module or some import from this python api then I take 
> error message : no module name some_module_from_program (for all other 
> modules celery and rabbitmq app work fine).
>
> that mean in celery and rabbitmq server loose that paths from batch file 
> and I take this error.
>
> my question how to can define rabbitmq server to start server with that 
> paths like batch file before to avoid this error?
>

Your question is quite hard to understand. But certainly instead of 
creating a batch file to set paths, you should be using a virtualenv.
-- 
DR

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/449617dd-a931-4608-b645-d7157227e1be%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: DJANGO app loose path from batch file

[Xristos Xristoou]

i am windows user

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/007d0f8d-d6f0-4c3e-9245-ad7a72b15551%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Problems with dumpdata/loaddata

[Matthijs Kooijman]

Hey folks,

I'm replying to an old thread about a problem with "manage.py loaddata" 
generating a BOM (byte-order-mark) that "manage.py dumpdata" refuses to 
load. I just ran into this same problem and found that, in my case, the 
problem is not "loaddata", but powershell which I was running under. 
Apparently powershell adds a BOM when redirecting output. For example, a 
simple "echo foo > somefile" also adds a BOM. Seeing the original poster 
was also using Windows, perhaps he was also using powershell (the regular 
Windows cmd.exe does not seem to have this behaviour).

For me, the solution was to just use a different shell, or use the 
"--output" option to dumpdata.

Gr.

Matthijs

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/227a6a31-4256-49a8-a11b-704b26d58dd3%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Problems with dumpdata/loaddata

[Jani Tiainen]

Hi.

Maybe it would be useful to create ticket to document this behavior in
Django documentation.

6.11.2017 8.21 ip. "Matthijs Kooijman"  kirjoitti:

> Hey folks,
>
> I'm replying to an old thread about a problem with "manage.py loaddata"
> generating a BOM (byte-order-mark) that "manage.py dumpdata" refuses to
> load. I just ran into this same problem and found that, in my case, the
> problem is not "loaddata", but powershell which I was running under.
> Apparently powershell adds a BOM when redirecting output. For example, a
> simple "echo foo > somefile" also adds a BOM. Seeing the original poster
> was also using Windows, perhaps he was also using powershell (the regular
> Windows cmd.exe does not seem to have this behaviour).
>
> For me, the solution was to just use a different shell, or use the
> "--output" option to dumpdata.
>
> Gr.
>
> Matthijs
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-users+unsubscr...@googlegroups.com.
> To post to this group, send email to django-users@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-users.
> To view this discussion on the web visit https://groups.google.com/d/
> msgid/django-users/227a6a31-4256-49a8-a11b-704b26d58dd3%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/CAHn91oeAM16pmqL3-Nwxo4rytVGJ85_6KLjxb1RJuy5-wPcqrA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Equivalent of multi-table JOIN (another post on reverse select_related)

[Samuel Abels]

I am working on a reporting feature that allows users for querying 
arbitrary models and fields, and present the result as a table. For 
example, consider the following object model:

Package
   |
   v
Device <- Component
   ^ ^
   | |
   | Interface <---2--- Connection
^^^
   / | \
  /  |  \
 /   |   \
  Sampling  IP   Policy

(The dash is the direction of a ForeignKey.)
To produce a report, I chose a three-step process:

1. The user interfaces returns a list of fields to be included in the 
report, such as

args = dict('Device.hostname__contains': 'localhost', 
'Package.name__icontains': 'unix', 'IP.address__contains': '192')

2. Given the list of args, find the shortest path that connects all 
required models. For the example above, the result is a tuple:

path = Device, Package, Interface, IP

3. In theory, I could now perform the following SQL request:

SELECT * FROM myapp_device d
LEFT JOIN myapp_package pa ON pa.device_id=d.id
LEFT JOIN myapp_interface ifc ON ifc.device_id=d.id
LEFT JOIN myapp_ip ip ON ip.interface_id=ifc.id;

But of course, I want to avoid the raw SQL. I considered the following 
options:

- Using Device.objects.select_related() does not work, because Device has a 
1:n relation to Package (and also to Unit), which Django's select_related() 
does not support.

- Using prefetch_related() does not work, because it prefetches everything, 
which is too much in our case (>100 million rows if a user queries on all 
tables), and it does not provide us with a total of the number of rows 
selected. In practice, I want to count(*) everything for displaying a 
total, and fetch only a subset, using LIMIT.

Our tests showed that the raw SQL query with LEFT JOIN is fast enough for 
production, regardless of what fields and objects are being queried. The 
craziest query I built took about 20 seconds, which is ok for what we are 
trying to do.

Any other options?

-Samuel

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/ec0e2d37-9b0f-4623-8f60-c6feeef0eb9e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Add a processing page between submit page and result page

[Ruifeng Hu]

Hi All,

I am writing a project, on the submit page, a user submits some parameters, 
then the server side will take a calculation that may take a while, I want 
to add a waiting page to tall users that it is running on the server. When 
the result is ready, it will automatically jump to the result page.
I do not know what I should do to get these!

Is there anyone can help me!

Thank You!

Best,
Ruifeng Hu   

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/2602e268-b76f-4b15-8f2c-c2c956da4755%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

RE: Equivalent of multi-table JOIN (another post on reverse select_related)

[Matthew Pava]

Though it doesn’t directly answer your query, you might be interested in this 
package:
https://github.com/burke-software/django-report-builder


From: django-users@googlegroups.com [mailto:django-users@googlegroups.com] On 
Behalf Of Samuel Abels
Sent: Monday, November 6, 2017 1:33 PM
To: Django users
Subject: Equivalent of multi-table JOIN (another post on reverse select_related)

I am working on a reporting feature that allows users for querying arbitrary 
models and fields, and present the result as a table. For example, consider the 
following object model:

Package
   |
   v
Device <- Component
   ^ ^
   | |
   | Interface <---2--- Connection
^^^
   / | \
  /  |  \
 /   |   \
  Sampling  IP   Policy

(The dash is the direction of a ForeignKey.)
To produce a report, I chose a three-step process:

1. The user interfaces returns a list of fields to be included in the report, 
such as

args = dict('Device.hostname__contains': 'localhost', 
'Package.name__icontains': 'unix', 'IP.address__contains': '192')

2. Given the list of args, find the shortest path that connects all required 
models. For the example above, the result is a tuple:

path = Device, Package, Interface, IP

3. In theory, I could now perform the following SQL request:

SELECT * FROM myapp_device d
LEFT JOIN myapp_package pa ON pa.device_id=d.id
LEFT JOIN myapp_interface ifc ON ifc.device_id=d.id
LEFT JOIN myapp_ip ip ON ip.interface_id=ifc.id;

But of course, I want to avoid the raw SQL. I considered the following options:

- Using Device.objects.select_related() does not work, because Device has a 1:n 
relation to Package (and also to Unit), which Django's select_related() does 
not support.

- Using prefetch_related() does not work, because it prefetches everything, 
which is too much in our case (>100 million rows if a user queries on all 
tables), and it does not provide us with a total of the number of rows 
selected. In practice, I want to count(*) everything for displaying a total, 
and fetch only a subset, using LIMIT.

Our tests showed that the raw SQL query with LEFT JOIN is fast enough for 
production, regardless of what fields and objects are being queried. The 
craziest query I built took about 20 seconds, which is ok for what we are 
trying to do.

Any other options?

-Samuel
--
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to 
django-users+unsubscr...@googlegroups.com.
To post to this group, send email to 
django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/ec0e2d37-9b0f-4623-8f60-c6feeef0eb9e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/296e72112fa946b6b3f13cf52b96303d%40ISS1.ISS.LOCAL.
For more options, visit https://groups.google.com/d/optout.

Re: Equivalent of multi-table JOIN (another post on reverse select_related)

[Samuel Abels]

Thanks, I have seen that and plan to use it, but for this particular 
feature I need something more tailored.

-Samuel

On Monday, November 6, 2017 at 8:56:29 PM UTC+1, Matthew Pava wrote:
>
> Though it doesn’t directly answer your query, you might be interested in 
> this package:
>
> https://github.com/burke-software/django-report-builder
>
>  
>
>  
>
> *From:* django...@googlegroups.com  [mailto:
> django...@googlegroups.com ] *On Behalf Of *Samuel Abels
> *Sent:* Monday, November 6, 2017 1:33 PM
> *To:* Django users
> *Subject:* Equivalent of multi-table JOIN (another post on reverse 
> select_related)
>
>  
>
> I am working on a reporting feature that allows users for querying 
> arbitrary models and fields, and present the result as a table. For 
> example, consider the following object model:
>
>  
>
> Package
>
>|
>
>v
>
> Device <- Component
>
>^ ^
>
>| |
>
>| Interface <---2--- Connection
>
> ^^^
>
>/ | \
>
>   /  |  \
>
>  /   |   \
>
>   Sampling  IP   Policy
>
>  
>
> (The dash is the direction of a ForeignKey.)
>
> To produce a report, I chose a three-step process:
>
>  
>
> 1. The user interfaces returns a list of fields to be included in the 
> report, such as
>
>  
>
> args = dict('Device.hostname__contains': 'localhost', 
> 'Package.name__icontains': 'unix', 'IP.address__contains': '192')
>
>  
>
> 2. Given the list of args, find the shortest path that connects all 
> required models. For the example above, the result is a tuple:
>
>  
>
> path = Device, Package, Interface, IP
>
>  
>
> 3. In theory, I could now perform the following SQL request:
>
>  
>
> SELECT * FROM myapp_device d
>
> LEFT JOIN myapp_package pa ON pa.device_id=d.id
>
> LEFT JOIN myapp_interface ifc ON ifc.device_id=d.id
>
> LEFT JOIN myapp_ip ip ON ip.interface_id=ifc.id;
>
>  
>
> But of course, I want to avoid the raw SQL. I considered the following 
> options:
>
>  
>
> - Using Device.objects.select_related() does not work, because Device has 
> a 1:n relation to Package (and also to Unit), which Django's 
> select_related() does not support.
>
>  
>
> - Using prefetch_related() does not work, because it prefetches 
> everything, which is too much in our case (>100 million rows if a user 
> queries on all tables), and it does not provide us with a total of the 
> number of rows selected. In practice, I want to count(*) everything for 
> displaying a total, and fetch only a subset, using LIMIT.
>
>  
>
> Our tests showed that the raw SQL query with LEFT JOIN is fast enough for 
> production, regardless of what fields and objects are being queried. The 
> craziest query I built took about 20 seconds, which is ok for what we are 
> trying to do.
>
>  
>
> Any other options?
>
>  
>
> -Samuel
>
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-users...@googlegroups.com .
> To post to this group, send email to djang...@googlegroups.com 
> .
> Visit this group at https://groups.google.com/group/django-users.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-users/ec0e2d37-9b0f-4623-8f60-c6feeef0eb9e%40googlegroups.com
>  
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/f692aa3a-cc23-4d78-8e3f-16ed30097038%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

RE: Equivalent of multi-table JOIN (another post on reverse select_related)

[Matthew Pava]

Maybe you are expecting too much from the user interface.  Shouldn’t you at 
least request from the user what primary object you are looking for?  Explicit 
is better than implicit.  Your example indicates that your primary object is a 
Device, but your UI dict gives no indication whatsoever that that’s what you 
want.

If you want to look for related fields in a model, check out the Model _meta 
API:
https://docs.djangoproject.com/en/1.11/ref/models/meta/

Ultimately, your Django ORM code would look like this for your example, if I 
follow your arrows correctly:
Device.objects.filter(hostname__contains= 'localhost ', 
package__name__contains= 'unix ', interface__IP__address__contains= '192')

It would seem that you want your UI to pass in the primary model and with all 
other models how they are related to the primary model.


From: django-users@googlegroups.com [mailto:django-users@googlegroups.com] On 
Behalf Of Samuel Abels
Sent: Monday, November 6, 2017 1:59 PM
To: Django users
Subject: Re: Equivalent of multi-table JOIN (another post on reverse 
select_related)

Thanks, I have seen that and plan to use it, but for this particular feature I 
need something more tailored.

-Samuel

On Monday, November 6, 2017 at 8:56:29 PM UTC+1, Matthew Pava wrote:
Though it doesn’t directly answer your query, you might be interested in this 
package:
https://github.com/burke-software/django-report-builder


From: django...@googlegroups.com 
[mailto:django...@googlegroups.com] On Behalf Of Samuel Abels
Sent: Monday, November 6, 2017 1:33 PM
To: Django users
Subject: Equivalent of multi-table JOIN (another post on reverse select_related)

I am working on a reporting feature that allows users for querying arbitrary 
models and fields, and present the result as a table. For example, consider the 
following object model:

Package
   |
   v
Device <- Component
   ^ ^
   | |
   | Interface <---2--- Connection
^^^
   / | \
  /  |  \
 /   |   \
  Sampling  IP   Policy

(The dash is the direction of a ForeignKey.)
To produce a report, I chose a three-step process:

1. The user interfaces returns a list of fields to be included in the report, 
such as

args = dict('Device.hostname__contains': 'localhost', 
'Package.name__icontains': 'unix', 'IP.address__contains': '192')

2. Given the list of args, find the shortest path that connects all required 
models. For the example above, the result is a tuple:

path = Device, Package, Interface, IP

3. In theory, I could now perform the following SQL request:

SELECT * FROM myapp_device d
LEFT JOIN myapp_package pa ON pa.device_id=d.id
LEFT JOIN myapp_interface ifc ON ifc.device_id=d.id
LEFT JOIN myapp_ip ip ON ip.interface_id=ifc.id;

But of course, I want to avoid the raw SQL. I considered the following options:

- Using Device.objects.select_related() does not work, because Device has a 1:n 
relation to Package (and also to Unit), which Django's select_related() does 
not support.

- Using prefetch_related() does not work, because it prefetches everything, 
which is too much in our case (>100 million rows if a user queries on all 
tables), and it does not provide us with a total of the number of rows 
selected. In practice, I want to count(*) everything for displaying a total, 
and fetch only a subset, using LIMIT.

Our tests showed that the raw SQL query with LEFT JOIN is fast enough for 
production, regardless of what fields and objects are being queried. The 
craziest query I built took about 20 seconds, which is ok for what we are 
trying to do.

Any other options?

-Samuel
--
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users...@googlegroups.com.
To post to this group, send email to djang...@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/ec0e2d37-9b0f-4623-8f60-c6feeef0eb9e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to 
django-users+unsubscr...@googlegroups.com.
To post to this group, send email to 
django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/f692aa3a-cc23-4d78-8e3f-16ed30097038%40googlegroups.com

Re: Equivalent of multi-table JOIN (another post on reverse select_related)

[Samuel Abels]

On Monday, November 6, 2017 at 9:15:09 PM UTC+1, Matthew Pava wrote:
>
> Maybe you are expecting too much from the user interface.  Shouldn’t you 
> at least request from the user what primary object you are looking for?
>

The primary model is always that one that is closest to "device"; step 2 of 
the process already takes this into account and returns the path in the 
best order, with the primary model being returned first.
 

> Ultimately, your Django ORM code would look like this for your example, if 
> I follow your arrows correctly:
>
> Device.objects.filter(hostname__contains= 'localhost ', 
> package__name__contains= 'unix ', interface__IP__address__contains= '192')
>

This would perform the right query, but would not provide me with a total. 
It would also be impossible to do paging, because slicing the result does 
not take into account that the LEFT JOIN multiplies the number of total 
rows.

-Samuel

>

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/f1c92f41-ce59-47c5-9a82-6af38467a536%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

RE: Equivalent of multi-table JOIN (another post on reverse select_related)

[Matthew Pava]

Is it really that bad?  Maybe I’m missing something in your situation.  I use 
my own custom page_queryset function.  I never got around to looking at the 
built-in Django way of doing it.  I think there is a generic view that can do 
paging.
q = Device.objects.filter(hostname__contains= 'localhost ', 
package__name__contains= 'unix ', interface__IP__address__contains= '192')
total_count = q.count()

def page_queryset(qs, page, count_per_page):
"""
:param qs: the queryset or list to slice
:param page: the current page to get records from (1-based)
:param count_per_page: how many items are part of each page
:return: a tuple: the index of the last page, the sliced queryset
"""
slice_begin = (page - 1) * count_per_page
slice_end = slice_begin + count_per_page

if type(qs) == QuerySet:
max_count = qs.count()
else:
max_count = len(qs)

slice_end = slice_end if slice_end < max_count else max_count
last_page = max_count // count_per_page + 1
return last_page, qs[slice_begin:slice_end]


From: django-users@googlegroups.com [mailto:django-users@googlegroups.com] On 
Behalf Of Samuel Abels
Sent: Monday, November 6, 2017 2:22 PM
To: Django users
Subject: Re: Equivalent of multi-table JOIN (another post on reverse 
select_related)



On Monday, November 6, 2017 at 9:15:09 PM UTC+1, Matthew Pava wrote:
Maybe you are expecting too much from the user interface.  Shouldn’t you at 
least request from the user what primary object you are looking for?

The primary model is always that one that is closest to "device"; step 2 of the 
process already takes this into account and returns the path in the best order, 
with the primary model being returned first.

Ultimately, your Django ORM code would look like this for your example, if I 
follow your arrows correctly:
Device.objects.filter(hostname__contains= 'localhost ', 
package__name__contains= 'unix ', interface__IP__address__contains= '192')

This would perform the right query, but would not provide me with a total. It 
would also be impossible to do paging, because slicing the result does not take 
into account that the LEFT JOIN multiplies the number of total rows.

-Samuel
--
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to 
django-users+unsubscr...@googlegroups.com.
To post to this group, send email to 
django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/f1c92f41-ce59-47c5-9a82-6af38467a536%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/0220e4a2f3c84809a51c952f60b8b7f9%40ISS1.ISS.LOCAL.
For more options, visit https://groups.google.com/d/optout.

Re: DJANGO app loose path from batch file

[Mike Morris]

On 11/06/2017 10:12 AM, Xristos Xristoou wrote:


i am windows user


My condolences.

--
You received this message because you are subscribed to the Google Groups "Django 
users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/7d3a7d6b-d259-9524-a63a-bfd8db007701%40musicplace.com.
For more options, visit https://groups.google.com/d/optout.

Re: Equivalent of multi-table JOIN (another post on reverse select_related)

[Samuel Abels]

On Monday, November 6, 2017 at 9:36:11 PM UTC+1, Matthew Pava wrote:
>
> Is it really that bad? Maybe I’m missing something in your situation. 
>

Ooooh, it isn't really. I incorrectly assumed that the query would perform 
like having an implicit DISTINCT(device.id). But it does in fact return 
absolutely the right thing:

>>> q = Device.objects.filter(pk='localhost', package__name__contains='i', 
unit__ip__address__contains='1')
>>> q.count()
1257

Yeah, that solves the issue. Thanks a lot!

-Samuel

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/7b0fbf85-5057-4e06-ab8c-6f16092b7fa3%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Django REST framework nested query.

[Shamim Hossain]

 I am switched to Djangorest framework from Nodejs. I don't know how can I 
start to solve following problem.

I have two django model named Category and Product. In Category model has 
"is_featured" boolean field. 
I need category list in JSON format where is_featured=true and the list of 
products. The models are like following.

from django.db import models

class Category(models.Model):
name=models.CharField(max_length=25, blank=False, null=False)
is_featured=models.BooleanField(default=False)
parent=models.ForeignKey('self', on_delete=models.CASCADE)

def __str__(self):
return self.name

class Product(models.Model):
name=models.CharField(max_length=25, blank=False, null=False)
category=models.ForeignKey(Category, on_delete=models.CASCADE)
uom=models.CharField(max_length=10)
price=models.FloatField()

def __str__(self):
return '%s %s %s %s' % (self.name, self.category.name, self.price, 
self.uom)

My expected category list (is_featured=true) result will be look like following.

{  
  "id":1,
  "name":"Foods",
  "products":[  
 {  
"id":2,
"name":"Chicken Fri",
"price":80
 },
 {  
"id":4,
"name":"Tomato Salad",
"price":20
 }
  ]
},
{  
  "id":3,
  "name":"Vegitable",
  "products":[  
 {  
"id":3,
"name":"Potato",
"price":20
 },
 {  
"id":6,
"name":"Bean",
"price":55
 }
  ]
}

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/207046d9-00fd-4919-a0b0-8a13e4922e6e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Django REST framework nested query.

[Xavier Ordoquy]

Hi,

You are looking for nested serializers. Documentation explains how to deal with 
them at: 
http://www.django-rest-framework.org/api-guide/relations/#nested-relationships 


Regards,
Xavier,
Linovia.


> Le 7 nov. 2017 à 04:30, Shamim Hossain  a écrit :
> 
>  I am switched to Djangorest framework from Nodejs. I don't know how can I 
> start to solve following problem.
> 
> I have two django model named Category and Product. In Category model has 
> "is_featured" boolean field. 
> I need category list in JSON format where is_featured=true and the list of 
> products. The models are like following.
> 
> from django.db import models
> 
> class Category(models.Model):
> name=models.CharField(max_length=25, blank=False, null=False)
> is_featured=models.BooleanField(default=False)
> parent=models.ForeignKey('self', on_delete=models.CASCADE)
> 
> def __str__(self):
> return self.name
> 
> class Product(models.Model):
> name=models.CharField(max_length=25, blank=False, null=False)
> category=models.ForeignKey(Category, on_delete=models.CASCADE)
> uom=models.CharField(max_length=10)
> price=models.FloatField()
> 
> def __str__(self):
> return '%s %s %s %s' % (self.name, self.category.name, 
> self.price, self.uom)
> My expected category list (is_featured=true) result will be look like 
> following.
> 
> {  
>   "id":1,
>   "name":"Foods",
>   "products":[  
>  {  
> "id":2,
> "name":"Chicken Fri",
> "price":80
>  },
>  {  
> "id":4,
> "name":"Tomato Salad",
> "price":20
>  }
>   ]
> },
> {  
>   "id":3,
>   "name":"Vegitable",
>   "products":[  
>  {  
> "id":3,
> "name":"Potato",
> "price":20
>  },
>  {  
> "id":6,
> "name":"Bean",
> "price":55
>  }
>   ]
> }
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-users+unsubscr...@googlegroups.com 
> .
> To post to this group, send email to django-users@googlegroups.com 
> .
> Visit this group at https://groups.google.com/group/django-users 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-users/207046d9-00fd-4919-a0b0-8a13e4922e6e%40googlegroups.com
>  
> .
> For more options, visit https://groups.google.com/d/optout 
> .

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/41352EFE-499C-42F2-BDE3-368742038186%40linovia.com.
For more options, visit https://groups.google.com/d/optout.