2000 is AL16UTF16. But since there should be a non-lossy conversion
between AL32UTF8 and AL16UTF16 you'd think that Oracle would perform
it and use the index. Puzzling.
Furthermore, when you look in the Oracle db driver you can see in
base.py where it sets NLS_LANG to ".UTF8". This is an old setti
That sounds only relevant if Django creates the tables tight? You cannot do
that with partitioned Oracle of this size. As much as I like to have django
create the schemas, legacy db is all I have. THe entire DB is over 2TB of
data, you cannot just change a column field type or add an index witho
You can try to put a db_index=True in markid field.
https://docs.djangoproject.com/en/1.7/ref/models/fields/#db-index
On Fri, Jan 23, 2015 at 9:59 AM, Jani Tiainen wrote:
> On Fri, 23 Jan 2015 03:38:19 -0800 (PST)
> Joris Benschop wrote:
>
> > And to keep replying to myself:
> >
> > This one i
On Fri, 23 Jan 2015 03:38:19 -0800 (PST)
Joris Benschop wrote:
> And to keep replying to myself:
>
> This one is slow:
> x=Marker.objects.raw("SELECT * from PROD_SCHEMA.MARKER WHERE MARKID=
> %s",[u'TO1'])
> print x[0]
>
> This one is fast:
> y=Marker.objects.raw("SELECT * from PROD_SCHEMA.MAR
On Fri, 23 Jan 2015 03:38:19 -0800 (PST)
Joris Benschop wrote:
> And to keep replying to myself:
>
> This one is slow:
> x=Marker.objects.raw("SELECT * from PROD_SCHEMA.MARKER WHERE MARKID=
> %s",[u'TO1'])
> print x[0]
>
> This one is fast:
> y=Marker.objects.raw("SELECT * from PROD_SCHEMA.MAR
And to keep replying to myself:
This one is slow:
x=Marker.objects.raw("SELECT * from PROD_SCHEMA.MARKER WHERE MARKID=
%s",[u'TO1'])
print x[0]
This one is fast:
y=Marker.objects.raw("SELECT * from PROD_SCHEMA.MARKER WHERE MARKID= 'TO1'")
print y[0]
If I copy the table and make the MARKID fiel
Skip that,
>>>x=Marker.objects.raw("SELECT * from VARIANT_CLONE.MARKER WHERE
MARKID='TO1'")
is fast
DEBUG (0.001) QUERY = u"SELECT * from SCHEMA_PROD.MARKER WHERE
MARKID='mTO1'" - PARAMS = (); args=()
--
You received this message because you are subscribed to the Google Groups
"Django use
As an addition:
>>>x=Marker.objects.raw("SELECT * from VARIANT_CLONE.MARKER WHERE
MARKID='TO1'")
This is also slow
so it indeed seems to be a locale issue
the database is AL32UTF8
and django uses national character set id "2000"
is this a locale issue?
--
You received this message because yo
This seems to be specific for partial text searches (LIKE, STARTSWITH etc).
Still, interesting addition.
--
You received this message because you are subscribed to the Google Groups
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to django-
On Fri, 23 Jan 2015 02:06:58 -0800 (PST)
Joris Benschop wrote:
> Very interesting point.
> In the backend the MARKID column is of type VARCHAR2(20 BYTE). This is not
> something I can change. But it does seem that the bindvar method forces a
> unicode lookup onto an ascii table.
>
> Is there
Very interesting point.
In the backend the MARKID column is of type VARCHAR2(20 BYTE). This is not
something I can change. But it does seem that the bindvar method forces a
unicode lookup onto an ascii table.
Is there any way to make django get this field in ascii to test this theory?
--
You
Could it be a character set issue? That is, perhaps the database
NLS_CHARACTERSET is not unicode friendly, your column is VARCHAR
instead of NVARCHAR2, and Django is sending unicode instead of the
native database charset (which cx_Oracle directly might use).
I'm just brainstorming on what might ca
THank you all for your help
there's a few things here:
* This Oracle table is highly partitioned and optimized.There's indexes
everywhere. There's two full time senior Oracle DBAs working on tuning this
DB.So a return of 0.1 sec is not strange for this setup.
(btw the DBAs are really impressed
Granted, I glazed over that fact. What I did see, though, is the blank=True
on a CharField(), which would indicate to me that multiple '' (empty
string) entries could potentially be inserted into the database, meaning
that it can't be unique (with implied null=False).
*confused*
I'm guessing the
> Den 23/01/2015 kl. 08.19 skrev James Schneider :
>
> How many results do you typically get back from that query?
There should be only one result, right? Since the "markid" field is defined as
unique and OP is filtering on that with a single value.
First thing to check is that the actual colu
How many results do you typically get back from that query?
It sounds like the DB is highly optimized if you can gather results from
tables that large that quickly running the query outside of Django. If you
are returning a large dataset, then it is more likely that Django is
furiously coercing ev
Do you fetch all the results when running the query directly against
> Oracle? Many Oracle SQL clients do not fetch all the results, instead they
> fetch just the top 100 or so rows.markid =
> models.CharField(unique=True, max_length=20, blank=True)
>
I missed the part where you said you a
Do you fetch all the results when running the query directly against
Oracle? Many Oracle SQL clients do not fetch all the results, instead they
fetch just the top 100 or so rows.
- Anssi
On Thursday, January 22, 2015 at 5:32:36 PM UTC+2, Joris Benschop wrote:
>
> Dear List,
>
> I'm trying to r
Hi,
1.5b rows is rather big data.
Could you test what .values() or .values_list() produces in terms of speed.
also print qs might actually pull in all selected records in Python memory so
it may take a while to constuct bunch of objects.
On Thu, 22 Jan 2015 07:32:36 -0800 (PST)
Joris Benschop
On 2015-01-22 17:14, Joris Benschop wrote:
> Thanks you for your response
> Unfortunately, yes, the 1.5b records is not something I have
> control over.
[snip]
> WHERE "PROD_SCHEMA"."MARKER"."MARKID" = :arg0
Just out of curiosity, do you have an index on MARKER.MARKID ? And
if so, is it combined
Thanks you for your response
Unfortunately, yes, the 1.5b records is not something I have control over.
I agree the SYSGUID field needs working. However, if I change this field to
any other field type, the query still takes over 40 seconds.
class Marker(models.Model):
marker_id = models.Binar
To many calculations in the from_db_value function. Need optimizations. And
what about query limitations: 1.5b is really needed?
class SYSGUID16Field(models.Field):
default_error_messages = {
'invalid': "'%(value)s' is not a valid SYS_GUID."
}
description = "A connector to the SYS_GUID
Dear List,
I'm trying to run a simple select on a very large Oracle table (1500
million records).
I create a standard django model:
--
class Marker(models.Model):
marker_id = SYSGUID16Field(primary_key=True) # This field type is a
guess.
markid = models.CharField(uniqu
23 matches
Mail list logo
