On Mon, Jun 8, 2015 at 3:32 AM, naren wrote:
> Memory Error while working with pandas dataframe.
>
> Description of Environment Windows 7 python 3.4.2 32-bit version pandas
> 0.16.0
>
> We are running into the error described below. Any help provided will be
> sincerely appreciated.
>
> We are ab
Jamie Mitchell writes:
> ...
> I then get a memory error:
>
> Traceback (most recent call last):
> File "", line 1, in
> File "/usr/local/sci/lib/python2.7/site-packages/scipy/stats/stats.py",
> line 2409, in pearsonr
> x = np.asarray(x)
> File "/usr/local/sci/lib/python2.7/site-packag
On 03/24/2014 04:32 AM, Jamie Mitchell wrote:
Hello all,
I'm afraid I am new to all this so bear with me...
I am looking to find the statistical significance between two large netCDF data
sets.
Firstly I've loaded the two files into python:
swh=netCDF4.Dataset('/data/cr1/jmitchel/Q0/swh/cont
On Monday, March 24, 2014 11:32:31 AM UTC, Jamie Mitchell wrote:
> Hello all,
>
>
>
> I'm afraid I am new to all this so bear with me...
>
>
>
> I am looking to find the statistical significance between two large netCDF
> data sets.
>
>
>
> Firstly I've loaded the two files into python:
>
On 23 January 2013 17:33, Isaac Won wrote:
> On Wednesday, January 23, 2013 10:51:43 AM UTC-6, Oscar Benjamin wrote:
>> On 23 January 2013 14:57, Isaac Won wrote:
>>
>> > On Wednesday, January 23, 2013 8:40:54 AM UTC-6, Oscar Benjamin wrote:
>>
>> Unless I've misunderstood how this function is su
On Wednesday, January 23, 2013 10:51:43 AM UTC-6, Oscar Benjamin wrote:
> On 23 January 2013 14:57, Isaac Won wrote:
>
> > On Wednesday, January 23, 2013 8:40:54 AM UTC-6, Oscar Benjamin wrote:
>
> >> On 23 January 2013 14:28, Isaac Won wrote:
>
> >>
>
> [SNIP]
>
> >
>
> > Following is full
On 23 January 2013 14:57, Isaac Won wrote:
> On Wednesday, January 23, 2013 8:40:54 AM UTC-6, Oscar Benjamin wrote:
>> On 23 January 2013 14:28, Isaac Won wrote:
>>
[SNIP]
>
> Following is full error message after I adjusted following Ulich's advice:
>
> interp = interp1d(indices[not_nan], x[not_
On Wednesday, January 23, 2013 8:40:54 AM UTC-6, Oscar Benjamin wrote:
> On 23 January 2013 14:28, Isaac Won wrote:
>
> > On Wednesday, January 23, 2013 4:08:13 AM UTC-6, Oscar Benjamin wrote:
>
> >
>
> > To Oscar
>
> > My actual error message is:
>
> > File
> > "/lustre/work/apps/python-2.7
On Wednesday, January 23, 2013 2:55:14 AM UTC-6, Ulrich Eckhardt wrote:
> Am 23.01.2013 05:06, schrieb Isaac Won:
>
> > I have tried to use different interpolation methods with Scipy. My
>
> > code seems just fine with linear interpolation, but shows memory
>
> > error with quadratic. I am a nov
On 23 January 2013 14:28, Isaac Won wrote:
> On Wednesday, January 23, 2013 4:08:13 AM UTC-6, Oscar Benjamin wrote:
>
> To Oscar
> My actual error message is:
> File
> "/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/interpolate/interpolate.py",
> line 311, in __init__
> se
On Wednesday, January 23, 2013 4:08:13 AM UTC-6, Oscar Benjamin wrote:
> On 23 January 2013 08:55, Ulrich Eckhardt
>
>
>
> > Am 23.01.2013 05:06, schrieb Isaac Won:
>
> >
>
> >> I have tried to use different interpolation methods with Scipy. My
>
> >> code seems just fine with linear interpol
On Wednesday, January 23, 2013 4:08:13 AM UTC-6, Oscar Benjamin wrote:
> On 23 January 2013 08:55, Ulrich Eckhardt
>
> wrote:
>
> > Am 23.01.2013 05:06, schrieb Isaac Won:
>
> >
>
> >> I have tried to use different interpolation methods with Scipy. My
>
> >> code seems just fine with linear i
On Tuesday, January 22, 2013 10:06:41 PM UTC-6, Isaac Won wrote:
> Hi all,
>
>
>
> I have tried to use different interpolation methods with Scipy. My code seems
> just fine with linear interpolation, but shows memory error with quadratic. I
> am a novice for python. I will appreciate any help.
On 23 January 2013 08:55, Ulrich Eckhardt
wrote:
> Am 23.01.2013 05:06, schrieb Isaac Won:
>
>> I have tried to use different interpolation methods with Scipy. My
>> code seems just fine with linear interpolation, but shows memory
>> error with quadratic. I am a novice for python. I will appreciat
Am 23.01.2013 05:06, schrieb Isaac Won:
I have tried to use different interpolation methods with Scipy. My
code seems just fine with linear interpolation, but shows memory
error with quadratic. I am a novice for python. I will appreciate any
help.
>
#code
f = open(filin, "r")
Check out the "w
Hello All,
I am still having trouble with memory errors when I try to process many
netcdf files.
Originally I would get the memory error as mentioned in the previous post
but when I added gc.collect() after each for loop I receive the error:
GEOS_ERROR: bad allocation
with no additional information
>
> But-- the image does say Pythonwin... are you running this from the
> Pythonwin editor/IDE? Does this script crash out if you run it through the
> normal 'python'(or pythonw) commands? If not, are you attempting to do any
> sort of GUI work in this script? That rarely works within Pythonwin
> d
From: python-list-bounces+shahmed=sfwmd@python.org
[mailto:python-list-bounces+shahmed=sfwmd@python.org] On Behalf Of Stephen
Hansen
Sent: Thursday, December 03, 2009 10:22 PM
To: python-list@python.org
Subject: Re: memory error
On Thu, Dec 3, 2009 at 5:51 AM, Ahmed, Shakir
On Thu, Dec 3, 2009 at 5:51 AM, Ahmed, Shakir wrote:
> I am getting a memory error while executing a script. Any idea is highly
> appreciated.
>
> Error message: " The instruction at "0x1b009032" referenced memory at
> "0x0804:, The memory could not be "written"
>
> This error is appearing an
On 12/4/2009 12:51 AM, Ahmed, Shakir wrote:
I am getting a memory error while executing a script. Any idea is highly
appreciated.
Error message: " The instruction at "0x1b009032" referenced memory at
"0x0804:, The memory could not be "written"
This error is appearing and I have to exit from
On Thursday 03 December 2009 05:51:05 Ahmed, Shakir wrote:
> I am getting a memory error while executing a script. Any idea is
> highly appreciated.
>
> Error message: " The instruction at "0x1b009032" referenced memory at
> "0x0804:, The memory could not be "written"
>
> This error is appea
On Mon, 13 Jul 2009 14:20:13 -0700, Aaron Scott wrote:
>> BTW, you should derive all your classes from something. If nothing
>> else, use object.
>> class textfile(object):
>
> Just out of curiousity... why is that? I've been coding in Python for a
> long time, and I never derive my base class
On Mon, Jul 13, 2009 at 2:51 PM, Vilya Harvey wrote:
> 2009/7/13 Aaron Scott :
>>> BTW, you should derive all your classes from something. If nothing
>>> else, use object.
>>> class textfile(object):
>>
>> Just out of curiousity... why is that? I've been coding in Python for
>> a long time, and
2009/7/13 Aaron Scott :
>> BTW, you should derive all your classes from something. If nothing
>> else, use object.
>> class textfile(object):
>
> Just out of curiousity... why is that? I've been coding in Python for
> a long time, and I never derive my base classes. What's the advantage
> to der
> BTW, you should derive all your classes from something. If nothing
> else, use object.
> class textfile(object):
Just out of curiousity... why is that? I've been coding in Python for
a long time, and I never derive my base classes. What's the advantage
to deriving them?
--
http://mail.python
sityee kong wrote:
Hi All,
I have a similar problem that many new python users might encounter. I would
really appreciate if you could help me fix the error.
I have a big text file with size more than 2GB. It turned out memory error
when reading in this file. Here is my python script, the error
phoebe> I have a big text file with size more than 2GB. It turned out
phoebe> memory error when reading in this file. Here is my python
phoebe> script, the error occurred at line -- self.fh.readlines().
phoebe> import math
phoebe> import time
phoebe> class textfile:
p
sityee kong wrote:
Hi All,
I have a similar problem that many new python users might encounter. I
would really appreciate if you could help me fix the error.
I have a big text file with size more than 2GB. It turned out memory
error when reading in this file. Here is my python script, the erro
On Nov 10, 4:47 pm, [EMAIL PROTECTED] wrote:
> Hello Everyone,
>
> I need to read a .csv file which has a size of 2.26 GB . And I wrote a
> Python script , where I need to read this file. And my Computer has 2
> GB RAM Please see the code as follows:
>
> """
> This program has been developed to ret
On Nov 11, 8:47 am, [EMAIL PROTECTED] wrote:
> import linecache
Why???
> reader2 = csv.reader(open(sys.argv[2],"rb"))
> reader2_list = []
> reader2_list.extend(reader2)
>
> for data2 in reader2_list:
> refSeqIDsinTransPro.append(data2[3])
> for data2 in reader2_list:
> promoterSequencesinT
On Tue, Nov 11, 2008 at 7:47 AM, <[EMAIL PROTECTED]> wrote:
> refSeqIDsinTransPro = []
> promoterSequencesinTransPro = []
> reader2 = csv.reader(open(sys.argv[2],"rb"))
> reader2_list = []
> reader2_list.extend(reader2)
Without testing, this looks like you're reading the _ENTIRE_
input stream int
Thank you very much Martin. It worked like a charm.
--
http://mail.python.org/mailman/listinfo/python-list
> I didn't have the problem with dumping as a string. When I tried to
> save this object to a file, memory error pops up.
That's not what the backtrace says. The backtrace says that the error
occurs inside pickle.dumps() (and it is consistent with the functions
being called, so it's plausible).
>
I didn't have the problem with dumping as a string. When I tried to
save this object to a file, memory error pops up.
I am sorry for the mention of size for a dictionary. What I meant by
65000X50 is that it has 65000 keys and each key has a list of 50
tuples.
I was able to save a dictionary objec
Nagu wrote:
> I am trying to save a dictionary of size 65000X50 to a local file and
> I get the memory error problem.
What do you mean by this size specification? When I interpreter X as
multiplication, I can't see a problem: the code
import pickle
d = {}
for i in xrange(65000*50):
d[i]=i
p
Nagu wrote:
I am trying to save a dictionary of size 65000X50 to a local file and
I get the memory error problem.
How do I go about resolving this? Is there way to partition the pickle
object and combine later if this is a problem due to limited resources
(memory) on the machine (it is 32 bit ma
> "lisa" == lisa engblom <[EMAIL PROTECTED]> writes:
lisa> Hi, I am using matplotlib with python to generate a bunch of
lisa> charts. My code works fine for a single iteration, which
lisa> creates and saves 4 different charts. The trouble is that
lisa> when I try to run it fo
It is hard to know what is wrong when we do not know how the
wrapper around the function works. The error could also be in
ConstructFigName or ConstructFigPath. Also please send the
specific error message when asking for help as that significantly
helps in tracking down the error.
Cheers
Tommy
Hari Sekhon wrote:
> I've seen people using everything from zip to touch, either out of
> laziness or out of the fact it wouldn't work very well in python, this
> zip case is a good example.
so based on a limitation in one library, and some random code you've
seen on the internet, you're makin
Fredrik Lundh wrote:
Hari Sekhon wrote:
I take it that it's still a work in progress to be able to pythonify
everything, and until then we're just gonna have to rely on shell and
those great C coded coreutils and stuff like that. Ok, I'm rather fond
of Bash+coreutils, highest r
Hari Sekhon wrote:
> I take it that it's still a work in progress to be able to pythonify
> everything, and until then we're just gonna have to rely on shell and
> those great C coded coreutils and stuff like that. Ok, I'm rather fond
> of Bash+coreutils, highest ratio of code lines to work I'v
Fredrik Lundh wrote:
Hari Sekhon wrote:
Is it me or is having to use os.system() all the time symtomatic of a
deficiency/things which are missing from python as a language?
it's you.
I take it that it's still a work in progress to be able to pythonify
everything
Hari Sekhon wrote:
> Is it me or is having to use os.system() all the time symtomatic of a
> deficiency/things which are missing from python as a language?
it's you.
--
http://mail.python.org/mailman/listinfo/python-list
On 20/05/06, Bruno Desthuilliers <[EMAIL PROTECTED]> wrote:
Roger Miller a écrit :> The basic problem is that the zipfile interface only reads and writes> whole files, so it may perform poorly or fail on huge files. At one> time I implemented a patch to allow reading files in chunks. However I
>
Roger Miller a écrit :
> The basic problem is that the zipfile interface only reads and writes
> whole files, so it may perform poorly or fail on huge files. At one
> time I implemented a patch to allow reading files in chunks. However I
> believe that the current interface has too many problems
The basic problem is that the zipfile interface only reads and writes
whole files, so it may perform poorly or fail on huge files. At one
time I implemented a patch to allow reading files in chunks. However I
believe that the current interface has too many problems to solve by
incremental patching,
Sion Arrowsmith wrote:
> Hari Sekhon <[EMAIL PROTECTED]> wrote:
(snip)
>>The python zipfile module is obviously broken...
>
> This isn't at all obvious to me.
zipfile.read() does not seem to take full advantage of zlib's
decompressobj's features. This could perhaps be improved (left as an
exerci
bruno at modulix <[EMAIL PROTECTED]> wrote:
>http://mail.zope.org/pipermail/zope/2004-October/153882.html
>"""
>MemoryError is raised by Python when an underlying (OS-level) allocation
>fails.
>(...)
>Normally this would mean that you were out of even virtual memory
>(swap), but it could also be a
Hari Sekhon <[EMAIL PROTECTED]> wrote:
>import zipfile
>zip=zipfile.ZipFile('d:\somepath\cdimage.zip')
>zip.namelist()
>['someimage.iso']
[ ... ]
>B) content=zip.read('someimage.iso')
>
>Traceback (most recent call last):
> File "", line 1, in ?
> File "D:\u\Python24\lib\zipfile.py", line 357
[EMAIL PROTECTED] wrote:
> Take a look at the pywin32 extension, which I believe has some lower
> level memory allocation and file capabilities that might help you in
> this situation.
But then the solution would not be portable, which would be a shame
since the zlib module (on which ZipFile reli
Take a look at the pywin32 extension, which I believe has some lower
level memory allocation and file capabilities that might help you in
this situation. If I'm completely wrong, someone please tell me XD.
Of course, you could just make the read() a step process, reading, O
lets say 8192 bytes at
Hari Sekhon wrote:
> I do
>
> import zipfile
> zip=zipfile.ZipFile('d:\somepath\cdimage.zip')
> zip.namelist()
> ['someimage.iso']
>
> then either of the two:
>
> A) file('someimage.iso','w').write(zip.read('someimage.iso'))
> or
> B) content=zip.read('someimage.iso')
>
> but both result in
> Hari Sekhon <[EMAIL PROTECTED]> writes:
> Traceback (most recent call last):
> File "", line 1, in ?
> File "D:\u\Python24\lib\zipfile.py", line 357, in read
> bytes = dc.decompress(bytes)
> MemoryError
Looks like the .iso file is huge. Even if it's only a CD image (approx
650MB), r
53 matches
Mail list logo
