Question about nested loop
Hi all,
I am a very novice for Python. Currently, I am trying to read continuous
columns repeatedly in the form of array.
my code is like below:
import numpy as np
b = []
c = 4
f = open("text.file", "r")
while c < 10:
c = c + 1
for columns in ( raw.strip().split() for raw in f ):
b.append(columns[c])
y = np.array(b, float)
print c, y
I thought that can get the arrays of the columns[5] to [10], but I only could
get repetition of same arrays of columns[5].
The result was something like:
5 [1 2 3 4 .., 10 9 8]
6 [1 2 3 4 .., 10 9 8]
7 [1 2 3 4 .., 10 9 8]
8 [1 2 3 4 .., 10 9 8]
9 [1 2 3 4 .., 10 9 8]
10 [1 2 3 4 .., 10 9 8]
What I can't understand is that even though c increased incrementally upto 10,
y arrays stay same.
Would someone help me to understand this problem more?
I really appreciate any help.
Thank you,
Isaac
--
http://mail.python.org/mailman/listinfo/python-list
Re: Question about nested loop
On Monday, December 31, 2012 5:25:16 AM UTC-6, Gisle Vanem wrote: > "Isaac Won" wrote: > > > > > while c < 10: > > >c = c + 1 > > > > > >for columns in ( raw.strip().split() for raw in f ): > > > > > > > > >b.append(columns[c]) > > > > > >y = np.array(b, float) > > >print c, y > > > > > > > > > I thought that can get the arrays of the columns[5] to [10], > > > but I only could get repetition of same arrays of columns[5]. > > > > I don't pretend to know list comprehension very well, but > > 'c' isn't incremented in the inner loop ( .. for raw in f). > > Hence you only append to columns[5]. > > > > Maybe you could use another 'd' indexer inside the inner-loop? > > But there must a more elegant way to solve your issue. (I'm a > > PyCommer myself). > > > > --gv Thank you for your advice. I agree with you and tried to increment in inner loop, but still not very succesful. Anyway many thanks for you. -- http://mail.python.org/mailman/listinfo/python-list
Re: Question about nested loop
On Monday, December 31, 2012 6:59:34 AM UTC-6, Hans Mulder wrote:
> On 31/12/12 11:02:56, Isaac Won wrote:
>
> > Hi all,
>
> > I am a very novice for Python. Currently, I am trying to read continuous
>
> > columns repeatedly in the form of array.
>
> > my code is like below:
>
> >
>
> > import numpy as np
>
> >
>
> > b = []
>
> > c = 4
>
> > f = open("text.file", "r")
>
> >
>
> > while c < 10:
>
> > c = c + 1
>
> >
>
> > for columns in ( raw.strip().split() for raw in f ):
>
> > b.append(columns[c])
>
> >
>
> > y = np.array(b, float)
>
> > print c, y
>
> >
>
> >
>
> > I thought that can get the arrays of the columns[5] to [10], but I only
>
> > could get repetition of same arrays of columns[5].
>
> >
>
> > The result was something like:
>
> >
>
> > 5 [1 2 3 4 .., 10 9 8]
>
> > 6 [1 2 3 4 .., 10 9 8]
>
> > 7 [1 2 3 4 .., 10 9 8]
>
> > 8 [1 2 3 4 .., 10 9 8]
>
> > 9 [1 2 3 4 .., 10 9 8]
>
> > 10 [1 2 3 4 .., 10 9 8]
>
> >
>
> >
>
> > What I can't understand is that even though c increased incrementally upto
> > 10,
>
> > y arrays stay same.
>
> >
>
> > Would someone help me to understand this problem more?
>
>
>
> That's because the inner loop read from a file until his reaches
>
> the end of the file. Since you're not resetting the file pointer,
>
> during the second and later runs of the outer loop, the inner loop
>
> starts at the end of the file and terminates without any action.
>
>
>
> You'd get more interesting results if you rewind the file:
>
>
>
> import numpy as np
>
>
>
> b = []
>
> c = 4
>
> f = open("text.file", "r")
>
>
>
> while c < 10:
>
> c = c + 1
>
>
>
> f.seek(0,0)
>
> for columns in ( raw.strip().split() for raw in f ):
>
> b.append(columns[c])
>
>
>
> y = np.array(b, float)
>
> print c, y
>
>
>
> It's a bit inefficient to read the same file several times.
>
> You might consider reading it just once. For example:
>
>
>
> import numpy as np
>
>
>
> b = []
>
>
>
> f = open("text.file", "r")
>
> data = [ line.strip().split() for line in f ]
>
> f.close()
>
>
>
> for c in xrange(5, 11):
>
> for row in data:
>
> b.append(row[c])
>
>
>
> y = np.array(b, float)
>
> print c, y
>
>
>
>
>
> Hope this helps,
>
>
>
> -- HansM
Hi Hans,
I appreciate your advice and kind tips.
The both codes which you gave seem pretty interesting.
Both look working for incrementing inner loop number, but the results of y are
added repeatedly such as [1,2,3],[1,2,3,4,5,6], [1,2,3,4,5,6,7,8,9]. Anyhow,
really thank you for your help and I will look at this problem more in detail.
Isaac
--
http://mail.python.org/mailman/listinfo/python-list
avoding the accumulation of array when using loop.
Hi all,
Thanks to Hans, I have had a good progress on my problem.
Followings are Hans's Idea:
import numpy as np
b = []
c = 4
f = open("text.file", "r")
while c < 10:
c = c + 1
f.seek(0,0)
for columns in ( raw.strip().split() for raw in f ):
b.append(columns[c])
y = np.array(b, float)
print c, y
It's a bit inefficient to read the same file several times.
You might consider reading it just once. For example:
import numpy as np
b = []
f = open("text.file", "r")
data = [ line.strip().split() for line in f ]
f.close()
for c in xrange(5, 11):
for row in data:
b.append(row[c])
y = np.array(b, float)
print c, y
---
It is a great idea, but I found some problems. I want each individual array of
y. However, these two codes prodce accumulated array such as [1,2,3],
[1,2,3,4,5,6], [1,2,3,4,5,6,7,8,9] and so on. I have tried to initialize for
loop for each time to produce array. This effort has not been very successful.
Do you guys have any idea? I will really appreciate ant help and idea.
Thanks,
Isaac
--
http://mail.python.org/mailman/listinfo/python-list
Re: avoding the accumulation of array when using loop.
On Wednesday, January 2, 2013 5:54:18 PM UTC-6, Dave Angel wrote:
> On 01/02/2013 05:21 PM, Isaac Won wrote:
>
> > Hi all,
>
> >
>
> > Thanks to Hans, I have had a good progress on my problem.
>
> >
>
> > Followings are Hans's Idea:
>
> >
>
> > import numpy as np
>
> >
>
> > b = []
>
> > c = 4
>
> > f = open("text.file", "r")
>
> >
>
> > while c < 10:
>
> > c = c + 1
>
> >
>
> >
>
> > f.seek(0,0)
>
> >
>
> > for columns in ( raw.strip().split() for raw in f ):
>
> > b.append(columns[c])
>
> >
>
> > y = np.array(b, float)
>
> > print c, y
>
> >
>
> >
>
> > It's a bit inefficient to read the same file several times.
>
>
>
> Don't bet on it. The OS and the libraries and Python each do some
>
> buffering, so it might be nearly as fast to just reread if it's a small
>
> file. And if it's a huge one, the list would be even bigger. So the
>
> only sizes where the second approach is likely better is the mid-size file.
>
>
>
> > You might consider reading it just once. For example:
>
> >
>
> >
>
> > import numpy as np
>
> >
>
> > b = []
>
> >
>
> >
>
> >
>
> > f = open("text.file", "r")
>
> >
>
> > data = [ line.strip().split() for line in f ]
>
> > f.close()
>
> >
>
> > for c in xrange(5, 11):
>
> > for row in data:
>
> > b.append(row[c])
>
> >
>
> >
>
> > y = np.array(b, float)
>
> > print c, y
>
> > ---
>
> >
>
> > It is a great idea, but I found some problems. I want each individual array
> > of y. However, these two codes prodce accumulated array such as [1,2,3],
> > [1,2,3,4,5,6], [1,2,3,4,5,6,7,8,9] and so on. I have tried to initialize
> > for loop for each time to produce array. This effort has not been very
> > successful.
>
> > Do you guys have any idea? I will really appreciate ant help and idea.
>
>
>
> Your description is very confusing. But i don't see why you just don't
>
> just set b=[] inside the outer loop, rather than doing it at the begin
>
> of the program.
>
>
>
> for c in xrange(5, 11):
>
> b = []
>
> for row in data:
>
> b.append(row[c])
>
>
>
>
>
>
>
> --
>
>
>
> DaveA
Hi Dave,
I really appreciate your advice. It was really helpful.
Isaac
--
http://mail.python.org/mailman/listinfo/python-list
Memory error with quadratic interpolation
Hi all, I have tried to use different interpolation methods with Scipy. My code seems just fine with linear interpolation, but shows memory error with quadratic. I am a novice for python. I will appreciate any help. #code f = open(filin, "r") for columns in ( raw.strip().split() for raw in f ): a.append(columns[5]) x = np.array(a, float) not_nan = np.logical_not(np.isnan(x)) indices = np.arange(len(x)) interp = interp1d(indices[not_nan], x[not_nan], kind = 'quadratic') p = interp(indices) The number of data is 31747. Thank you, Isaac -- http://mail.python.org/mailman/listinfo/python-list
Re: Memory error with quadratic interpolation
On Tuesday, January 22, 2013 10:06:41 PM UTC-6, Isaac Won wrote: > Hi all, > > > > I have tried to use different interpolation methods with Scipy. My code seems > just fine with linear interpolation, but shows memory error with quadratic. I > am a novice for python. I will appreciate any help. > > > > #code > > f = open(filin, "r") > > for columns in ( raw.strip().split() for raw in f ): > > a.append(columns[5]) > > x = np.array(a, float) > > > > > > not_nan = np.logical_not(np.isnan(x)) > > indices = np.arange(len(x)) > > interp = interp1d(indices[not_nan], x[not_nan], kind = 'quadratic') > > > > p = interp(indices) > > > > > > The number of data is 31747. > > > > Thank you, > > > > Isaac I really appreciate to both Ulich and Oscar. To Oscar My actual error message is: File "/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/interpolate/interpolate.py", line 311, in __init__ self._spline = splmake(x,oriented_y,order=order) File "/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/interpolate/interpolate.py", line 809, in splmake coefs = func(xk, yk, order, conds, B) File "/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/interpolate/interpolate.py", line 530, in _find_smoothest u,s,vh = np.dual.svd(B) File "/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/linalg/decomp_svd.py", line 91, in svd full_matrices=full_matrices, overwrite_a = overwrite_a) MemoryError -- Thank you, Hoonill -- http://mail.python.org/mailman/listinfo/python-list
Re: Memory error with quadratic interpolation
On Wednesday, January 23, 2013 4:08:13 AM UTC-6, Oscar Benjamin wrote: > On 23 January 2013 08:55, Ulrich Eckhardt > > wrote: > > > Am 23.01.2013 05:06, schrieb Isaac Won: > > > > > >> I have tried to use different interpolation methods with Scipy. My > > >> code seems just fine with linear interpolation, but shows memory > > >> error with quadratic. I am a novice for python. I will appreciate any > > >> help. > > > > > [SNIP] > > > > > > > > > Concerning the rest of your problems, there is lots of code and the datafile > > > missing. However, there is also too much of it, try replacing the file with > > > generated data and remove everything from the code that is not absolutely > > > necessary. > > > > Also please copy paste the actual error message rather than paraphrasing it. > > > > > > Oscar I really appreciate to both Ulich and Oscar. To Oscar My actual error message is: File "/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/interpolate/interpolate.py", line 311, in __init__ self._spline = splmake(x,oriented_y,order=order) File "/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/interpolate/interpolate.py", line 809, in splmake coefs = func(xk, yk, order, conds, B) File "/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/interpolate/interpolate.py", line 530, in _find_smoothest u,s,vh = np.dual.svd(B) File "/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/linalg/decomp_svd.py", line 91, in svd full_matrices=full_matrices, overwrite_a = overwrite_a) MemoryError -- Thank you, Isaac -- http://mail.python.org/mailman/listinfo/python-list
Re: Memory error with quadratic interpolation
On Wednesday, January 23, 2013 4:08:13 AM UTC-6, Oscar Benjamin wrote: > On 23 January 2013 08:55, Ulrich Eckhardt > > > > > Am 23.01.2013 05:06, schrieb Isaac Won: > > > > > >> I have tried to use different interpolation methods with Scipy. My > > >> code seems just fine with linear interpolation, but shows memory > > >> error with quadratic. I am a novice for python. I will appreciate any > > >> help. > > > > > [SNIP] > > > > > > > > > Concerning the rest of your problems, there is lots of code and the datafile > > > missing. However, there is also too much of it, try replacing the file with > > > generated data and remove everything from the code that is not absolutely > > > necessary. > > > > Also please copy paste the actual error message rather than paraphrasing it. > > > > > > Oscar I really appreciate to both Ulich and Oscar. To Oscar My actual error message is: File "/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/interpolate/interpolate.py", line 311, in __init__ self._spline = splmake(x,oriented_y,order=order) File "/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/interpolate/interpolate.py", line 809, in splmake coefs = func(xk, yk, order, conds, B) File "/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/interpolate/interpolate.py", line 530, in _find_smoothest u,s,vh = np.dual.svd(B) File "/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/linalg/decomp_svd.py", line 91, in svd full_matrices=full_matrices, overwrite_a = overwrite_a) MemoryError -- Thank you, Hoonill -- http://mail.python.org/mailman/listinfo/python-list
Re: Memory error with quadratic interpolation
On Wednesday, January 23, 2013 2:55:14 AM UTC-6, Ulrich Eckhardt wrote: > Am 23.01.2013 05:06, schrieb Isaac Won: > > > I have tried to use different interpolation methods with Scipy. My > > > code seems just fine with linear interpolation, but shows memory > > > error with quadratic. I am a novice for python. I will appreciate any > > > help. > > > > > > #code > > > f = open(filin, "r") > > > > Check out the "with open(...) as f" syntax. > > > > > > > for columns in ( raw.strip().split() for raw in f ): > > > > For the record, this first builds a sequence and then iterates over that > > sequence. This is not very memory-efficient, try this instead: > > > > for line in f: > > columns = line.strip().split() > > > > > > Concerning the rest of your problems, there is lots of code and the > > datafile missing. However, there is also too much of it, try replacing > > the file with generated data and remove everything from the code that is > > not absolutely necessary. > > > > Good luck! > > > > Uli Hi Ulich, I tried to change the code following your advice, but it doesn't seem to work still. My adjusted code is: a = [] with open(filin, "r") as f: for line in f: columns = line.strip().split() a.append(columns[5]) x = np.array(a, float) not_nan = np.logical_not(np.isnan(x)) indices = np.arange(len(x)) interp = interp1d(indices[not_nan], x[not_nan], kind = 'quadratic') p = interp(indices) - And full error message is: interp = interp1d(indices[not_nan], x[not_nan], kind = 'quadratic') File "/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/interpolate/interpolate.py", line 311, in __init__ self._spline = splmake(x,oriented_y,order=order) File "/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/interpolate/interpolate.py", line 809, in splmake coefs = func(xk, yk, order, conds, B) File "/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/interpolate/interpolate.py", line 530, in _find_smoothest u,s,vh = np.dual.svd(B) File "/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/linalg/decomp_svd.py", line 91, in svd full_matrices=full_matrices, overwrite_a = overwrite_a) MemoryError --- Could you give me some advice for this situation? Thank you always, Isaac -- http://mail.python.org/mailman/listinfo/python-list
Re: Memory error with quadratic interpolation
On Wednesday, January 23, 2013 8:40:54 AM UTC-6, Oscar Benjamin wrote: > On 23 January 2013 14:28, Isaac Won wrote: > > > On Wednesday, January 23, 2013 4:08:13 AM UTC-6, Oscar Benjamin wrote: > > > > > > To Oscar > > > My actual error message is: > > > File > > "/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/interpolate/interpolate.py", > > line 311, in __init__ > > > self._spline = splmake(x,oriented_y,order=order) > > > File > > "/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/interpolate/interpolate.py", > > line 809, in splmake > > > coefs = func(xk, yk, order, conds, B) > > > File > > "/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/interpolate/interpolate.py", > > line 530, in _find_smoothest > > > u,s,vh = np.dual.svd(B) > > > File > > "/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/linalg/decomp_svd.py", > > line 91, in svd > > > full_matrices=full_matrices, overwrite_a = overwrite_a) > > > MemoryError > > > > Are you sure that's the *whole* error message? The traceback only > > refers to the scipy modules. I can't see the line from your code that > > is generating the error. > > > > > > Oscar Dear Oscar, Following is full error message after I adjusted following Ulich's advice: interp = interp1d(indices[not_nan], x[not_nan], kind = 'quadratic') File "/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/interpolate/interpolate.py", line 311, in __init__ self._spline = splmake(x,oriented_y,order=order) File "/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/interpolate/interpolate.py", line 809, in splmake coefs = func(xk, yk, order, conds, B) File "/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/interpolate/interpolate.py", line 530, in _find_smoothest u,s,vh = np.dual.svd(B) File "/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/linalg/decomp_svd.py", line 91, in svd full_matrices=full_matrices, overwrite_a = overwrite_a) MemoryError -- Thank you, Hoonill -- http://mail.python.org/mailman/listinfo/python-list
Re: Memory error with quadratic interpolation
On Wednesday, January 23, 2013 10:51:43 AM UTC-6, Oscar Benjamin wrote:
> On 23 January 2013 14:57, Isaac Won wrote:
>
> > On Wednesday, January 23, 2013 8:40:54 AM UTC-6, Oscar Benjamin wrote:
>
> >> On 23 January 2013 14:28, Isaac Won wrote:
>
> >>
>
> [SNIP]
>
> >
>
> > Following is full error message after I adjusted following Ulich's advice:
>
> >
>
> > interp = interp1d(indices[not_nan], x[not_nan], kind = 'quadratic')
>
> > File
> > "/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/interpolate/interpolate.py",
> > line 311, in __init__
>
> > self._spline = splmake(x,oriented_y,order=order)
>
> > File
> > "/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/interpolate/interpolate.py",
> > line 809, in splmake
>
> > coefs = func(xk, yk, order, conds, B)
>
> > File
> > "/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/interpolate/interpolate.py",
> > line 530, in _find_smoothest
>
> > u,s,vh = np.dual.svd(B)
>
> > File
> > "/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/linalg/decomp_svd.py",
> > line 91, in svd
>
> > full_matrices=full_matrices, overwrite_a = overwrite_a)
>
> > MemoryError
>
>
>
> Where is the new code? You should show full working code (with the
>
> import statements) and the full error that is generated by exactly
>
> that code. If possible you should also write code that someone else
>
> could run even without having access to your data files. If you did
>
> that in your first post, you'd probably have an answer to your problem
>
> by now.
>
>
>
> Here is a version of your code that many people on this list can test
>
> straight away:
>
>
>
> import numpy as np
>
> from scipy.interpolate import interp1d
>
> x = np.array(31747 * [0.0], float)
>
> indices = np.arange(len(x))
>
> interp = interp1d(indices, x, kind='quadratic')
>
>
>
> Running this gives the following error:
>
>
>
> ~$ python tmp.py
>
> Traceback (most recent call last):
>
> File "tmp.py", line 5, in
>
> interp = interp1d(indices, x, kind='quadratic')
>
> File "/usr/lib/python2.7/dist-packages/scipy/interpolate/interpolate.py",
>
> line 308, in __init__
>
> self._spline = splmake(x,oriented_y,order=order)
>
> File "/usr/lib/python2.7/dist-packages/scipy/interpolate/interpolate.py",
>
> line 805, in splmake
>
> B = _fitpack._bsplmat(order, xk)
>
> MemoryError
>
>
>
> Unless I've misunderstood how this function is supposed to be used, it
>
> just doesn't really seem to work for arrays of much more than a few
>
> hundred elements.
>
>
>
>
>
> Oscar
Thank you Oscar for your help and advice.
I agree with you. So, I tried to find the way to solve this problem.
My full code adjusted is:
from scipy.interpolate import interp1d
import numpy as np
import matplotlib.pyplot as plt
with open(filin, "r") as f:
for line in f:
columns = line.strip().split()
a.append(columns[5])
x = np.array(a, float)
not_nan = np.logical_not(np.isnan(x))
indices = np.arange(len(x))
interp = interp1d(indices[not_nan], x[not_nan], kind = 'quadratic')
p = interp(indices)
k = np.arange(31747)
plt.subplot(211)
plt.plot(k, p)
plt.xlabel('Quadratic interpolation')
plt.subplot(212)
plt.plot(k, x)
plt.show()
-
Whole error message was:
Traceback (most recent call last):
File "QI1.py", line 22, in
interp = interp1d(indices[not_nan], x[not_nan], kind = 'quadratic')
File
"/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/interpolate/interpolate.py",
line 311, in __init__
self._spline = splmake(x,oriented_y,order=order)
File
"/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/interpolate/interpolate.py",
line 809, in splmake
coefs = func(xk, yk, order, conds, B)
File
"/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/interpolate/interpolate.py",
line 530, in _find_smoothest
u,s,vh = np.dual.svd(B)
File
"/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/linalg/decomp_svd.py",
line 91, in svd
full_matrices=full_matrices, overwrite_a = overwrite_a)
MemoryError
--
Thank you again Oscar,
Isaac
--
http://mail.python.org/mailman/listinfo/python-list
Plot a contour inside a contour
I tried to plot one smaller contour inside of the other larger contour. I have two different 2D-arrays. One is with smaller grid spacing and smaller domain size and the other is with larger spacing and larger domain size. So, I tried to use fig.add_axes function as follows: fig = plt.figure() ax1 = fig.add_axes([0.1,0.1,0.8,0.8]) . . dx = 450 NX = SHFX_plt.shape[1] NY = SHFX_plt.shape[0] xdist= (np.arange(NX)*dx+dx/2.)/1000. ydist= (np.arange(NY)*dx+dx/2.)/1000. myPLT = plt.pcolor(xdist,ydist,SHFX_plt) . . ax2 = fig.add_axes([8.,8.,18.,18.]) dx1 = 150 NX1 = SHFX_plt1.shape[1] NY1 = SHFX_plt1.shape[0] print 'NX1=',NX1,'NY1=',NY1 xdist1= (np.arange(NX1)*dx1+dx1/2.)/1000. ydist1= (np.arange(NY1)*dx1+dx1/2.)/1000. myPLT1 = plt.pcolor(xdist1,ydist1,SHFX_plt1) plt.show() My intention is to plot ax2 on the top of ax1 from xdist and ydist = 8 with 18 by 18 size. However, the result seems only showing ax1. I will really appreciate any help or idea. Thank you, Isaac -- https://mail.python.org/mailman/listinfo/python-list
Re: Plot a contour inside a contour
On Thursday, November 14, 2013 2:01:39 PM UTC-8, John Ladasky wrote: > On Thursday, November 14, 2013 11:39:37 AM UTC-8, Isaac Won wrote: > > > I tried to plot one smaller contour inside of the other larger contour. > > > > Using what software? A plotting package is not part of the Python standard > library. > Thanks John, I am using Matplotlib package. I will ask the question in the > matplotlib-users discussion group as you suggested. Thank you again, Isaac > > > You did not show the import statements in your code. If I had to guess, I > would say that you are using the Matplotlib package. Questions which are > specific to matplotlib should be asked in the matplotlib-users discussion > group: > > > > https://lists.sourceforge.net/lists/listinfo/matplotlib-users -- https://mail.python.org/mailman/listinfo/python-list
Using MFDataset to combine netcdf files in python
I am trying to combine netcdf files, but it contifuously shows " File "CBL_plot.py", line 11, in f = MFDataset(fili) File "utils.pyx", line 274, in netCDF4.MFDataset.init (netCDF4.c:3822) IOError: master dataset THref_11:00.nc does not have a aggregation dimension." So, I checked only one netcdf files and the information of a netcdf file is as below: float64 th_ref(u't',) unlimited dimensions = () current size = (30,) It looks there is no aggregation dimension. However, I would like to combine those netcdf files rather than just using one by one. Is there any way to create aggregation dimension to make this MFData set work? Below is the python code I used: import numpy as np from netCDF4 import MFDataset varn = 'th_ref' fili = THref_*nc' f= MFDataset(fili) Th = f.variables[varn] Th_ref=np.array(Th[:]) print Th.shape I will really appreciate any help, idea, and hint. Thank you, Isaac -- https://mail.python.org/mailman/listinfo/python-list
Drawing shaded area depending on distance with latitude and altitude coordinate
I have tried to make a plot of points with longitude and latitude coordinate,
and draw shaded area with distance from one point. So, I thought that I could
uae contourf function from matplotlibrary. My code is:
import haversine
import numpy as np
import matplotlib.pyplot as plt
with open(filin, 'r') as f:
arrays = [map(float, line.split()) for line in f]
newa = [[x[1],-x[2]] for x in arrays]
lat = np.zeros(275)
lon = np.zeros(275)
for c in range(0,275):
lat[c] = newa[c][0]
lon[c] = newa[c][1]
with open(filin, 'r') as f:
arrays = [map(float, line.split()) for line in f]
newa = [[x[1],-x[2]] for x in arrays]
lat = np.zeros(275)
lon = np.zeros(275)
for c in range(0,275):
lat[c] = newa[c][0]
lon[c] = newa[c][1]
dis = np.zeros(275)
for c in range(0,275):
dis[c] = haversine.distance(newa[0],[lat[c],lon[c]])
dis1 = [[]]*1
for c in range(0,275):
dis1[0].append(dis[c])
cs = plt.contourf(lon,lat,dis1)
cb = plt.colorbar(cs)
plt.plot(-lon[0],lat[0],'ro')
plt.plot(-lon[275],lat[275],'ko')
plt.plot(-lon[1:275],lat[1:275],'bo')
plt.xlabel('Longitude(West)')
plt.ylabel('Latitude(North)')
plt.gca().invert_xaxis()
plt.show()
My idea in this code was that I could made a shaded contour by distance from a
certain point which was noted as newa[0] in the code. I calculated distances
between newa[0] and other points by haversine module which calculate distances
with longitudes and latitudes of two points. However, whenever I ran this code,
I got the error related to X, Y or Z in contourf such as:
TypeError: Length of x must be number of columns in z, and length of y must
be number of rows.
IF I use meshgrid for X and Y, I also get:
TypeError: Inputs x and y must be 1D or 2D.
I just need to draw shaded contour with distance from one point on the top of
the plot of each point.
If you give any idea or hint, I will really apprecite. Thank you, Isaac
--
https://mail.python.org/mailman/listinfo/python-list
Extracting the value from Netcdf file with longitude and lattitude
Hi,
My question may be confusing.
Now I would like to extract temperature values from model output with python.
My model output have separate temperature, longitude and latitude variables.
So, I overlap these three grid variables on one figure to show temperature with
longitude and latitude through model domain.
Up to this point, everything is fine. The problem is to extract temperature
value at certain longitude and latitude.
Temperature variable doesn't have coordinate, but only values with grid.
Do you have idea about this issue?
Below is my code for the 2 D plot with temperature on model domain.
varn1 = 'T2'
varn2 = 'XLONG'
varn3 = 'XLAT'
Temp = read_netcdf(filin,varn1)
Lon = read_netcdf(filin,varn2)
Lat = read_netcdf(filin,varn3)
Temp_plt = Temp[12,:,:]
Lon_plt = Lon[12,:,:]
Lat_plt = Lat[12,:,:]
x = Lon_plt
y = Lat_plt
Temp_c = Temp_plt-273.15
myPLT = plt.pcolor(x,y,Temp_c)
mxlabel = plt.xlabel('Latitude')
mylabel = plt.ylabel('Longitude')
plt.xlim(126.35,127.35)
plt.ylim(37.16,37.84)
myBAR = plt.colorbar(myPLT)
myBAR.set_label('Temperature ($^\circ$C)')
plt.show()
--
read_netcdf is a code for extracting values of [time, x,y] format.
I think that the point is to bind x, y in Temp_plt with x, y in Lon_plt and
Lat_plt to extract temperature values with longitude and latitude input.
This question might be confusing. If you can't understand please let me know.
Any idea or help will be really appreciated.
Best regards,
Hoonill
--
https://mail.python.org/mailman/listinfo/python-list
About a value error called 'ValueError: A value in x_new is below the interpolation range'
Dear all,
I am trying to calculate correlation coefficients between one time series data
and other time series. However,there are some missing values. So, I
interploated each time series with 1d interpolation in scipy and got
correlation coefficients between them. This code works well for some data sets,
but doesn't for some others. Following is actual error I got:
0.0708904109589
0.0801369863014
0.0751141552511
0.0938356164384
0.0769406392694
Traceback (most recent call last):
File "error_removed.py", line 56, in
i2 = interp(indices)
File
"/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/interpolate/interpolate.py",
line 394, in __call__
out_of_bounds = self._check_bounds(x_new)
File
"/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/interpolate/interpolate.py",
line 449, in _check_bounds
raise ValueError("A value in x_new is below the interpolation "
ValueError: A value in x_new is below the interpolation range.
This time is 'x_new is below the interpolation range", but some times, it shows
"above interpolation range.'
I would like to make some self-contained code, but, I am not sure how to make
it to represent my case well.
I just put all of my code here. I apologize for this inconvenience.
---
-
a = []
c = 4
with open(filin1, 'r') as f1:
arrays = [map(float, line.split()) for line in f1]
newa = [[x[1],x[2]] for x in arrays]
o = newa[58]
f = open(filin, "r")
percent1 = []
for columns in ( raw.strip().split() for raw in f ):
a.append(columns[63])
x = np.array(a, float)
not_nan = np.logical_not(np.isnan(x))
indices = np.arange(len(x))
interp = interp1d(indices[not_nan], x[not_nan])
#interp = np.interp(indices, indices[not_nan], x[not_nan])
i1 = interp(indices)
f.close
h1 = []
p1 = []
while c <278:
c = c + 1
d = c - 5
b = []
f.seek(0,0)
for columns in ( raw.strip().split() for raw in f ):
b.append(columns[c])
y = np.array(b, float)
h = haversine.distance(o, newa[d])
n = len(y)
l = b.count('nan')
percent = l/8760.
percent1 = percent1 + [percent]
#print l, percent
if percent < 0.1:
not_nan = np.logical_not(np.isnan(y))
indices = np.arange(len(y))
interp = interp1d(indices[not_nan], y[not_nan])
#interp = np.interp(indices, indices[not_nan], x[not_nan])
i2 = interp(indices)
pearcoef = sp.pearsonr(i1,i2)
p = pearcoef[0]
p1 = p1 + [p]
h1 = h1 + [h]
print percent
print h1
print p1
print len(p1)
plt.plot(h1, p1, 'o')
plt.xlabel('Distance(km)')
plt.ylabel('Correlation coefficient')
plt.grid(True)
plt.show()
---
For any help or advice, I will really appreciate.
Best regards,
Isaac
--
http://mail.python.org/mailman/listinfo/python-list
Triple nested loop python (While loop insde of for loop inside of while loop)
try to make my triple nested loop working. My code would be: c = 4 y1 = [] m1 = [] std1 = [] while c <24: c = c + 1 a = [] f.seek(0,0) for columns in ( raw.strip().split() for raw in f ): a.append(columns[c]) x = np.array(a, float) not_nan = np.logical_not(np.isnan(x)) indices = np.arange(len(x)) interp = interp1d(indices[not_nan], x[not_nan], kind = 'nearest') p = interp(indices) N = len(p) dt = 900.0 #Time step (seconds) fs = 1./dt #Sampling frequency KA,PSD = oned_Fourierspectrum(p,dt) # Call Song's 1D FS function time_axis = np.linspace(0.0,N,num = N,endpoint = False)*15/(60*24) plot_freq = 24*3600.*KA #Convert to cycles per day plot_period = 1.0/plot_freq # convert to days/cycle fpsd = plot_freq*PSD d = -1 while d <335: d = d + 1 y = fpsd[d] y1 = y1 + [y] m = np.mean(y1) m1 = m1 + [m] print m1 My purpose is make a list of [mean(fpsd[0]), mean(fpsd[1]), mean(fpsd[2]).. mean(fpsd[335])]. Each y1 would be the list of fpsd[d]. I check it is working pretty well before second while loop and I can get individual mean of fpsd[d]. However, with second whole loop, it produces definitely wrong numbers. Would you help me this problem? -- http://mail.python.org/mailman/listinfo/python-list
Re: Triple nested loop python (While loop insde of for loop inside of while loop)
Thank you, Chris. I just want to acculate value from y repeatedly. If y = 1,2,3...10, just have a [1,2,3...10] at onece. On Friday, March 1, 2013 7:41:05 AM UTC-6, Chris Angelico wrote: > On Fri, Mar 1, 2013 at 7:59 PM, Isaac Won wrote: > > > while c <24: > > > for columns in ( raw.strip().split() for raw in f ): > > > while d <335: > > > > Note your indentation levels: the code does not agree with your > > subject line. The third loop is not actually inside your second. > > Should it be? > > > > ChrisA -- http://mail.python.org/mailman/listinfo/python-list
Re: Triple nested loop python (While loop insde of for loop inside of while loop)
On Friday, March 1, 2013 7:41:05 AM UTC-6, Chris Angelico wrote: > On Fri, Mar 1, 2013 at 7:59 PM, Isaac Won wrote: > > > while c <24: > > > for columns in ( raw.strip().split() for raw in f ): > > > while d <335: > > > > Note your indentation levels: the code does not agree with your > > subject line. The third loop is not actually inside your second. > > Should it be? > > > > ChrisA Yes, the thiird lood should be inside of my whole loop. Thank you, Isaac -- http://mail.python.org/mailman/listinfo/python-list
Re: Triple nested loop python (While loop insde of for loop inside of while loop)
Thank you Ulich for reply, What I really want to get from this code is m1 as I told. For this purpose, for instance, values of fpsd upto second loop and that from third loop should be same, but they are not. Actually it is my main question. Thank you, Isaac On Friday, March 1, 2013 6:00:42 AM UTC-6, Ulrich Eckhardt wrote: > Am 01.03.2013 09:59, schrieb Isaac Won: > > > try to make my triple nested loop working. My code would be: > > > c = 4 > > [...] > > > while c <24: > > > c = c + 1 > > > > This is bad style and you shouldn't do that in python. The question that > > comes up for me is whether something else is modifying "c" in that loop, > > but I think the answer is "no". For that reason, use Python's way: > > > >for c in range(5, 25): > >... > > > > That way it is also clear that the first value in the loop is 5, while > > the initial "c = 4" seems to suggest something different. Also, the last > > value is 24, not 23. > > > > > > > > > while d <335: > > > d = d + 1 > > > y = fpsd[d] > > > y1 = y1 + [y] > > > m = np.mean(y1) > > > m1 = m1 + [m] > > > > Apart from the wrong indention (don't mix tabs and spaces, see PEP 8!) > > and the that "d in range(336)" is better style, you don't start with an > > empty "y1", except on the first iteration of the outer loop. > > > > I'm not really sure if that answers your problem. In any case, please > > drop everything not necessary to demostrate the problem before posting. > > This makes it easier to see what is going wrong both for you and others. > > Also make sure that others can actually run the code. > > > > > > Greetings from Hamburg! > > > > Uli -- http://mail.python.org/mailman/listinfo/python-list
Putting the loop inside of loop properly
I just would like to make my previous question simpler and I bit adjusted my code with help with Ulich and Chris. The basic structure of my code is: for c in range(5,25): for columns in ( raw.strip().split() for raw in f ): a.append(columns[c]) x = np.array(a, float) not_nan = np.logical_not(np.isnan(x)) indices = np.arange(len(x)) interp = interp1d(indices[not_nan], x[not_nan], kind = 'nearest') p = interp(indices) N = len(p) fpsd = plot_freq*PSD f.seek(0,0) for d in range(336): y = fpsd[d] y1 = y1 + [y] m = np.mean(y1) m1 = m1 + [m] -- I just removed seemingly unnecesary lines. I expect that last loop can produce the each order values (first, second, last(336th)) of fpsd from former loop. fpsd would be 20 lists. So, fpsd[0] in third loop shoul be first values from 20 lists and it expects to be accumulated to y1. So, y1 should be the list of first values from 20 fpsd lists. and m is mean of y1. I expect to repeat 356 times and accumulated to m1. However, it doesn't work and fpsd values in and out of the last loop are totally different. My question is clear? Any help or questions would be really appreciated. Isaac -- http://mail.python.org/mailman/listinfo/python-list
