[Numpy-discussion] How to speed up array generating

[klark--kent]

Hello. There is a random 1D array m_0 with size 3000, for example:m_0 = np.array([0, 1, 2])
I need to generate two 1D arrays:m_1 = np.array([0, 1, 2, 0, 1, 2, 0, 1, 2])
m_2 = np.array([0, 0, 0, 1, 1, 1, 2, 2, 2])
Is there faster way to do it than this one:import numpy as np
import time
N = 3
m_0 = np.arange(N)

t = time.time()
m_1 = np.tile(m_0, N)
m_2 = np.repeat(m_0, N)
t = time.time() - tI tried other ways but they are slower or have the same time. Other NumPy operations in my code 10-100 times faster. Why the repeating an array is so slow? I need 10 times speed up. Thank you for your attantion to my problem.___
NumPy-Discussion mailing list
NumPy-Discussion@python.org
https://mail.python.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] How to speed up array generating

[Joseph Fox-Rabinovitz]

What other ways have you tried?

On Sat, Jan 9, 2021 at 2:15 PM  wrote:

> Hello. There is a random 1D array m_0 with size 3000, for example:
>
> m_0 = np.array([0, 1, 2])
>
> I need to generate two 1D arrays:
>
> m_1 = np.array([0, 1, 2, 0, 1, 2, 0, 1, 2])
> m_2 = np.array([0, 0, 0, 1, 1, 1, 2, 2, 2])
>
> Is there faster way to do it than this one:
>
> import numpy as npimport time
> N = 3
> m_0 = np.arange(N)
>
> t = time.time()
> m_1 = np.tile(m_0, N)
> m_2 = np.repeat(m_0, N)
> t = time.time() - t
>
> I tried other ways but they are slower or have the same time. Other NumPy 
> operations in my code 10-100 times faster. Why the repeating an array is so 
> slow? I need 10 times speed up. Thank you for your attantion to my problem.
>
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
>
___
NumPy-Discussion mailing list
NumPy-Discussion@python.org
https://mail.python.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] How to speed up array generating

[klark--kent]

np.meshgrid, indexing, reshape09.01.2021, 22:30, "Joseph Fox-Rabinovitz" :What other ways have you tried?On Sat, Jan 9, 2021 at 2:15 PM  wrote:Hello. There is a random 1D array m_0 with size 3000, for example:m_0 = np.array([0, 1, 2])
I need to generate two 1D arrays:m_1 = np.array([0, 1, 2, 0, 1, 2, 0, 1, 2])
m_2 = np.array([0, 0, 0, 1, 1, 1, 2, 2, 2])
Is there faster way to do it than this one:import numpy as np
import time
N = 3
m_0 = np.arange(N)

t = time.time()
m_1 = np.tile(m_0, N)
m_2 = np.repeat(m_0, N)
t = time.time() - tI tried other ways but they are slower or have the same time. Other NumPy operations in my code 10-100 times faster. Why the repeating an array is so slow? I need 10 times speed up. Thank you for your attantion to my problem.___
NumPy-Discussion mailing list
NumPy-Discussion@python.org
https://mail.python.org/mailman/listinfo/numpy-discussion

,___NumPy-Discussion mailing listNumPy-Discussion@python.orghttps://mail.python.org/mailman/listinfo/numpy-discussion___
NumPy-Discussion mailing list
NumPy-Discussion@python.org
https://mail.python.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] How to speed up array generating

[Kevin Sheppard]

What about arange and then an integer divide or mod?

Kevin


On Sat, Jan 9, 2021, 20:54  wrote:

> np.meshgrid, indexing, reshape
>
> 09.01.2021, 22:30, "Joseph Fox-Rabinovitz" :
>
> What other ways have you tried?
>
> On Sat, Jan 9, 2021 at 2:15 PM  wrote:
>
> Hello. There is a random 1D array m_0 with size 3000, for example:
>
> m_0 = np.array([0, 1, 2])
>
> I need to generate two 1D arrays:
>
> m_1 = np.array([0, 1, 2, 0, 1, 2, 0, 1, 2])
> m_2 = np.array([0, 0, 0, 1, 1, 1, 2, 2, 2])
>
> Is there faster way to do it than this one:
>
> import numpy as npimport time
> N = 3
> m_0 = np.arange(N)
>
> t = time.time()
> m_1 = np.tile(m_0, N)
> m_2 = np.repeat(m_0, N)
> t = time.time() - t
>
> I tried other ways but they are slower or have the same time. Other NumPy 
> operations in my code 10-100 times faster. Why the repeating an array is so 
> slow? I need 10 times speed up. Thank you for your attantion to my problem.
>
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
>
> ,
>
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
>
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
>
___
NumPy-Discussion mailing list
NumPy-Discussion@python.org
https://mail.python.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] How to speed up array generating

[Kevin Sheppard]

Actually I would try a broadcast multiply followed by ravel first.

Kevin


On Sat, Jan 9, 2021, 21:12 Kevin Sheppard 
wrote:

> What about arange and then an integer divide or mod?
>
> Kevin
>
>
> On Sat, Jan 9, 2021, 20:54  wrote:
>
>> np.meshgrid, indexing, reshape
>>
>> 09.01.2021, 22:30, "Joseph Fox-Rabinovitz" :
>>
>> What other ways have you tried?
>>
>> On Sat, Jan 9, 2021 at 2:15 PM  wrote:
>>
>> Hello. There is a random 1D array m_0 with size 3000, for example:
>>
>> m_0 = np.array([0, 1, 2])
>>
>> I need to generate two 1D arrays:
>>
>> m_1 = np.array([0, 1, 2, 0, 1, 2, 0, 1, 2])
>> m_2 = np.array([0, 0, 0, 1, 1, 1, 2, 2, 2])
>>
>> Is there faster way to do it than this one:
>>
>> import numpy as npimport time
>> N = 3
>> m_0 = np.arange(N)
>>
>> t = time.time()
>> m_1 = np.tile(m_0, N)
>> m_2 = np.repeat(m_0, N)
>> t = time.time() - t
>>
>> I tried other ways but they are slower or have the same time. Other NumPy 
>> operations in my code 10-100 times faster. Why the repeating an array is so 
>> slow? I need 10 times speed up. Thank you for your attantion to my problem.
>>
>> ___
>> NumPy-Discussion mailing list
>> NumPy-Discussion@python.org
>> https://mail.python.org/mailman/listinfo/numpy-discussion
>>
>> ,
>>
>> ___
>> NumPy-Discussion mailing list
>> NumPy-Discussion@python.org
>> https://mail.python.org/mailman/listinfo/numpy-discussion
>>
>> ___
>> NumPy-Discussion mailing list
>> NumPy-Discussion@python.org
>> https://mail.python.org/mailman/listinfo/numpy-discussion
>>
>
___
NumPy-Discussion mailing list
NumPy-Discussion@python.org
https://mail.python.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] How to speed up array generating

[Sebastian Berg]

On Sat, 2021-01-09 at 23:45 +0300, klark--k...@yandex.ru wrote:
> np.meshgrid, indexing, reshape

I would expect e.g. reshape helps if you make use of the fact that this
is very clear in 2-D. The other thing is ensuring to use the methods
`arr.reshape`, `arr.repeat`, which avoids overheads (some of which may
be optimized away in the future though).

That said, with this type of code and the small arrays in the example,
there are fairly significant overheads.  So if  you want to get serious
input or a good solution, I would suggest to give more context and
include how those lines of code are used in your program/function.

Chances are that there is a much faster and possibly more elegant way
to write it, e.g. by solving many of these at the same time, but nobody
will be able to tell unless they know how you need it. Also with more
context, the other possibility is that somone may hint at cython or
numba/transonic.

Cheers,

Sebastian


> 
> 09.01.2021, 22:30, "Joseph Fox-Rabinovitz"
> :
> What other ways have you tried?
> 
> On Sat, Jan 9, 2021 at 2:15 PM  wrote:
> > Hello. There is a random 1D array m_0 with size 3000, for example:
> > m_0 = np.array([0, 1, 2])
> > I need to generate two 1D arrays:
> > m_1 = np.array([0, 1, 2, 0, 1, 2, 0, 1, 2])
> > m_2 = np.array([0, 0, 0, 1, 1, 1, 2, 2, 2])
> > Is there faster way to do it than this one:
> > import numpy as np
> > import time
> > N = 3
> > m_0 = np.arange(N)
> > 
> > t = time.time()
> > m_1 = np.tile(m_0, N)
> > m_2 = np.repeat(m_0, N)
> > t = time.time() - t
> > I tried other ways but they are slower or have the same time. Other
> > NumPy operations in my code 10-100 times faster. Why the repeating
> > an array is so slow? I need 10 times speed up. Thank you for your
> > attantion to my problem.
> > ___
> > NumPy-Discussion mailing list
> > NumPy-Discussion@python.org
> > https://mail.python.org/mailman/listinfo/numpy-discussion
> ,
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion



signature.asc
Description: This is a digitally signed message part
___
NumPy-Discussion mailing list
NumPy-Discussion@python.org
https://mail.python.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] How to speed up array generating

[V. Armando Sole]

I guess the speed up, if any, will be machine dependent, but you can
give a try at: 

import numpy as np
import time
N = 3000
m_0 = np.arange(N)
t = time.time()
a = np.ones(N)
m_1 = np.outer(m_0, a).ravel()
m_2 = np.outer(a, m_0).ravel()
t = time.time() - t

On 2021-01-09 20:07, klark--k...@yandex.ru wrote:

> Hello. There is a random 1D array m_0 with size 3000, for example:
> 
> m_0 = np.array([0, 1, 2])
> 
> I need to generate two 1D arrays:
> 
> m_1 = np.array([0, 1, 2, 0, 1, 2, 0, 1, 2])
> m_2 = np.array([0, 0, 0, 1, 1, 1, 2, 2, 2])
> 
> Is there faster way to do it than this one:
> 
> import numpy as np
> import time
> N = 3
> m_0 = np.arange(N)
> 
> t = time.time()
> m_1 = np.tile(m_0, N)
> m_2 = np.repeat(m_0, N)
> t = time.time() - t
> 
> I tried other ways but they are slower or have the same time. Other NumPy 
> operations in my code 10-100 times faster. Why the repeating an array is so 
> slow? I need 10 times speed up. Thank you for your attantion to my problem.
> 
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion___
NumPy-Discussion mailing list
NumPy-Discussion@python.org
https://mail.python.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] How to speed up array generating

[klark--kent]

Thank you but the same time:   import numpy as np    import time    N = 3000    m_0 = np.arange(N)    a = np.ones(N)#= first =    t1 = time.time()    m_1 = np.outer(m_0, a).ravel()    m_2 = np.outer(a, m_0).ravel()    t1 = time.time() - t1#= second ===    t2 = time.time()    m_3 = np.tile(m_0, N)    m_4  = np.repeat(m_0, N)    t2 = time.time() - t2#==    np.sum(m_1 - m_3)    np.sum(m_2 - m_4)    t1    t2 10.01.2021, 08:37, "V. Armando Sole" :I guess the speed up, if any, will be machine dependent, but you can give a try at:import numpy as npimport timeN = 3000m_0 = np.arange(N)t = time.time()a = np.ones(N)m_1 = np.outer(m_0, a).ravel()m_2 = np.outer(a, m_0).ravel()t = time.time() - t  On 2021-01-09 20:07, klark--k...@yandex.ru wrote:Hello. There is a random 1D array m_0 with size 3000, for example:m_0 = np.array([0, 1, 2])
I need to generate two 1D arrays:m_1 = np.array([0, 1, 2, 0, 1, 2, 0, 1, 2])
m_2 = np.array([0, 0, 0, 1, 1, 1, 2, 2, 2])
Is there faster way to do it than this one:import numpy as np
import time
N = 3
m_0 = np.arange(N)

t = time.time()
m_1 = np.tile(m_0, N)
m_2 = np.repeat(m_0, N)
t = time.time() - tI tried other ways but they are slower or have the same time. Other NumPy operations in my code 10-100 times faster. Why the repeating an array is so slow? I need 10 times speed up. Thank you for your attantion to my problem. ___NumPy-Discussion mailing listNumPy-Discussion@python.orghttps://mail.python.org/mailman/listinfo/numpy-discussion,___NumPy-Discussion mailing listNumPy-Discussion@python.orghttps://mail.python.org/mailman/listinfo/numpy-discussion___
NumPy-Discussion mailing list
NumPy-Discussion@python.org
https://mail.python.org/mailman/listinfo/numpy-discussion