For DataFrames, it depends on what you want to do. It is difficult to get
performance with DataArrays as columns using the current implementation.
With the ongoing work by John Myles White on the use of a Nullable type,
that should be much better. Also, you can use standard Arrays as columns of
a DataFrame. It's not documented well, but it can be done.
Also, if you want to treat a DataFrame like a matrix, then generally the
answer is no. With some "trickery", you can store a view to a matrix in a
DataFrame. Basically, you have to create column views into the matrix. Here
is an example. It might be useful if you want to treat all or part of a
DataFrame as a matrix.
julia> using DataFrames
julia> m = rand(5,5)
5x5 Array{Float64,2}:
0.186736 0.247699 0.0968634 0.471383 0.145244
0.985306 0.966015 0.663865 0.0468244 0.0471465
0.981947 0.707241 0.0841202 0.0539529 0.692217
0.918222 0.0415162 0.646298 0.581983 0.653881
0.515692 0.0344289 0.0821672 0.877258 0.653756
julia> d = DataFrame(Any[sub(m, :, i) for i in 1:size(m, 2)], [:a, :b, :c,
:d, :e])
5x5 DataFrames.DataFrame
| Row | a | b | c | d | e |
|-----|----------|-----------|-----------|-----------|-----------|
| 1 | 0.186736 | 0.247699 | 0.0968634 | 0.471383 | 0.145244 |
| 2 | 0.985306 | 0.966015 | 0.663865 | 0.0468244 | 0.0471465 |
| 3 | 0.981947 | 0.707241 | 0.0841202 | 0.0539529 | 0.692217 |
| 4 | 0.918222 | 0.0415162 | 0.646298 | 0.581983 | 0.653881 |
| 5 | 0.515692 | 0.0344289 | 0.0821672 | 0.877258 | 0.653756 |
julia> d[:b]
5-element SubArray{Float64,1,Array{Float64,2},(Colon,Int64),2}:
0.247699
0.966015
0.707241
0.0415162
0.0344289
On Fri, Dec 5, 2014 at 8:37 AM, Tim Holy <[email protected]> wrote:
> The new SubArrays steal mercilessly from the good ideas of both the old
> SubArray and ArrayViews, and then add some new tricks of their own. In
> theory,
> they should be a strict improvement on both of their predecessors.
>
> --Tim
>
> On Friday, December 05, 2014 05:13:28 AM David van Leeuwen wrote:
> > So what is the relation between ArrayViews and 0.4 `SubArray revamp'?
> Are
> > they targeting different use cases or is one of them going to be phased
> > out?
> >
> > On Friday, December 5, 2014 1:28:16 PM UTC+1, Tim Holy wrote:
> > > In 0.4, the views are a revamp of SubArray. So if NamedArrays already
> > > interacts well with SubArrays, you're basically set.
> > >
> > > Most likely not, as SubArrays are new to me (I've tried sub() in
> >
> > production code in the past, but it was always better to write out a
> > loop)).
> >
> > > FYI the implementation in 0.4 is largely backwards-compatible, but
> there
> > > are
> > > some important differences. If you need to dig into the internal
> > > implementation
> > > of SubArrays, you can find documentation on the changes for 0.4 here:
> > > http://docs.julialang.org/en/latest/devdocs/subarrays/
> > >
> > > OK, I'll have to study this in order to understand the implications for
> >
> > NamedArrays.
> >
> > ---david
> >
> > > --Tim
> > >
> > > On Friday, December 05, 2014 04:18:08 AM David van Leeuwen wrote:
> > > > Hi,
> > > >
> > > > On Friday, December 5, 2014 8:47:22 AM UTC+1, Ján Dolinský wrote:
> > > > > Hi,
> > > > >
> > > > > I am exploring DataFrames and NamedArrays packages and I would
> like to
> > >
> > > ask
> > >
> > > > > whether their are suitable for heavier computations and whether I
> can
> > >
> > > use
> > >
> > > > > them directly in BLAS calls (e.g. gemv() etc.). In addition, is it
> > > > > possible
> > > > > to create views of e.g. DataFrames or NamedArrays ?
> > > > >
> > > > > I can only speak for NamedArrays. On the one hand the deployment
> of
> > >
> > > BLAS
> > >
> > > > should be transparant and the use of NamedArray vs Array not lead to
> > >
> > > much
> > >
> > > > degradation in performance. E.g., "a * b" with `a` and `b` a
> > >
> > > NamedArray,
> > >
> > > > effectively calls "a.array * b.array" which Base implements with
> > > > BLAS.gemm(). There is just a little overhead of filling in sensible
> > >
> > > names
> > >
> > > > in the result---so if you have small matrices in an inner loop,
> you're
> > > > going to get hurt.
> > > >
> > > > On the other hand, I am not sure how much of the Julia BLAS
> cleverness
> > >
> > > is
> > >
> > > > retained in NamedArrays---but the intention of the package is that
> it is
> > > > completely transparent, and if you notice bad performance for a
> > >
> > > particular
> > >
> > > > situation then you should file an issue (or make a PR:-). Individual
> > > > element indexing of a NamedArray with integers is just a little bit
> > >
> > > slower
> > >
> > > > than that of an Array. Indexing by name is quite a bit slower---you
> may
> > > > try a different Associative than the standard Dict.
> > > >
> > > > Incidentally, I've been toying with the idea of NamedArrays `*`
> check on
> > > > consistency of index and dimension names, but my guess is that people
> > >
> > > would
> > >
> > > > find such a thing annoying.
> > > >
> > > > ArrayViews are currently not aware of NameArrays. I believe the
> views
> > >
> > > are
> > >
> > > > going to be part ov julia-0.4, so then it would be a task for
> NamedArray
> > >
> > > to
> > >
> > > > implement views of NamedArrays I gather.
> > > >
> > > > Cheers,
> > > >
> > > > ---david
> > > >
> > > > > Thanks,
> > > > > Jan
>
>
