At least with the non-1 arrays I know of now, you can always get the OneTo
array form with parent(A). However, that's not something I feel comfortable
saying will be guaranteed forever.
In terms of the "automatic conversion" part of your proposal, I'm a bit
uncomfortable because in my view the indices mean something: how should
copy!(A, B)
work if B's indices are not contained within A's indices? To me the indices
*are* the identity of a location in the array, so making it too easy to rename
things (esp. automatically) seems like a formula for serious bugs: if I wanted
to copy my data to indices -5:5, it's a bug if it gets stuck into the slots
1:11 instead because of some silent index-renaming.
If I understand correctly, regarding complexity, your biggest concern is about
the range types "affiliated" with a non-1 array type. First, those are
optional,
in the sense that many algorithms can be written without them. What makes them
seemingly necessary in some circumstances is `similar` and the fact that Julia
(unlike Fortran) has a type system with significant performance implications.
For example, suppose you have defined a "sparse" AbstractArray type with non-1
indices, and a matching dense array type. Let's say you pass in one of these
sparse arrays S to some *generic* algorithm (perhaps one that someone else
wrote, without your use case in mind) which needs to allocate some kind of
dense output. Then what you need to say is "allocate me an array similar to S,
but make it dense, with the same indices as S." `similar(S)` might not work,
because that might create a sparse output; nor would `Array{eltype(S)}
(size(S))` because `S` might have non-1 indices and `Array` doesn't support
that.
So the adopted solution is to allow you to say
similar(dims->Array{eltype(S)}(dims), indices(S))
and your package needs to specialize this method to do what you want it to do.
Notice here that you have only one mechanism to control *which* type of array
gets created: the type returned by `indices(S)`. If it's a OneTo-tuple this
will just create an Array, but if it's something else then something else will
be created. The author of a non-1 array package gets to customize this if s/he
takes advantage of the opportunity to define a new index type; alternatively,
you could just make sure that the `OffsetArrays` package has been loaded, and
then use UnitRange for the return value of `indices`, and it will create an
`OffsetArray` for you. If you're happy with that, you don't have to define a
custom indices type.
Now, we could get around this by declaring "one true non-1 array type," but
this might be constraining. For example, https://github.com/JuliaArrays/
CatIndices.jl defines a BidirectionalVector array type that causes `shift!` and
`unshift!` to adjust the indices of the first element, complementary to how
`push!` and `pop!` adjust the indices of the last element; I don't think this
is something you could assume was desired in general, but it's very useful in
particular contexts.
I'm all in favor of simplifying this where possible, and would love ideas for
how to do it. But I'm not sure that Fortran is a perfect model, because the
flexibility of Julia's type system allows useful behaviors that make Fortran
look so...static.
Best,
--Tim
On Thursday, October 27, 2016 5:21:26 PM CDT Bob Portmann wrote:
> TL;DR I think the type system should take care of converting between
> different types of arrays and thus free the user to code against the type
> of array they want (OneTo, ZeroTo, Offset, etc).
>
> Maybe I have a deep mis-understanding of how the new generalized offset
> arrays are suppose to work in Julia. If so I'd be happy to be set straight.
> If not then I think the present system will lead to unnecessary complexity
> and bug-ridden code. In the present system one can define array types that
> are not one-based but can have arbitrary offsets. This is a great idea but
> to make it work a very general system for indexing relative to the ends of
> the array has been devised. If one writes a function that accepts an
> `AbstractArray` then one MUST use this more general system or produce bugs
> when e.g. a `ZeroToArray` is passed in. This puts a large burden on writers
> of library code that works on arrays and I think is an unnecessary
> complexity.
>
> Since Fortran arrays are a nice example let look at the situation in
> Fortran. If I define an array in a function `real arr(-10:10)` then I would
> index it using indices that range from -10 to 10. If I pass this into
> another function that declared the array using its total size (usually
> passed in separately or in a parameter statement) `real arr(21)` then in
> that subroutine one would index the array using "normal" indices that range
> from 1 to 21. I.E., you state in the function how you want to treat the
> array and are not forced to work with in offset form unless you want to. In
> my opinion, this is what makes using offset arrays in Fortran sane and is
> the key thing that Julia should try to emulate.
>
> One way to get this behavior in Julia would be to use the type system and
> `convert`. Since `AbstractArray` has meant until 0.5 a `OneTo` array I
> think it is wise that it remain that way so that all old code will work
> unchanged (even with the new array types). Thus, I propose adding a new
> top-level type above AbstractArray type to capture all arrays something
> like:
>
> ```
> Array{T,N} <: DenseArray{T,N} <: AbstractArray{T,N} <:
> AbstractAllArray{T,N} <: Any
> ```
>
> And the offset arrays would be subtypes of `AbstractAllArrays` on a
> different branch
>
> ```
> OffsetArray{T,N} <: AbstractOffsetArray{T,N} <: AbstractAllArray{T,N} <: Any
> ```
>
> And similarly for `ZeroToArray`.
>
> Then, if one declares an Array as
>
> ```
> function func(arr::AbstractArray)
> ```
>
> one can safely assume it is a 1-based Array inside `func`. If an offset
> array is passed into `func` then it is automatically converted to a `OneTo`
> array inside `func`. This is the key point of this proposal and is similar
> to the auto-conversion of, e.g., Int to Floats in a function call.
> Similarly if one declares an array as a `ZeroToArray` in a function and
> passes in an `Array` then it will be converted to a `ZeroToArray` in the
> function. Some conversions would need more information and thus would be
> disallowed (e.g., `Array` to `OffsetArray`). I think this system would go a
> long way towards making `ZeroToArray`s and `OffsetArray`s simple to use in
> Julia w/o using the generalized indexing techniques introduced in Julia
> 0.5. Note that it would still be possible to use the `AbstractAllArray`
> type and accept all arrays w/o conversion and use the generalized indexing
> techniques.
>
> I'm curious what people think about this.
>
> Bob
>
> ps Paste into github to see in formatted form. I'm not sure how to get that
> in an email.
