Re: Using arrow for sparse data

2020-08-21 Thread Rok Mihevc
Hi Niranda, There's some examples in tests: https://github.com/apache/arrow/blob/master/cpp/src/arrow/sparse_tensor_test.cc#L187 , https://github.com/apache/arrow/blob/master/python/pyarrow/tests/test_sparse_tensor.py If you have more questions just ask. Questions are good input for documentation

Using arrow for sparse data

2020-08-20 Thread Niranda Perera
Hi, I was wondering how we could use arrow arrays for sparse matrix data representation? I felt like the arrow structure suits well for CSC (compressed sparse column) format which is used heavily in sparse matrix operations. I saw that there was a 'sparse tensor' on the arrow website. https://arr