Hey everyone, Just wanted to put out the announcement of the release of DataStreams v0.1. (it was actually tagged a few weeks ago, but I've been letting a few last things shake out before announcing).
I've written up a blog post on the updates and release here: http://quinnj.github.io/datastreams-jl-v0-1/ The TL;DR is DataStreams.jl now defines concrete interfaces for Data.Sources and Data.Sinks, with each being completely decoupled from the other. This has also allowed some cool new features like appending to Data.Sinks and allowing simple transform functions to be applied to data "in-transit". I included release notes of existing packages in the blog post, but I'll copy-paste here below for easier access: Do note that the DataStreams.jl framework is now Julia 0.5-only. - *CSV.jl* - *Docs* <http://juliadata.github.io/CSV.jl/stable/> - Supports a wide variety of delimited file options such as delim, quotechar, escapechar, custom null strings; a header can be provided manually or on a specified row or range of rows; types can be provided manually, and results can be requested as nullable or not ( nullable=true by default); and the # of rows can be provided manually (if known) for efficiency. - CSV.parsefield(io::IO, ::Type{T}) can be called directly on any IOtype to tap into the delimited-parsing functionality manually - *SQLite.jl* - *Docs* <http://juliadb.github.io/SQLite.jl/stable/> - Query results will now use the declared table column type by default, which can help resultset column typing in some cases - Parameterized SQL statements are fully supported, with the ability to bind julia values to be sent to the DB - Full serialization/deserialization of native and custom Julia types is supported; so Complex{Int128} can be stored in its own SQLite table column and retrieved without any issue - Pure Julia scalar and aggregation functions can be registered with an SQLite database and then called from within SQL statements: full docs here <http://juliadb.github.io/SQLite.jl/stable/#User-Defined-Functions-1> - *Feather.jl* - *Docs* <http://juliastats.github.io/Feather.jl/stable/> - Full support for feather release v0.3.0 to ensure compatibility - Full support for returning "factor" or "category" type columns as native CategoricalArray and NullableCategoricalArray types in Julia, thanks to the new CategoricalArrays.jl <https://github.com/JuliaData/CategoricalArrays.jl> package - nullable::Bool=true keyword argument; if false, columns without null values will be returned as Vector{T} instead of NullableVector{T} - Feather.Sink now supports appending, so multiple DataFrames or CSV.Source or any Data.Source can all be streamed to a single feather file - *ODBC.jl* - *Docs* <http://juliadb.github.io/ODBC.jl/stable/> - A new ODBC.DSN type that represents a valid, open connection to a database; used in all subsequent api calls; it can be constructed using a previously configured system/user dsn w/ username and password, or as a full custom connection string - Full support for the DataStreams.jl framework through the ODBC.Sourceand ODBC.Sink types, along with their high-level convenience methods ODBC.query and ODBC.load - A new ODBC.prepare(dsn, sql) => ODBC.Statement method which can send an sql statement to the database to be compiled and planned before executed 1 or more times. SQL statements can include parameters to be prepared that can have dynamic values bound before each execution.
