If you're using a third-party parquet encoding library, you'll need to pass null/not-null using whatever convention that library requires.
I took a quick look at xitongsys/parquet-go's front-page documentation, and it seems it handles null (OPTIONAL) using pointers: https://github.com/xitongsys/parquet-go#repetition-type That is, an int32 which maybe null is passed as *int32, with a nil pointer for null. On Sunday, 22 January 2023 at 22:22:10 UTC Nick White wrote: > On Sun, Jan 22, 2023 at 08:56:12PM +0000, Rory Campbell-Lange wrote: > > I have implemented a fairly efficient line-wise dump file reader which > includes a rudimentary table typer. However I'm not sure how to represent > nulls in my go code so that I can pass these onto parquet encoders. > > > > Is a struct something like the following a good idea for each type? > > > > type pgInt64 struct { > > val int64 > > isNull bool > > } > > I'm not experienced with Parquet at all, but that struct plan is > almost exactly the same as the format go's database/sql package > reads into: > > $ go doc sql NullInt64 > package sql // import "database/sql" > > type NullInt64 struct { > Int64 int64 > Valid bool // Valid is true if Int64 is not NULL > } > > ... > > So yes, it seems like a reasonable way to represent data that may > contain nulls. Whether this is the easiest way to pass to parquet, I > don't know, but I'd imagine it shouldn't be too bad. > > Nick > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/1c7052ce-9667-43d7-a2a6-529c1f17783bn%40googlegroups.com.
