I'm encountering errors in Hive 2.3.2 when reading sets of Parquet files,
where the schema has evolved.

The error I'm seeing is :
Failed with exception java.io.IOException:java.lang.RuntimeException: Hive
internal error: conversion of string to array<string>not supported yet.

My schema has a top-level column of struct type: that has changed from:

myColumn struct<c1:string, c2:string, c3:string>

To

myColumn struct<c1:string, c2:string, new_column:array<string>, c3:string>

I've update my table with the new column type using the DDL below but then
see the aforementioned error when selecting the data.

I've tried to force column lookup by name rather than by index using the
setting:

parquet.column.index.access=false

But I see the same error. Are these kind of schema evolutions supported
(nested column insertion)? What are my options for resolving this issue?

Many thanks,

Patrick.

Reply via email to