Hello,
when you said your pandas Dataframe has 10 rows, does that mean it contains
10 images? Because if that's the case, then you'd want ro only use 3 layers
of ArrayType when you define the schema.
Best regards,
Adrian
On Thu, Jul 27, 2023, 11:04 second_co...@yahoo.com.INVALID
<second_co...@yahoo.com.invalid> wrote:
> i have panda dataframe with column 'image' using numpy.ndarray. shape is (500,
> 333, 3) per image. my panda dataframe has 10 rows, thus, shape is (10,
> 500, 333, 3)
>
> when using spark.createDataframe(panda_dataframe, schema), i need to
> specify the schema,
>
> schema = StructType([
> StructField("image",
> ArrayType(ArrayType(ArrayType(ArrayType(IntegerType())))), nullable=False)
> ])
>
>
> i get error
>
> raise TypeError(
> , TypeError: field image:
> ArrayType(ArrayType(ArrayType(ArrayType(IntegerType(), True), True), True),
> True) can not accept object array([[[14, 14, 14],
>
> ...
>
> Can advise how to set schema for image with numpy.ndarray ?
>
>
>
>
