[
https://issues.apache.org/jira/browse/SPARK-19059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15796213#comment-15796213
]
Remon K commented on SPARK-19059:
---------------------------------
I was not able to reproduce it
{code}
spark-2.1.0-bin-hadoop2.7$ bin/spark-shell
Setting default log level to "WARN".
To adjust logging level use sc.setLogLevel(newLevel).
17/01/03 13:10:45 WARN NativeCodeLoader: Unable to load native-hadoop library
for your platform... using builtin-java classes where applicable
17/01/03 13:10:46 WARN SparkContext: Use an existing SparkContext, some
configuration may not take effect.
Spark context Web UI available at http://localhost:4040
Spark context available as 'sc' (master = local[*], app id =
local-1483477846511).
Spark session available as 'spark'.
Welcome to
____ __
/ __/__ ___ _____/ /__
_\ \/ _ \/ _ `/ __/ '_/
/___/ .__/\_,_/_/ /_/\_\ version 2.0.0
/_/
Using Scala version 2.11.8 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_66)
Type in expressions to have them evaluated.
Type :help for more information.
scala> spark.sql("CREATE TABLE `_a`(i INT) USING parquet").show
17/01/03 13:11:20 WARN ObjectStore: Version information not found in metastore.
hive.metastore.schema.verification is not enabled so recording the schema
version 1.2.0
17/01/03 13:11:21 WARN ObjectStore: Failed to get database default, returning
NoSuchObjectException
17/01/03 13:11:23 WARN HiveMetaStore: Location:
file:/$HOME/Downloads/spark-2.1.0-bin-hadoop2.7/spark-warehouse/_a specified
for non-external table:_a
++
||
++
++
scala> spark.sql("INSERT INTO `_a` VALUES (1), (2), (3)").show
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further
details.
++
||
++
++
scala> spark.sql("SELECT * FROM `_a`").show
+---+
| i|
+---+
| 1|
| 2|
| 3|
+—+
{code}
> Unable to retrieve data from a parquet table whose name starts with underscore
> ------------------------------------------------------------------------------
>
> Key: SPARK-19059
> URL: https://issues.apache.org/jira/browse/SPARK-19059
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
> Affects Versions: 2.1.0
> Reporter: Giambattista
> Fix For: 2.1.1
>
>
> It looks like there is some bug introduced in Spark 2.1.0 preventing to read
> data from a parquet table (hive support is enabled) whose name starts with
> underscore. CREATE and INSERT statements on the same table instead seems to
> work as expected.
> The problem can be reproduced from spark-shell through the following steps:
> 1) Create a table with some values
> scala> spark.sql("CREATE TABLE `_a`(i INT) USING parquet").show
> scala> spark.sql("INSERT INTO `_a` VALUES (1), (2), (3)").show
> 2) Select data from the just created and filled table --> no results
> scala> spark.sql("SELECT * FROM `_a`").show
> +---+
> | i|
> +---+
> +---+
> 3) rename the table so that the prefixing underscore disappears
> scala> spark.sql("ALTER TABLE `_a` RENAME TO `a`").show
> 4) select data from the just renamed table --> results are shown
> scala> spark.sql("SELECT * FROM `a`").show
> +---+
> | i|
> +---+
> | 1|
> | 2|
> | 3|
> +---+
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
