[
https://issues.apache.org/jira/browse/SPARK-21507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maciej Bryński updated SPARK-21507:
-----------------------------------
Description:
When more than one process is using packages option it's possible to create
following exception
{code}
INFO - Ivy Default Cache set to: /home/bi/.ivy2/cache
INFO - The jars for the packages stored in: /home/bi/.ivy2/jars
INFO - :: loading settings :: url =
jar:file:/opt/spark-2.2.0/jars/ivy-2.4.0.jar!/org/apache/ivy/core/settings/ivysettings.xml
INFO - datastax#spark-cassandra-connector added as a dependency
INFO - :: resolving dependencies :: org.apache.spark#spark-submit-parent;1.0
INFO - confs: [default]
INFO - found datastax#spark-cassandra-connector;2.0.3-s_2.11 in spark-packages
INFO - found commons-beanutils#commons-beanutils;1.9.3 in central
INFO - found commons-collections#commons-collections;3.2.2 in local-m2-cache
INFO - found joda-time#joda-time;2.3 in central
INFO - found org.joda#joda-convert;1.2 in central
INFO - found com.twitter#jsr166e;1.1.0 in central
INFO - found io.netty#netty-all;4.0.33.Final in central
INFO - found org.scala-lang#scala-reflect;2.11.8 in local-m2-cache
INFO - :: resolution report :: resolve 1042ms :: artifacts dl 51ms
INFO - :: modules in use:
INFO - com.twitter#jsr166e;1.1.0 from central in [default]
INFO - commons-beanutils#commons-beanutils;1.9.3 from central in [default]
INFO - commons-collections#commons-collections;3.2.2 from local-m2-cache in
[default]
INFO - datastax#spark-cassandra-connector;2.0.3-s_2.11 from spark-packages in
[default]
INFO - io.netty#netty-all;4.0.33.Final from central in [default]
INFO - joda-time#joda-time;2.3 from central in [default]
INFO - org.joda#joda-convert;1.2 from central in [default]
INFO - org.scala-lang#scala-reflect;2.11.8 from local-m2-cache in [default]
INFO - ---------------------------------------------------------------------
INFO - | | modules || artifacts |
INFO - | conf | number| search|dwnlded|evicted|| number|dwnlded|
INFO - ---------------------------------------------------------------------
INFO - | default | 8 | 0 | 0 | 0 || 8 | 0 |
INFO - ---------------------------------------------------------------------
INFO - :: retrieving :: org.apache.spark#spark-submit-parent
INFO - confs: [default]
INFO - Exception in thread "main" java.lang.RuntimeException: problem during
retrieve of org.apache.spark#spark-submit-parent: java.text.ParseException:
failed to parse report:
/home/bi/.ivy2/cache/org.apache.spark-spark-submit-parent-default.xml:
Premature end of file.
INFO - at
org.apache.ivy.core.retrieve.RetrieveEngine.retrieve(RetrieveEngine.java:249)
INFO - at
org.apache.ivy.core.retrieve.RetrieveEngine.retrieve(RetrieveEngine.java:83)
INFO - at org.apache.ivy.Ivy.retrieve(Ivy.java:551)
INFO - at
org.apache.spark.deploy.SparkSubmitUtils$.resolveMavenCoordinates(SparkSubmit.scala:1180)
INFO - at
org.apache.spark.deploy.SparkSubmit$.prepareSubmitEnvironment(SparkSubmit.scala:298)
INFO - at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:153)
INFO - at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:119)
INFO - at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
INFO - Caused by: java.text.ParseException: failed to parse report:
/home/bi/.ivy2/cache/org.apache.spark-spark-submit-parent-default.xml:
Premature end of file.
INFO - at
org.apache.ivy.plugins.report.XmlReportParser.parse(XmlReportParser.java:293)
INFO - at
org.apache.ivy.core.retrieve.RetrieveEngine.determineArtifactsToCopy(RetrieveEngine.java:329)
INFO - at
org.apache.ivy.core.retrieve.RetrieveEngine.retrieve(RetrieveEngine.java:118)
INFO - ... 7 more
INFO - Caused by: org.xml.sax.SAXParseException; Premature end of file.
INFO - at
org.apache.xerces.util.ErrorHandlerWrapper.createSAXParseException(Unknown
Source)
INFO - at org.apache.xerces.util.ErrorHandlerWrapper.fatalError(Unknown Source)
INFO - at org.apache.xerces.impl.XMLErrorReporter.reportError(Unknown Source)
INFO - at org.apache.xerces.impl.XMLErrorReporter.reportError(Unknown Source)
INFO - at org.apache.xerces.impl.XMLErrorReporter.reportError(Unknown Source)
INFO - at org.apache.xerces.impl.XMLVersionDetector.determineDocVersion(Unknown
Source)
INFO - at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
INFO - at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
INFO - at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)
INFO - at org.apache.xerces.parsers.AbstractSAXParser.parse(Unknown Source)
INFO - at org.apache.xerces.jaxp.SAXParserImpl$JAXPSAXParser.parse(Unknown
Source)
INFO - at org.apache.xerces.jaxp.SAXParserImpl.parse(Unknown Source)
INFO - at javax.xml.parsers.SAXParser.parse(SAXParser.java:328)
INFO - at
org.apache.ivy.plugins.report.XmlReportParser$SaxXmlReportParser.parse(XmlReportParser.java:249)
INFO - at
org.apache.ivy.plugins.report.XmlReportParser.parse(XmlReportParser.java:291)
INFO - ... 9 more
{code}
was:
When more than one process is using packages option it's possible to create
exception
{code}
INFO - Ivy Default Cache set to: /home/bi/.ivy2/cache
INFO - The jars for the packages stored in: /home/bi/.ivy2/jars
INFO - :: loading settings :: url =
jar:file:/opt/spark-2.2.0/jars/ivy-2.4.0.jar!/org/apache/ivy/core/settings/ivysettings.xml
INFO - datastax#spark-cassandra-connector added as a dependency
INFO - :: resolving dependencies :: org.apache.spark#spark-submit-parent;1.0
INFO - confs: [default]
INFO - found datastax#spark-cassandra-connector;2.0.3-s_2.11 in spark-packages
INFO - found commons-beanutils#commons-beanutils;1.9.3 in central
INFO - found commons-collections#commons-collections;3.2.2 in local-m2-cache
INFO - found joda-time#joda-time;2.3 in central
INFO - found org.joda#joda-convert;1.2 in central
INFO - found com.twitter#jsr166e;1.1.0 in central
INFO - found io.netty#netty-all;4.0.33.Final in central
INFO - found org.scala-lang#scala-reflect;2.11.8 in local-m2-cache
INFO - :: resolution report :: resolve 1042ms :: artifacts dl 51ms
INFO - :: modules in use:
INFO - com.twitter#jsr166e;1.1.0 from central in [default]
INFO - commons-beanutils#commons-beanutils;1.9.3 from central in [default]
INFO - commons-collections#commons-collections;3.2.2 from local-m2-cache in
[default]
INFO - datastax#spark-cassandra-connector;2.0.3-s_2.11 from spark-packages in
[default]
INFO - io.netty#netty-all;4.0.33.Final from central in [default]
INFO - joda-time#joda-time;2.3 from central in [default]
INFO - org.joda#joda-convert;1.2 from central in [default]
INFO - org.scala-lang#scala-reflect;2.11.8 from local-m2-cache in [default]
INFO - ---------------------------------------------------------------------
INFO - | | modules || artifacts |
INFO - | conf | number| search|dwnlded|evicted|| number|dwnlded|
INFO - ---------------------------------------------------------------------
INFO - | default | 8 | 0 | 0 | 0 || 8 | 0 |
INFO - ---------------------------------------------------------------------
INFO - :: retrieving :: org.apache.spark#spark-submit-parent
INFO - confs: [default]
INFO - Exception in thread "main" java.lang.RuntimeException: problem during
retrieve of org.apache.spark#spark-submit-parent: java.text.ParseException:
failed to parse report:
/home/bi/.ivy2/cache/org.apache.spark-spark-submit-parent-default.xml:
Premature end of file.
INFO - at
org.apache.ivy.core.retrieve.RetrieveEngine.retrieve(RetrieveEngine.java:249)
INFO - at
org.apache.ivy.core.retrieve.RetrieveEngine.retrieve(RetrieveEngine.java:83)
INFO - at org.apache.ivy.Ivy.retrieve(Ivy.java:551)
INFO - at
org.apache.spark.deploy.SparkSubmitUtils$.resolveMavenCoordinates(SparkSubmit.scala:1180)
INFO - at
org.apache.spark.deploy.SparkSubmit$.prepareSubmitEnvironment(SparkSubmit.scala:298)
INFO - at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:153)
INFO - at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:119)
INFO - at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
INFO - Caused by: java.text.ParseException: failed to parse report:
/home/bi/.ivy2/cache/org.apache.spark-spark-submit-parent-default.xml:
Premature end of file.
INFO - at
org.apache.ivy.plugins.report.XmlReportParser.parse(XmlReportParser.java:293)
INFO - at
org.apache.ivy.core.retrieve.RetrieveEngine.determineArtifactsToCopy(RetrieveEngine.java:329)
INFO - at
org.apache.ivy.core.retrieve.RetrieveEngine.retrieve(RetrieveEngine.java:118)
INFO - ... 7 more
INFO - Caused by: org.xml.sax.SAXParseException; Premature end of file.
INFO - at
org.apache.xerces.util.ErrorHandlerWrapper.createSAXParseException(Unknown
Source)
INFO - at org.apache.xerces.util.ErrorHandlerWrapper.fatalError(Unknown Source)
INFO - at org.apache.xerces.impl.XMLErrorReporter.reportError(Unknown Source)
INFO - at org.apache.xerces.impl.XMLErrorReporter.reportError(Unknown Source)
INFO - at org.apache.xerces.impl.XMLErrorReporter.reportError(Unknown Source)
INFO - at org.apache.xerces.impl.XMLVersionDetector.determineDocVersion(Unknown
Source)
INFO - at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
INFO - at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
INFO - at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)
INFO - at org.apache.xerces.parsers.AbstractSAXParser.parse(Unknown Source)
INFO - at org.apache.xerces.jaxp.SAXParserImpl$JAXPSAXParser.parse(Unknown
Source)
INFO - at org.apache.xerces.jaxp.SAXParserImpl.parse(Unknown Source)
INFO - at javax.xml.parsers.SAXParser.parse(SAXParser.java:328)
INFO - at
org.apache.ivy.plugins.report.XmlReportParser$SaxXmlReportParser.parse(XmlReportParser.java:249)
INFO - at
org.apache.ivy.plugins.report.XmlReportParser.parse(XmlReportParser.java:291)
INFO - ... 9 more
{code}
> Exception when using spark.jars.packages
> -----------------------------------------
>
> Key: SPARK-21507
> URL: https://issues.apache.org/jira/browse/SPARK-21507
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
> Affects Versions: 2.2.0
> Reporter: Maciej Bryński
> Priority: Minor
>
> When more than one process is using packages option it's possible to create
> following exception
> {code}
> INFO - Ivy Default Cache set to: /home/bi/.ivy2/cache
> INFO - The jars for the packages stored in: /home/bi/.ivy2/jars
> INFO - :: loading settings :: url =
> jar:file:/opt/spark-2.2.0/jars/ivy-2.4.0.jar!/org/apache/ivy/core/settings/ivysettings.xml
> INFO - datastax#spark-cassandra-connector added as a dependency
> INFO - :: resolving dependencies :: org.apache.spark#spark-submit-parent;1.0
> INFO - confs: [default]
> INFO - found datastax#spark-cassandra-connector;2.0.3-s_2.11 in spark-packages
> INFO - found commons-beanutils#commons-beanutils;1.9.3 in central
> INFO - found commons-collections#commons-collections;3.2.2 in local-m2-cache
> INFO - found joda-time#joda-time;2.3 in central
> INFO - found org.joda#joda-convert;1.2 in central
> INFO - found com.twitter#jsr166e;1.1.0 in central
> INFO - found io.netty#netty-all;4.0.33.Final in central
> INFO - found org.scala-lang#scala-reflect;2.11.8 in local-m2-cache
> INFO - :: resolution report :: resolve 1042ms :: artifacts dl 51ms
> INFO - :: modules in use:
> INFO - com.twitter#jsr166e;1.1.0 from central in [default]
> INFO - commons-beanutils#commons-beanutils;1.9.3 from central in [default]
> INFO - commons-collections#commons-collections;3.2.2 from local-m2-cache in
> [default]
> INFO - datastax#spark-cassandra-connector;2.0.3-s_2.11 from spark-packages in
> [default]
> INFO - io.netty#netty-all;4.0.33.Final from central in [default]
> INFO - joda-time#joda-time;2.3 from central in [default]
> INFO - org.joda#joda-convert;1.2 from central in [default]
> INFO - org.scala-lang#scala-reflect;2.11.8 from local-m2-cache in [default]
> INFO - ---------------------------------------------------------------------
> INFO - | | modules || artifacts |
> INFO - | conf | number| search|dwnlded|evicted|| number|dwnlded|
> INFO - ---------------------------------------------------------------------
> INFO - | default | 8 | 0 | 0 | 0 || 8 | 0 |
> INFO - ---------------------------------------------------------------------
> INFO - :: retrieving :: org.apache.spark#spark-submit-parent
> INFO - confs: [default]
> INFO - Exception in thread "main" java.lang.RuntimeException: problem during
> retrieve of org.apache.spark#spark-submit-parent: java.text.ParseException:
> failed to parse report:
> /home/bi/.ivy2/cache/org.apache.spark-spark-submit-parent-default.xml:
> Premature end of file.
> INFO - at
> org.apache.ivy.core.retrieve.RetrieveEngine.retrieve(RetrieveEngine.java:249)
> INFO - at
> org.apache.ivy.core.retrieve.RetrieveEngine.retrieve(RetrieveEngine.java:83)
> INFO - at org.apache.ivy.Ivy.retrieve(Ivy.java:551)
> INFO - at
> org.apache.spark.deploy.SparkSubmitUtils$.resolveMavenCoordinates(SparkSubmit.scala:1180)
> INFO - at
> org.apache.spark.deploy.SparkSubmit$.prepareSubmitEnvironment(SparkSubmit.scala:298)
> INFO - at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:153)
> INFO - at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:119)
> INFO - at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
> INFO - Caused by: java.text.ParseException: failed to parse report:
> /home/bi/.ivy2/cache/org.apache.spark-spark-submit-parent-default.xml:
> Premature end of file.
> INFO - at
> org.apache.ivy.plugins.report.XmlReportParser.parse(XmlReportParser.java:293)
> INFO - at
> org.apache.ivy.core.retrieve.RetrieveEngine.determineArtifactsToCopy(RetrieveEngine.java:329)
> INFO - at
> org.apache.ivy.core.retrieve.RetrieveEngine.retrieve(RetrieveEngine.java:118)
> INFO - ... 7 more
> INFO - Caused by: org.xml.sax.SAXParseException; Premature end of file.
> INFO - at
> org.apache.xerces.util.ErrorHandlerWrapper.createSAXParseException(Unknown
> Source)
> INFO - at org.apache.xerces.util.ErrorHandlerWrapper.fatalError(Unknown
> Source)
> INFO - at org.apache.xerces.impl.XMLErrorReporter.reportError(Unknown Source)
> INFO - at org.apache.xerces.impl.XMLErrorReporter.reportError(Unknown Source)
> INFO - at org.apache.xerces.impl.XMLErrorReporter.reportError(Unknown Source)
> INFO - at
> org.apache.xerces.impl.XMLVersionDetector.determineDocVersion(Unknown Source)
> INFO - at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
> INFO - at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
> INFO - at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)
> INFO - at org.apache.xerces.parsers.AbstractSAXParser.parse(Unknown Source)
> INFO - at org.apache.xerces.jaxp.SAXParserImpl$JAXPSAXParser.parse(Unknown
> Source)
> INFO - at org.apache.xerces.jaxp.SAXParserImpl.parse(Unknown Source)
> INFO - at javax.xml.parsers.SAXParser.parse(SAXParser.java:328)
> INFO - at
> org.apache.ivy.plugins.report.XmlReportParser$SaxXmlReportParser.parse(XmlReportParser.java:249)
> INFO - at
> org.apache.ivy.plugins.report.XmlReportParser.parse(XmlReportParser.java:291)
> INFO - ... 9 more
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
