hi, i read this post recommending to use twitter4j-3.0.3* and it is not
working for me. I want to load this jars within spark-shell without any
lucky. This is the output
1. *MacBook-Pro-Retina-de-Alonso:spark-1.6 aironman$ ls *jar*
2. mysql-connector-java-5.1.30.jar twitter4j-async-3.0.3.jar
twitter4j-examples-3.0.3.jar twitter4j-stream-3.0.3.jar
3. spark-streaming-twitter_2.10-1.0.0.jar twitter4j-core-3.0.3.jar
twitter4j-media-support-3.0.3.jar
4. *MacBook-Pro-Retina-de-Alonso:spark-1.6 aironman$* bin/spark-shell
--jars spark-streaming-twitter_2.10-1.0.0.jar twitter4j-async-3.0.3.jar
twitter4j-core-3.0.3.jar twitter4j-examples-3.0.3.jar
twitter4j-media-support-3.0.3.jar twitter4j-stream-3.0.3.jar
5. log4j:WARN No appenders could be found for logger
(org.apache.hadoop.metrics2.lib.MutableMetricsFactory).
6. log4j:WARN Please initialize the log4j system properly.
7. log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig
for more info.
8. 2016-03-01 18:00:33.542 java[35711:11354206] Unable to load realm
info from SCDynamicStore
9. Using Spark's repl log4j profile:
org/apache/spark/log4j-defaults-repl.properties
10. To adjust logging level use sc.setLogLevel("INFO")
11. error: error while loading <root>, error in opening zip file
12.
13. Failed to initialize compiler: object scala.runtime in compiler
mirror not found.
14. ** Note that as of 2.8 scala does not assume use of the java
classpath.
15. ** For the old behavior pass -usejavacp to scala, or if using a
Settings
16. ** object programatically, settings.usejavacp.value = true.
17. 16/03/01 18:00:34 WARN SparkILoop$SparkILoopInterpreter: Warning:
compiler accessed before init set up. Assuming no postInit code.
18.
19. Failed to initialize compiler: object scala.runtime in compiler
mirror not found.
20. ** Note that as of 2.8 scala does not assume use of the java
classpath.
21. ** For the old behavior pass -usejavacp to scala, or if using a
Settings
22. ** object programatically, settings.usejavacp.value = true.
23. Exception in thread "main" java.lang.AssertionError: assertion
failed: null
24. at scala.Predef$.assert(Predef.scala:179)
25. at
org.apache.spark.repl.SparkIMain.initializeSynchronous(SparkIMain.scala:247)
26. at
org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1.apply$mcZ$sp(SparkILoop.scala:990)
27. at
org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1.apply(SparkILoop.scala:945)
28. at
org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1.apply(SparkILoop.scala:945)
29. at
scala.tools.nsc.util.ScalaClassLoader$.savingContextLoader(ScalaClassLoader.scala:135)
30. at org.apache.spark.repl.SparkILoop.org
$apache$spark$repl$SparkILoop$$process(SparkILoop.scala:945)
31. at
org.apache.spark.repl.SparkILoop.process(SparkILoop.scala:1059)
32. at org.apache.spark.repl.Main$.main(Main.scala:31)
33. at org.apache.spark.repl.Main.main(Main.scala)
34. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
35. at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
36. at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
37. at java.lang.reflect.Method.invoke(Method.java:606)
38. at
org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:731)
39. at
org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:181)
40. at
org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:206)
41. at
org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:121)
42. at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
43. MacBook-Pro-Retina-de-Alonso:spark-1.6 aironman$
Do i have to update the jars version? because i see that actual version of
twitter4j is 4.0.4...
thanks
Alonso Isidoro Roman.
Mis citas preferidas (de hoy) :
"Si depurar es el proceso de quitar los errores de software, entonces
programar debe ser el proceso de introducirlos..."
- Edsger Dijkstra
My favorite quotes (today):
"If debugging is the process of removing software bugs, then programming
must be the process of putting ..."
- Edsger Dijkstra
"If you pay peanuts you get monkeys"
