Re: [discuss] dropping Python 2.6 support
Sasha, it is more complicated than that: many RHEL 6 OS utilities rely on Python 2.6. Upgrading it to 2.7 breaks the system. For large enterprises migrating to another server OS means re-certifying (re-testing) hundreds of applications, so yes, they do prefer to stay where they are until the benefits of migrating outweigh the overhead. Long story short: you cannot simply upgrade built-in Python 2.6 in RHEL 6 and it will take years for enterprises to migrate to RHEL 7. Having said that, I don't think that it is a problem though, because Python 2.6 and Python 2.7 can easily co-exist in the same environment. For example, we use virtualenv to run Spark with Python 2.7 and do not touch system Python 2.6. Thank you, Dmitry 09.01.2016, 06:36, "Sasha Kacanski" : > +1 > Companies that use stock python in redhat 2.6 will need to upgrade or install > fresh version wich is total of 3.5 minutes so no issues ... > > On Tue, Jan 5, 2016 at 2:17 AM, Reynold Xin wrote: >> Does anybody here care about us dropping support for Python 2.6 in Spark 2.0? >> >> Python 2.6 is ancient, and is pretty slow in many aspects (e.g. json >> parsing) when compared with Python 2.7. Some libraries that Spark depend on >> stopped supporting 2.6. We can still convince the library maintainers to >> support 2.6, but it will be extra work. I'm curious if anybody still uses >> Python 2.6 to run Spark. >> >> Thanks. > > -- > Aleksandar Kacanski - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org
BUILD FAILURE...again?! :( Spark Project External Flume on fire
Hi,
It appears that the last commit [1] broke the build. Is anyone working
on it? I can when told so.
➜ spark git:(master) ✗ ./build/mvn -Pyarn -Phadoop-2.6
-Dhadoop.version=2.7.1 -Dscala-2.11 -Phive -Phive-thriftserver
-DskipTests clean install
...
[info] Compiling 8 Scala sources and 1 Java source to
/Users/jacek/dev/oss/spark/external/flume/target/scala-2.11/classes...
[error]
/Users/jacek/dev/oss/spark/external/flume/src/main/scala/org/apache/spark/streaming/flume/FlumeInputDStream.scala:33:
object jboss is not a member of package org
[error] import org.jboss.netty.handler.codec.compression._
[error]^
[error]
/Users/jacek/dev/oss/spark/external/flume/src/main/scala/org/apache/spark/streaming/flume/FlumeInputDStream.scala:31:
object jboss is not a member of package org
[error] import org.jboss.netty.channel.{ChannelPipeline,
ChannelPipelineFactory, Channels}
[error]^
[error]
/Users/jacek/dev/oss/spark/external/flume/src/main/scala/org/apache/spark/streaming/flume/FlumeInputDStream.scala:32:
object jboss is not a member of package org
[error] import org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory
[error]^
[warn] Class org.jboss.netty.channel.ChannelFactory not found -
continuing with a stub.
[warn] Class org.jboss.netty.channel.ChannelFactory not found -
continuing with a stub.
[warn] Class org.jboss.netty.channel.ChannelPipelineFactory not found
- continuing with a stub.
[warn] Class org.jboss.netty.handler.execution.ExecutionHandler not
found - continuing with a stub.
[warn] Class org.jboss.netty.channel.ChannelFactory not found -
continuing with a stub.
[warn] Class org.jboss.netty.handler.execution.ExecutionHandler not
found - continuing with a stub.
[warn] Class org.jboss.netty.channel.group.ChannelGroup not found -
continuing with a stub.
[error]
/Users/jacek/dev/oss/spark/external/flume/src/main/scala/org/apache/spark/streaming/flume/FlumeInputDStream.scala:149:
not found: type NioServerSocketChannelFactory
[error] val channelFactory = new
NioServerSocketChannelFactory(Executors.newCachedThreadPool(),
[error]^
[error]
/Users/jacek/dev/oss/spark/external/flume/src/main/scala/org/apache/spark/streaming/flume/FlumeInputDStream.scala:196:
not found: type ChannelPipelineFactory
[error] class CompressionChannelPipelineFactory extends
ChannelPipelineFactory {
[error] ^
[error] Class org.jboss.netty.channel.ChannelFactory not found -
continuing with a stub.
[error] Class org.jboss.netty.channel.ChannelPipelineFactory not found
- continuing with a stub.
[error] Class org.jboss.netty.handler.execution.ExecutionHandler not
found - continuing with a stub.
[error]
/Users/jacek/dev/oss/spark/external/flume/src/main/scala/org/apache/spark/streaming/flume/FlumeInputDStream.scala:197:
not found: type ChannelPipeline
[error] def getPipeline(): ChannelPipeline = {
[error]^
[error]
/Users/jacek/dev/oss/spark/external/flume/src/main/scala/org/apache/spark/streaming/flume/FlumeInputDStream.scala:198:
not found: value Channels
[error] val pipeline = Channels.pipeline()
[error] ^
[error]
/Users/jacek/dev/oss/spark/external/flume/src/main/scala/org/apache/spark/streaming/flume/FlumeInputDStream.scala:199:
not found: type ZlibEncoder
[error] val encoder = new ZlibEncoder(6)
[error] ^
[error]
/Users/jacek/dev/oss/spark/external/flume/src/main/scala/org/apache/spark/streaming/flume/FlumePollingInputDStream.scala:29:
object jboss is not a member of package org
[error] import org.jboss.netty.channel.socket.nio.NioClientSocketChannelFactory
[error]^
[error]
/Users/jacek/dev/oss/spark/external/flume/src/main/scala/org/apache/spark/streaming/flume/FlumePollingInputDStream.scala:73:
not found: type NioClientSocketChannelFactory
[error] new NioClientSocketChannelFactory(channelFactoryExecutor,
channelFactoryExecutor)
[error] ^
[warn] Class org.jboss.netty.channel.ChannelFuture not found -
continuing with a stub.
[warn] Class org.jboss.netty.channel.ChannelFactory not found -
continuing with a stub.
[warn] Class org.jboss.netty.channel.ChannelFactory not found -
continuing with a stub.
[warn] Class org.jboss.netty.channel.ChannelFactory not found -
continuing with a stub.
[warn] Class org.jboss.netty.channel.ChannelFactory not found -
continuing with a stub.
[warn] Class org.jboss.netty.channel.ChannelUpstreamHandler not found
- continuing with a stub.
[error] Class org.jboss.netty.channel.ChannelFactory not found -
continuing with a stub.
[error]
/Users/jacek/dev/oss/spark/external/flume/src/main/scala/org/apache/spark/streaming/flume/FlumeTestUtils.scala:33:
object jboss is not a member of package org
[error] import org.jboss.netty.channel.ChannelPipeline
[error]^
[error]
/Users/jacek/dev/oss/spark/external/flume/src/main/scala/org/apache/spark/streaming/fl
