Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/8054#discussion_r38480468
--- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala ---
@@ -2017,18 +2011,23 @@ private[spark] object Utils extends Logging {
conf: SparkConf,
serviceName: String = ""): (T, Int) = {
+ val maxPort = 65536
+ val failedPorts = new ArrayBuffer[Int]()
require(startPort == 0 || (1024 <= startPort && startPort < 65536),
"startPort should be between 1024 and 65535 (inclusive), or 0 for a
random free port.")
-
val serviceString = if (serviceName.isEmpty) "" else s" '$serviceName'"
val maxRetries = portMaxRetries(conf)
for (offset <- 0 to maxRetries) {
// Do not increment port if startPort is 0, which is treated as a
special port
val tryPort = if (startPort == 0) {
startPort
- } else {
- // If the new port wraps around, do not try a privilege port
- ((startPort + offset - 1024) % (65536 - 1024)) + 1024
+ }
+ else {
+ // If the new port wraps around, do not try a privilege port or a
failedPort
+ while(!failedPorts.contains((startPort + Math.random() * (maxPort
- startPort + 1))) &&
--- End diff --
This is an array buffer, so contains is linear time, which is pretty
expensive. Also this code is very difficult to understand.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
