Hi Patrick, Thank you very much for your response. I am almost there, but am not sure about my conclusion. Let me try to approach it from a different angle.
I would like to time the impact of a particular lambda function, or if possible, more broadly measure the the impact of any map function. I would like to do this by inserting timers into the code. What are my options for this? I thought of two: - write an actual function instead of the lamdba, and start and end this function with a timer, writing it's value into an accumulator. - in RDD.scala, insert a start and stop timer into compute(). The difference being that in the first option, I will measure more closely the effects of the computation (but not the computation of my lambda, I would measure the computation of a function equal to the lambda), whereas in the second I would measure my lambda function but also measure more overhead. Are there any more options? Did I miss any pro's or con's? Thanks again, Tom -- View this message in context: http://apache-spark-developers-list.1001551.n3.nabble.com/What-is-the-location-in-the-source-code-of-the-computation-of-the-elements-in-a-map-transformation-tp11971p12305.html Sent from the Apache Spark Developers List mailing list archive at Nabble.com. --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org
