> On Jan. 16, 2015, 10:35 p.m., Xuefu Zhang wrote: > > spark-client/src/main/java/org/apache/hive/spark/client/JobHandleImpl.java, > > line 179 > > <https://reviews.apache.org/r/29954/diff/1-2/?file=823286#file823286line179> > > > > Here sparkJobIds.add() is in the synchronized block. However, we have > > code accessing the same variable (sparkJobIds) such as in > > RemoteSparkJobStatus class. Does that also needs protection? > > Marcelo Vanzin wrote: > No, we don't. The job id list itself is thread-safe. The synchronization > happens here so that we notify all listeners of everything. We don't want a > listener being registered concurrently with a new spark job arriving to miss > that event. > > (That reminds me that I probably should switch the order of events around > if a listener is added after the handle is in a final state. Stay tuned.) > > Xuefu Zhang wrote: > In that case, can we move sparkJobIds.add() outside the sync block?
I don't think that works well. That can cause two different conditions depending on where "outside" means: - if you do it before the synchronized block, the listener may be notified twice of the same Spark job - if you do it after the synchronized block, the listener will be called with a Spark job that is not yet listed in `handle.getSparkJobIds()`. Since I don't belived this will cause any performance issue at all, I'd rather keep the behavior consistent. - Marcelo ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/29954/#review68492 ----------------------------------------------------------- On Jan. 16, 2015, 9:22 p.m., Marcelo Vanzin wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/29954/ > ----------------------------------------------------------- > > (Updated Jan. 16, 2015, 9:22 p.m.) > > > Review request for hive, Brock Noland, chengxiang li, and Xuefu Zhang. > > > Bugs: HIVE-9179 > https://issues.apache.org/jira/browse/HIVE-9179 > > > Repository: hive-git > > > Description > ------- > > HIVE-9179. Add listener API to JobHandle. > > > Diffs > ----- > > spark-client/pom.xml 77016df61a0bcbd94058bcbd2825c6c210a70e14 > spark-client/src/main/java/org/apache/hive/spark/client/BaseProtocol.java > f9c10b196ab47b5b4f4c0126ad455869ab68f0ca > spark-client/src/main/java/org/apache/hive/spark/client/JobHandle.java > e760ce35d92bedf4d301b08ec57d1c2dc37a39f0 > spark-client/src/main/java/org/apache/hive/spark/client/JobHandleImpl.java > 1b8feedb0b23aa7897dc6ac37ea5c0209e71d573 > spark-client/src/main/java/org/apache/hive/spark/client/RemoteDriver.java > 0d49ed3d9e33ca08d6a7526c1c434a0dd0a06a67 > > spark-client/src/main/java/org/apache/hive/spark/client/SparkClientImpl.java > a30d8cbbaae9d25b1cffdc286b546f549e439545 > spark-client/src/test/java/org/apache/hive/spark/client/TestJobHandle.java > PRE-CREATION > > spark-client/src/test/java/org/apache/hive/spark/client/TestSparkClient.java > 795d62c776cec5e9da2a24b7d40bc749a03186ab > > Diff: https://reviews.apache.org/r/29954/diff/ > > > Testing > ------- > > > Thanks, > > Marcelo Vanzin > >
