Re: [ANNOUNCE] New committer: Bharathkrishna Guruvayoor Murali

2018-12-02 Thread Sahil Takiar
uruvayoor Murali to become a committer, and we are pleased to announce > > that > > he has accepted. > > > > Bharath, welcome, thank you for your contributions, and we look forward > > your > > further interactions with the community! > > > > Ashutosh

Re: Review Request 69107: HIVE-20512

2018-11-05 Thread Sahil Takiar
/SparkRecordHandler.java Lines 135 (patched) <https://reviews.apache.org/r/69107/#comment294997> Should call `Thread.currentThread().interrupt();` after this - see https://docs.oracle.com/javase/7/docs/api/java/util/concurrent/ExecutorService.html - Sahil Takiar On Oct. 31, 2018, 11:

[jira] [Created] (HIVE-20828) Upgrade to Spark 2.4.0

2018-10-29 Thread Sahil Takiar (JIRA)
Sahil Takiar created HIVE-20828: --- Summary: Upgrade to Spark 2.4.0 Key: HIVE-20828 URL: https://issues.apache.org/jira/browse/HIVE-20828 Project: Hive Issue Type: Improvement

[jira] [Created] (HIVE-20809) Parse Spark error blacklist errors

2018-10-25 Thread Sahil Takiar (JIRA)
Sahil Takiar created HIVE-20809: --- Summary: Parse Spark error blacklist errors Key: HIVE-20809 URL: https://issues.apache.org/jira/browse/HIVE-20809 Project: Hive Issue Type: Sub-task

Re: Review Request 69107: HIVE-20512

2018-10-25 Thread Sahil Takiar
tor.DiscardPolicy` - Sahil Takiar On Oct. 24, 2018, 8:55 p.m., Bharathkrishna Guruvayoor Murali wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https

Re: Review Request 69107: HIVE-20512

2018-10-25 Thread Sahil Takiar
> On Oct. 23, 2018, 7:50 p.m., Sahil Takiar wrote: > > ql/src/java/org/apache/hadoop/hive/ql/exec/spark/SparkRecordHandler.java > > Line 49 (original), 52 (patched) > > <https://reviews.apache.org/r/69107/diff/1/?file=2101701#file2101701line52> > > > >

Re: Review Request 69107: HIVE-20512

2018-10-25 Thread Sahil Takiar
-mail. To reply, visit: > https://reviews.apache.org/r/69107/ > --- > > (Updated Oct. 24, 2018, 8:55 p.m.) > > > Review request for hive, Antal Sinkovits, Sahil Takiar, and Vihang > Karajgaonkar. > > >

[jira] [Created] (HIVE-20790) SparkSession should be able to close a session while it is being opened

2018-10-23 Thread Sahil Takiar (JIRA)
Sahil Takiar created HIVE-20790: --- Summary: SparkSession should be able to close a session while it is being opened Key: HIVE-20790 URL: https://issues.apache.org/jira/browse/HIVE-20790 Project: Hive

Re: Review Request 69107: HIVE-20512

2018-10-23 Thread Sahil Takiar
its own method? ql/src/java/org/apache/hadoop/hive/ql/exec/spark/SparkReduceRecordHandler.java Line 601 (original), 597 (patched) <https://reviews.apache.org/r/69107/#comment294577> move to top of method - Sahil Takiar On Oct.

Re: Review Request 69022: HIVE-20737: Local SparkContext is shared between user sessions and should be closed only when there is no active

2018-10-18 Thread Sahil Takiar
> On Oct. 16, 2018, 1:47 p.m., Sahil Takiar wrote: > > ql/src/java/org/apache/hadoop/hive/ql/exec/spark/LocalHiveSparkClient.java > > Line 73 (original), 73 (patched) > > <https://reviews.apache.org/r/69022/diff/2/?file=2097990#file2097990line75> > > > >

Re: Review Request 68474: HIVE-20440

2018-10-16 Thread Sahil Takiar
they could at the granularity of a `MapJoinOperator`? For example, confirm that starting a new query actually evicts everything from the cache? We want to make sure we aren't accidentally leaking small tables. - Sahil Takiar On Oct. 10, 2018, 1:20 p.m., Antal Sinko

Re: Review Request 68474: HIVE-20440

2018-10-16 Thread Sahil Takiar
`String`. I don't think the interface needs to be tied to reading data to a folder on HDFS. ql/src/java/org/apache/hadoop/hive/ql/exec/spark/SmallTableCache.java Lines 131 (patched) <https://reviews.apache.org/r/68474/#comment294165> why do we run the action just for the l2 cache?

Re: Review Request 69022: HIVE-20737: Local SparkContext is shared between user sessions and should be closed only when there is no active

2018-10-16 Thread Sahil Takiar
> On Oct. 16, 2018, 1:47 p.m., Sahil Takiar wrote: > > ql/src/java/org/apache/hadoop/hive/ql/exec/spark/LocalHiveSparkClient.java > > Line 73 (original), 73 (patched) > > <https://reviews.apache.org/r/69022/diff/2/?file=2097990#file2097990line75> > > > >

Re: Review Request 69022: HIVE-20737: Local SparkContext is shared between user sessions and should be closed only when there is no active

2018-10-16 Thread Sahil Takiar
> On Oct. 16, 2018, 1:47 p.m., Sahil Takiar wrote: > > ql/src/java/org/apache/hadoop/hive/ql/exec/spark/session/SparkSessionImpl.java > > Line 125 (original), 120 (patched) > > <https://reviews.apache.org/r/69022/diff/2/?file=2097991#file2097991line125> > > >

Re: Review Request 69022: HIVE-20737: Local SparkContext is shared between user sessions and should be closed only when there is no active

2018-10-16 Thread Sahil Takiar
tps://reviews.apache.org/r/69022/#comment294145> why remove this? - Sahil Takiar On Oct. 15, 2018, 7:21 p.m., denys kuzmenko wrote: > > --- > This is an automatically generated e-mail. To reply,

Re: Review Request 68474: HIVE-20440: Create better cache eviction policy for SmallTableCache

2018-10-01 Thread Sahil Takiar
Cleanup Thread" ql/src/java/org/apache/hadoop/hive/ql/exec/spark/SmallTableCache.java Lines 117 (patched) <https://reviews.apache.org/r/68474/#comment293418> replace with `cacheL1.get(key, valueLoader)` where `valueLoader` loads from `cacheL2` - Sahil Ta

[jira] [Created] (HIVE-20519) Remove 30m min value for hive.spark.session.timeout

2018-09-07 Thread Sahil Takiar (JIRA)
Sahil Takiar created HIVE-20519: --- Summary: Remove 30m min value for hive.spark.session.timeout Key: HIVE-20519 URL: https://issues.apache.org/jira/browse/HIVE-20519 Project: Hive Issue Type

[jira] [Created] (HIVE-20512) Improve record and memory usage logging in SparkRecordHandler

2018-09-06 Thread Sahil Takiar (JIRA)
Sahil Takiar created HIVE-20512: --- Summary: Improve record and memory usage logging in SparkRecordHandler Key: HIVE-20512 URL: https://issues.apache.org/jira/browse/HIVE-20512 Project: Hive

Re: [ANNOUNCE] New committer: Andrew Sherman

2018-09-04 Thread Sahil Takiar
gt;> to become a committer, and we are pleased to announce that he has > > >accepted. > > >> > > >> Andrew, welcome, thank you for your contributions, and we look > > >forward to > > >> your > > >> further interactions with the community! > > >> > > >> Ashutosh Chauhan (on behalf of the Apache Hive PMC) > > >> > > > > > > > -- Sahil Takiar Software Engineer takiar.sa...@gmail.com | (510) 673-0309

[jira] [Created] (HIVE-20495) Set hive.spark.dynamic.partition.pruning.map.join.only to true by default

2018-08-31 Thread Sahil Takiar (JIRA)
Sahil Takiar created HIVE-20495: --- Summary: Set hive.spark.dynamic.partition.pruning.map.join.only to true by default Key: HIVE-20495 URL: https://issues.apache.org/jira/browse/HIVE-20495 Project: Hive

[jira] [Created] (HIVE-20488) SparkSubmitSparkClient#launchDriver should parse exceptions, not just errors

2018-08-30 Thread Sahil Takiar (JIRA)
Sahil Takiar created HIVE-20488: --- Summary: SparkSubmitSparkClient#launchDriver should parse exceptions, not just errors Key: HIVE-20488 URL: https://issues.apache.org/jira/browse/HIVE-20488 Project

[jira] [Created] (HIVE-20280) JobResultSerializer uses wrong registration id in KyroMessageCodec

2018-07-31 Thread Sahil Takiar (JIRA)
Sahil Takiar created HIVE-20280: --- Summary: JobResultSerializer uses wrong registration id in KyroMessageCodec Key: HIVE-20280 URL: https://issues.apache.org/jira/browse/HIVE-20280 Project: Hive

[jira] [Created] (HIVE-20273) Spark jobs aren't cancelled if getSparkJobInfo or getSparkStagesInfo

2018-07-30 Thread Sahil Takiar (JIRA)
Sahil Takiar created HIVE-20273: --- Summary: Spark jobs aren't cancelled if getSparkJobInfo or getSparkStagesInfo Key: HIVE-20273 URL: https://issues.apache.org/jira/browse/HIVE-20273 Project:

[jira] [Created] (HIVE-20271) Improve HoS query cancellation handling

2018-07-30 Thread Sahil Takiar (JIRA)
Sahil Takiar created HIVE-20271: --- Summary: Improve HoS query cancellation handling Key: HIVE-20271 URL: https://issues.apache.org/jira/browse/HIVE-20271 Project: Hive Issue Type: Improvement

[jira] [Created] (HIVE-20270) Don't serialize hashCode for groupByKey

2018-07-30 Thread Sahil Takiar (JIRA)
Sahil Takiar created HIVE-20270: --- Summary: Don't serialize hashCode for groupByKey Key: HIVE-20270 URL: https://issues.apache.org/jira/browse/HIVE-20270 Project: Hive Issue Type

Re: [ANNOUNCE] New PMC Member : Sahil Takiar

2018-07-30 Thread Sahil Takiar
t; > >> On behalf of the Hive PMC I am delighted to announce Sahil Takiar is > >> joining Hive PMC. > >> Thanks Sahil for all your contributions till now. Looking forward to > many > >> more. > >> > >> Welcome, Sahil! > >> > >> Thanks, > >> Ashutosh > > > > -- Sahil Takiar Software Engineer takiar.sa...@gmail.com | (510) 673-0309

[jira] [Created] (HIVE-20243) Ptests spend an excessive amount of time in GC

2018-07-25 Thread Sahil Takiar (JIRA)
Sahil Takiar created HIVE-20243: --- Summary: Ptests spend an excessive amount of time in GC Key: HIVE-20243 URL: https://issues.apache.org/jira/browse/HIVE-20243 Project: Hive Issue Type: Sub

[jira] [Created] (HIVE-20211) DPP call to remove PartitionDescs from aliasToPartnInfo doesn't do anything

2018-07-19 Thread Sahil Takiar (JIRA)
Sahil Takiar created HIVE-20211: --- Summary: DPP call to remove PartitionDescs from aliasToPartnInfo doesn't do anything Key: HIVE-20211 URL: https://issues.apache.org/jira/browse/HIVE-20211 Pr

[jira] [Created] (HIVE-20141) Turn hive.spark.use.groupby.shuffle off by default

2018-07-11 Thread Sahil Takiar (JIRA)
Sahil Takiar created HIVE-20141: --- Summary: Turn hive.spark.use.groupby.shuffle off by default Key: HIVE-20141 URL: https://issues.apache.org/jira/browse/HIVE-20141 Project: Hive Issue Type

[jira] [Created] (HIVE-20134) Improve logging when HoS Driver is killed due to exceeding memory limits

2018-07-10 Thread Sahil Takiar (JIRA)
Sahil Takiar created HIVE-20134: --- Summary: Improve logging when HoS Driver is killed due to exceeding memory limits Key: HIVE-20134 URL: https://issues.apache.org/jira/browse/HIVE-20134 Project: Hive

[jira] [Created] (HIVE-20125) Typo in MetricsCollection for OutputMetrics

2018-07-09 Thread Sahil Takiar (JIRA)
Sahil Takiar created HIVE-20125: --- Summary: Typo in MetricsCollection for OutputMetrics Key: HIVE-20125 URL: https://issues.apache.org/jira/browse/HIVE-20125 Project: Hive Issue Type: Sub-task

[jira] [Created] (HIVE-20124) Re-add HIVE-19787: Log message when spark-submit has completed

2018-07-09 Thread Sahil Takiar (JIRA)
Sahil Takiar created HIVE-20124: --- Summary: Re-add HIVE-19787: Log message when spark-submit has completed Key: HIVE-20124 URL: https://issues.apache.org/jira/browse/HIVE-20124 Project: Hive

[jira] [Created] (HIVE-20108) Investigate alternatives to groupByKey

2018-07-06 Thread Sahil Takiar (JIRA)
Sahil Takiar created HIVE-20108: --- Summary: Investigate alternatives to groupByKey Key: HIVE-20108 URL: https://issues.apache.org/jira/browse/HIVE-20108 Project: Hive Issue Type: Improvement

Re: Review Request 67815: HIVE-19733 : RemoteSparkJobStatus#getSparkStageProgress inefficient implementation

2018-07-06 Thread Sahil Takiar
/RemoteSparkJobStatus.java Line 194 (original), 197 (patched) <https://reviews.apache.org/r/67815/#comment288716> I don't think this makes sense given than the exceptions thrown in this method are specific to run a `GetJobInfoJob`. - Sahil Takiar On July 5, 2018, 7:34 p.m., Bha

Re: Review Request 67815: HIVE-19733 : RemoteSparkJobStatus#getSparkStageProgress inefficient implementation

2018-07-03 Thread Sahil Takiar
ks very similar to `GetJobInfoJob`, can the two be re-factored to share common code? - Sahil Takiar On July 3, 2018, 6:32 p.m., Bharathkrishna Guruvayoor Murali wrote: > > --- > This is an automatically generated e-mail. To reply, visit: >

[jira] [Created] (HIVE-20056) SparkPartitionPruner shouldn't be triggered by Spark tasks

2018-07-02 Thread Sahil Takiar (JIRA)
Sahil Takiar created HIVE-20056: --- Summary: SparkPartitionPruner shouldn't be triggered by Spark tasks Key: HIVE-20056 URL: https://issues.apache.org/jira/browse/HIVE-20056 Project: Hive

[jira] [Created] (HIVE-20054) Propagate ExecutionExceptions from the driver thread to the client

2018-07-02 Thread Sahil Takiar (JIRA)
Sahil Takiar created HIVE-20054: --- Summary: Propagate ExecutionExceptions from the driver thread to the client Key: HIVE-20054 URL: https://issues.apache.org/jira/browse/HIVE-20054 Project: Hive

Re: Review Request 67636: HIVE-19176 : Add HoS support to progress bar on Beeline client.

2018-07-02 Thread Sahil Takiar
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/67636/#review205643 --- Ship it! Ship It! - Sahil Takiar On June 22, 2018, 8:01 p.m

[jira] [Created] (HIVE-20032) Don

2018-06-28 Thread Sahil Takiar (JIRA)
Sahil Takiar created HIVE-20032: --- Summary: Don Key: HIVE-20032 URL: https://issues.apache.org/jira/browse/HIVE-20032 Project: Hive Issue Type: Improvement Reporter: Sahil Takiar

Re: Review Request 67636: HIVE-19176 : Add HoS support to progress bar on Beeline client.

2018-06-21 Thread Sahil Takiar
/r/67636/#comment288134> why is this necessary? - Sahil Takiar On June 21, 2018, 3:50 p.m., Bharathkrishna Guruvayoor Murali wrote: > > --- > This is an automatically generate

Re: New committer announcement : Marta Kuczora

2018-06-21 Thread Sahil Takiar
t; to become a committer, and we are pleased to announce that he has > accepted. > > > > Marta, welcome, thank you for your contributions, and we look forward > your > > further interactions with the community! > > > > Ashutosh Chauhan (on behalf of the Apache

Re: [ANNOUNCE] New committer: Adam Szita

2018-06-21 Thread Sahil Takiar
become a committer, and we are pleased to announce that he has > accepted. > > > > Adam, welcome, thank you for your contributions, and we look forward your > > further interactions with the community! > > > > Ashutosh Chauhan (on behalf of the Apache Hive PMC) > > -- Sahil Takiar Software Engineer takiar.sa...@gmail.com | (510) 673-0309

Re: Review Request 67636: HIVE-19176 : Add HoS support to progress bar on Beeline client.

2018-06-19 Thread Sahil Takiar
che.org/r/67636/#comment287899> how much of this code is new coded vs. copied from other classes? ql/src/java/org/apache/hadoop/hive/ql/exec/spark/status/RenderStrategy.java Lines 209 (patched) <https://reviews.apache.org/r/67636/#comment287898> javadocs please - Sahil T

Re: Ptest timeouts

2018-06-19 Thread Sahil Takiar
1:55 AM, Prasanth Jayachandran > wrote: > > Precommit tests have started to timeout again. Could it be because of > disk space issue? Should we need restart again? Sometimes >90% disk usage > may also result in unhealthy node where no containers can be launched. > > >

[jira] [Created] (HIVE-19937) Intern JobConf objects in Spark tasks

2018-06-18 Thread Sahil Takiar (JIRA)
Sahil Takiar created HIVE-19937: --- Summary: Intern JobConf objects in Spark tasks Key: HIVE-19937 URL: https://issues.apache.org/jira/browse/HIVE-19937 Project: Hive Issue Type: Improvement

Re: Review Request 67468: HIVE-18118: provide supportability support for Erasure Coding Update number of Erasure Coded Files in a directory as part of Basic (aka Quick) Stats This information is then

2018-06-15 Thread Sahil Takiar
a list? - Sahil Takiar On June 6, 2018, 12:46 a.m., Andrew Sherman wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.

[jira] [Created] (HIVE-19910) hive.spark.log.dir isn't honored for TestSparkCliDriver

2018-06-15 Thread Sahil Takiar (JIRA)
Sahil Takiar created HIVE-19910: --- Summary: hive.spark.log.dir isn't honored for TestSparkCliDriver Key: HIVE-19910 URL: https://issues.apache.org/jira/browse/HIVE-19910 Project: Hive

[jira] [Created] (HIVE-19821) Distributed HiveServer2

2018-06-06 Thread Sahil Takiar (JIRA)
Sahil Takiar created HIVE-19821: --- Summary: Distributed HiveServer2 Key: HIVE-19821 URL: https://issues.apache.org/jira/browse/HIVE-19821 Project: Hive Issue Type: New Feature

Re: Review Request 67263: HIVE-19602

2018-06-05 Thread Sahil Takiar
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/67263/#review204332 --- Ship it! Ship It! - Sahil Takiar On June 4, 2018, 6:34 p.m

[jira] [Created] (HIVE-19797) Enable Spark InProcessLauncher on secure clusters

2018-06-05 Thread Sahil Takiar (JIRA)
Sahil Takiar created HIVE-19797: --- Summary: Enable Spark InProcessLauncher on secure clusters Key: HIVE-19797 URL: https://issues.apache.org/jira/browse/HIVE-19797 Project: Hive Issue Type

[jira] [Created] (HIVE-19788) Flaky test: TestHCatLoaderComplexSchema

2018-06-04 Thread Sahil Takiar (JIRA)
Sahil Takiar created HIVE-19788: --- Summary: Flaky test: TestHCatLoaderComplexSchema Key: HIVE-19788 URL: https://issues.apache.org/jira/browse/HIVE-19788 Project: Hive Issue Type: Sub-task

[jira] [Created] (HIVE-19787) Log message when spark-submit has completed

2018-06-04 Thread Sahil Takiar (JIRA)
Sahil Takiar created HIVE-19787: --- Summary: Log message when spark-submit has completed Key: HIVE-19787 URL: https://issues.apache.org/jira/browse/HIVE-19787 Project: Hive Issue Type: Sub-task

[jira] [Created] (HIVE-19786) RpcServer cancelTask log message is incorrect

2018-06-04 Thread Sahil Takiar (JIRA)
Sahil Takiar created HIVE-19786: --- Summary: RpcServer cancelTask log message is incorrect Key: HIVE-19786 URL: https://issues.apache.org/jira/browse/HIVE-19786 Project: Hive Issue Type: Sub

[jira] [Created] (HIVE-19785) Race condition when timeout task is invoked during SASL negotation

2018-06-04 Thread Sahil Takiar (JIRA)
Sahil Takiar created HIVE-19785: --- Summary: Race condition when timeout task is invoked during SASL negotation Key: HIVE-19785 URL: https://issues.apache.org/jira/browse/HIVE-19785 Project: Hive

[jira] [Created] (HIVE-19784) Regression test selection framework for ptest

2018-06-04 Thread Sahil Takiar (JIRA)
Sahil Takiar created HIVE-19784: --- Summary: Regression test selection framework for ptest Key: HIVE-19784 URL: https://issues.apache.org/jira/browse/HIVE-19784 Project: Hive Issue Type: Sub

Re: Review Request 67415: HIVE-19525 : Spark task logs print PLAN PATH excessive number of times

2018-06-01 Thread Sahil Takiar
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/67415/#review204193 --- Ship it! Ship It! - Sahil Takiar On June 1, 2018, 3:51 p.m

[jira] [Created] (HIVE-19765) Add Parquet specific tests to BlobstoreCliDriver

2018-06-01 Thread Sahil Takiar (JIRA)
Sahil Takiar created HIVE-19765: --- Summary: Add Parquet specific tests to BlobstoreCliDriver Key: HIVE-19765 URL: https://issues.apache.org/jira/browse/HIVE-19765 Project: Hive Issue Type: Sub

[jira] [Created] (HIVE-19764) Add --SORT_QUERY_RESULTS to hive-blobstore/map_join.q.out

2018-06-01 Thread Sahil Takiar (JIRA)
Sahil Takiar created HIVE-19764: --- Summary: Add --SORT_QUERY_RESULTS to hive-blobstore/map_join.q.out Key: HIVE-19764 URL: https://issues.apache.org/jira/browse/HIVE-19764 Project: Hive Issue

[jira] [Created] (HIVE-19760) Flaky test: TestJdbcWithMiniLlapArrow.testDataTypes

2018-06-01 Thread Sahil Takiar (JIRA)
Sahil Takiar created HIVE-19760: --- Summary: Flaky test: TestJdbcWithMiniLlapArrow.testDataTypes Key: HIVE-19760 URL: https://issues.apache.org/jira/browse/HIVE-19760 Project: Hive Issue Type

[jira] [Created] (HIVE-19759) Flaky test: TestRpc#testServerPort

2018-06-01 Thread Sahil Takiar (JIRA)
Sahil Takiar created HIVE-19759: --- Summary: Flaky test: TestRpc#testServerPort Key: HIVE-19759 URL: https://issues.apache.org/jira/browse/HIVE-19759 Project: Hive Issue Type: Sub-task

[jira] [Created] (HIVE-19752) PerfLogger integration for critical Hive-on-S3 paths

2018-05-31 Thread Sahil Takiar (JIRA)
Sahil Takiar created HIVE-19752: --- Summary: PerfLogger integration for critical Hive-on-S3 paths Key: HIVE-19752 URL: https://issues.apache.org/jira/browse/HIVE-19752 Project: Hive Issue Type

Re: Review Request 67263: HIVE-19602

2018-05-31 Thread Sahil Takiar
/java/org/apache/hadoop/hive/ql/exec/spark/status/SparkJobMonitor.java Line 238 (original), 245 (patched) <https://reviews.apache.org/r/67263/#comment286518> can be private - Sahil Takiar On May 29, 2018, 10:53 p.m., Bharathkrishna Guruvayoor Murali

[jira] [Created] (HIVE-19733) RemoteSparkJobStatus#getSparkStageProgress inefficient implementation

2018-05-29 Thread Sahil Takiar (JIRA)
Sahil Takiar created HIVE-19733: --- Summary: RemoteSparkJobStatus#getSparkStageProgress inefficient implementation Key: HIVE-19733 URL: https://issues.apache.org/jira/browse/HIVE-19733 Project: Hive

Re: Review Request 67263: HIVE-19602

2018-05-29 Thread Sahil Takiar
ent286393> can you add some javadocs for this class ql/src/java/org/apache/hadoop/hive/ql/exec/spark/status/SparkProgressMonitor.java Lines 18 (patched) <https://reviews.apache.org/r/67263/#comment286392> this can be package private right? - Sahil Takiar On May 23, 2018, 5:32 a.

Re: Review Request 67263: HIVE-19602

2018-05-29 Thread Sahil Takiar
> On May 25, 2018, 4:43 p.m., Sahil Takiar wrote: > > ql/src/java/org/apache/hadoop/hive/ql/exec/spark/status/SparkJobMonitor.java > > Line 70 (original), 67 (patched) > > <https://reviews.apache.org/r/67263/diff/1/?file=2027540#file2027540line72> >

[jira] [Created] (HIVE-19716) Set spark.local.dir for a few more HoS integration tests

2018-05-25 Thread Sahil Takiar (JIRA)
Sahil Takiar created HIVE-19716: --- Summary: Set spark.local.dir for a few more HoS integration tests Key: HIVE-19716 URL: https://issues.apache.org/jira/browse/HIVE-19716 Project: Hive Issue

Re: Review Request 67263: HIVE-19602

2018-05-25 Thread Sahil Takiar
e impact of having an extra argument? does the formatting change at all? ql/src/java/org/apache/hadoop/hive/ql/exec/spark/status/SparkProgressMonitor.java Lines 77 (patched) <https://reviews.apache.org/r/67263/#comment286254> the current states LGTM - Sahil Takiar O

Re: May 2018 Hive User Group Meeting

2018-05-23 Thread Sahil Takiar
/gwX3KpHa2j0 - Hive-on-Spark at Uber: Efficiency & Scale - Xuefu Zhang - https://www.slideshare.net/sahiltakiar/hive-on-spark-at-uber-scale - Hive-on-S3 Performance: Past, Present, and Future - Sahil Takiar - https://www.slideshare.net/sahiltakiar/hive-ons3-performance-past-present-and-fu

[jira] [Created] (HIVE-19676) Ability to selectively run tests in TestBlobstoreCliDriver

2018-05-23 Thread Sahil Takiar (JIRA)
Sahil Takiar created HIVE-19676: --- Summary: Ability to selectively run tests in TestBlobstoreCliDriver Key: HIVE-19676 URL: https://issues.apache.org/jira/browse/HIVE-19676 Project: Hive Issue

Re: Release branch updates

2018-05-22 Thread Sahil Takiar
mmits. > Please ping me if you would like to get your commit to branch-3 keeping in > mind that we should target only minor features and bug fixes in there. > Master will continue to be used for major features in 4.0. > > > Thanks, > Vineet Garg > -- Sahil Takiar Software Engineer takiar.sa...@gmail.com | (510) 673-0309

[jira] [Created] (HIVE-19603) Decrease batch size of TestMinimrCliDriver

2018-05-18 Thread Sahil Takiar (JIRA)
Sahil Takiar created HIVE-19603: --- Summary: Decrease batch size of TestMinimrCliDriver Key: HIVE-19603 URL: https://issues.apache.org/jira/browse/HIVE-19603 Project: Hive Issue Type: Test

Re: Review Request 67023: HIVE-18117: Add a new Test Driver "TestErasureCodingHDFSCliDriver" that can be used to run tests over hdfs directories that employ Erasure Coding.

2018-05-16 Thread Sahil Takiar
> On May 10, 2018, noon, Sahil Takiar wrote: > > itests/util/src/main/java/org/apache/hadoop/hive/cli/control/CliConfigs.java > > Lines 674 (patched) > > <https://reviews.apache.org/r/67023/diff/1/?file=2018287#file2018287line674> > > > > is this nece

[jira] [Created] (HIVE-19571) Ability to run multiple pre-commit jobs on a ptest server

2018-05-16 Thread Sahil Takiar (JIRA)
Sahil Takiar created HIVE-19571: --- Summary: Ability to run multiple pre-commit jobs on a ptest server Key: HIVE-19571 URL: https://issues.apache.org/jira/browse/HIVE-19571 Project: Hive Issue

Re: Review Request 67023: HIVE-18117: Add a new Test Driver "TestErasureCodingHDFSCliDriver" that can be used to run tests over hdfs directories that employ Erasure Coding.

2018-05-15 Thread Sahil Takiar
> On May 10, 2018, noon, Sahil Takiar wrote: > > itests/util/src/main/java/org/apache/hadoop/hive/cli/control/CliConfigs.java > > Lines 674 (patched) > > <https://reviews.apache.org/r/67023/diff/1/?file=2018287#file2018287line674> > > > > is this nece

Re: Review Request 67023: HIVE-18117: Add a new Test Driver "TestErasureCodingHDFSCliDriver" that can be used to run tests over hdfs directories that employ Erasure Coding.

2018-05-15 Thread Sahil Takiar
> On May 14, 2018, 4:57 p.m., Sahil Takiar wrote: > > itests/src/test/resources/testconfiguration.properties > > Line 1693 (original), 1693 (patched) > > <https://reviews.apache.org/r/67023/diff/1-2/?file=2018286#file2018286line1693> > > > > why would

[jira] [Created] (HIVE-19562) Flaky test: TestMiniSparkOnYarn FileNotFoundException in spark-submit

2018-05-15 Thread Sahil Takiar (JIRA)
Sahil Takiar created HIVE-19562: --- Summary: Flaky test: TestMiniSparkOnYarn FileNotFoundException in spark-submit Key: HIVE-19562 URL: https://issues.apache.org/jira/browse/HIVE-19562 Project: Hive

[jira] [Created] (HIVE-19559) SparkClientImpl shouldn't name redirector thread "RemoteDriver"

2018-05-15 Thread Sahil Takiar (JIRA)
Sahil Takiar created HIVE-19559: --- Summary: SparkClientImpl shouldn't name redirector thread "RemoteDriver" Key: HIVE-19559 URL: https://issues.apache.org/jira/browse/HIVE-19559

Re: [VOTE] Stricter commit guidelines

2018-05-15 Thread Sahil Takiar
I see it, by enforcing that we always get a +1 from the testing > > infra > > >>> before committing, 1) we will have a more stable project, and 2) we > > will > > >>> have another incentive as a community to create a more robust testing > > >>&g

Re: Review Request 67023: HIVE-18117: Add a new Test Driver "TestErasureCodingHDFSCliDriver" that can be used to run tests over hdfs directories that employ Erasure Coding.

2018-05-15 Thread Sahil Takiar
> On May 10, 2018, noon, Sahil Takiar wrote: > > itests/util/src/main/java/org/apache/hadoop/hive/cli/control/CliConfigs.java > > Lines 674 (patched) > > <https://reviews.apache.org/r/67023/diff/1/?file=2018287#file2018287line674> > > > > is this nece

Re: Review Request 67023: HIVE-18117: Add a new Test Driver "TestErasureCodingHDFSCliDriver" that can be used to run tests over hdfs directories that employ Erasure Coding.

2018-05-15 Thread Sahil Takiar
> On May 14, 2018, 4:57 p.m., Sahil Takiar wrote: > > itests/src/test/resources/testconfiguration.properties > > Line 1693 (original), 1693 (patched) > > <https://reviews.apache.org/r/67023/diff/1-2/?file=2018286#file2018286line1693> > > > > why would

Re: Review Request 67023: HIVE-18117: Add a new Test Driver "TestErasureCodingHDFSCliDriver" that can be used to run tests over hdfs directories that employ Erasure Coding.

2018-05-14 Thread Sahil Takiar
ent285102> why would we want to run this on the local fs? - Sahil Takiar On May 11, 2018, 11:38 p.m., Andrew Sherman wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://re

Re: Review Request 67023: HIVE-18117: Add a new Test Driver "TestErasureCodingHDFSCliDriver" that can be used to run tests over hdfs directories that employ Erasure Coding.

2018-05-14 Thread Sahil Takiar
> On May 10, 2018, noon, Sahil Takiar wrote: > > itests/util/src/main/java/org/apache/hadoop/hive/cli/control/CliConfigs.java > > Lines 674 (patched) > > <https://reviews.apache.org/r/67023/diff/1/?file=2018287#file2018287line674> > > > > is this nece

Re: Review Request 66290: HIVE-14388 : Add number of rows inserted message after insert command in Beeline

2018-05-14 Thread Sahil Takiar
than copying it. - Sahil Takiar On May 12, 2018, 12:30 a.m., Bharathkrishna Guruvayoor Murali wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache

[jira] [Created] (HIVE-19525) Spark task logs print PLAN PATH excessive number of times

2018-05-14 Thread Sahil Takiar (JIRA)
Sahil Takiar created HIVE-19525: --- Summary: Spark task logs print PLAN PATH excessive number of times Key: HIVE-19525 URL: https://issues.apache.org/jira/browse/HIVE-19525 Project: Hive Issue

Re: May 2018 Hive User Group Meeting

2018-05-14 Thread Sahil Takiar
che.org > *Subject:* Re: May 2018 Hive User Group Meeting > > > > Hey everyone, > > > > Was the meeting recorded by any chance? > > Luis > > > On May 8, 2018, at 5:31 PM, Sahil Takiar wrote: > > Hey Everyone, > > > > Almost time for t

[jira] [Created] (HIVE-19515) TestRpc.testServerPort is consistently failing

2018-05-13 Thread Sahil Takiar (JIRA)
Sahil Takiar created HIVE-19515: --- Summary: TestRpc.testServerPort is consistently failing Key: HIVE-19515 URL: https://issues.apache.org/jira/browse/HIVE-19515 Project: Hive Issue Type: Test

[jira] [Created] (HIVE-19508) SparkJobMonitor getReport doesn't print stage progress in order

2018-05-11 Thread Sahil Takiar (JIRA)
Sahil Takiar created HIVE-19508: --- Summary: SparkJobMonitor getReport doesn't print stage progress in order Key: HIVE-19508 URL: https://issues.apache.org/jira/browse/HIVE-19508 Project:

Re: Review Request 67073: HIVE-19370 : Retain time part in add_months function on timestamp datatype fields in hive

2018-05-11 Thread Sahil Takiar
> On May 11, 2018, 3:22 p.m., Sahil Takiar wrote: > > common/src/java/org/apache/hive/common/util/DateUtils.java > > Lines 39 (patched) > > <https://reviews.apache.org/r/67073/diff/1/?file=2019551#file2019551line39> > > > > why does this need to be

Re: Review Request 67073: HIVE-19370 : Retain time part in add_months function on timestamp datatype fields in hive

2018-05-11 Thread Sahil Takiar
52 (patched) <https://reviews.apache.org/r/67073/#comment285002> Perhaps we should change the docs too. - Sahil Takiar On May 10, 2018, 9:55 p.m., Bharathkrishna Guruvayoor Murali wrote: > > --- > This is an automatically

Re: Review Request 66290: HIVE-14388 : Add number of rows inserted message after insert command in Beeline

2018-05-10 Thread Sahil Takiar
an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/66290/ > ------- > > (Updated May 9, 2018, 8:46 p.m.) > > > Review request for hive, Sahil Takiar and Vihang Karajgaonkar. > > > Bugs: HIVE-14388 > https:/

Re: Review Request 66290: HIVE-14388 : Add number of rows inserted message after insert command in Beeline

2018-05-10 Thread Sahil Takiar
57> don't think these imports need to be re-ordered? - Sahil Takiar On May 9, 2018, 8:46 p.m., Bharathkrishna Guruvayoor Murali wrote: > > --- > This is an automatically generated e-mail. To reply, visit: >

Re: May 2018 Hive User Group Meeting

2018-05-08 Thread Sahil Takiar
tarts. For those attending in person, there will be guest wifi: Login: HiveMeetup Password: ClouderaHive On Mon, May 7, 2018 at 12:48 PM, Sahil Takiar wrote: > Hey Everyone, > > The meetup is only a day away! Here > <https://docs.google.com/document/d/1v8iERias-LOq8-q4BrCNSUsOarCw

Re: Integrating Yetus with Precommit job

2018-05-07 Thread Sahil Takiar
d a couple of jiras: > > >>>> -HIVE-17995 <https://issues.apache.org/jira/browse/HIVE-17995> Run > > >>>> checkstyle on standalone-metastore module with proper configuration > > >>>> -HIVE-17996 <https://issues.apache.org/jira/browse/HIVE-17996> Fix > > ASF > > >>>> headers > > >>>> -HIVE-17997 <https://issues.apache.org/jira/browse/HIVE-17997> Add > > rat > > >>>> plugin and configuration to standalone metastore pom > > >>>> > > >>>> Sahil: there is an umbrella jira (HIVE-13503 > > >>>> <https://issues.apache.org/jira/browse/HIVE-13503>) for test > > >>> improvements, > > >>>> the Yetus integration itself is also a subtask of it. I think any > > >>> further > > >>>> improvements on what Yetus features we want to enable should go here > > >>> too. > > >>>> > > >>>> Adam > > >>>> > > >>>> > > >>>> > > >>>> > > >>>> > > >>> > > >> > > >> > > > > > -- Sahil Takiar Software Engineer takiar.sa...@gmail.com | (510) 673-0309

Re: May 2018 Hive User Group Meeting

2018-05-07 Thread Sahil Takiar
https://www.meetup.com/Hive-User-Group-Meeting/events/249641278/ On Wed, May 2, 2018 at 7:48 AM, Sahil Takiar wrote: > Hey Everyone, > > The agenda for the meetup has been set and I'm excited to say we have lots > of interesting talks scheduled! Below is final agenda, the full list

Re: PTest Maintenance

2018-05-06 Thread Sahil Takiar
Maintenance has been completed. There should be no missed Hive QA runs. On Sun, May 6, 2018 at 5:25 PM, Sahil Takiar wrote: > Will be performing some maintenance on PTest this evening to > deploy HIVE-19212. If there are any issues, I will ping the individual > JIRAs affected. &g

PTest Maintenance

2018-05-06 Thread Sahil Takiar
Will be performing some maintenance on PTest this evening to deploy HIVE-19212. If there are any issues, I will ping the individual JIRAs affected. --Sahil

[jira] [Created] (HIVE-19422) Create Docker env for running HoS locally

2018-05-04 Thread Sahil Takiar (JIRA)
Sahil Takiar created HIVE-19422: --- Summary: Create Docker env for running HoS locally Key: HIVE-19422 URL: https://issues.apache.org/jira/browse/HIVE-19422 Project: Hive Issue Type: Sub-task

[jira] [Created] (HIVE-19405) AddPartitionDesc should intern its fields

2018-05-03 Thread Sahil Takiar (JIRA)
Sahil Takiar created HIVE-19405: --- Summary: AddPartitionDesc should intern its fields Key: HIVE-19405 URL: https://issues.apache.org/jira/browse/HIVE-19405 Project: Hive Issue Type: Bug

Re: May 2018 Hive User Group Meeting

2018-05-02 Thread Sahil Takiar
- Xuefu Zhang - Hive-on-S3 Performance: Past, Present, and Future - Sahil Takiar - Dali: Data Access Layer at LinkedIn - Adwait Tumbde - Parquet Vectorization in Hive - Vihang Karajgaonkar - ORC Column Level Encryption - Owen O’Malley - Running Hive at Scale @ Lyft - Sharanya Santhana

Re: Review Request 66290: HIVE-14388 : Add number of rows inserted message after insert command in Beeline

2018-05-01 Thread Sahil Takiar
. Right now it would print "No rows affected..." if an overflow happens, which would be odd. - Sahil Takiar On April 27, 2018, 11:05 p.m., Bharathkrishna Guruvayoor Murali wrote: > > --- > This is an automatically ge

  1   2   3   4   >