Re: [VOTE] SPARK 2.2.3 (RC1)

[Takeshi Yamamuro]

Hi, Dongjoon,

We don't need to include https://github.com/apache/spark/pull/23456 in this
release?
The query there fails in v2.x while it passes in v1.6.

On Thu, Jan 10, 2019 at 10:16 AM Sean Owen  wrote:

> Hm OK those other profiles should be unrelated. I'll see if I can
> figure it out, but it's likely this is specific to the machine I am
> testing on somehow.
> For that reason, I'll say +1 on the basis that these tests really do pass.
>
> On Wed, Jan 9, 2019 at 6:05 PM Dongjoon Hyun 
> wrote:
> >
> > I tested with Maven and `-Pyarn -Phadoop-2.7 -Pkinesis-asl -Phive
> -Phive-thriftserver` on CentOS/JDK8.
> >
> > The difference seems to be `-Pmesos -Psparkr` from your and
> `-Pkinesis-asl` from mine.
> >
> > Do you think it's related? BTW, at least, we have a green balls on
> Jenkins.
> >
> >
> https://amplab.cs.berkeley.edu/jenkins/view/Spark%20QA%20Test%20(Dashboard)/job/spark-branch-2.2-test-maven-hadoop-2.7/591/
> >
> >
> > On Wed, Jan 9, 2019 at 3:37 PM Sean Owen  wrote:
> >>
> >> BTW did you run with the same profiles, I wonder; I test with,
> >> generally, -Pyarn -Phadoop-2.7 -Phive -Phive-thriftserver -Pmesos
> >> -Psparkr
> >>
> >> I am checking mostly because none of that weird error would happen
> >> without testing hive-thriftserver.
> >>
> >> The others are probably just flakiness or something else odd, and I'd
> >> look past them if others are not seeing them.
> >>
> >> The licenses and signatures looked fine, and it built correctly, at
> least.
> >>
> >> On Wed, Jan 9, 2019 at 5:09 PM Dongjoon Hyun 
> wrote:
> >> >
> >> > Hi, Sean.
> >> >
> >> > It looks strange. I didn't hit them. I'm not sure but it looks like
> some flakiness at 2.2.x era.
> >> > For me, those test passes. (I ran twice before starting a vote and
> during this voting from the source tar file)
> >> >
> >> > Bests,
> >> > Dongjoon
> >> >
> >> > On Wed, Jan 9, 2019 at 1:42 PM Sean Owen  wrote:
> >> >>
> >> >> I wonder if anyone else is seeing the following issues, or whether
> >> >> it's specific to my environment:
> >> >>
> >> >> With -Phive-thriftserver, it compiles fine. However during tests, I
> get ...
> >> >> [error]
> /home/ubuntu/spark-2.2.3/sql/hive-thriftserver/src/main/java/org/apache/hive/service/cli/thrift/ThriftCLIService.java:64:
> >> >> error: package org.eclipse.jetty.server does not exist
> >> >> [error]   protected org.eclipse.jetty.server.Server httpServer;
> >> >> [error] ^
> >> >>
> >> >> That's weird. I'd have to dig into the POM to see if this dependency
> >> >> for some reason would not be available at test time. But does this
> >> >> profile pass for anyone else?
> >> >>
> >> >> I'm also seeing test failures like the following. Yes, there's more,
> >> >> just seeing if anyone sees these?
> >> >>
> >> >> - event ordering *** FAILED ***
> >> >>   The code passed to failAfter did not complete within 10 seconds.
> >> >> (StreamingQueryListenerSuite.scala:411)
> >> >>
> >> >> - HDFSMetadataLog: metadata directory collision *** FAILED ***
> >> >>   The await method on Waiter timed out.
> (HDFSMetadataLogSuite.scala:201)
> >> >>
> >> >> - recovery *** FAILED ***
> >> >>   == Results ==
> >> >>   !== Correct Answer - 1 ==   == Spark Answer - 0 ==
> >> >>   !struct<_1:int,_2:int>  struct<>
> >> >>   ![10,5]
> >> >>
> >> >>
> >> >>
> >> >> On Tue, Jan 8, 2019 at 1:14 PM Dongjoon Hyun <
> dongjoon.h...@gmail.com> wrote:
> >> >> >
> >> >> > Please vote on releasing the following candidate as Apache Spark
> version 2.2.3.
> >> >> >
> >> >> > The vote is open until January 11 11:30AM (PST) and passes if a
> majority +1 PMC votes are cast, with
> >> >> > a minimum of 3 +1 votes.
> >> >> >
> >> >> > [ ] +1 Release this package as Apache Spark 2.2.3
> >> >> > [ ] -1 Do not release this package because ...
> >> >> >
> >> >> > To learn more about Apache Spark, please see
> http://spark.apache.org/
> >> >> >
> >> >> > The tag to be voted on is v2.2.3-rc1 (commit
> 4acb6ba37b94b90aac445e6546426145a5f9eba2):
> >> >> > https://github.com/apache/spark/tree/v2.2.3-rc1
> >> >> >
> >> >> > The release files, including signatures, digests, etc. can be
> found at:
> >> >> > https://dist.apache.org/repos/dist/dev/spark/v2.2.3-rc1-bin/
> >> >> >
> >> >> > Signatures used for Spark RCs can be found in this file:
> >> >> > https://dist.apache.org/repos/dist/dev/spark/KEYS
> >> >> >
> >> >> > The staging repository for this release can be found at:
> >> >> >
> https://repository.apache.org/content/repositories/orgapachespark-1295
> >> >> >
> >> >> > The documentation corresponding to this release can be found at:
> >> >> > https://dist.apache.org/repos/dist/dev/spark/v2.2.3-rc1-docs/
> >> >> >
> >> >> > The list of bug fixes going into 2.2.3 can be found at the
> following URL:
> >> >> > https://issues.apache.org/jira/projects/SPARK/versions/12343560
> >> >> >
> >> >> > FAQ
> >> >> >
> >> >> > =
> >> >> > How can I help test this release?
> >> >> > ==

Re: [VOTE] SPARK 2.2.3 (RC1)

[Sean Owen]

Is that the right link? that is marked as a minor bug, maybe. From
what you describe it's not a regression from 2.2.2 either.

On Thu, Jan 10, 2019 at 6:37 AM Takeshi Yamamuro  wrote:
>
> Hi, Dongjoon,
>
> We don't need to include https://github.com/apache/spark/pull/23456 in this 
> release?
> The query there fails in v2.x while it passes in v1.6.
>

-
To unsubscribe e-mail: dev-unsubscr...@spark.apache.org

[GitHub] srowen closed pull request #165: Update 2018-12-19-spark-ai-summit-apr-2019-agenda-posted.md

[GitBox]

srowen closed pull request #165: Update 
2018-12-19-spark-ai-summit-apr-2019-agenda-posted.md
URL: https://github.com/apache/spark-website/pull/165
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/news/_posts/2018-12-19-spark-ai-summit-apr-2019-agenda-posted.md 
b/news/_posts/2018-12-19-spark-ai-summit-apr-2019-agenda-posted.md
index b4c967c05..615a08cb4 100644
--- a/news/_posts/2018-12-19-spark-ai-summit-apr-2019-agenda-posted.md
+++ b/news/_posts/2018-12-19-spark-ai-summit-apr-2019-agenda-posted.md
@@ -1,6 +1,6 @@
 ---
 layout: post
-title: Spark+AI Summit (April 23-25th, 2018, San Francisco) agenda posted
+title: Spark+AI Summit (April 23-25th, 2019, San Francisco) agenda posted
 categories:
 - News
 tags: []
diff --git a/site/committers.html b/site/committers.html
index 3771ba93a..f41921844 100644
--- a/site/committers.html
+++ b/site/committers.html
@@ -162,7 +162,7 @@
   Latest News
   
 
-  Spark+AI Summit (April 
23-25th, 2018, San Francisco) agenda posted
+  Spark+AI Summit (April 
23-25th, 2019, San Francisco) agenda posted
   (Dec 19, 2018)
 
   Spark 2.4.0 
released
diff --git a/site/community.html b/site/community.html
index 2548315ea..db666cec7 100644
--- a/site/community.html
+++ b/site/community.html
@@ -162,7 +162,7 @@
   Latest News
   
 
-  Spark+AI Summit (April 
23-25th, 2018, San Francisco) agenda posted
+  Spark+AI Summit (April 
23-25th, 2019, San Francisco) agenda posted
   (Dec 19, 2018)
 
   Spark 2.4.0 
released
diff --git a/site/contributing.html b/site/contributing.html
index 018d1bb75..05cd6306a 100644
--- a/site/contributing.html
+++ b/site/contributing.html
@@ -162,7 +162,7 @@
   Latest News
   
 
-  Spark+AI Summit (April 
23-25th, 2018, San Francisco) agenda posted
+  Spark+AI Summit (April 
23-25th, 2019, San Francisco) agenda posted
   (Dec 19, 2018)
 
   Spark 2.4.0 
released
diff --git a/site/developer-tools.html b/site/developer-tools.html
index b043c75c7..5fc463fcc 100644
--- a/site/developer-tools.html
+++ b/site/developer-tools.html
@@ -162,7 +162,7 @@
   Latest News
   
 
-  Spark+AI Summit (April 
23-25th, 2018, San Francisco) agenda posted
+  Spark+AI Summit (April 
23-25th, 2019, San Francisco) agenda posted
   (Dec 19, 2018)
 
   Spark 2.4.0 
released
diff --git a/site/documentation.html b/site/documentation.html
index 4d54ac622..3ac0bb799 100644
--- a/site/documentation.html
+++ b/site/documentation.html
@@ -162,7 +162,7 @@
   Latest News
   
 
-  Spark+AI Summit (April 
23-25th, 2018, San Francisco) agenda posted
+  Spark+AI Summit (April 
23-25th, 2019, San Francisco) agenda posted
   (Dec 19, 2018)
 
   Spark 2.4.0 
released
diff --git a/site/downloads.html b/site/downloads.html
index 68b9a048c..f9daa8010 100644
--- a/site/downloads.html
+++ b/site/downloads.html
@@ -162,7 +162,7 @@
   Latest News
   
 
-  Spark+AI Summit (April 
23-25th, 2018, San Francisco) agenda posted
+  Spark+AI Summit (April 
23-25th, 2019, San Francisco) agenda posted
   (Dec 19, 2018)
 
   Spark 2.4.0 
released
diff --git a/site/examples.html b/site/examples.html
index ba71cdc7f..006c2cae2 100644
--- a/site/examples.html
+++ b/site/examples.html
@@ -162,7 +162,7 @@
   Latest News
   
 
-  Spark+AI Summit (April 
23-25th, 2018, San Francisco) agenda posted
+  Spark+AI Summit (April 
23-25th, 2019, San Francisco) agenda posted
   (Dec 19, 2018)
 
   Spark 2.4.0 
released
diff --git a/site/faq.html b/site/faq.html
index 58c693b9a..0afcee05b 100644
--- a/site/faq.html
+++ b/site/faq.html
@@ -162,7 +162,7 @@
   Latest News
   
 
-  Spark+AI Summit (April 
23-25th, 2018, San Francisco) agenda posted
+  Spark+AI Summit (April 
23-25th, 2019, San Francisco) agenda posted
   (Dec 19, 2018)
 
   Spark 2.4.0 
released
diff --git a/site/graphx/index.html b/site/graphx/index.html
index 675b46511..2364dcf62 100644
--- a/site/graphx/index.html
+++ b/site/graphx/index.html
@@ -165,7 +165,7 @@
   Latest News
   
 
-  Spark+AI Summit (April 
23-25th, 2018, San Francisco) agenda posted
+  Spark+AI Summit (April 
23-25th, 2019, San Francisco) agenda posted
   (Dec 19, 2018)
 
   Spark 2.4.0 
released
diff --git a/site/history.html b/site/history.html
index b3b91c6ed..ff2f099df 100644
--- a/site/history.html
+++ b/site/history.html
@@ -162,7 +162,7 @@
   Latest News

[GitHub] srowen opened a new pull request #166: Further expand and update the merge and commit process for committers

[GitBox]

srowen opened a new pull request #166: Further expand and update the merge and 
commit process for committers
URL: https://github.com/apache/spark-website/pull/166
 
 
   Following up on 
https://github.com/apache/spark-website/commit/eb0aa14df472cff092b35ea1b894a0d880185561#r31886611
 with additional changes.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe e-mail: dev-unsubscr...@spark.apache.org

Re: [VOTE] SPARK 2.2.3 (RC1)

[Dongjoon Hyun]

Hi, Takeshi.

Yep. It's not a release blocker. We don't need that as Sean mentioned
already.
Since you are the release manager of 2.3.3, you may include that in the
scope of Spark 2.3.3 before it starts.

Bests,
Dongjoon.


On Thu, Jan 10, 2019 at 5:44 AM Sean Owen  wrote:

> Is that the right link? that is marked as a minor bug, maybe. From
> what you describe it's not a regression from 2.2.2 either.
>
> On Thu, Jan 10, 2019 at 6:37 AM Takeshi Yamamuro 
> wrote:
> >
> > Hi, Dongjoon,
> >
> > We don't need to include https://github.com/apache/spark/pull/23456 in
> this release?
> > The query there fails in v2.x while it passes in v1.6.
> >
>

[GitHub] dongjoon-hyun commented on a change in pull request #166: Further expand and update the merge and commit process for committers

[GitBox]

dongjoon-hyun commented on a change in pull request #166: Further expand and 
update the merge and commit process for committers
URL: https://github.com/apache/spark-website/pull/166#discussion_r246836939
 
 

 ##
 File path: committers.md
 ##
 @@ -127,13 +127,41 @@ Git history for that code to see who reviewed patches 
before. You can do this us
 Changes pushed to the master branch on Apache cannot be removed; that is, we 
can't force-push to 
 it. So please don't add any test commits or anything like that, only real 
patches.
 
-All merges should be done using the 
-[dev/merge_spark_pr.py](https://github.com/apache/spark/blob/master/dev/merge_spark_pr.py)
 
-script, which squashes the pull request's changes into one commit. To use this 
script, you 
+Setting up Remotes
+
+To use the `merge_spark_pr.py` script described below, you 
 will need to add a git remote called `apache` at 
`https://github.com/apache/spark`, 
-as well as one called "apache-github" at `git://github.com/apache/spark`. For 
the `apache` repo, 
-you can authenticate using your ASF username and password. Ask 
`dev@spark.apache.org` if you have trouble with 
-this or want help doing your first merge.
+as well as one called `apache-github` at `git://github.com/apache/spark`.
+
+You will likely also have a remote `origin` pointing to your fork of Spark, and
+`upstream` pointing to the `apache/spark` GitHub repo. 
+
+If correct, your `git remote -v` should look like:
+
+```
+apache https://github.com/apache/spark-website.git (fetch)
+apache https://github.com/apache/spark-website.git (push)
+apache-github  git://github.com/apache/spark-website (fetch)
+apache-github  git://github.com/apache/spark-website (push)
+origin https://github.com/[your username]/spark-website.git (fetch)
+origin https://github.com/[your username]/spark-website.git (push)
+upstream   https://github.com/apache/spark-website.git (fetch)
+upstream   https://github.com/apache/spark-website.git (push)
 
 Review comment:
   In this context, these should be `spark.git` instead of `spark-website.git`.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe e-mail: dev-unsubscr...@spark.apache.org

[GitHub] dongjoon-hyun commented on issue #166: Further expand and update the merge and commit process for committers

[GitBox]

dongjoon-hyun commented on issue #166: Further expand and update the merge and 
commit process for committers
URL: https://github.com/apache/spark-website/pull/166#issuecomment-453167761
 
 
   Great! Thank you for updating, @srowen .


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe e-mail: dev-unsubscr...@spark.apache.org

[GitHub] srowen closed pull request #166: Further expand and update the merge and commit process for committers

[GitBox]

srowen closed pull request #166: Further expand and update the merge and commit 
process for committers
URL: https://github.com/apache/spark-website/pull/166
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/committers.md b/committers.md
index 0eaad06e0..c3daf10fd 100644
--- a/committers.md
+++ b/committers.md
@@ -127,13 +127,41 @@ Git history for that code to see who reviewed patches 
before. You can do this us
 Changes pushed to the master branch on Apache cannot be removed; that is, we 
can't force-push to 
 it. So please don't add any test commits or anything like that, only real 
patches.
 
-All merges should be done using the 
-[dev/merge_spark_pr.py](https://github.com/apache/spark/blob/master/dev/merge_spark_pr.py)
 
-script, which squashes the pull request's changes into one commit. To use this 
script, you 
+Setting up Remotes
+
+To use the `merge_spark_pr.py` script described below, you 
 will need to add a git remote called `apache` at 
`https://github.com/apache/spark`, 
-as well as one called "apache-github" at `git://github.com/apache/spark`. For 
the `apache` repo, 
-you can authenticate using your ASF username and password. Ask 
`dev@spark.apache.org` if you have trouble with 
-this or want help doing your first merge.
+as well as one called `apache-github` at `git://github.com/apache/spark`.
+
+You will likely also have a remote `origin` pointing to your fork of Spark, and
+`upstream` pointing to the `apache/spark` GitHub repo. 
+
+If correct, your `git remote -v` should look like:
+
+```
+apache https://github.com/apache/spark.git (fetch)
+apache https://github.com/apache/spark.git (push)
+apache-github  git://github.com/apache/spark (fetch)
+apache-github  git://github.com/apache/spark (push)
+origin https://github.com/[your username]/spark.git (fetch)
+origin https://github.com/[your username]/spark.git (push)
+upstream   https://github.com/apache/spark.git (fetch)
+upstream   https://github.com/apache/spark.git (push)
+```
+
+For the `apache` repo, you will need to set up command-line authentication to 
GitHub. This may
+include setting up an SSH key and/or personal access token. See:
+
+- https://help.github.com/articles/connecting-to-github-with-ssh/
+- 
https://help.github.com/articles/creating-a-personal-access-token-for-the-command-line/
+
+Ask `dev@spark.apache.org` if you have trouble with these steps, or want help 
doing your first merge.
+
+Merge Script
+
+All merges should be done using the 
+[dev/merge_spark_pr.py](https://github.com/apache/spark/blob/master/dev/merge_spark_pr.py),
+which squashes the pull request's changes into one commit.
 
 The script is fairly self explanatory and walks you through steps and options 
interactively.
 
@@ -144,29 +172,12 @@ Then, in a separate window, modify the code and push a 
commit. Run `git rebase -
 You can verify the result is one change with `git log`. Then resume the script 
in the other window.
 
 Also, please remember to set Assignee on JIRAs where applicable when they are 
resolved. The script 
-can't do this automatically.
-Once a PR is merged please leave a comment on the PR stating which branch(es) 
it has been merged with.
+can do this automatically in most cases. However where the contributor is not 
yet a part of the
+Contributors group for the Spark project in ASF JIRA, it won't work until they 
are added. Ask
+an admin to add the person to Contributors at 
+https://issues.apache.org/jira/plugins/servlet/project-config/SPARK/roles .
 
-
+Once a PR is merged please leave a comment on the PR stating which branch(es) 
it has been merged with.
 
 Policy on Backporting Bug Fixes
 
diff --git a/site/committers.html b/site/committers.html
index 3771ba93a..3b3f47112 100644
--- a/site/committers.html
+++ b/site/committers.html
@@ -532,13 +532,42 @@ How to Merge a Pull Request
 Changes pushed to the master branch on Apache cannot be removed; that is, 
we can’t force-push to 
 it. So please don’t add any test commits or anything like that, only 
real patches.
 
-All merges should be done using the 
-https://github.com/apache/spark/blob/master/dev/merge_spark_pr.py";>dev/merge_spark_pr.py
 
-script, which squashes the pull request’s changes into one commit. To 
use this script, you 
+Setting up Remotes
+
+To use the merge_spark_pr.py script described below, you 
 will need to add a git remote called apache at 
https://github.com/apache/spark, 
-as well as one called “apache-github” at 
git://github.com/apache/spark. For the apache repo, 
-you can authenticate using your ASF username and password. Ask 
dev@spark.apache.org if you have trouble with 
-this or want help doing your first merge.
+as well as one called apache-github at 
git://github.com/apache/spark.
+
+You will likely also have

DataSourceV2 sync notes

[Ryan Blue]

Here are my notes from the DSv2 sync last night.

*As usual, I didn’t take great notes because I was participating in the
discussion. Feel free to send corrections or clarification.*

*Attendees*:
Ryan Blue
John Zhuge
Xiao Li
Reynold Xin
Felix Cheung
Anton Okolnychyi
Bruce Robbins
Dale Richardson
Dongjoon Hyun
Genliang Wang
Matt Cheah
Russel Spitzer
Wenchen Fan
Maryann Xue
Jacky Lee

*Topics*:

   - Passing query ID to DSv2 write builder
   - OutputMode in DSv2 writes
   - Catalog API framing to help clarify the discussion
   - Catalog API plugin system
   - Catlaog API approach (separate TableCatalog, FunctionCatalog,
   UDFCatalog, etc.)
   - Proposed TableCatalog API for catalog plugins (SPIP
   

   )
   - Migration to new Catalog API from ExternalCatalog
   - Proposed user-facing API for table catalog operations (SPIP
   

   )

*Notes*:

   - Passing query ID to DSv2 write builder
  - Wenchen: query ID is needed for some sources
  - Reynold: this may be useful for batch if batch ever supports
  recovery or resuming a write
  - Ryan: this may also be useful in sources to identify the outputs of
  a particular batch. Iceberg already generates a UUID for each write.
  - Wenchen proposed adding query ID to the newWriteBuilder factory
  method
  - Ryan: if args to newWriteBuilder will change over time, then it is
  easier to maintain them as builder methods
  - Reynold: what does the read side do? [Note: it passes no arguments
  to newScanBuilder]
  - No one knew, so we tabled further discussion
   - OutputMode
  - Wenchen: OutputMode instructs current sources how to handle rows
  written to them.
  - Reynold: discussed with Michael and OutputMode doesn’t need to be
  in this new public API.
  - Ryan: the problem with OutputMode, specifically Update, is that it
  is ambiguous like SaveMode. Update mode doesn’t tell a sink how
to identify
  the records to replace.
  - Reynold: nothing implements Update and it is only defined
  internally, only Append and Complete are needed
  - Ryan: that works for source implementations, but users can pass the
  mode in the API, so Spark should still throw an exception
  - Reynold added SupportsTruncate (similar to SupportsOverwrite) to
  the proposal to configure Complete mode.
  - Ryan added SupportsUpdate to the proposal for how Update mode could
  be configured
  - Discussion clarified that truncate, like overwrite, is not required
  to happen immediately when the builder is configured. This would allow
  sources to implement truncate and append in a single atomic operation for
  each epoch.
   - *Catalog API framing*: there are several issues to discuss. To make
   discussions more effective, let’s try to keep these areas separate:
  - A plugin system for catalog implementations
  - The approach to replace ExternalCatalog with multiple interfaces
  for specific purposes (TableCatalog, FunctionCatalog, etc.)
  - The proposed TableCatalog methods
  - How to migrate from ExternalCatalog to a new catalog API
  - A public API that can handle operations that are currently
  supported only in DDL
  - *Let’s try to keep these discussions separate to make progress on
  each area*
   - *TableCatalog API proposal*: an interface for plugins
   

   to expose create/load/alter/drop operations for tables.
  - Ryan provided a summary and pointed out that the part under
  discussion is the plugin API, TableCatalog, NOT the public API
  - Maryann: the proposed API uses 2-level identifiers, but many
  databases support 3 levels
  - Ryan: 2-level identifiers in a catalog is a reasonable
  simplification? Presto does this. A catalog could be used for
each entry in
  the top level in a 3-level database. Otherwise, would Spark support 4
  levels?
  - Someone?: identifiers should support arbitrary levels and the
  problem should be delegated to the catalog. Flexible table identifiers
  could be used to also support path-based tables in the same API
  - Consensus seemed to be around using arbitrary identifiers. [Will
  follow up with a DISCUSS thread]
  - Ryan: identifier is actually orthogonal to most of the proposal; it
  is enough that we assume there is some identifier. Let’s consider this
  proposal with TableIdentifier as a place-holder. Any other concerns about
  this API?
  - Maryann: what about replacing an entire schema [Note: Hive syntax
  supports this]. Alter table passes individual changes, which is difficult
  for some 

Re: [VOTE] SPARK 2.2.3 (RC1)

[Takeshi Yamamuro]

ok, thanks for the check.

best,
takeshi

On Fri, Jan 11, 2019 at 1:37 AM Dongjoon Hyun 
wrote:

> Hi, Takeshi.
>
> Yep. It's not a release blocker. We don't need that as Sean mentioned
> already.
> Since you are the release manager of 2.3.3, you may include that in the
> scope of Spark 2.3.3 before it starts.
>
> Bests,
> Dongjoon.
>
>
> On Thu, Jan 10, 2019 at 5:44 AM Sean Owen  wrote:
>
>> Is that the right link? that is marked as a minor bug, maybe. From
>> what you describe it's not a regression from 2.2.2 either.
>>
>> On Thu, Jan 10, 2019 at 6:37 AM Takeshi Yamamuro 
>> wrote:
>> >
>> > Hi, Dongjoon,
>> >
>> > We don't need to include https://github.com/apache/spark/pull/23456 in
>> this release?
>> > The query there fails in v2.x while it passes in v1.6.
>> >
>>
>

-- 
---
Takeshi Yamamuro

Re: [VOTE] SPARK 2.2.3 (RC1)

[Hyukjin Kwon]

+1

Thanks.

2019년 1월 11일 (금) 오전 7:01, Takeshi Yamamuro 님이 작성:

> ok, thanks for the check.
>
> best,
> takeshi
>
> On Fri, Jan 11, 2019 at 1:37 AM Dongjoon Hyun 
> wrote:
>
>> Hi, Takeshi.
>>
>> Yep. It's not a release blocker. We don't need that as Sean mentioned
>> already.
>> Since you are the release manager of 2.3.3, you may include that in the
>> scope of Spark 2.3.3 before it starts.
>>
>> Bests,
>> Dongjoon.
>>
>>
>> On Thu, Jan 10, 2019 at 5:44 AM Sean Owen  wrote:
>>
>>> Is that the right link? that is marked as a minor bug, maybe. From
>>> what you describe it's not a regression from 2.2.2 either.
>>>
>>> On Thu, Jan 10, 2019 at 6:37 AM Takeshi Yamamuro 
>>> wrote:
>>> >
>>> > Hi, Dongjoon,
>>> >
>>> > We don't need to include https://github.com/apache/spark/pull/23456
>>> in this release?
>>> > The query there fails in v2.x while it passes in v1.6.
>>> >
>>>
>>
>
> --
> ---
> Takeshi Yamamuro
>