>
>
> maybe discuss having a list @ release time. As an example, s3 and
> encryption at rest shipped in beta stage... what's in 2.8 that "we don't
> yet trust ourselves?". Me, I'd put erasure coding in there just because
> I've no familiarity with it
>
> Quick clarification, EC isn't scheduled for
> On 25 Nov 2015, at 22:01, Vinod Kumar Vavilapalli wrote:
>
> Tx for your comments, Andrew!
>
> I did talk about it in a few discussions in the past related to this but yes,
> we never codified the feature-level alpha/beta tags. Part of the reason why I
> never pushed for such a codification
> On Nov 25, 2015, at 11:23 AM, Vinod Kumar Vavilapalli
> wrote:
>
> There are 40 odd incompatible changes in 3.x:
> https://issues.apache.org/jira/issues/?jql=project%20in%20%28HADOOP%2C%20YARN%2C%20HDFS%2C%20MAPREDUCE%29%20AND%20resolution%20%3D%20Fixed%20AND%20fixVersion%20%3D%203.0.0%20AND
SGTM, thanks Vinod! LMK if you need reviews on any of that.
Regarding the release checklist, another item I'd add is updating the
release notes in the project documentation, we've forgotten in the past.
On Wed, Nov 25, 2015 at 2:01 PM, Vinod Kumar Vavilapalli wrote:
> Tx for your comments, Andr
+1. Thanks, Vinod.
--Chris Nauroth
On 11/25/15, 1:45 PM, "Vinod Kumar Vavilapalli" wrote:
>Okay, tx for this clarification Chris! I dug more into this and now
>realized the actual scope of this. Given the the limited nature of this
>feature (non-Namenode etc) and the WIP nature of the large
Tx for your comments, Andrew!
I did talk about it in a few discussions in the past related to this but yes,
we never codified the feature-level alpha/beta tags. Part of the reason why I
never pushed for such a codification is that (a) it is a subjective decision
that the feature contributors us
Okay, tx for this clarification Chris! I dug more into this and now realized
the actual scope of this. Given the the limited nature of this feature
(non-Namenode etc) and the WIP nature of the larger umbrella HADOOP-11744, we
will ship the feature but I’ll stop calling this out as a notable feat
Hi Vinod,
The HDFS-8155 work is complete in branch-2 already, so feel free to
include it in the roadmap.
For those watching the thread that aren't familiar with HDFS-8155, I want
to call out that it was a client-side change only. The WebHDFS client is
capable of obtaining OAuth2 tokens and passi
Regarding interface visibility/stability, I'm aware of 2 relevant JIRAs
right now.
HADOOP-10776 proposes to mark Public some of the security plumbing like
the FileSystem delegation token methods, UserGroupInformation, Token and
Credentials. At this point, I think we are only fooling ourselves try
Hey Vinod,
I'm fine with the idea of alpha/beta marking in the abstract, but had a
question: do we define these terms in our compatibility policy or
elsewhere? I think it's commonly understood among us developers (alpha
means not fully tested and API unstable, beta means it's not fully tested
but
Hi Vinod,
Thanks for driving this. Can you add YARN-2573 which includes the work done
to integrate ReservationSystem with the RM failover mechanism to your list.
This can be reviewed and committed (branch-2) also about a month back.
Cheers,
Subru
On Wed, Nov 25, 2015 at 11:37 AM, Vinod Kumar Vav
I think we’ve converged at a high level w.r.t 2.8. And as I just sent out an
email, I updated the Roadmap wiki reflecting the same:
https://wiki.apache.org/hadoop/Roadmap
I plan to create a 2.8 branch EOD today.
The goal for all of us should be to restrict improvements & fixes to only (a)
the
Steve,
> There's a lot of stuff in 2.8; I note that I'd like to see the s3a perf
> improvements & openstack fixes in there: for which I need reviewers. I don't
> have the spare time to do this myself.
If you think they are useful, it helps to file tickets (or point out existing
tickets), star
There are 40 odd incompatible changes in 3.x:
https://issues.apache.org/jira/issues/?jql=project%20in%20%28HADOOP%2C%20YARN%2C%20HDFS%2C%20MAPREDUCE%29%20AND%20resolution%20%3D%20Fixed%20AND%20fixVersion%20%3D%203.0.0%20AND%20fixVersion%20not%20in%20%282.6.2%2C%202.6.3%2C%202.7.1%2C%202.7.2%2C%202
Haohui,
It’ll help to document this whole line of discussion about hdfs jar change and
its impact/non-impact for existing users so there is less confusion.
Thanks
+Vinod
> On Nov 11, 2015, at 3:26 PM, Haohui Mai wrote:
>
> bq. If and only if they take the Hadoop class path at face value.
> M
This is the current state from the feedback I gathered.
- Support priorities across applications within the same queue YARN-1963
— Can push as an alpha / beta feature per Sunil
- YARN-1197 Support changing resources of an allocated container:
— Can push as an alpha/beta feature per Wangda
I reviewed the current state of the YARN-2928 changes regarding its impact
if the timeline service v.2 is disabled. It does appear that there are a
lot of things that still do get created and enabled unconditionally
regardless of configuration. While this is understandable when we were
working to i
Did we consider cutting a branch-3 that borrows relatively compatible
patches from trunk to run the 3.x line? That said, I would like for us to
really tighten our compatibility policies and actually stick to them
starting the next major release.
On Wed, Nov 11, 2015 at 1:11 PM, Vinod Vavilapalli
I am really against the notion of calling x.y.0 releases alpha/beta; it is
very confusing. If we think a release is alpha/beta quality, why not
release it as x.y.0-alpha or x.y.0-beta, and follow it up eventually with
x.y.0 GA.
I am in favor of labeling any of the newer features to be of alpha/bet
Thank you Vinod for starting this discussion.
+1 for getting a beta/alpha version for Application Priority (YARN-1963).
Major patches are already in and MAPREDUCE-5870 (making MR also to use
app-priority) is in final review stages. This feature is also tested
in-house. With 2.8.0 alpha, I feel we
There's a lot of stuff in 2.8; I note that I'd like to see the s3a perf
improvements & openstack fixes in there: for which I need reviewers. I don't
have the spare time to do this myself.
I've already been building & testing both Apache Slider (incubating) and Apache
Spark against both 2.8.0-S
On Wed, Nov 11, 2015 at 12:13 PM, Vinod Vavilapalli wrote:
> — YARN Timeline Service Next generation: YARN-2928: Lots of momentum,
> but clearly a work in progress. Two options here
> — If it is safe to ship it into 2.8 in a disable manner, we can
> get the early code into trunk and a
bq. If and only if they take the Hadoop class path at face value.
Many applications don’t because of conflicting dependencies and
instead import specific jars.
We do make the assumptions that applications need to pick up all the
dependency (either automatically or manually). The situation is
simil
bq. currently pulling in hadoop-client gives downstream apps
hadoop-hdfs-client, but not hadoop-hdfs server side, right?
Right now hadoop-client pulls in hadoop-hdfs directly to ensure a
smooth transition. Maybe we can revisit the decision in the 2.9 / 3.x?
On Wed, Nov 11, 2015 at 3:00 PM, Steve
> On 11 Nov 2015, at 22:15, Haohui Mai wrote:
>
> bq. it basically makes the assumption that everyone recompiles for
> every minor release.
>
> I don't think that the statement holds. HDFS-6200 keeps classes in the
> same package. hdfs-client becomes a transitive dependency of the
> original h
> On Nov 11, 2015, at 2:15 PM, Haohui Mai wrote:
>
> bq. it basically makes the assumption that everyone recompiles for
> every minor release.
>
> I don't think that the statement holds. HDFS-6200 keeps classes in the
> same package. hdfs-client becomes a transitive dependency of the
> origina
bq. it basically makes the assumption that everyone recompiles for
every minor release.
I don't think that the statement holds. HDFS-6200 keeps classes in the
same package. hdfs-client becomes a transitive dependency of the
original hdfs jar.
Applications continue to work without recompilation a
> On Nov 11, 2015, at 1:11 PM, Vinod Vavilapalli
> wrote:
>
> I’ll let others comment on specific features.
>
> Regarding the 3.x vs 2.x point, as I noted before on other threads, given all
> the incompatibilities in trunk it will be ways off before users can run their
> production workloads
Thanks to Vinod for starting this discussion!
+1 to add YARN-1197 (container resizing) to 2.8.0, it is end-to-end tested.
I'd prefer to push it as an Alpha feature before wilder testing.
And also agree to call first release of 2.8 as an Alpha release according
to the number of new features / code
On Nov 11, 2015, at 12:13, Vinod Vavilapalli
mailto:vino...@hortonworks.com>> wrote:
— YARN Timeline Service v1.5 - YARN-4233: A short term bridge before
YARN-2928 comes around. I think this should go in given the tremendous activity
recently.
+1, let’s target ATS v1.5 work to 2.8. Most c
I’ll let others comment on specific features.
Regarding the 3.x vs 2.x point, as I noted before on other threads, given all
the incompatibilities in trunk it will be ways off before users can run their
production workloads on a 3.x release. Therefore, as I was proposing before, we
should contin
> On Nov 11, 2015, at 12:13 PM, Vinod Vavilapalli
> wrote:
>
>— HDFS-6200 Create a separate jar for hdfs-client: Compatible improvement
> - no dimension of alpha/betaness here.
IMO: this feels like a massive break in backwards compatibility. Anyone
who is looking for specific met
Agreed on not mixing this with major release discussions.
Okay, I just finished my review of 2.8 content.
A quick summary follows.
Current state of originally planned items
- Nearly Done / Done and so need to close down quickly
— Support *both* JDK7 and JDK8 runtimes HADOOP-11090
— Sup
I think it makes sense to have a 2.8 release since there are a
tremendous number of JIRAs in 2.8 that are not in 2.7. Doing a 3.x
release seems like something we should consider separately since it
would not have the same compatibility guarantees as a 2.8 release.
There's a pretty big delta betwee
With two active sustaining branches (2.6, 2.7), what would you think
of releasing trunk as 3.x instead of pushing 2.8? There are many new
features (EC, Y1197, etc.), and trunk could be the source of several
alpha/beta releases before we fork the 3.x line. -C
On Sat, Sep 26, 2015 at 12:49 PM, Vinod
As you may have noted, 2.8.0 got completely derailed what with 2.7.x and the
unusually long 2.6.1 release.
With 2.6.1 out of the way, and two parallel threads in progress for 2.6.2 and
2.7.2, it’s time for us to look back at where we are with Hadoop 2.8.
I’ll do a quick survey of where the indi
e.org; mapreduce-...@hadoop.apache.org
> Cc: vino...@apache.org
> Subject: [DISCUSS] Looking to a 2.8.0 release
>
> With 2.7.0 out of the way, and with more maintenance releases to stabilize
> it, I propose we start thinking about 2.8.0.
>
> Here's my first cut of the pro
Sure, I agree it's better to have clear guidelines and scheme. Let me fork this
thread about that.
Re 2.7.0, I just forgot about the naming initially though I was clear in the
discussion/voting. I so had to end up calling it alpha outside of the release
artifact naming.
Thanks
+Vinod
On Apr
Thanks for your comment Karthik. Here's my take.
Feature based release is going to put us back into the same prolonged release
cycle. That is why I am proposing that we look at 2-3 months time and get
whatever is ready. If we don't have even a single feature in by then, clearly
we can drop the
I would also like to support Karthik's proposal on the release thread about
version numbering. 2.7.0 being "alpha" is pretty confusing since all of the
other 2.x releases since GA have been stable. I think users would prefer a
version like "2.8.0-alpha1" instead, which is very clear and similar to
The feature set here seems pretty long, even for 2 - 3 months. Can we come
up with a minimum set of features (or a number of features) that justify a
new minor release, and start stabilizing as soon as those are in?
On Tue, Apr 21, 2015 at 2:39 PM, Vinod Kumar Vavilapalli wrote:
> With 2.7.0 out
With 2.7.0 out of the way, and with more maintenance releases to stabilize
it, I propose we start thinking about 2.8.0.
Here's my first cut of the proposal, will update the Roadmap wiki.
- Support *both* JDK7 and JDK8 runtimes: HADOOP-11090
- Compatibility tools to catch backwards, forwards comp
42 matches
Mail list logo
