On a related note, any objections to use
http://code.google.com/p/thread-weaver/ in our unit tests?
__Luke
On Fri, May 7, 2010 at 4:35 PM, Todd Lipcon wrote:
> Hi Cos,
>
> This looks great, and I'm excited to have more ways of finding these tricky
> bugs. Are there any examples of bugs found alr
Working on the new metrics framework
(https://issues.apache.org/jira/browse/HADOOP-6728 and its sub-tasks),
I'd like to invite some feedback from the community on how to evolve
Hadoop metrics. Here is a quick summary of changes:
1. Simplified metrics instrumentation
- Declarative annotations f
That's because of the missing jersey dependencies in hadoop-core pom
and that you're using the minicluster stuff that will pull in the
webhdfs classes. A work around would be adding jersey-* dependencies
(cf. hadoop ivy.xml) to your pom in test scope.
There is no perfect solution to this problem (
Hey Devaraj,
Thanks for testing IBM JDK. Can you do the one line fix for IBM JDK as
well for HADOOP-8251, as Todd only fixed Sun JDK. It's not blocker for
1.0.3. But much easier for you to submit a patch. I already have 14
patches in the queue waiting for our legal approval. Doing a code scan
for
Sorry, meant for Devaraj only :)
On Tue, May 15, 2012 at 2:20 PM, Luke Lu wrote:
> Hey Devaraj,
>
> Thanks for testing IBM JDK. Can you do the one line fix for IBM JDK as
> well for HADOOP-8251, as Todd only fixed Sun JDK. It's not blocker for
> 1.0.3. But much easier for y
-1, +1, -1.
If we want to introduce a "platform independent" scripting language, we
should not choose python, as it has a bad track record for compatibility
(between versions/platforms).
+1 to use groovy, as we can control the version of groovy jars included in
our distribution.
__Luke
On Mon,
Thanks for the heads up Mostafa. I've added commment/questions on the JIRA.
The main concern is that now you have to decide what metrics system to use
depending on the process is long running or not. Why not just enhance the
existing metrics system itself to support on-demand publishing? So that yo
Another more serious issue is MAPREDUCE-4478. It manifests as some TTs
burns CPU even if nothing is running. This can only be fixed by manually
restart the affected TTs.
This is a trivial patch. I think it would be good to include in all 1.x
branches.
Sorry for missing the vote. I was on vacation
Thanks for the voting thread. Otherwise, many committers would have missed
it.
I agree that this is a superset of code change that has larger impact than
typical code change.
On Thu, Nov 29, 2012 at 6:26 PM, Matt Foley wrote:
> > Finally, this is code change, so I'm not sure why we are doing a
v vars) to call the
cross-platform jruby scripts.
__Luke
On Fri, Nov 30, 2012 at 3:21 AM, Luke Lu wrote:
> Thanks for the voting thread. Otherwise, many committers would have missed
> it.
>
> I agree that this is a superset of code change that has larger impact than
> typical
On Fri, Nov 30, 2012 at 5:29 AM, Steve Loughran wrote:
> where do you set your env vars... and what about JAVA_HOME
>
There should be only two env vars (JAVA_HOME and HADOOP_HOME) to deal with
in the native scripts (.bat on windows and .sh on unix platforms) to
boostrap jruby scripts, which deal
On Fri, Nov 30, 2012 at 5:29 AM, Steve Loughran wrote:
> Yes, you need python at build time, but that's no worse than having a
> protoc
> compiler, gcc and the automake toolchain.
>
The problem is that python is known to have _backward_ compatibility issues
on various platforms. It would be very
+1.
On Mon, Mar 11, 2013 at 11:52 AM, Colin McCabe wrote:
> +1 (non-binding)
>
> best,
> Colin
>
>
> On Sun, Mar 10, 2013 at 8:38 PM, Matt Foley wrote:
>
> > Hi all,
> > I have created branch-1.2 from branch-1, and propose to cut the first
> > release candidate for 1.2.0 on Monday 3/18 (a week
If protocol compatibility of v2 and v3 is a goal, HADOOP-8990 should be a
blocker for v2.
__Luke
On Fri, Apr 26, 2013 at 12:07 PM, Eli Collins wrote:
> On Fri, Apr 26, 2013 at 11:15 AM, Arun C Murthy
> wrote:
> >
> > On Apr 25, 2013, at 7:31 PM, Roman Shaposhnik wrote:
> >
> >> On Thu, Apr 25,
+1. guava is the new apache commons, maintained by java experts with
comprehensive test coverage.
I also propose to deprecate any existing utils in hadoop-common with
duplicate functionality.
On Thu, Feb 10, 2011 at 10:25 PM, Jakob Homan wrote:
> +1
>
> On Thu, Feb 10, 2011 at 10:04 PM, Konstant
Chris had the exact the same problem and we discovered the same
reason/solution (ivy-2.0.0rc2.jar in $HOME/.ant/lib) by turning on ant
debug (ant -d -v). I didn't know that it's a prevalent issue though.
On Tue, May 24, 2011 at 10:50 PM, Todd Lipcon wrote:
> Hey folks,
>
> I've seen this error po
ld however mess up the current versioning system, but I think
would be good to support a stable version that doesn't require
patching on the end user/consumer side, especially when the patches
seem readily available, but not officially taged as a release.
Thanks,
Luke
dea goes.
Any takers or ideas on this already been kicked around? If for
anything else, people learning about hadoop have to reformat their
data when they're just testing different versions, which is a shame
for me :)
Luke
To work around the hadoop-project pom issue. Just use mvn install ...
instead of mvn package ...
On Mon, Oct 3, 2011 at 1:38 AM, Ravi Teja wrote:
> Hi Tim,
> You have set your JAVA_HOME to JRE(Java Runtime env) instead of setting it
> to JDK (Java Dev Kit). The tools jar is not available in the j
Try to do a mvn install -DskipTests -P-cbuild before mvn eclipse:eclipse..
On Mon, Oct 3, 2011 at 10:08 PM, Alexandre de Assis Bento Lima
wrote:
> Hi,
>
> I'm following the steps suggested by the wiki page:
>
> EclipseEnvironment - Hadoop Wiki
> (http://wiki.apache.org/hadoop/EclipseEnvironment)
The problem is due to relnotes.py generating the html containing some CRLF
(from JIRA) and the release manager not using git-svn, which caused the
html with mixed eol getting checked in. The problem would then manifest for
git users due to text=auto in .gitattributes (see HADOOP-8912) that auto
con
BTW, I just checked in the eol fix for the releasenotes.html, so git users
wouldn't be annoyed until the next relnotes.py run :)
On Sat, Jun 29, 2013 at 5:18 PM, Luke Lu wrote:
> The problem is due to relnotes.py generating the html containing some CRLF
> (from JIRA) and the rele
+1 (binding).
Verified digests and gpg signature. Started a non-secure single node
cluster; ran a few fs commands, some example mr jobs. Verified node-group
topology functional tests on a 4 physical host 8 virtual node cluster.
On Wed, Jul 24, 2013 at 2:08 PM, Matt Foley wrote:
> Colleagues,
>
I've verified that it's only a version issue (tested on a Ubuntu 12.04 VM)
: as long as you have 2.5.0 protoc, it works. BTW, the version check is a
little "too" strict. I had protobuf 2.5.1 (trunk) installed for 2.5 tests
and the exact check broke my build.
On Tue, Aug 13, 2013 at 3:29 PM, Steve
+1.
- Verified signatures and checksums of source and binary tarballs
- Deployed binary tarball on a 4 node virtual cluster with simple auth
- Ran a few hadoop-1 and 2 examples jobs successfully
On Wed, Oct 9, 2013 at 2:34 PM, Roman Shaposhnik wrote:
> On Mon, Oct 7, 2013 at 12:00 AM, Arun C M
1. File and Ganglia are the only bundled sinks, though there are
socket/json (for chukwa) and graphite sinks patches in the works.
2. Hadoop metrics (and metrics2) is mostly designed for system/process
metrics, which means you'll need to attach jconsole to your map/reduce task
processes to see your
Luke Lu created HADOOP-8418:
---
Summary: Fix UGI for IBM JDK running on Windows
Key: HADOOP-8418
URL: https://issues.apache.org/jira/browse/HADOOP-8418
Project: Hadoop Common
Issue Type: Bug
Luke Lu created HADOOP-8419:
---
Summary: GzipCodec NPE upon reset with IBM JDK
Key: HADOOP-8419
URL: https://issues.apache.org/jira/browse/HADOOP-8419
Project: Hadoop Common
Issue Type: Bug
Luke Lu created HADOOP-8425:
---
Summary: Upgrade commons-math version to 2.2
Key: HADOOP-8425
URL: https://issues.apache.org/jira/browse/HADOOP-8425
Project: Hadoop Common
Issue Type: Bug
Luke Lu created HADOOP-8515:
---
Summary: Upgrade Jetty to the current Jetty 7 release
Key: HADOOP-8515
URL: https://issues.apache.org/jira/browse/HADOOP-8515
Project: Hadoop Common
Issue Type
Luke Lu created HADOOP-8560:
---
Summary: Change TestFTPFileSystem to use non-SNAPSHOT dependencies
Key: HADOOP-8560
URL: https://issues.apache.org/jira/browse/HADOOP-8560
Project: Hadoop Common
Luke Lu created HADOOP-8561:
---
Summary: Introduce HADOOP_PROXY_USER for secure impersonation in
child hadoop client processes
Key: HADOOP-8561
URL: https://issues.apache.org/jira/browse/HADOOP-8561
Project
[
https://issues.apache.org/jira/browse/HADOOP-9051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Luke Lu resolved HADOOP-9051.
-
Resolution: Fixed
Fix Version/s: 1.1.2
1.2.0
Committed trivial patch to
[
https://issues.apache.org/jira/browse/HADOOP-9128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Luke Lu resolved HADOOP-9128.
-
Resolution: Fixed
Fix Version/s: 1.0.0
0.23.0
2.0.0-alpha
[
https://issues.apache.org/jira/browse/HADOOP-8389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Luke Lu resolved HADOOP-8389.
-
Resolution: Fixed
Fix Version/s: 1.0.0
0.23.0
2.0.0-alpha
Luke Lu created HADOOP-9160:
---
Summary: Change management protocol to JMX
Key: HADOOP-9160
URL: https://issues.apache.org/jira/browse/HADOOP-9160
Project: Hadoop Common
Issue Type: Improvement
Luke Lu created HADOOP-9194:
---
Summary: RPC Support for QoS
Key: HADOOP-9194
URL: https://issues.apache.org/jira/browse/HADOOP-9194
Project: Hadoop Common
Issue Type: New Feature
[
https://issues.apache.org/jira/browse/HADOOP-9395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Luke Lu resolved HADOOP-9395.
-
Resolution: Duplicate
Fix Version/s: (was: 1.1.3)
(was: 1.2.0)
Thanks
[
https://issues.apache.org/jira/browse/HADOOP-8817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Luke Lu resolved HADOOP-8817.
-
Resolution: Fixed
This jira is covered by JIRAs it "contains".
> Bac
[
https://issues.apache.org/jira/browse/HADOOP-9434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Luke Lu resolved HADOOP-9434.
-
Resolution: Fixed
Hadoop Flags: Reviewed
> Backport HADOOP-9267 to branc
Luke Lu created HADOOP-9630:
---
Summary: Remove IpcSerializationType
Key: HADOOP-9630
URL: https://issues.apache.org/jira/browse/HADOOP-9630
Project: Hadoop Common
Issue Type: Sub-task
Common portion of MAPREDUCE-1545
Key: HADOOP-6657
URL: https://issues.apache.org/jira/browse/HADOOP-6657
Project: Hadoop Common
Issue Type: Improvement
Reporter: Luke Lu
Assignee
Overhaul metric framework
-
Key: HADOOP-6728
URL: https://issues.apache.org/jira/browse/HADOOP-6728
Project: Hadoop Common
Issue Type: Improvement
Reporter: Luke Lu
Fix For: 0.20.3
Per
(Leopard)
Reporter: Luke Lu
Fix For: 0.22.0
TestNetUtils fails consistently after HADOOP-6722 on Mac OS X Leopard 10.5.8:
{{
- Standard Error -
local address: /127.0.0.1
local port: 64991
- ---
Testcase
Reporter: Luke Lu
The GlobFilter class (especially the error-prone setRegex method) is duplicated
in both FileContext and Filesystem. And I'd like to be able to reuse the most
of the setRegex code for glob pattern matching in metrics filter configurations.
We could at least f
j
-
Key: HADOOP-6918
URL: https://issues.apache.org/jira/browse/HADOOP-6918
Project: Hadoop Common
Issue Type: Improvement
Reporter: Luke Lu
--
This message is automatically generated by JIRA.
-
You can
Reporter: Luke Lu
Assignee: Luke Lu
Fix For: 0.22.0
The jira tracks the new metrics framework only changes, i.e., it doesn't track
the instrumentation changes (and compatibility issues) to existing code.
--
This message is automatically generated by JIRA.
-
Yo
Metrics2: metrics instrumentation
-
Key: HADOOP-6920
URL: https://issues.apache.org/jira/browse/HADOOP-6920
Project: Hadoop Common
Issue Type: Sub-task
Affects Versions: 0.20.2
Reporter: Luke
metrics2: metrics plugins
-
Key: HADOOP-6921
URL: https://issues.apache.org/jira/browse/HADOOP-6921
Project: Hadoop Common
Issue Type: Sub-task
Affects Versions: 0.20.2
Reporter: Luke Lu
Reporter: Luke Lu
Fix For: 0.22.0
The problem is introduced by HADOOP-6486. Which have caused a lot of
mysterious artifact issues (unable to downgrade or do parallel dev, without
wiping out both m2 and ivy caches etc.) wasting countless hours of dev (many
people's) time to track dow
: Luke Lu
Fix For: 0.23.0
Much of the Hadoop metrics and status info is available via JMX, especially
since 0.20.100, and 0.22+ (HDFS-1318, HADOOP-6728 etc.) For operations staff
not familiar JMX setup, especially JMX with SSL and firewall tunnelling, the
usage can be daunting. Using a
[
https://issues.apache.org/jira/browse/HADOOP-6918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Luke Lu resolved HADOOP-6918.
-
Resolution: Fixed
Fix Version/s: 0.23.0
Release Note: Harmonizing metrics names
Hadoop
[
https://issues.apache.org/jira/browse/HADOOP-6918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Luke Lu reopened HADOOP-6918:
-
> Make metrics naming consistent
> --
>
> Key:
: 0.21.0
Reporter: Luke Lu
Fix For: 0.23.0
Needed by mapreduce 2.0 avro support. Maybe we could jump to Avro 1.5. There is
incompatible API changes from 1.3x to 1.4x (Utf8 to CharSequence in user facing
APIs) not sure about 1.5x though.
--
This message is automatically
Deprecate metrics v1
Key: HADOOP-7266
URL: https://issues.apache.org/jira/browse/HADOOP-7266
Project: Hadoop Common
Issue Type: Sub-task
Reporter: Luke Lu
Assignee: Luke Lu
--
This message is
Move JMXGet from hdfs to common
---
Key: HADOOP-7273
URL: https://issues.apache.org/jira/browse/HADOOP-7273
Project: Hadoop Common
Issue Type: Improvement
Affects Versions: 0.21.0
Reporter: Luke
[
https://issues.apache.org/jira/browse/HADOOP-6921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Luke Lu resolved HADOOP-6921.
-
Resolution: Fixed
Fix Version/s: 0.23.0
Release Note: Metrics names are standardized to
[
https://issues.apache.org/jira/browse/HADOOP-6887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Luke Lu resolved HADOOP-6887.
-
Resolution: Fixed
Fix Version/s: 0.23.0
> Need a separate metrics per garbage collec
Reporter: Luke Lu
Assignee: Luke Lu
Priority: Minor
Fix For: 0.23.0
The TestSinkQueue is racy (Thread.yield is not enough to guarantee other
intended thread getting run), though it's the first time (from HADOOP-7289) I
saw it manifested here.
--
Components: metrics
Affects Versions: 0.23.0
Reporter: Luke Lu
Assignee: Luke Lu
Fix For: 0.23.0
Per experience and discussion with HDFS-1922, it seems preferable to treat
missing metrics config file as empty/default config, which is more compatible
with
[
https://issues.apache.org/jira/browse/HADOOP-6508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Luke Lu resolved HADOOP-6508.
-
Resolution: Fixed
Fix Version/s: 0.23.0
> Incorrect values for metrics with CompositeCont
: 0.20.203.0, 0.23.0
Reporter: Luke Lu
Fix For: 0.20.204.0, 0.23.0
Although, all metrics in metrics v2 are exposed via the standard JMX
mechanisms, there are a few users who prefer using Ganglia to collect metrics.
--
This message is automatically generated by JIRA.
For more
-7330
Project: Hadoop Common
Issue Type: Bug
Components: metrics
Affects Versions: 0.20.203.0
Reporter: Luke Lu
Assignee: Luke Lu
Fix For: 0.20.203.1
The MetricsSourceAdapter#getAttribute in 0.20.203 is returning the attribute
Components: build
Affects Versions: 0.23.0
Reporter: Luke Lu
Assignee: Luke Lu
Fix For: 0.23.0
In 0.20.x, only one jar (hadoop-core-*.jar) really matters. The
o.a.h.util.VersionInfo combined with saveVersion.sh script (generating a
package level annotation
Common
Issue Type: Improvement
Reporter: Luke Lu
Assignee: Luke Lu
Fix For: 0.23.0
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[
https://issues.apache.org/jira/browse/HADOOP-7311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Luke Lu resolved HADOOP-7311.
-
Resolution: Duplicate
Fix Version/s: 0.20.204.0
204 should be feature parity (and then some
Hadoop web UI 2.0
-
Key: HADOOP-7532
URL: https://issues.apache.org/jira/browse/HADOOP-7532
Project: Hadoop Common
Issue Type: Improvement
Affects Versions: 0.23.0
Reporter: Luke Lu
Assignee: Luke
Reporter: Luke Lu
Fix For: 0.23.0
HttpServer is a wrapper for servlet containers (JSR-154), which jetty is the
current implementation. HADOOP-7594 violates the abstraction by adding an
implementation specific method addJerseyResource which is tied to a specific
JSR-311 implementation
Components: fs
Reporter: Luke Lu
Fix For: 0.23.0
Much of the hadoop-common dependencies is from the fs implementations. We more
fs implementations on the way (ceph, lafs etc). I propose that we move all the
fs implementations to a hadoop-fs module under hadoop-common-project
[
https://issues.apache.org/jira/browse/HADOOP-7630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Luke Lu reopened HADOOP-7630:
-
> hadoop-metrics2.properties should have a property *.period set to a default
> value foe m
Luke Lu created HADOOP-9662:
---
Summary: More unit tests for RPC v9
Key: HADOOP-9662
URL: https://issues.apache.org/jira/browse/HADOOP-9662
Project: Hadoop Common
Issue Type: Improvement
Luke Lu created HADOOP-9663:
---
Summary: Nonblocking RPC client
Key: HADOOP-9663
URL: https://issues.apache.org/jira/browse/HADOOP-9663
Project: Hadoop Common
Issue Type: Improvement
Luke Lu created HADOOP-9664:
---
Summary: Documentation for RPC v9
Key: HADOOP-9664
URL: https://issues.apache.org/jira/browse/HADOOP-9664
Project: Hadoop Common
Issue Type: Improvement
Luke Lu created HADOOP-9683:
---
Summary: Wrap IpcConnectionContext in RPC headers
Key: HADOOP-9683
URL: https://issues.apache.org/jira/browse/HADOOP-9683
Project: Hadoop Common
Issue Type: Sub-task
[
https://issues.apache.org/jira/browse/HADOOP-9785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Luke Lu resolved HADOOP-9785.
-
Resolution: Duplicate
Fix Version/s: (was: 2.0.4-alpha)
(was: 3.0.0
Luke Lu created HADOOP-9871:
---
Summary: Fix intermittent findbug warnings in DefaultMetricsSystem
Key: HADOOP-9871
URL: https://issues.apache.org/jira/browse/HADOOP-9871
Project: Hadoop Common
[
https://issues.apache.org/jira/browse/HADOOP-10040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Luke Lu reopened HADOOP-10040:
--
Woah, this completely mess up git.
Short answer: you should svn propset windows file as eol-style
Luke Miner created HADOOP-14101:
---
Summary: MultiObjectDeleteException thrown when writing directly
to s3a
Key: HADOOP-14101
URL: https://issues.apache.org/jira/browse/HADOOP-14101
Project: Hadoop
Luke Miner created HADOOP-14161:
---
Summary: Failed to rename S3AFileStatus
Key: HADOOP-14161
URL: https://issues.apache.org/jira/browse/HADOOP-14161
Project: Hadoop Common
Issue Type: Bug
Luke Lu created HADOOP-11152:
Summary: Better random number generator
Key: HADOOP-11152
URL: https://issues.apache.org/jira/browse/HADOOP-11152
Project: Hadoop Common
Issue Type: Improvement
: metrics
Affects Versions: 1.0.0
Reporter: Luke Lu
Assignee: Luke Lu
For hysterical raisins, metrics2 code in hadoop 1.x branch and trunk diverged.
It looks like HBase needs to support both 1.x and 0.23+ for a while, hence the
more urgent need to clean up the situation
[
https://issues.apache.org/jira/browse/HADOOP-7211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Luke Lu resolved HADOOP-7211.
-
Resolution: Duplicate
Target Version/s: 1.0.3, 2.0.0
This jira is incorporated by the patches
: Hadoop Common
Issue Type: Improvement
Components: metrics
Affects Versions: 0.23.0
Reporter: Luke Lu
Fix For: 0.23.0
HADOOP-7734 expressed the desire to support downstream projects that depends on
both 0.23+ and 0.20-security releases.
In order to
83 matches
Mail list logo
