ctubbsii commented on code in PR #454:
URL: https://github.com/apache/accumulo-website/pull/454#discussion_r2036030116
##########
_posts/release/2025-05-01-accumulo-2.1.4.md:
##########
Review Comment:
Because of the quirky way Jekyll organizes posts, this page won't even be
rendered (not even as a draft) if the date is in the future. It needs to have a
date in the past on the filename. The file can be renamed upon release, when it
is no longer draft.
##########
_posts/release/2025-05-01-accumulo-2.1.4.md:
##########
@@ -0,0 +1,139 @@
+---
+title: Apache Accumulo 2.1.4
+sortableversion: '02.01.04'
+draft: true
+LTM: true
+---
+## About
+
+Apache Accumulo 2.1.4 is a patch release of the 2.1 LTM line. It contains bug
+fixes and minor enhancements. This version supersedes 2.1.3. Users upgrading to
+2.1 should upgrade directly to this version instead of 2.1.3.
+
+## Notable Changes
+
+* {% ghi 5446 %} Many dependencies were updated, to include protobuf-java which
+ was updated from 3.22.0 to 3.25.6 due to a vulnerability. Users may run into
+ a runtime exception added in
https://github.com/protocolbuffers/protobuf/pull/20084
+ when using old generated Protobuf files. Users are advised to regenerate
their
+ protobuf files using protobuf 25.6 or later. Alternatively, users can set a
+ system property (see #20084 for details) to suppress the exception.
+
+* {% ghi 5073 %} {% ghi 5034 %} Added `accumulo check-accumulo-properties`
command that can be
+ run before an instance is initialized to check the properties file.
+
Review Comment:
We don't need a blank line between each bullet. I think that creates
separate lists in Markdown. We want one list, so no blank lines between the
bullets.
##########
_posts/release/2025-05-01-accumulo-2.1.4.md:
##########
@@ -0,0 +1,139 @@
+---
+title: Apache Accumulo 2.1.4
+sortableversion: '02.01.04'
+draft: true
+LTM: true
+---
+## About
+
+Apache Accumulo 2.1.4 is a patch release of the 2.1 LTM line. It contains bug
+fixes and minor enhancements. This version supersedes 2.1.3. Users upgrading to
+2.1 should upgrade directly to this version instead of 2.1.3.
+
+## Notable Changes
+
+* {% ghi 5446 %} Many dependencies were updated, to include protobuf-java which
+ was updated from 3.22.0 to 3.25.6 due to a vulnerability. Users may run into
+ a runtime exception added in
https://github.com/protocolbuffers/protobuf/pull/20084
+ when using old generated Protobuf files. Users are advised to regenerate
their
+ protobuf files using protobuf 25.6 or later. Alternatively, users can set a
+ system property (see #20084 for details) to suppress the exception.
+
+* {% ghi 5073 %} {% ghi 5034 %} Added `accumulo check-accumulo-properties`
command that can be
+ run before an instance is initialized to check the properties file.
+
+* {% ghi 5193 %} Added `accumulo admin signalShutdown` command to signal the
server
+ process to initiate a graceful shutdown. Compactors will finish the major
compaction
+ that they are currently working on, then will exit. Scan Servers will return
a busy
+ signal to all clients for new scan requests, and will shutdown when all
in-progress
+ scans have closed. Tablet Servers will signal the Manager that they are
shutting down,
+ which should prevent assignment of tablets to that server, then they will
unload all
+ tablets and shut down. The Monitor, Manager, GarbageCollector, and
CompactionCoordinator
+ will shut down also. The last step in the shutdown process for all servers
is to remove
+ their lock in ZooKeeper.
+
+* {% ghi 5438 %} Added `accumulo upgrade --prepare` command which should be
used after
+ shutting down an instance in preparation for an upgrade. This will check
that no Fate
+ transactions exist, delete any ZooKeeper locks for server processes, and
prohibit any
+ server processes from being started.
+
+* {% ghi 4898 %} New way to compute bulk load plans (TODO: Needs text)
+
+* {% ghi 5169 %} {% ghi 5170 %} The TabletServer will halt itself when a walog
write or
+ minc failure occurs and the TabletServer lock is not held in ZooKeeper.
+
+* {% ghi 5145 %} {% ghi 5132 %} The Manager and TabletServer processes now
have a background
+ thread that can be enabled via the property
`general.server.lock.verification.interval` to
+ validate that the process is holding the lock in ZooKeeper. The existing
mechanism of relying
+ on a Watcher to notify the process that the lock has been lost can be a
problem due to the
+ fact that there is a single thread in the ZooKeeper client that fires
Watcher events. If the
+ thread is currently waiting on a hung Watcher, then subsequent Watcher
events will not fire
+ and could leave the server running without a lock.
+
+* {% ghi 5174 %} Improvements to the `accumulo-cluster` and `accumulo-service`
scripts have been
+ backported from the main branch. This includes syntax changes.
Review Comment:
This may require reviewing accumulo-env.sh if users were using some of the
syntax in there for these scripts.
##########
_posts/release/2025-05-01-accumulo-2.1.4.md:
##########
@@ -0,0 +1,139 @@
+---
+title: Apache Accumulo 2.1.4
+sortableversion: '02.01.04'
+draft: true
+LTM: true
+---
+## About
+
+Apache Accumulo 2.1.4 is a patch release of the 2.1 LTM line. It contains bug
+fixes and minor enhancements. This version supersedes 2.1.3. Users upgrading to
+2.1 should upgrade directly to this version instead of 2.1.3.
+
+## Notable Changes
+
+* {% ghi 5446 %} Many dependencies were updated, to include protobuf-java which
+ was updated from 3.22.0 to 3.25.6 due to a vulnerability. Users may run into
+ a runtime exception added in
https://github.com/protocolbuffers/protobuf/pull/20084
+ when using old generated Protobuf files. Users are advised to regenerate
their
+ protobuf files using protobuf 25.6 or later. Alternatively, users can set a
+ system property (see #20084 for details) to suppress the exception.
+
+* {% ghi 5073 %} {% ghi 5034 %} Added `accumulo check-accumulo-properties`
command that can be
+ run before an instance is initialized to check the properties file.
+
+* {% ghi 5193 %} Added `accumulo admin signalShutdown` command to signal the
server
Review Comment:
camelCase in command-line options are really annoying, I think. We should
rename this to `signal-shutdown` or just `shutdown`.
##########
_posts/release/2025-05-01-accumulo-2.1.4.md:
##########
@@ -0,0 +1,139 @@
+---
+title: Apache Accumulo 2.1.4
+sortableversion: '02.01.04'
+draft: true
+LTM: true
+---
+## About
+
+Apache Accumulo 2.1.4 is a patch release of the 2.1 LTM line. It contains bug
+fixes and minor enhancements. This version supersedes 2.1.3. Users upgrading to
+2.1 should upgrade directly to this version instead of 2.1.3.
+
+## Notable Changes
+
+* {% ghi 5446 %} Many dependencies were updated, to include protobuf-java which
+ was updated from 3.22.0 to 3.25.6 due to a vulnerability. Users may run into
+ a runtime exception added in
https://github.com/protocolbuffers/protobuf/pull/20084
+ when using old generated Protobuf files. Users are advised to regenerate
their
+ protobuf files using protobuf 25.6 or later. Alternatively, users can set a
+ system property (see #20084 for details) to suppress the exception.
+
+* {% ghi 5073 %} {% ghi 5034 %} Added `accumulo check-accumulo-properties`
command that can be
+ run before an instance is initialized to check the properties file.
+
+* {% ghi 5193 %} Added `accumulo admin signalShutdown` command to signal the
server
+ process to initiate a graceful shutdown. Compactors will finish the major
compaction
+ that they are currently working on, then will exit. Scan Servers will return
a busy
+ signal to all clients for new scan requests, and will shutdown when all
in-progress
+ scans have closed. Tablet Servers will signal the Manager that they are
shutting down,
+ which should prevent assignment of tablets to that server, then they will
unload all
+ tablets and shut down. The Monitor, Manager, GarbageCollector, and
CompactionCoordinator
+ will shut down also. The last step in the shutdown process for all servers
is to remove
+ their lock in ZooKeeper.
+
+* {% ghi 5438 %} Added `accumulo upgrade --prepare` command which should be
used after
+ shutting down an instance in preparation for an upgrade. This will check
that no Fate
+ transactions exist, delete any ZooKeeper locks for server processes, and
prohibit any
+ server processes from being started.
+
+* {% ghi 4898 %} New way to compute bulk load plans (TODO: Needs text)
+
+* {% ghi 5169 %} {% ghi 5170 %} The TabletServer will halt itself when a walog
write or
+ minc failure occurs and the TabletServer lock is not held in ZooKeeper.
+
+* {% ghi 5145 %} {% ghi 5132 %} The Manager and TabletServer processes now
have a background
+ thread that can be enabled via the property
`general.server.lock.verification.interval` to
+ validate that the process is holding the lock in ZooKeeper. The existing
mechanism of relying
+ on a Watcher to notify the process that the lock has been lost can be a
problem due to the
+ fact that there is a single thread in the ZooKeeper client that fires
Watcher events. If the
+ thread is currently waiting on a hung Watcher, then subsequent Watcher
events will not fire
+ and could leave the server running without a lock.
+
+* {% ghi 5174 %} Improvements to the `accumulo-cluster` and `accumulo-service`
scripts have been
+ backported from the main branch. This includes syntax changes.
+
+* {% ghi 5384 %} {% ghi 5386 %} Fixed an issue with multiple threads
performing AES decryption leading to corrupt data.
+
+### Configurable Improvements
+
+Many notable improvements have been added that include a means for users to
configure them via new
+or existing properties to improve system performance or behavior. These
include:
+
+* {% ghi 4723 %} {% ghi 5239 %} Changes to properties
`tserver.session.idle.max` and
+ `tserver.session.update.idle.max` no longer require a TabletServer restart.
+
+* {% ghi 5397 %} {% ghi 5399 %} Added property `gc.threads.delete.wal` to
control the number of threads to use for
+ deleting write-ahead logs and recovery files.
+
+### Notable Bug Fixes
+
+* {% ghi 5033 %} {% ghi 5038 %} Group names in cluster.yaml must conform to
bash variable name rules
+ or an error will be raised when parsing the cluster.yaml file.
+
+* {% ghi 5221 %} The shell will now print to stdout, instead of stderr, when
there is no terminal set. This
+ will help with piping the output of the shell to other commands such as grep.
+
+* {% ghi 5341 %} Created an optimization in Bulk Import v2 to enable faster
processing of bulk import files
+ when loading into a sparse set of tablets in a large table. See new table
property `table.bulk.metadata.skip.distance`.
+
+* {% ghi 5358 %} Fixed issue with balancing metadata tablets.
+
+* {% ghi 5396 %} Modified `accumulo-cluster` to stop GC and ScanServer
processes first when stopping the
+ cluster. These processes write to the metadata table, which can lead to
slower shutdown times.
+
+* {% ghi 4868 %} {% ghi 4871 %} Fixed listscans so that it shows a scan
session id for batch scans instead
+ of always showing zero.
+
+* {% ghi 4845 %} InstanceOperations.getActiveCompaction(String) did not handle
the address of a
+ Compactor process being passed in the parameter. This has been fixed.
+
+* {% ghi 5445 %} The `cf` and `cq` options for the DeleteMany and Grep shell
commands were not being
+ handled properly.
+
+### Metrics Improvements
+
+* {% ghi 4756 %} {% ghi 4757 %} {% ghi 4840 %} Added metric to indicate how
many zombie scan threads are running
+ in the TabletServer.
+
+* {% ghi 5011 %} Added queue tag to metrics emitted from the External
Compactor processes.
+
+* {% ghi 5025 %} {% ghi 4922 %} Added property
`general.micrometer.log.metrics` to enable metrics on
+ the log4j2 or logback logging frameworks. Disabled by default.
+
+### Other Improvements
+
+* {% ghi 4755 %} {% ghi 5220 %} Fixed the help flag for the admin command so
that it works for subcommands.
+
+* {% ghi 4819 %} Tablets that are closing will no longer wait on scans to
complete. Instead they
+ will try to interrupt the scan and then continue closing the tablet so that
it can be migrated.
+ It's possible that a zombie scan thread could remain in the TabletServer, if
the scan thread is
+ blocked on something and does not die. Users should monitor the new zombie
scan metric mentioned
+ in the section above.
+
+* {% ghi 4867 %} Lowered the memory burden of listing a large number of
External Compactors in the Monitor.
+
+* {% ghi 5026 %} Modified CompactionJobPrioritizer.createPriority to give a
higher priority to tablets that
+ have more files than the maximum number of files per tablet.
+
+* {% ghi 5400 %} Modified AESCryptoService to prevent the creation of
overlapping streams.
+
+## Requirements
+
+Accumulo 2.1.4 now requires JDK 17 to build, but still supports Java 11
runtime.
+
+## Upgrading
+
+View the [Upgrading Accumulo documentation][upgrade] for guidance.
+
+## Useful Links
+
+* [All Changes since 2.1.3][all-changes]
+* [All tickets related to this release][milestone]
+
+This release also contains bug fixes from 1.10.4, which was released after
2.1.2.
Review Comment:
Can omit this line. There are no other bug fixes from 1.x that users need to
reference to know what's in this this.
```suggestion
```
##########
_posts/release/2025-05-01-accumulo-2.1.4.md:
##########
@@ -0,0 +1,139 @@
+---
+title: Apache Accumulo 2.1.4
+sortableversion: '02.01.04'
+draft: true
+LTM: true
+---
+## About
+
+Apache Accumulo 2.1.4 is a patch release of the 2.1 LTM line. It contains bug
+fixes and minor enhancements. This version supersedes 2.1.3. Users upgrading to
+2.1 should upgrade directly to this version instead of 2.1.3.
Review Comment:
It looks like I started using the phrase "supersedes" for 2.1.1, to
emphasize the part about upgrading directly to 2.1.1 instead of to 2.1.0 first,
which some people might think. After that, it was carried forward as-is. I'm
not sure it aged well, and doesn't quite match the message I was going for
anymore.
I think it should say:
```suggestion
fixes and minor enhancements. This version supersedes earlier 2.1 releases
and users upgrading to 2.1 should upgrade directly to this version instead of
any prior 2.1 release.
```
##########
_posts/release/2025-05-01-accumulo-2.1.4.md:
##########
@@ -0,0 +1,139 @@
+---
+title: Apache Accumulo 2.1.4
+sortableversion: '02.01.04'
+draft: true
+LTM: true
+---
+## About
+
+Apache Accumulo 2.1.4 is a patch release of the 2.1 LTM line. It contains bug
+fixes and minor enhancements. This version supersedes 2.1.3. Users upgrading to
+2.1 should upgrade directly to this version instead of 2.1.3.
+
+## Notable Changes
+
+* {% ghi 5446 %} Many dependencies were updated, to include protobuf-java which
+ was updated from 3.22.0 to 3.25.6 due to a vulnerability. Users may run into
+ a runtime exception added in
https://github.com/protocolbuffers/protobuf/pull/20084
+ when using old generated Protobuf files. Users are advised to regenerate
their
+ protobuf files using protobuf 25.6 or later. Alternatively, users can set a
+ system property (see #20084 for details) to suppress the exception.
+
+* {% ghi 5073 %} {% ghi 5034 %} Added `accumulo check-accumulo-properties`
command that can be
+ run before an instance is initialized to check the properties file.
+
+* {% ghi 5193 %} Added `accumulo admin signalShutdown` command to signal the
server
+ process to initiate a graceful shutdown. Compactors will finish the major
compaction
+ that they are currently working on, then will exit. Scan Servers will return
a busy
+ signal to all clients for new scan requests, and will shutdown when all
in-progress
+ scans have closed. Tablet Servers will signal the Manager that they are
shutting down,
+ which should prevent assignment of tablets to that server, then they will
unload all
+ tablets and shut down. The Monitor, Manager, GarbageCollector, and
CompactionCoordinator
+ will shut down also. The last step in the shutdown process for all servers
is to remove
+ their lock in ZooKeeper.
+
+* {% ghi 5438 %} Added `accumulo upgrade --prepare` command which should be
used after
+ shutting down an instance in preparation for an upgrade. This will check
that no Fate
+ transactions exist, delete any ZooKeeper locks for server processes, and
prohibit any
+ server processes from being started.
+
+* {% ghi 4898 %} New way to compute bulk load plans (TODO: Needs text)
+
+* {% ghi 5169 %} {% ghi 5170 %} The TabletServer will halt itself when a walog
write or
+ minc failure occurs and the TabletServer lock is not held in ZooKeeper.
+
+* {% ghi 5145 %} {% ghi 5132 %} The Manager and TabletServer processes now
have a background
+ thread that can be enabled via the property
`general.server.lock.verification.interval` to
+ validate that the process is holding the lock in ZooKeeper. The existing
mechanism of relying
+ on a Watcher to notify the process that the lock has been lost can be a
problem due to the
+ fact that there is a single thread in the ZooKeeper client that fires
Watcher events. If the
+ thread is currently waiting on a hung Watcher, then subsequent Watcher
events will not fire
+ and could leave the server running without a lock.
+
+* {% ghi 5174 %} Improvements to the `accumulo-cluster` and `accumulo-service`
scripts have been
+ backported from the main branch. This includes syntax changes.
+
+* {% ghi 5384 %} {% ghi 5386 %} Fixed an issue with multiple threads
performing AES decryption leading to corrupt data.
+
+### Configurable Improvements
+
+Many notable improvements have been added that include a means for users to
configure them via new
+or existing properties to improve system performance or behavior. These
include:
+
+* {% ghi 4723 %} {% ghi 5239 %} Changes to properties
`tserver.session.idle.max` and
+ `tserver.session.update.idle.max` no longer require a TabletServer restart.
+
+* {% ghi 5397 %} {% ghi 5399 %} Added property `gc.threads.delete.wal` to
control the number of threads to use for
+ deleting write-ahead logs and recovery files.
+
+### Notable Bug Fixes
+
+* {% ghi 5033 %} {% ghi 5038 %} Group names in cluster.yaml must conform to
bash variable name rules
+ or an error will be raised when parsing the cluster.yaml file.
+
+* {% ghi 5221 %} The shell will now print to stdout, instead of stderr, when
there is no terminal set. This
+ will help with piping the output of the shell to other commands such as grep.
+
+* {% ghi 5341 %} Created an optimization in Bulk Import v2 to enable faster
processing of bulk import files
+ when loading into a sparse set of tablets in a large table. See new table
property `table.bulk.metadata.skip.distance`.
Review Comment:
This is in the bugfix section, but should be in a different section...
perhaps the configuration section.
##########
_posts/release/2025-05-01-accumulo-2.1.4.md:
##########
@@ -0,0 +1,139 @@
+---
+title: Apache Accumulo 2.1.4
+sortableversion: '02.01.04'
+draft: true
+LTM: true
+---
+## About
+
+Apache Accumulo 2.1.4 is a patch release of the 2.1 LTM line. It contains bug
+fixes and minor enhancements. This version supersedes 2.1.3. Users upgrading to
+2.1 should upgrade directly to this version instead of 2.1.3.
+
+## Notable Changes
+
+* {% ghi 5446 %} Many dependencies were updated, to include protobuf-java which
+ was updated from 3.22.0 to 3.25.6 due to a vulnerability. Users may run into
+ a runtime exception added in
https://github.com/protocolbuffers/protobuf/pull/20084
+ when using old generated Protobuf files. Users are advised to regenerate
their
+ protobuf files using protobuf 25.6 or later. Alternatively, users can set a
+ system property (see #20084 for details) to suppress the exception.
+
+* {% ghi 5073 %} {% ghi 5034 %} Added `accumulo check-accumulo-properties`
command that can be
+ run before an instance is initialized to check the properties file.
+
+* {% ghi 5193 %} Added `accumulo admin signalShutdown` command to signal the
server
+ process to initiate a graceful shutdown. Compactors will finish the major
compaction
+ that they are currently working on, then will exit. Scan Servers will return
a busy
+ signal to all clients for new scan requests, and will shutdown when all
in-progress
+ scans have closed. Tablet Servers will signal the Manager that they are
shutting down,
+ which should prevent assignment of tablets to that server, then they will
unload all
+ tablets and shut down. The Monitor, Manager, GarbageCollector, and
CompactionCoordinator
+ will shut down also. The last step in the shutdown process for all servers
is to remove
+ their lock in ZooKeeper.
+
+* {% ghi 5438 %} Added `accumulo upgrade --prepare` command which should be
used after
+ shutting down an instance in preparation for an upgrade. This will check
that no Fate
+ transactions exist, delete any ZooKeeper locks for server processes, and
prohibit any
+ server processes from being started.
+
+* {% ghi 4898 %} New way to compute bulk load plans (TODO: Needs text)
+
+* {% ghi 5169 %} {% ghi 5170 %} The TabletServer will halt itself when a walog
write or
+ minc failure occurs and the TabletServer lock is not held in ZooKeeper.
+
+* {% ghi 5145 %} {% ghi 5132 %} The Manager and TabletServer processes now
have a background
+ thread that can be enabled via the property
`general.server.lock.verification.interval` to
+ validate that the process is holding the lock in ZooKeeper. The existing
mechanism of relying
+ on a Watcher to notify the process that the lock has been lost can be a
problem due to the
+ fact that there is a single thread in the ZooKeeper client that fires
Watcher events. If the
+ thread is currently waiting on a hung Watcher, then subsequent Watcher
events will not fire
+ and could leave the server running without a lock.
+
+* {% ghi 5174 %} Improvements to the `accumulo-cluster` and `accumulo-service`
scripts have been
+ backported from the main branch. This includes syntax changes.
+
+* {% ghi 5384 %} {% ghi 5386 %} Fixed an issue with multiple threads
performing AES decryption leading to corrupt data.
+
+### Configurable Improvements
+
+Many notable improvements have been added that include a means for users to
configure them via new
+or existing properties to improve system performance or behavior. These
include:
+
+* {% ghi 4723 %} {% ghi 5239 %} Changes to properties
`tserver.session.idle.max` and
+ `tserver.session.update.idle.max` no longer require a TabletServer restart.
+
+* {% ghi 5397 %} {% ghi 5399 %} Added property `gc.threads.delete.wal` to
control the number of threads to use for
+ deleting write-ahead logs and recovery files.
+
+### Notable Bug Fixes
+
+* {% ghi 5033 %} {% ghi 5038 %} Group names in cluster.yaml must conform to
bash variable name rules
+ or an error will be raised when parsing the cluster.yaml file.
+
+* {% ghi 5221 %} The shell will now print to stdout, instead of stderr, when
there is no terminal set. This
+ will help with piping the output of the shell to other commands such as grep.
+
+* {% ghi 5341 %} Created an optimization in Bulk Import v2 to enable faster
processing of bulk import files
+ when loading into a sparse set of tablets in a large table. See new table
property `table.bulk.metadata.skip.distance`.
+
+* {% ghi 5358 %} Fixed issue with balancing metadata tablets.
Review Comment:
I'd probably skip mentioning this one. It's kind of down in the weeds, and
doesn't really inform the user of anything specific that they need to do. If we
keep it, it might be useful to mention the kinds of balancing issues users
might have seen that would have been fixed by this. If it's not something
they're likely to have noticed, I'd skip it.
##########
_posts/release/2025-05-01-accumulo-2.1.4.md:
##########
@@ -0,0 +1,139 @@
+---
+title: Apache Accumulo 2.1.4
+sortableversion: '02.01.04'
+draft: true
+LTM: true
+---
+## About
+
+Apache Accumulo 2.1.4 is a patch release of the 2.1 LTM line. It contains bug
+fixes and minor enhancements. This version supersedes 2.1.3. Users upgrading to
+2.1 should upgrade directly to this version instead of 2.1.3.
+
+## Notable Changes
+
+* {% ghi 5446 %} Many dependencies were updated, to include protobuf-java which
+ was updated from 3.22.0 to 3.25.6 due to a vulnerability. Users may run into
+ a runtime exception added in
https://github.com/protocolbuffers/protobuf/pull/20084
+ when using old generated Protobuf files. Users are advised to regenerate
their
+ protobuf files using protobuf 25.6 or later. Alternatively, users can set a
+ system property (see #20084 for details) to suppress the exception.
+
+* {% ghi 5073 %} {% ghi 5034 %} Added `accumulo check-accumulo-properties`
command that can be
+ run before an instance is initialized to check the properties file.
+
+* {% ghi 5193 %} Added `accumulo admin signalShutdown` command to signal the
server
+ process to initiate a graceful shutdown. Compactors will finish the major
compaction
+ that they are currently working on, then will exit. Scan Servers will return
a busy
+ signal to all clients for new scan requests, and will shutdown when all
in-progress
+ scans have closed. Tablet Servers will signal the Manager that they are
shutting down,
+ which should prevent assignment of tablets to that server, then they will
unload all
+ tablets and shut down. The Monitor, Manager, GarbageCollector, and
CompactionCoordinator
+ will shut down also. The last step in the shutdown process for all servers
is to remove
+ their lock in ZooKeeper.
+
+* {% ghi 5438 %} Added `accumulo upgrade --prepare` command which should be
used after
+ shutting down an instance in preparation for an upgrade. This will check
that no Fate
+ transactions exist, delete any ZooKeeper locks for server processes, and
prohibit any
+ server processes from being started.
+
+* {% ghi 4898 %} New way to compute bulk load plans (TODO: Needs text)
+
+* {% ghi 5169 %} {% ghi 5170 %} The TabletServer will halt itself when a walog
write or
+ minc failure occurs and the TabletServer lock is not held in ZooKeeper.
+
+* {% ghi 5145 %} {% ghi 5132 %} The Manager and TabletServer processes now
have a background
+ thread that can be enabled via the property
`general.server.lock.verification.interval` to
+ validate that the process is holding the lock in ZooKeeper. The existing
mechanism of relying
+ on a Watcher to notify the process that the lock has been lost can be a
problem due to the
+ fact that there is a single thread in the ZooKeeper client that fires
Watcher events. If the
+ thread is currently waiting on a hung Watcher, then subsequent Watcher
events will not fire
+ and could leave the server running without a lock.
+
+* {% ghi 5174 %} Improvements to the `accumulo-cluster` and `accumulo-service`
scripts have been
+ backported from the main branch. This includes syntax changes.
+
+* {% ghi 5384 %} {% ghi 5386 %} Fixed an issue with multiple threads
performing AES decryption leading to corrupt data.
Review Comment:
I think this wasn't an actual issue in our code. It merely added some checks
in our SPI implementation to prevent users from using the code directly in ways
that were not intended, and which Accumulo did not do. This might not be
important enough to mention, since it's not an actual Accumulo bug and would
not have affected anybody except people trying to use our SPI implementation
classes outside the framework it was written for... which is not a case we need
to document.
##########
_posts/release/2025-05-01-accumulo-2.1.4.md:
##########
@@ -0,0 +1,139 @@
+---
+title: Apache Accumulo 2.1.4
+sortableversion: '02.01.04'
+draft: true
+LTM: true
+---
+## About
+
+Apache Accumulo 2.1.4 is a patch release of the 2.1 LTM line. It contains bug
+fixes and minor enhancements. This version supersedes 2.1.3. Users upgrading to
+2.1 should upgrade directly to this version instead of 2.1.3.
+
+## Notable Changes
+
+* {% ghi 5446 %} Many dependencies were updated, to include protobuf-java which
+ was updated from 3.22.0 to 3.25.6 due to a vulnerability. Users may run into
+ a runtime exception added in
https://github.com/protocolbuffers/protobuf/pull/20084
+ when using old generated Protobuf files. Users are advised to regenerate
their
+ protobuf files using protobuf 25.6 or later. Alternatively, users can set a
+ system property (see #20084 for details) to suppress the exception.
Review Comment:
This advises users to regenerate their proto files. However, this version
bump of protobuf-java has nothing to do with user files generated by protoc...
it has to do with *our* protoc generated files. In 2.1, we have replication
code that is deprecated, that has two proto files. If we have correctly
regenerated our files (I don't know if they were during that version bump),
then there is nothing to note to the users regarding that, except something
like `Users may need to address any compatibility issues with this version of
protobuf-java, if their code also makes use of this library`
##########
_posts/release/2025-05-01-accumulo-2.1.4.md:
##########
@@ -0,0 +1,139 @@
+---
+title: Apache Accumulo 2.1.4
+sortableversion: '02.01.04'
+draft: true
+LTM: true
+---
+## About
+
+Apache Accumulo 2.1.4 is a patch release of the 2.1 LTM line. It contains bug
+fixes and minor enhancements. This version supersedes 2.1.3. Users upgrading to
+2.1 should upgrade directly to this version instead of 2.1.3.
+
+## Notable Changes
+
+* {% ghi 5446 %} Many dependencies were updated, to include protobuf-java which
+ was updated from 3.22.0 to 3.25.6 due to a vulnerability. Users may run into
+ a runtime exception added in
https://github.com/protocolbuffers/protobuf/pull/20084
+ when using old generated Protobuf files. Users are advised to regenerate
their
+ protobuf files using protobuf 25.6 or later. Alternatively, users can set a
+ system property (see #20084 for details) to suppress the exception.
+
+* {% ghi 5073 %} {% ghi 5034 %} Added `accumulo check-accumulo-properties`
command that can be
+ run before an instance is initialized to check the properties file.
+
+* {% ghi 5193 %} Added `accumulo admin signalShutdown` command to signal the
server
+ process to initiate a graceful shutdown. Compactors will finish the major
compaction
+ that they are currently working on, then will exit. Scan Servers will return
a busy
+ signal to all clients for new scan requests, and will shutdown when all
in-progress
+ scans have closed. Tablet Servers will signal the Manager that they are
shutting down,
+ which should prevent assignment of tablets to that server, then they will
unload all
+ tablets and shut down. The Monitor, Manager, GarbageCollector, and
CompactionCoordinator
+ will shut down also. The last step in the shutdown process for all servers
is to remove
+ their lock in ZooKeeper.
+
+* {% ghi 5438 %} Added `accumulo upgrade --prepare` command which should be
used after
+ shutting down an instance in preparation for an upgrade. This will check
that no Fate
+ transactions exist, delete any ZooKeeper locks for server processes, and
prohibit any
+ server processes from being started.
+
+* {% ghi 4898 %} New way to compute bulk load plans (TODO: Needs text)
+
+* {% ghi 5169 %} {% ghi 5170 %} The TabletServer will halt itself when a walog
write or
+ minc failure occurs and the TabletServer lock is not held in ZooKeeper.
+
+* {% ghi 5145 %} {% ghi 5132 %} The Manager and TabletServer processes now
have a background
+ thread that can be enabled via the property
`general.server.lock.verification.interval` to
+ validate that the process is holding the lock in ZooKeeper. The existing
mechanism of relying
+ on a Watcher to notify the process that the lock has been lost can be a
problem due to the
+ fact that there is a single thread in the ZooKeeper client that fires
Watcher events. If the
+ thread is currently waiting on a hung Watcher, then subsequent Watcher
events will not fire
+ and could leave the server running without a lock.
+
+* {% ghi 5174 %} Improvements to the `accumulo-cluster` and `accumulo-service`
scripts have been
+ backported from the main branch. This includes syntax changes.
+
+* {% ghi 5384 %} {% ghi 5386 %} Fixed an issue with multiple threads
performing AES decryption leading to corrupt data.
+
+### Configurable Improvements
+
+Many notable improvements have been added that include a means for users to
configure them via new
+or existing properties to improve system performance or behavior. These
include:
Review Comment:
These words don't add anything to the section. The section title and
individual points are sufficient.
```suggestion
```
##########
_posts/release/2025-05-01-accumulo-2.1.4.md:
##########
@@ -0,0 +1,139 @@
+---
+title: Apache Accumulo 2.1.4
+sortableversion: '02.01.04'
+draft: true
+LTM: true
+---
+## About
+
+Apache Accumulo 2.1.4 is a patch release of the 2.1 LTM line. It contains bug
+fixes and minor enhancements. This version supersedes 2.1.3. Users upgrading to
+2.1 should upgrade directly to this version instead of 2.1.3.
+
+## Notable Changes
+
+* {% ghi 5446 %} Many dependencies were updated, to include protobuf-java which
+ was updated from 3.22.0 to 3.25.6 due to a vulnerability. Users may run into
+ a runtime exception added in
https://github.com/protocolbuffers/protobuf/pull/20084
+ when using old generated Protobuf files. Users are advised to regenerate
their
+ protobuf files using protobuf 25.6 or later. Alternatively, users can set a
+ system property (see #20084 for details) to suppress the exception.
+
+* {% ghi 5073 %} {% ghi 5034 %} Added `accumulo check-accumulo-properties`
command that can be
+ run before an instance is initialized to check the properties file.
+
+* {% ghi 5193 %} Added `accumulo admin signalShutdown` command to signal the
server
+ process to initiate a graceful shutdown. Compactors will finish the major
compaction
+ that they are currently working on, then will exit. Scan Servers will return
a busy
+ signal to all clients for new scan requests, and will shutdown when all
in-progress
+ scans have closed. Tablet Servers will signal the Manager that they are
shutting down,
+ which should prevent assignment of tablets to that server, then they will
unload all
+ tablets and shut down. The Monitor, Manager, GarbageCollector, and
CompactionCoordinator
+ will shut down also. The last step in the shutdown process for all servers
is to remove
+ their lock in ZooKeeper.
+
+* {% ghi 5438 %} Added `accumulo upgrade --prepare` command which should be
used after
+ shutting down an instance in preparation for an upgrade. This will check
that no Fate
+ transactions exist, delete any ZooKeeper locks for server processes, and
prohibit any
+ server processes from being started.
+
+* {% ghi 4898 %} New way to compute bulk load plans (TODO: Needs text)
+
+* {% ghi 5169 %} {% ghi 5170 %} The TabletServer will halt itself when a walog
write or
+ minc failure occurs and the TabletServer lock is not held in ZooKeeper.
+
+* {% ghi 5145 %} {% ghi 5132 %} The Manager and TabletServer processes now
have a background
+ thread that can be enabled via the property
`general.server.lock.verification.interval` to
+ validate that the process is holding the lock in ZooKeeper. The existing
mechanism of relying
+ on a Watcher to notify the process that the lock has been lost can be a
problem due to the
+ fact that there is a single thread in the ZooKeeper client that fires
Watcher events. If the
+ thread is currently waiting on a hung Watcher, then subsequent Watcher
events will not fire
+ and could leave the server running without a lock.
+
+* {% ghi 5174 %} Improvements to the `accumulo-cluster` and `accumulo-service`
scripts have been
+ backported from the main branch. This includes syntax changes.
+
+* {% ghi 5384 %} {% ghi 5386 %} Fixed an issue with multiple threads
performing AES decryption leading to corrupt data.
+
+### Configurable Improvements
Review Comment:
This sounds like the it's configurable whether or not you get the
improvement.
```suggestion
### Configuration Improvements
```
##########
_posts/release/2025-05-01-accumulo-2.1.4.md:
##########
@@ -0,0 +1,139 @@
+---
+title: Apache Accumulo 2.1.4
+sortableversion: '02.01.04'
+draft: true
+LTM: true
+---
+## About
+
+Apache Accumulo 2.1.4 is a patch release of the 2.1 LTM line. It contains bug
+fixes and minor enhancements. This version supersedes 2.1.3. Users upgrading to
+2.1 should upgrade directly to this version instead of 2.1.3.
+
+## Notable Changes
+
+* {% ghi 5446 %} Many dependencies were updated, to include protobuf-java which
+ was updated from 3.22.0 to 3.25.6 due to a vulnerability. Users may run into
+ a runtime exception added in
https://github.com/protocolbuffers/protobuf/pull/20084
+ when using old generated Protobuf files. Users are advised to regenerate
their
+ protobuf files using protobuf 25.6 or later. Alternatively, users can set a
+ system property (see #20084 for details) to suppress the exception.
+
+* {% ghi 5073 %} {% ghi 5034 %} Added `accumulo check-accumulo-properties`
command that can be
+ run before an instance is initialized to check the properties file.
+
+* {% ghi 5193 %} Added `accumulo admin signalShutdown` command to signal the
server
+ process to initiate a graceful shutdown. Compactors will finish the major
compaction
+ that they are currently working on, then will exit. Scan Servers will return
a busy
+ signal to all clients for new scan requests, and will shutdown when all
in-progress
+ scans have closed. Tablet Servers will signal the Manager that they are
shutting down,
+ which should prevent assignment of tablets to that server, then they will
unload all
+ tablets and shut down. The Monitor, Manager, GarbageCollector, and
CompactionCoordinator
+ will shut down also. The last step in the shutdown process for all servers
is to remove
+ their lock in ZooKeeper.
+
+* {% ghi 5438 %} Added `accumulo upgrade --prepare` command which should be
used after
+ shutting down an instance in preparation for an upgrade. This will check
that no Fate
+ transactions exist, delete any ZooKeeper locks for server processes, and
prohibit any
+ server processes from being started.
+
+* {% ghi 4898 %} New way to compute bulk load plans (TODO: Needs text)
+
+* {% ghi 5169 %} {% ghi 5170 %} The TabletServer will halt itself when a walog
write or
+ minc failure occurs and the TabletServer lock is not held in ZooKeeper.
+
+* {% ghi 5145 %} {% ghi 5132 %} The Manager and TabletServer processes now
have a background
+ thread that can be enabled via the property
`general.server.lock.verification.interval` to
+ validate that the process is holding the lock in ZooKeeper. The existing
mechanism of relying
+ on a Watcher to notify the process that the lock has been lost can be a
problem due to the
+ fact that there is a single thread in the ZooKeeper client that fires
Watcher events. If the
+ thread is currently waiting on a hung Watcher, then subsequent Watcher
events will not fire
+ and could leave the server running without a lock.
+
+* {% ghi 5174 %} Improvements to the `accumulo-cluster` and `accumulo-service`
scripts have been
+ backported from the main branch. This includes syntax changes.
+
+* {% ghi 5384 %} {% ghi 5386 %} Fixed an issue with multiple threads
performing AES decryption leading to corrupt data.
+
+### Configurable Improvements
+
+Many notable improvements have been added that include a means for users to
configure them via new
+or existing properties to improve system performance or behavior. These
include:
+
+* {% ghi 4723 %} {% ghi 5239 %} Changes to properties
`tserver.session.idle.max` and
+ `tserver.session.update.idle.max` no longer require a TabletServer restart.
+
+* {% ghi 5397 %} {% ghi 5399 %} Added property `gc.threads.delete.wal` to
control the number of threads to use for
+ deleting write-ahead logs and recovery files.
+
+### Notable Bug Fixes
+
+* {% ghi 5033 %} {% ghi 5038 %} Group names in cluster.yaml must conform to
bash variable name rules
+ or an error will be raised when parsing the cluster.yaml file.
+
+* {% ghi 5221 %} The shell will now print to stdout, instead of stderr, when
there is no terminal set. This
+ will help with piping the output of the shell to other commands such as grep.
+
+* {% ghi 5341 %} Created an optimization in Bulk Import v2 to enable faster
processing of bulk import files
+ when loading into a sparse set of tablets in a large table. See new table
property `table.bulk.metadata.skip.distance`.
+
+* {% ghi 5358 %} Fixed issue with balancing metadata tablets.
+
+* {% ghi 5396 %} Modified `accumulo-cluster` to stop GC and ScanServer
processes first when stopping the
+ cluster. These processes write to the metadata table, which can lead to
slower shutdown times.
+
+* {% ghi 4868 %} {% ghi 4871 %} Fixed listscans so that it shows a scan
session id for batch scans instead
+ of always showing zero.
+
+* {% ghi 4845 %} InstanceOperations.getActiveCompaction(String) did not handle
the address of a
+ Compactor process being passed in the parameter. This has been fixed.
+
+* {% ghi 5445 %} The `cf` and `cq` options for the DeleteMany and Grep shell
commands were not being
+ handled properly.
+
+### Metrics Improvements
+
+* {% ghi 4756 %} {% ghi 4757 %} {% ghi 4840 %} Added metric to indicate how
many zombie scan threads are running
+ in the TabletServer.
+
+* {% ghi 5011 %} Added queue tag to metrics emitted from the External
Compactor processes.
+
+* {% ghi 5025 %} {% ghi 4922 %} Added property
`general.micrometer.log.metrics` to enable metrics on
+ the log4j2 or logback logging frameworks. Disabled by default.
+
+### Other Improvements
+
+* {% ghi 4755 %} {% ghi 5220 %} Fixed the help flag for the admin command so
that it works for subcommands.
+
+* {% ghi 4819 %} Tablets that are closing will no longer wait on scans to
complete. Instead they
+ will try to interrupt the scan and then continue closing the tablet so that
it can be migrated.
+ It's possible that a zombie scan thread could remain in the TabletServer, if
the scan thread is
+ blocked on something and does not die. Users should monitor the new zombie
scan metric mentioned
+ in the section above.
+
+* {% ghi 4867 %} Lowered the memory burden of listing a large number of
External Compactors in the Monitor.
+
+* {% ghi 5026 %} Modified CompactionJobPrioritizer.createPriority to give a
higher priority to tablets that
+ have more files than the maximum number of files per tablet.
+
+* {% ghi 5400 %} Modified AESCryptoService to prevent the creation of
overlapping streams.
+
+## Requirements
+
+Accumulo 2.1.4 now requires JDK 17 to build, but still supports Java 11
runtime.
+
+## Upgrading
+
+View the [Upgrading Accumulo documentation][upgrade] for guidance.
+
+## Useful Links
+
+* [All Changes since 2.1.3][all-changes]
+* [All tickets related to this release][milestone]
+
+This release also contains bug fixes from 1.10.4, which was released after
2.1.2.
+
+
+[upgrade]: /docs/2.x/administration/upgrading
Review Comment:
That doc should be updated to mention the new `upgrade --prepare` and
`upgrade --start` paths when upgrading to the major or minor release since
2.1.4, but not necessary for bugfix releases.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
