From: corwin
The dm-vdo target provides inline deduplication, compression, zero-block
elimination, and thin provisioning. A dm-vdo target can be backed by up to
256TB of storage, and can present a logical size of up to 4PB. This target
was originally developed at Permabit Technology Corp. startin
From: corwin
This adds the admin-guide documentation for dm-vdo.
vdo.rst is the guide to using dm-vdo. vdo-design is an overview of the
design of dm-vdo.
Signed-off-by: corwin
---
.../admin-guide/device-mapper/vdo-design.rst | 390 ++
.../admin-guide/device-mapper/vdo.rst
From: corwin
This adds dm-vdo to the drivers/md Kconfig and Makefile.
Signed-off-by: corwin
---
drivers/md/Kconfig | 16
drivers/md/Makefile | 2 ++
2 files changed, 18 insertions(+)
diff --git a/drivers/md/Kconfig b/drivers/md/Kconfig
index b0a22e99bad..9fa9dec1029 100644
From: corwin
This adds the dm-vdo target.
Signed-off-by: corwin
---
drivers/md/dm-vdo-target.c | 2983
1 file changed, 2983 insertions(+)
create mode 100644 drivers/md/dm-vdo-target.c
diff --git a/drivers/md/dm-vdo-target.c b/drivers/md/dm-vdo-target.c
ne
rm. There is no such hash function already in the
kernel.
J. corwin Coburn (39):
Add documentation for dm-vdo.
Add the MurmurHash3 fast hashing algorithm.
Add memory allocation utilities.
Add basic logging and support utilities.
Add vdo type declarations, constants, and simple data str
.
Signed-off-by: J. corwin Coburn
---
drivers/md/dm-vdo/murmurhash3.c | 175
drivers/md/dm-vdo/murmurhash3.h | 15 +++
2 files changed, 190 insertions(+)
create mode 100644 drivers/md/dm-vdo/murmurhash3.c
create mode 100644 drivers/md/dm-vdo/murmurhash3.h
diff
This adds the admin-guide documentation for dm-vdo.
vdo.rst is the guide to using dm-vdo. vdo-design is an overview of the
design of dm-vdo.
Signed-off-by: J. corwin Coburn
---
.../admin-guide/device-mapper/vdo-design.rst | 390 ++
.../admin-guide/device-mapper/vdo.rst
allocate memory, since memory allocation during certain critical
code sections can cause the vdo target to deadlock.
Signed-off-by: J. corwin Coburn
---
drivers/md/dm-vdo/memory-alloc.c | 447 +++
drivers/md/dm-vdo/memory-alloc.h | 181 +
2 files changed, 628
Signed-off-by: J. corwin Coburn
---
drivers/md/dm-vdo/constants.c| 15 +
drivers/md/dm-vdo/constants.h| 102 +++
drivers/md/dm-vdo/release-versions.h | 20 ++
drivers/md/dm-vdo/status-codes.c | 126 +
drivers/md/dm-vdo/status-codes.h | 112
drivers
Add various support utilities for the vdo target and deduplication index,
including logging utilities, string and time management, and index-specific
error codes.
Signed-off-by: J. corwin Coburn
---
drivers/md/dm-vdo/errors.c | 316 +++
drivers/md/dm-vdo
This patch adds utilities for managing and using named threads, as well as
several locking and sychronization utilities. These utilities help dm-vdo
minimize thread transitions nad manage cross-thread interations.
Signed-off-by: J. corwin Coburn
---
drivers/md/dm-vdo/thread-cond-var.c | 46
Add structures which record the configuration of various deduplication
index parameters. This also includes facilities for saving and loading the
configuration and validating its integrity.
Signed-off-by: J. corwin Coburn
---
drivers/md/dm-vdo/config.c | 389
dispatching of many short-running tasks. The work_queue also supports
priorities. Finally, this patch adds vdo_completion, the structure which is
enqueued on work_queues.
Signed-off-by: J. corwin Coburn
---
drivers/md/dm-vdo/completion.c| 141 +++
drivers/md/dm-vdo/completion.h| 155
. corwin Coburn
---
drivers/md/dm-vdo/index-layout.c | 1775 ++
drivers/md/dm-vdo/index-layout.h | 42 +
drivers/md/dm-vdo/io-factory.c | 458
drivers/md/dm-vdo/io-factory.h | 66 ++
drivers/md/dm-vdo/numeric.h | 78 ++
5 files changed, 2419
of
deltas in order to find a given record. The delta index reduces this lookup
cost by splitting its key space into many sub-lists, each starting at a
fixed key value, so that each individual list is short.
Signed-off-by: J. corwin Coburn
---
drivers/md/dm-vdo/delta-index.c | 2018
Signed-off-by: J. corwin Coburn
---
drivers/md/dm-vdo/vio.c | 525
drivers/md/dm-vdo/vio.h | 221 +
2 files changed, 746 insertions(+)
create mode 100644 drivers/md/dm-vdo/vio.c
create mode 100644 drivers/md/dm-vdo/vio.h
diff --git a
. corwin Coburn
---
drivers/md/dm-vdo/action-manager.c | 410 +++
drivers/md/dm-vdo/action-manager.h | 117 +++
drivers/md/dm-vdo/admin-state.c| 512 +
drivers/md/dm-vdo/admin-state.h| 180 ++
4 files changed, 1219 insertions
Signed-off-by: J. corwin Coburn
---
drivers/md/dm-vdo/data-vio.c | 2076 ++
drivers/md/dm-vdo/data-vio.h | 683 +++
2 files changed, 2759 insertions(+)
create mode 100644 drivers/md/dm-vdo/data-vio.c
create mode 100644 drivers/md/dm-vdo/data-vio.h
diff
Signed-off-by: J. corwin Coburn
---
drivers/md/dm-vdo/dedupe.c | 622 +
drivers/md/dm-vdo/dedupe.h | 26 ++
2 files changed, 648 insertions(+)
diff --git a/drivers/md/dm-vdo/dedupe.c b/drivers/md/dm-vdo/dedupe.c
index 18c7509ef90..6ffecabd772 100644
--- a
The io_submitter handles bio submission from vdo data store to the storage
below. It will merge bios when possible.
Signed-off-by: J. corwin Coburn
---
drivers/md/dm-vdo/io-submitter.c | 483 +++
drivers/md/dm-vdo/io-submitter.h | 52
2 files changed, 535
blocks when bins fill.
Signed-off-by: J. corwin Coburn
---
drivers/md/dm-vdo/packer.c | 794 +
drivers/md/dm-vdo/packer.h | 123 ++
2 files changed, 917 insertions(+)
create mode 100644 drivers/md/dm-vdo/packer.c
create mode 100644 drivers/md/dm-vdo
f
its hash.
Signed-off-by: J. corwin Coburn
---
drivers/md/dm-vdo/dedupe.c | 2451
drivers/md/dm-vdo/dedupe.h | 93 ++
2 files changed, 2544 insertions(+)
create mode 100644 drivers/md/dm-vdo/dedupe.c
create mode 100644 drivers/md/dm-vdo/dedupe.h
di
cost of updating individual reference blocks.
This patch adds the slab structure as well as the slab journal and
reference counters.
Signed-off-by: J. corwin Coburn
---
drivers/md/dm-vdo/slab-depot.c | 2458
drivers/md/dm-vdo/slab-depot.h | 253
2 files
writes which
are in progress at the time of the notification have completed in all
zones, the flush bio is then allowed to complete.
Signed-off-by: J. corwin Coburn
---
drivers/md/dm-vdo/flush.c| 563 +++
drivers/md/dm-vdo/flush.h| 44 +++
drivers/md
Signed-off-by: J. corwin Coburn
---
drivers/md/dm-vdo/slab-depot.c | 964 +
drivers/md/dm-vdo/slab-depot.h | 121 +
2 files changed, 1085 insertions(+)
diff --git a/drivers/md/dm-vdo/slab-depot.c b/drivers/md/dm-vdo/slab-depot.c
index 47707497eb5..0c3d66578e0
it can in order to
return to a writable state. Although some data may be lost, this process
will ensure that the vdo's own metadata is self-consistent.
Signed-off-by: J. corwin Coburn
---
drivers/md/dm-vdo/repair.c | 1775
drivers/md/dm-vdo/repair.h |
Add support for dumping detailed vdo state to the kernel log via a dmsetup
message. The dump code is not thread-safe and is generally intended for use
only when the vdo is hung.
Signed-off-by: J. corwin Coburn
---
drivers/md/dm-vdo/dump.c | 288 +++
drivers
Signed-off-by: J. corwin Coburn
---
drivers/md/dm-vdo/message-stats.c | 1222 +
drivers/md/dm-vdo/message-stats.h | 13 +
drivers/md/dm-vdo/statistics.h| 279 +++
3 files changed, 1514 insertions(+)
create mode 100644 drivers/md/dm-vdo/message-stats.c
This adds dm-vdo to the drivers/md Kconfig and Makefile.
Signed-off-by: J. corwin Coburn
---
drivers/md/Kconfig | 16
drivers/md/Makefile | 2 ++
2 files changed, 18 insertions(+)
diff --git a/drivers/md/Kconfig b/drivers/md/Kconfig
index b0a22e99bad..9fa9dec1029 100644
slots in the compressed block this logical address maps to.
Signed-off-by: J. corwin Coburn
---
drivers/md/dm-vdo/block-map.c | 2151 +
drivers/md/dm-vdo/block-map.h | 232
2 files changed, 2383 insertions(+)
create mode 100644 drivers/md/dm-vdo/block-map.c
priority heap is used to minimize the search time for
free blocks.
Signed-off-by: J. corwin Coburn
---
drivers/md/dm-vdo/int-map.c| 710 +
drivers/md/dm-vdo/int-map.h| 40 ++
drivers/md/dm-vdo/pointer-map.c| 691
drivers
determined that none is available), the target can resume normal operation
in a degraded mode. Read and write requests can be serviced, perhaps with
degraded performance, while the remainder of the dirty slabs are recovered.
Signed-off-by: J. corwin Coburn
---
drivers/md/dm-vdo/slab-dep
Signed-off-by: J. corwin Coburn
---
drivers/md/dm-vdo/encodings.c | 1523 +
drivers/md/dm-vdo/encodings.h | 1307
2 files changed, 2830 insertions(+)
create mode 100644 drivers/md/dm-vdo/encodings.c
create mode 100644 drivers/md/dm
index's memory budget.
The volume index is composed of two subindexes in order to handle sparse
hook names separately from regular names. If sparse indexing is not
enabled, the sparse hook portion of the volume index is not used or
instantiated.
Signed-off-by: J. corwin Coburn
---
drivers/md/d
isk before the
other metadata structures may be updated to reflect the operation.
Signed-off-by: J. corwin Coburn
---
drivers/md/dm-vdo/recovery-journal.c | 1772 ++
drivers/md/dm-vdo/recovery-journal.h | 313 +
2 files changed, 2085 insertions(+)
create mode 1006
Each slab is independent of every other. They are assigned to "physical
zones" in round-robin fashion. If there are P physical zones, then slab n
is assigned to zone n mod P. The set of slabs in each physical zone is
managed by a block allocator.
Signed-off-by: J. corwin Coburn
---
-by: J. corwin Coburn
---
drivers/md/dm-vdo/index.c| 1403 ++
drivers/md/dm-vdo/index.h| 83 ++
drivers/md/dm-vdo/sparse-cache.c | 595 +
drivers/md/dm-vdo/sparse-cache.h | 49 ++
4 files changed, 2130 insertions(+)
create mode 100644
index properties at
runtime.
Signed-off-by: J. corwin Coburn
---
drivers/md/dm-vdo/index-session.c | 815 ++
drivers/md/dm-vdo/index-session.h | 84 +++
drivers/md/dm-vdo/uds-sysfs.c | 185 +++
drivers/md/dm-vdo/uds-sysfs.h | 12 +
drivers/md/dm-vdo/uds.h
The set of leaf pages of the block map tree is too large to fit in memory,
so each block map zone maintains a cache of leaf pages. This patch adds the
implementation of that cache.
Signed-off-by: J. corwin Coburn
---
drivers/md/dm-vdo/block-map.c | 1230
Signed-off-by: J. corwin Coburn
---
drivers/md/dm-vdo/pool-sysfs-stats.c | 2063 ++
drivers/md/dm-vdo/pool-sysfs.c | 193 +++
drivers/md/dm-vdo/pool-sysfs.h | 19 +
drivers/md/dm-vdo/sysfs.c| 84 ++
4 files changed, 2359 insertions(+)
create
searching and reading. The closed
chapter includes a delta index, called the chapter index, which maps each
record name to the record page containing the record and allows the index
to read at most one record page when looking up a record.
Signed-off-by: J. corwin Coburn
---
drivers/md/dm-vdo
Signed-off-by: J. corwin Coburn
---
drivers/md/dm-vdo/vdo.c | 1846 +++
drivers/md/dm-vdo/vdo.h | 381
2 files changed, 2227 insertions(+)
create mode 100644 drivers/md/dm-vdo/vdo.c
create mode 100644 drivers/md/dm-vdo/vdo.h
diff --git a/drivers
This adds the dm-vdo target.
Signed-off-by: J. corwin Coburn
---
drivers/md/dm-vdo-target.c | 2983
1 file changed, 2983 insertions(+)
create mode 100644 drivers/md/dm-vdo-target.c
diff --git a/drivers/md/dm-vdo-target.c b/drivers/md/dm-vdo-target.c
new
page they need, in order to minimize latency and I/O
requests when records have to be read from storage. The cache and queues
also coordinate with the volume index to ensure that the volume does not
waste resources reading pages that are no longer valid.
Signed-off-by: J. corwin Coburn
---
drivers
44 matches
Mail list logo
