Your message dated Mon, 07 Oct 2024 17:00:10 +0000
with message-id <e1sxr5e-000ioz...@fasolo.debian.org>
and subject line Bug#1083032: fixed in borgstore 0.0.4-1
has caused the Debian Bug report #1083032,
regarding ITP: borgstore -- A simple key/value store implementation in Python
to be marked as done.
This means that you claim that the problem has been dealt with.
If this is not the case it is now your responsibility to reopen the
Bug report if necessary, and/or fix the problem forthwith.
(NB: If you are a system administrator and have no idea what this
message is talking about, this may indicate a serious mail system
misconfiguration somewhere. Please contact ow...@bugs.debian.org
immediately.)
--
1083032: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1083032
Debian Bug Tracking System
Contact ow...@bugs.debian.org with problems
--- Begin Message ---
Package: wnpp
Severity: wishlist
Owner: Gianfranco Costamagna <locutusofb...@debian.org>
* Package name : borgstore
Version : 0.0.4
Upstream Author : Thomas Waldmann <t...@waldmann-edv.de>
* URL : https://github.com/borgbackup/borgstore
* License : BSD-3-Clause
Programming Lang: Python
Description : A simple key/value store implementation in Python
Binary package names: python3-borgstore
A key/value store implementation in Python, supporting multiple backends,
data redundancy and distribution.
.
Keys
----
.
A key (str) can look like:
.
- 0123456789abcdef... (usually a long, hex-encoded hash value)
- Any other pure ASCII string without "/" or ".." or " ".
.
.
Namespaces
----------
.
To keep stuff apart, keys should get prefixed with a namespace, like:
.
- config/settings
- meta/0123456789abcdef...
- data/0123456789abcdef...
.
Please note:
.
1. you should always use namespaces.
2. nested namespaces like namespace1/namespace2/key are not supported.
3. the code could work without a namespace (namespace ""), but then you
can't add another namespace later, because then you would have created
nested namespaces.
.
Values
------
.
Values can be any arbitrary binary data (bytes).
.
Store Operations
----------------
.
The high-level Store API implementation transparently deals with nesting and
soft deletion, so the caller doesn't have to care much for that and the Backend
API can be much simpler:
.
- create/destroy: initialize or remove the whole store.
- list: flat list of the items in the given namespace, with or without soft
deleted items.
- store: write a new item into the store (giving its key/value pair)
- load: read a value from the store (giving its key), partial loads giving
offset and/or size are supported.
- info: get information about an item via its key (exists? size? ...)
- delete: immediately remove an item from the store (giving its key)
- move: implements rename, soft delete / undelete, move to current
nesting level
- stats: api call counters, time spent in api methods, data volume/throughput
- latency/bandwidth emulator: can emulate higher latency (via BORGSTORE_LATENCY
[us]) and lower bandwidth (via BORGSTORE_BANDWIDTH [bit/s]) than what is
actually provided by the backend.
.
Automatic Nesting
-----------------
.
For the Store user, items have names like e.g.:
.
namespace/0123456789abcdef...
namespace/abcdef0123456789...
.
If there are very many items in the namespace, this could lead to scalability
issues in the backend, thus the Store implementation offers transparent
nesting, so that internally the Backend API will be called with
names like e.g.:
.
namespace/01/23/56/0123456789abcdef...
namespace/ab/cd/ef/abcdef0123456789...
.
The nesting depth can be configured from 0 (= no nesting) to N levels and
there can be different nesting configurations depending on the namespace.
.
The Store supports operating at different nesting levels in the same
namespace at the same time.
.
Soft deletion
-------------
.
To soft delete an item (so its value could be still read or it could be
undeleted), the store just renames the item, appending ".del" to its name.
.
Undelete reverses this by removing the ".del" suffix from the name.
.
Some store operations have a boolean flag "deleted" to choose whether they
shall consider soft deleted items.
.
Backends
--------
.
The backend API is rather simple, one only needs to provide some very
basic operations.
.
Currently, these storage backends are implemented:
.
- POSIX filesystems (namespaces: directories, values: in key-named files)
- SFTP (access a server via sftp, namespaces: directories, values: in
key-named files)
- Rclone - access any of the 100s of cloud providers
[rclone](https://rclone.org/) supports
- (more might come in future)
.
MStore
------
.
API of MStore is very similar to Store, but instead of directly using one
backend
only (like Store does), it uses multiple Stores internally to implement:
.
- redundancy (keep same data at multiple places)
- distribution (keep different data at multiple places)
.
Scalability
-----------
.
- Count of key/value pairs stored in a namespace: automatic nesting is
provided for keys to address common scalability issues.
- Key size: there are no special provisions for extremely long keys (like:
more than backend limitations). Usually this is not a problem though.
- Value size: there are no special provisions for dealing with large value
sizes (like: more than free memory, more than backend storage limitations,
etc.). If one deals with very large values, one usually cuts them into
chunks before storing them into the store.
- Partial loads improve performance by avoiding a full load if only a part
of the value is needed (e.g. a header with metadata).
.
Want a demo?
------------
.
Run this to get instructions how to run the demo:
.
python3 -m borgstore
.
State of this project
---------------------
.
**API is still unstable and expected to change as development goes on.**
.
**There will be no data migration tools involving development/testing releases,
like e.g. upgrading a store from alpha1 to alpha2 or beta13 to release.**
.
There are tests and they succeed for the basic functionality, so some of the
stuff is already working well.
.
There might be missing features or optimization potential, feedback welcome!
.
There are a lot of possible, but still missing backends (like e.g. for cloud
storage). If you want to create and support one: pull requests are welcome.
.
Borg?
-----
.
Please note that this code is currently **not** used by the stable release of
BorgBackup (aka "borg"), but only by borg2 beta 10+ and master branch.
.
OpenPGP_signature.asc
Description: OpenPGP digital signature
--- End Message ---
--- Begin Message ---
Source: borgstore
Source-Version: 0.0.4-1
Done: Gianfranco Costamagna <locutusofb...@debian.org>
We believe that the bug you reported is fixed in the latest version of
borgstore, which is due to be installed in the Debian FTP archive.
A summary of the changes between this version and the previous one is
attached.
Thank you for reporting the bug, which will now be closed. If you
have further comments please address them to 1083...@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.
Debian distribution maintenance software
pp.
Gianfranco Costamagna <locutusofb...@debian.org> (supplier of updated borgstore
package)
(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmas...@ftp-master.debian.org)
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Format: 1.8
Date: Mon, 30 Sep 2024 10:28:52 +0000
Source: borgstore
Binary: python3-borgstore
Architecture: source all
Version: 0.0.4-1
Distribution: unstable
Urgency: low
Maintainer: Gianfranco Costamagna <locutusofb...@debian.org>
Changed-By: Gianfranco Costamagna <locutusofb...@debian.org>
Description:
python3-borgstore - Simple key/value store implementation in Python
Closes: 1083032
Changes:
borgstore (0.0.4-1) unstable; urgency=low
.
* Initial release (Closes: #1083032)
Checksums-Sha1:
7f4188d3718b9880303fb574bbd50b1702785602 1883 borgstore_0.0.4-1.dsc
12be836a72b9b739b98c85f0eaa3bf52db75d5a2 22561 borgstore_0.0.4.orig.tar.gz
bab7038b8b88267a92371338bc1a589bcf9db31a 4032 borgstore_0.0.4-1.debian.tar.xz
fe97c0a10783624dbe45f229bfbc372ec2ef944c 7466 borgstore_0.0.4-1_amd64.buildinfo
8076b72c8b77e1ddf62d823c54be5e28800bfa43 23864
python3-borgstore_0.0.4-1_all.deb
Checksums-Sha256:
2e689e89cbf3d8d3a7b9068367874f59bcf80500910eab6a4197e844cf2a685e 1883
borgstore_0.0.4-1.dsc
3adfa48c676a952fbecce4f375b2b3ca5b63cc6edd14844ecc69a2824e8cbea5 22561
borgstore_0.0.4.orig.tar.gz
d1a842b89fad891b6852a0beaa2b2940ac1cffd07d4cb6e0337ff9ee0000a091 4032
borgstore_0.0.4-1.debian.tar.xz
4c500b20b0f65b970b87add914ca5bdd4df6057f2d128e018e835bf8c5d89f24 7466
borgstore_0.0.4-1_amd64.buildinfo
86c4cfd6b494fefba0b763af322175c04c8a5a185554c6b49b72a0aab2d654ab 23864
python3-borgstore_0.0.4-1_all.deb
Files:
5aaa0b9c87a6c2643da8e6c244e3fb5d 1883 python optional borgstore_0.0.4-1.dsc
84ea96c1adaf3fb02e3db97ab2e16e9c 22561 python optional
borgstore_0.0.4.orig.tar.gz
4bb3a0bb6a52e52cdf21621f3442104d 4032 python optional
borgstore_0.0.4-1.debian.tar.xz
6b407a53559be479afac4c603db9268f 7466 python optional
borgstore_0.0.4-1_amd64.buildinfo
e3c9e8f67ae7db76d770bfa8acb1d391 23864 python optional
python3-borgstore_0.0.4-1_all.deb
-----BEGIN PGP SIGNATURE-----
iQIzBAEBCAAdFiEEkpeKbhleSSGCX3/w808JdE6fXdkFAmb6giEACgkQ808JdE6f
XdmNChAAtsqdlfTi4pldxVgjyzX2iaOze8t0MKUu3Qnq7eB0BwW19BER9JgemWLR
CAwfw1LfWMOMUR8BpJ7Zg6YA+2NdmBPiNPz3qh314Z/IwDQsrVFgVkJvAYeUi3M1
hExPbGsGrzDuLlkF7NQijuIarC3I3FjnR+cz/DV4x7IQrgha95mKaL76udzBbbSb
dtiuv4s9eBlNAsFMauDs5mFiYTxGK1YWNJoGGG7MLG4rle0DiicOfNW3nl0h+fOy
4vb9Kqdj/f/vvhcsjSzU1ocrMfsj6Sfgn4x9B/cEOLn57pRKvyBXopfItwBCttrJ
GD6RY15UWQBXySDiyZpmI/cLmcb/tSxeeMa2KLb7ULFudPAfax7QLDUx/POgyj6A
GA6+RBJCuSjIq0ms3SmPz0S5vtfzAF8W4OhqS8ht6NEu1MhsVmaloQ3tAlDrzynL
RG7mCBSJwCLRTak6yxELGhyChuzrcqFWzmY88ssjNKFn4/QUu4ceDE0i2VdOte+Z
FQ8S6xahBiBraZOPqaNzNRbVhA0vBTRMVHM9NRaLE/Y/SPyeF6h0rv7wTOJnUEIA
se2b7UCyEI0W81NYHQ+xV1B9ll/+qY5B1T1n2RdCfWLur0Tt3CwtuYpYfQq9xKKN
BquDhjBAotaCmSx59zje91XCHZBqcgnOiVlITZCgIBsnf4IDeU0=
=PE/j
-----END PGP SIGNATURE-----
pgpEN1K0UhKzw.pgp
Description: PGP signature
--- End Message ---
