Re: [PATCH v4 rebased 1/2] fs/erofs: Add support for EROFS
Hi Daniel, On 2024/3/1 01:59, Daniel Kiper wrote: On Mon, Feb 19, 2024 at 02:22:23PM +0800, Yifan Zhao wrote: EROFS [1] is a lightweight read-only filesystem designed for performance which has already been shipped in most Linux distributions as well as widely used in several scenarios, such as Android system partitions, container images, and rootfs for embedded devices. This patch brings EROFS uncompressed support. Now, it's possible to boot directly through GRUB with an EROFS rootfs. EROFS compressed files will be supported later since it has more work to polish. [1] https://erofs.docs.kernel.org Reviewed-by: Glenn Washburn Reviewed-by: Gao Xiang Did they review the patch set here? If no these lines should be dropped. If yes they should go behind Signed-off-by below... Thanks for the reply! As for Glenn's review, see: https://lore.kernel.org/grub-devel/20230720121950.3d3039d4@crass-HP-ZBook-15-G2/ As for my own review, see the last part of: https://lore.kernel.org/grub-devel/03018833-5420-720e-ee51-d68c5dc11...@linux.alibaba.com/ I'm not sure if they are outdated, but Glenn is also in the loop, so feel free to reply if there are some objection here. For The remaining part, let's wait Yifan to address. Thanks, Gao Xiang ___ Grub-devel mailing list Grub-devel@gnu.org https://lists.gnu.org/mailman/listinfo/grub-devel
Re: [PATCH v4 rebased 1/2] fs/erofs: Add support for EROFS
Hi Glenn, On 2024/3/1 04:52, Glenn Washburn wrote: Hi Yifan, I would suggest that the next version of this series have a cover letter which includes the output of --range-diff or --interdiff (as appropriate). Cover letters for multipatch submissions are standard on this list. They allow the tracking of what changed in series revisions. Please see recent cover letter on this list with subject "[PATCH v9 00/22] Automatic Disk Unlock with TPM2" as an example. Including the diff from the last version of the series allows the reviewer to easily see what changed. For instance, I did review the (I think) first version of the series, but I don't have the time to review this series from scratch right now. I might have been able to review it again just looking at the diffs. By making things easier for the review you increase the chances that someone will review the series. Thanks for your confirmation, yes, that would be better to address the diff part. Hopefully Yifan could address these in the next version. Thanks, Gao Xiang Glenn ___ Grub-devel mailing list Grub-devel@gnu.org https://lists.gnu.org/mailman/listinfo/grub-devel
Re: [PATCH v5 1/2] fs/erofs: Add support for EROFS
On 2024/3/4 01:15, Yifan Zhao wrote: EROFS [1] is a lightweight read-only filesystem designed for performance which has already been shipped in most Linux distributions as well as widely used in several scenarios, such as Android system partitions, container images, and rootfs for embedded devices. This patch brings EROFS uncompressed support. Now, it's possible to boot directly through GRUB with an EROFS rootfs. EROFS compressed files will be supported later since it has more work to polish. [1] https://erofs.docs.kernel.org Signed-off-by: Yifan Zhao After checking the v4->v5 diff, it still looks good to me, Reviewed-by: Gao Xiang BTW, Glenn, would you mind taking some time checking if this version looks good to you? Thanks a lot! Thanks, Gao Xiang ___ Grub-devel mailing list Grub-devel@gnu.org https://lists.gnu.org/mailman/listinfo/grub-devel
Re: [PATCH v5 1/2] fs/erofs: Add support for EROFS
Hi Gleen, Daniel, On 2024/4/5 04:56, Daniel Kiper wrote: On Wed, Mar 06, 2024 at 10:18:21AM +0800, Gao Xiang wrote: On 2024/3/4 01:15, Yifan Zhao wrote: EROFS [1] is a lightweight read-only filesystem designed for performance which has already been shipped in most Linux distributions as well as widely used in several scenarios, such as Android system partitions, container images, and rootfs for embedded devices. This patch brings EROFS uncompressed support. Now, it's possible to boot directly through GRUB with an EROFS rootfs. EROFS compressed files will be supported later since it has more work to polish. [1] https://erofs.docs.kernel.org Signed-off-by: Yifan Zhao After checking the v4->v5 diff, it still looks good to me, Reviewed-by: Gao Xiang BTW, Glenn, would you mind taking some time checking if this version looks good to you? Thanks a lot! Glenn, Do you have any comments on this version of EROFS patch set? Yeah, friendly ping.. Thanks in advance! There is the diffs from the latest reviewed version (v2): v2(last reviewed)->v4: https://lore.kernel.org/r/20240303171739.801243-1-zhaoyi...@sjtu.edu.cn v4->v5: https://lore.kernel.org/r/20240303171558.800691-1-zhaoyi...@sjtu.edu.cn Could we have some progress on uncompressed EROFS so that it can be used for GRUB? Then we will focus on upstreaming compression support. Thanks again! Gao Xiang Daniel ___ Grub-devel mailing list Grub-devel@gnu.org https://lists.gnu.org/mailman/listinfo/grub-devel
Re: [PATCH v8 0/2] Introduce EROFS support
Hi Glenn, Daniel,
On 2024/4/24 17:30, Yifan Zhao wrote:
EROFS [1] is a lightweight read-only filesystem designed for performance
which has already been shipped in most Linux distributions as well as widely
used in several scenarios, such as Android system partitions, container
images, and rootfs for embedded devices.
This patch brings EROFS uncompressed support together with related tests.
Now, it's possible to boot directly through GRUB with an EROFS rootfs.
EROFS compressed files will be supported later since it has more work to
polish.
[1] https://erofs.docs.kernel.org
changelog since v7:
- check if log2_blksz is too small in erofs_mount()
- handle bondary case and memleak in erofs_iterate_dir()
- fix incorrect grub_strnlen() implementation
Could you help have another look at this latest version?
Is it ready for merging (Yifan has been working on for almost one year
in his leisure time) so that kernel/initrd can be loaded then..
Any comment would be much helpful.
Thanks,
Gao Xiang
Yifan Zhao (2):
fs/erofs: Add support for EROFS
fs/erofs: Add tests for EROFS in grub-fs-tester
.gitignore | 1 +
INSTALL | 8 +-
Makefile.util.def| 7 +
docs/grub.texi | 3 +-
grub-core/Makefile.core.def | 5 +
grub-core/fs/erofs.c | 984 +++
grub-core/kern/misc.c| 14 +
include/grub/misc.h | 1 +
tests/erofs_test.in | 20 +
tests/util/grub-fs-tester.in | 32 +-
10 files changed, 1063 insertions(+), 12 deletions(-)
create mode 100644 grub-core/fs/erofs.c
create mode 100644 tests/erofs_test.in
Interdiff against v7:
diff --git a/grub-core/fs/erofs.c b/grub-core/fs/erofs.c
index 5b89b7924..13f92e71a 100644
--- a/grub-core/fs/erofs.c
+++ b/grub-core/fs/erofs.c
@@ -101,6 +101,7 @@ struct grub_erofs_inode_chunk_info
#define EROFS_NULL_ADDR 1
#define EROFS_NAME_LEN255
+#define EROFS_MIN_LOG2_BLOCK_SIZE 9
#define EROFS_MAX_LOG2_BLOCK_SIZE 16
struct grub_erofs_inode_chunk_index
@@ -558,7 +559,7 @@ erofs_iterate_dir (grub_fshelp_node_t dir,
grub_fshelp_iterate_dir_hook_t hook,
goto not_found;
nameoff = grub_le_to_cpu16 (de->nameoff);
- if (nameoff < sizeof (struct grub_erofs_dirent) || nameoff > maxsize)
+ if (nameoff < sizeof (struct grub_erofs_dirent) || nameoff >= maxsize)
{
grub_error (GRUB_ERR_BAD_FS,
"invalid nameoff %u @ inode %" PRIuGRUB_UINT64_T,
@@ -587,11 +588,12 @@ erofs_iterate_dir (grub_fshelp_node_t dir,
grub_fshelp_iterate_dir_hook_t hook,
fdiro->inode_loaded = false;
nameoff = grub_le_to_cpu16 (de->nameoff);
- if (nameoff < sizeof (struct grub_erofs_dirent) || nameoff > maxsize)
+ if (nameoff < sizeof (struct grub_erofs_dirent) || nameoff >= maxsize)
{
grub_error (GRUB_ERR_BAD_FS,
"invalid nameoff %u @ inode %" PRIuGRUB_UINT64_T,
nameoff, dir->ino);
+ grub_free (fdiro);
goto not_found;
}
@@ -607,6 +609,7 @@ erofs_iterate_dir (grub_fshelp_node_t dir, grub_fshelp_iterate_dir_hook_t hook,
"invalid de_namelen %" PRIuGRUB_SIZE
" @ inode %" PRIuGRUB_UINT64_T,
de_namelen, dir->ino);
+ grub_free (fdiro);
goto not_found;
}
@@ -700,6 +703,7 @@ erofs_mount (grub_disk_t disk, bool read_root)
if (err != GRUB_ERR_NONE)
return NULL;
if (sb.magic != grub_cpu_to_le32_compile_time (EROFS_MAGIC) ||
+ grub_le_to_cpu32 (sb.log2_blksz) < EROFS_MIN_LOG2_BLOCK_SIZE ||
grub_le_to_cpu32 (sb.log2_blksz) > EROFS_MAX_LOG2_BLOCK_SIZE)
{
grub_error (GRUB_ERR_BAD_FS, "not a valid erofs filesystem");
diff --git a/grub-core/kern/misc.c b/grub-core/kern/misc.c
index ecb27fa5a..67240d64d 100644
--- a/grub-core/kern/misc.c
+++ b/grub-core/kern/misc.c
@@ -602,7 +602,7 @@ grub_strnlen (const char *s, grub_size_t n)
if (n == 0)
return 0;
- while (*p && n--)
+ while (n-- && *p)
p++;
return p - s;
___
Grub-devel mailing list
Grub-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/grub-devel
Re: [PATCH v10 0/2] Introduce EROFS support
Hi Daniel, Glenn, Vladimir,
On 2024/5/2 15:01, Yifan Zhao wrote:
EROFS [1] is a lightweight read-only filesystem designed for performance
which has already been shipped in most Linux distributions as well as widely
used in several scenarios, such as Android system partitions, container
images, and rootfs for embedded devices.
This patch brings EROFS uncompressed support together with related tests.
Now, it's possible to boot directly through GRUB with an EROFS rootfs.
EROFS compressed files will be supported later since it has more work to
polish.
[1] https://erofs.docs.kernel.org
changelog since v9:
- fix overflow and style issues according to Serbinenko's review comments
- add a upper bound for symlink length according to Daniel's advice
Tested-by Link (Commit 1):
https://lists.gnu.org/archive/html/grub-devel/2024-05/msg1.html
Reviewed-by Link (Commit 2):
https://lists.gnu.org/archive/html/grub-devel/2024-04/msg00101.html
Does this version look good you? Could you help take some
time look at the latest version if possible?
Thanks,
Gao Xiang
Yifan Zhao (2):
fs/erofs: Add support for EROFS
fs/erofs: Add tests for EROFS in grub-fs-tester
.gitignore |1 +
INSTALL |8 +-
Makefile.util.def|7 +
docs/grub.texi |3 +-
grub-core/Makefile.core.def |5 +
grub-core/fs/erofs.c | 1007 ++
tests/erofs_test.in | 20 +
tests/util/grub-fs-tester.in | 32 +-
8 files changed, 1071 insertions(+), 12 deletions(-)
create mode 100644 grub-core/fs/erofs.c
create mode 100644 tests/erofs_test.in
Interdiff against v9:
diff --git a/grub-core/fs/erofs.c b/grub-core/fs/erofs.c
index 9c2678796..b82212b16 100644
--- a/grub-core/fs/erofs.c
+++ b/grub-core/fs/erofs.c
@@ -101,6 +101,7 @@ struct grub_erofs_inode_chunk_info
#define EROFS_NULL_ADDR 1
#define EROFS_NAME_LEN255
+#define EROFS_PATH_LEN 4096
#define EROFS_MIN_LOG2_BLOCK_SIZE 9
#define EROFS_MAX_LOG2_BLOCK_SIZE 16
@@ -160,6 +161,12 @@ struct grub_erofs_inode_extended
grub_uint8_t i_reserved2[16];
} GRUB_PACKED;
+union grub_erofs_inode
+{
+ struct grub_erofs_inode_compact c;
+ struct grub_erofs_inode_extended e;
+} GRUB_PACKED;
+
#define EROFS_FT_UNKNOWN 0
#define EROFS_FT_REG_FILE 1
#define EROFS_FT_DIR 2
@@ -197,7 +204,7 @@ struct grub_erofs_xattr_ibody_header
struct grub_fshelp_node
{
struct grub_erofs_data *data;
- struct grub_erofs_inode_extended inode;
+ union grub_erofs_inode inode;
grub_uint64_t ino;
grub_uint8_t inode_type;
@@ -236,26 +243,27 @@ erofs_iloc (grub_fshelp_node_t node)
{
struct grub_erofs_super *sb = &node->data->sb;
- return (grub_le_to_cpu32 (sb->meta_blkaddr) << sb->log2_blksz) + (node->ino << EROFS_ISLOTBITS);
+ return ((grub_uint64_t) grub_le_to_cpu32 (sb->meta_blkaddr) <<
sb->log2_blksz) +
+(node->ino << EROFS_ISLOTBITS);
}
static grub_err_t
erofs_read_inode (struct grub_erofs_data *data, grub_fshelp_node_t node)
{
- struct grub_erofs_inode_compact *dic;
+ union grub_erofs_inode *di;
grub_err_t err;
grub_uint16_t i_format;
grub_uint64_t addr = erofs_iloc (node);
- dic = (struct grub_erofs_inode_compact *) &node->inode;
+ di = (union grub_erofs_inode *) &node->inode;
err = grub_disk_read (data->disk, addr >> GRUB_DISK_SECTOR_BITS,
addr & (GRUB_DISK_SECTOR_SIZE - 1),
- sizeof (struct grub_erofs_inode_compact), dic);
+ sizeof (struct grub_erofs_inode_compact), &di->c);
if (err != GRUB_ERR_NONE)
return err;
- i_format = grub_le_to_cpu16 (dic->i_format);
+ i_format = grub_le_to_cpu16 (di->c.i_format);
node->inode_type = (i_format >> EROFS_I_VERSION_BIT) &
EROFS_I_VERSION_MASKS;
node->inode_datalayout = (i_format >> EROFS_I_DATALAYOUT_BIT) &
EROFS_I_DATALAYOUT_MASKS;
@@ -267,7 +275,7 @@ erofs_read_inode (struct grub_erofs_data *data, grub_fshelp_node_t node)
data->disk, addr >> GRUB_DISK_SECTOR_BITS,
addr & (GRUB_DISK_SECTOR_SIZE - 1),
sizeof (struct grub_erofs_inode_extended) - sizeof (struct
grub_erofs_inode_compact),
- (grub_uint8_t *) dic + sizeof (struct grub_erofs_inode_compact));
+ (grub_uint8_t *) di + sizeof (struct grub_erofs_inode_compact));
if (err != GRUB_ERR_NONE)
return err;
break;
@@ -294,17 +302,17 @@ erofs_inode_size (grub_fshelp_node_t node)
static grub_uint64_t
erofs_inode_file_size (grub_fshelp_node_t node)
{
- struct grub_erofs_inode_compact *dic = (struct grub_erofs_inode_compact *)
&node->inode;
+ union grub_erofs_inode
Re: [PATCH v10 0/2] Introduce EROFS support
ad_inode (ctx->data, node);
- grub_errno = GRUB_ERR_NONE;
+ err = erofs_read_inode (ctx->data, node);
+ if (err != GRUB_ERR_NONE)
+return 0;
}
793 erofs_read_inode (ctx->data, node);
794 grub_errno = GRUB_ERR_NONE;
795 }
796
797 if (node->inode_loaded)
798 {
*** CID 460610: Resource leaks (RESOURCE_LEAK)
/grub-core/fs/erofs.c: 957 in grub_erofs_label()
951 *label = NULL;
952 return grub_errno;
953 }
954
955 *label = grub_strndup ((char *) data->sb.volume_name, sizeof
(data->sb.volume_name));
956 if (!*label)
CID 460610: Resource leaks (RESOURCE_LEAK)
Variable "data" going out of scope leaks the storage it points to.
957 return grub_errno;
958
959 grub_free (data);
Will fix as below:
@@ -953,11 +955,10 @@ grub_erofs_label (grub_device_t device, char **label)
}
*label = grub_strndup ((char *) data->sb.volume_name, sizeof
(data->sb.volume_name));
- if (!*label)
-return grub_errno;
-
grub_free (data);
+ if (!*label)
+return grub_errno;
return GRUB_ERR_NONE;
}
Do these look okay?
Thanks,
Gao Xiang
960
961 return GRUB_ERR_NONE;
962 }
___
Grub-devel mailing list
Grub-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/grub-devel
[PATCH v11 2/2] fs/erofs: Add tests for EROFS in grub-fs-tester
From: Yifan Zhao
In this patch, three tests of EROFS are introduced and they cover
compact, extended and chunk-based inodes, respectively.
Signed-off-by: Yifan Zhao
Reviewed-by: Glenn Washburn
Signed-off-by: Gao Xiang
---
Reviewed-by Link:
https://lists.gnu.org/archive/html/grub-devel/2024-04/msg00101.html
.gitignore | 1 +
Makefile.util.def| 6 ++
tests/erofs_test.in | 20
tests/util/grub-fs-tester.in | 32 +---
4 files changed, 52 insertions(+), 7 deletions(-)
create mode 100644 tests/erofs_test.in
diff --git a/.gitignore b/.gitignore
index 11fcecf5c..4c1f91db8 100644
--- a/.gitignore
+++ b/.gitignore
@@ -104,6 +104,7 @@ widthspec.bin
/docs/version-dev.texi
/docs/version.texi
/ehci_test
+/erofs_test
/example_grub_script_test
/example_scripted_test
/example_unit_test
diff --git a/Makefile.util.def b/Makefile.util.def
index 8d3bc107f..0f74a1680 100644
--- a/Makefile.util.def
+++ b/Makefile.util.def
@@ -764,6 +764,12 @@ script = {
dependencies = 'garbage-gen$(BUILD_EXEEXT)';
};
+script = {
+ testcase = native;
+ name = erofs_test;
+ common = tests/erofs_test.in;
+};
+
script = {
testcase = native;
name = ext234_test;
diff --git a/tests/erofs_test.in b/tests/erofs_test.in
new file mode 100644
index 0..5627a
--- /dev/null
+++ b/tests/erofs_test.in
@@ -0,0 +1,20 @@
+#!@BUILD_SHEBANG@
+
+set -e
+
+if [ "x$EUID" = "x" ] ; then
+ EUID=`id -u`
+fi
+
+if [ "$EUID" != 0 ] ; then
+ exit 99
+fi
+
+if ! which mkfs.erofs >/dev/null 2>&1; then
+ echo "mkfs.erofs not installed; cannot test erofs."
+ exit 99
+fi
+
+"@builddir@/grub-fs-tester" erofs_compact
+"@builddir@/grub-fs-tester" erofs_extended
+"@builddir@/grub-fs-tester" erofs_chunk
diff --git a/tests/util/grub-fs-tester.in b/tests/util/grub-fs-tester.in
index ea8b2d1f6..df5dc7542 100644
--- a/tests/util/grub-fs-tester.in
+++ b/tests/util/grub-fs-tester.in
@@ -227,6 +227,10 @@ for LOGSECSIZE in $(range "$MINLOGSECSIZE"
"$MAXLOGSECSIZE" 1); do
xsquash*)
MINBLKSIZE=4096
MAXBLKSIZE=1048576;;
+ x"erofs_"*)
+ MINBLKSIZE=4096
+ MAXBLKSIZE=4096
+ ;;
xxfs|xf2fs)
MINBLKSIZE=$SECSIZE
# OS Limitation: GNU/Linux doesn't accept > 4096
@@ -382,8 +386,8 @@ for LOGSECSIZE in $(range "$MINLOGSECSIZE" "$MAXLOGSECSIZE"
1); do
FSLABEL="g;/_é䏌䐓䏕䎛䎾䏴кит u"
#FSLABEL="g;/_é莭莽😁кит u"
;;
- # FS LIMITATION: reiserfs, extN and jfs label is at most 16
UTF-8 characters
- x"reiserfs_old" | x"reiserfs" | x"ext"* | x"lvm"* | x"luks"* |
x"mdraid"* | x"jfs" | x"jfs_caseins")
+ # FS LIMITATION: reiserfs, extN, jfs and erofs label is at most
16 UTF-8 characters
+ x"reiserfs_old" | x"reiserfs" | x"ext"* | x"lvm"* | x"luks"* |
x"mdraid"* | x"jfs" | x"jfs_caseins" | x"erofs_"*)
FSLABEL="g;/éт 莭😁";;
# FS LIMITATION: No underscore, space, semicolon, slash or
international characters in UFS* in label. Limited to 32 UTF-8 characters
x"ufs1" | x"ufs1_sun" | x"ufs2")
@@ -661,7 +665,7 @@ for LOGSECSIZE in $(range "$MINLOGSECSIZE" "$MAXLOGSECSIZE"
1); do
x"tarfs" | x"cpio_"*| x"ziso9660" | x"romfs" | x"squash4_"*\
| x"iso9660" | xjoliet | xrockridge | xrockridge_joliet \
| x"iso9660_1999" | xjoliet_1999 | xrockridge_1999 \
- | xrockridge_joliet_1999)
+ | xrockridge_joliet_1999 | x"erofs_"*)
MNTPOINTRW="$MASTER"
MNTPOINTRO="$MASTER"
mkdir -p "$MASTER";;
@@ -805,7 +809,7 @@ for LOGSECSIZE in $(range "$MINLOGSECSIZE" "$MAXLOGSECSIZE"
1); do
sleep 1
"zfs" create "$FSLABEL"/"grub fs"
sleep 1;;
- x"tarfs" | x"cpio_"* | x"iso9660" | xjoliet | xrockridge |
xrockridge_joliet | x"iso9660_1999" | xjoliet_1999 | xrockridge_1999 |
xrockridge_joliet_1999 | x"ziso9660" | x"romfs" | x"squash4_"*)
+ x"tarfs" | x"cpio_"* | x"iso9660" | xjoliet | xrockridge |
xrockridge_joliet | x"iso9660_1999" | xjoliet
[PATCH v11 0/2] Introduce EROFS support
EROFS [1] is a lightweight read-only filesystem designed for performance
which has already been shipped in most Linux distributions as well as widely
used in several scenarios, such as Android system partitions, container
images, and rootfs for embedded devices.
This patch brings EROFS uncompressed support together with related tests.
Now, it's possible to boot directly through GRUB with an EROFS rootfs.
EROFS compressed files will be supported later since it has more work to
polish.
[1] https://erofs.docs.kernel.org
changelog since v10:
- Fix coverity warnings reported by Daniel:
https://lore.kernel.org/grub-devel/20240509175714.fccferxvffylk...@tomti.i.net-space.pl
Tested-by Link (Commit 1):
https://lists.gnu.org/archive/html/grub-devel/2024-05/msg1.html
Reviewed-by Link (Commit 2):
https://lists.gnu.org/archive/html/grub-devel/2024-04/msg00101.html
Yifan Zhao (2):
fs/erofs: Add support for EROFS
fs/erofs: Add tests for EROFS in grub-fs-tester
.gitignore |1 +
INSTALL |8 +-
Makefile.util.def|7 +
docs/grub.texi |3 +-
grub-core/Makefile.core.def |5 +
grub-core/fs/erofs.c | 1008 ++
tests/erofs_test.in | 20 +
tests/util/grub-fs-tester.in | 32 +-
8 files changed, 1072 insertions(+), 12 deletions(-)
create mode 100644 grub-core/fs/erofs.c
create mode 100644 tests/erofs_test.in
Interdiff against v10:
diff --git a/grub-core/fs/erofs.c b/grub-core/fs/erofs.c
index b82212b16..14c86f435 100644
--- a/grub-core/fs/erofs.c
+++ b/grub-core/fs/erofs.c
@@ -396,7 +396,7 @@ erofs_map_blocks_chunkmode (grub_fshelp_node_t node,
unit = EROFS_BLOCK_MAP_ENTRY_SIZE;
chunkbits = node->data->sb.log2_blksz + (chunk_format &
EROFS_CHUNK_FORMAT_BLKBITS_MASK);
- if (chunkbits > 64)
+ if (chunkbits > 63)
return grub_error (GRUB_ERR_BAD_FS, "invalid chunkbits %u @ inode %"
PRIuGRUB_UINT64_T,
chunkbits, node->ino);
@@ -787,11 +787,13 @@ erofs_dir_iter (const char *filename, enum
grub_fshelp_filetype filetype,
{
struct grub_erofs_dir_ctx *ctx = data;
struct grub_dirhook_info info = {0};
+ grub_err_t err;
if (!node->inode_loaded)
{
- erofs_read_inode (ctx->data, node);
- grub_errno = GRUB_ERR_NONE;
+ err = erofs_read_inode (ctx->data, node);
+ if (err != GRUB_ERR_NONE)
+return 0;
}
if (node->inode_loaded)
@@ -953,11 +955,10 @@ grub_erofs_label (grub_device_t device, char **label)
}
*label = grub_strndup ((char *) data->sb.volume_name, sizeof
(data->sb.volume_name));
- if (!*label)
-return grub_errno;
-
grub_free (data);
+ if (!*label)
+return grub_errno;
return GRUB_ERR_NONE;
}
@@ -1004,4 +1005,4 @@ GRUB_MOD_INIT (erofs)
GRUB_MOD_FINI (erofs)
{
grub_fs_unregister (&grub_erofs_fs);
}
--
2.39.3
___
Grub-devel mailing list
Grub-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/grub-devel
[PATCH v11 1/2] fs/erofs: Add support for EROFS
From: Yifan Zhao
EROFS [1] is a lightweight read-only filesystem designed for performance
which has already been shipped in most Linux distributions as well as widely
used in several scenarios, such as Android system partitions, container
images, and rootfs for embedded devices.
This patch brings EROFS uncompressed support. Now, it's possible to boot
directly through GRUB with an EROFS rootfs.
EROFS compressed files will be supported later since it has more work to
polish.
[1] https://erofs.docs.kernel.org
Signed-off-by: Yifan Zhao
Tested-by: Daniel Axtens # fuzz testing only
Signed-off-by: Gao Xiang
---
Tested-by Link:
https://lists.gnu.org/archive/html/grub-devel/2024-05/msg1.html
INSTALL |8 +-
Makefile.util.def |1 +
docs/grub.texi |3 +-
grub-core/Makefile.core.def |5 +
grub-core/fs/erofs.c| 1008 +++
5 files changed, 1020 insertions(+), 5 deletions(-)
create mode 100644 grub-core/fs/erofs.c
diff --git a/INSTALL b/INSTALL
index 8d9207c84..84030c9f4 100644
--- a/INSTALL
+++ b/INSTALL
@@ -77,15 +77,15 @@ Prerequisites for make-check:
* If running a Linux kernel the following modules must be loaded:
- fuse, loop
- - btrfs, ext4, f2fs, fat, hfs, hfsplus, jfs, mac-roman, minix, nilfs2,
+ - btrfs, erofs, ext4, f2fs, fat, hfs, hfsplus, jfs, mac-roman, minix, nilfs2,
reiserfs, udf, xfs
- On newer kernels, the exfat kernel modules may be used instead of the
exfat FUSE filesystem
* The following are Debian named packages required mostly for the full
suite of filesystem testing (but some are needed by other tests as well):
- - btrfs-progs, dosfstools, e2fsprogs, exfat-utils, f2fs-tools, genromfs,
-hfsprogs, jfsutils, nilfs-tools, ntfs-3g, reiserfsprogs, squashfs-tools,
-reiserfsprogs, udftools, xfsprogs, zfs-fuse
+ - btrfs-progs, dosfstools, e2fsprogs, erofs-utils, exfat-utils, f2fs-tools,
+genromfs, hfsprogs, jfsutils, nilfs-tools, ntfs-3g, reiserfsprogs,
+squashfs-tools, reiserfsprogs, udftools, xfsprogs, zfs-fuse
- exfat-fuse, if not using the exfat kernel module
- gzip, lzop, xz-utils
- attr, cpio, g++, gawk, parted, recode, tar, util-linux
diff --git a/Makefile.util.def b/Makefile.util.def
index 9432365a9..8d3bc107f 100644
--- a/Makefile.util.def
+++ b/Makefile.util.def
@@ -98,6 +98,7 @@ library = {
common = grub-core/fs/cpio_be.c;
common = grub-core/fs/odc.c;
common = grub-core/fs/newc.c;
+ common = grub-core/fs/erofs.c;
common = grub-core/fs/ext2.c;
common = grub-core/fs/fat.c;
common = grub-core/fs/exfat.c;
diff --git a/docs/grub.texi b/docs/grub.texi
index d32266f69..b198d963d 100644
--- a/docs/grub.texi
+++ b/docs/grub.texi
@@ -353,6 +353,7 @@ blocklist notation. The currently supported filesystem
types are @dfn{Amiga
Fast FileSystem (AFFS)}, @dfn{AtheOS fs}, @dfn{BeFS},
@dfn{BtrFS} (including raid0, raid1, raid10, gzip and lzo),
@dfn{cpio} (little- and big-endian bin, odc and newc variants),
+@dfn{EROFS} (only uncompressed support for now),
@dfn{Linux ext2/ext3/ext4}, @dfn{DOS FAT12/FAT16/FAT32},
@dfn{exFAT}, @dfn{F2FS}, @dfn{HFS}, @dfn{HFS+},
@dfn{ISO9660} (including Joliet, Rock-ridge and multi-chunk files),
@@ -6276,7 +6277,7 @@ assumed to be encoded in UTF-8.
NTFS, JFS, UDF, HFS+, exFAT, long filenames in FAT, Joliet part of
ISO9660 are treated as UTF-16 as per specification. AFS and BFS are read
as UTF-8, again according to specification. BtrFS, cpio, tar, squash4, minix,
-minix2, minix3, ROMFS, ReiserFS, XFS, ext2, ext3, ext4, FAT (short names),
+minix2, minix3, ROMFS, ReiserFS, XFS, EROFS, ext2, ext3, ext4, FAT (short
names),
F2FS, RockRidge part of ISO9660, nilfs2, UFS1, UFS2 and ZFS are assumed
to be UTF-8. This might be false on systems configured with legacy charset
but as long as the charset used is superset of ASCII you should be able to
diff --git a/grub-core/Makefile.core.def b/grub-core/Makefile.core.def
index 8e1b1d9f3..7fa9446bd 100644
--- a/grub-core/Makefile.core.def
+++ b/grub-core/Makefile.core.def
@@ -1442,6 +1442,11 @@ module = {
common = fs/odc.c;
};
+module = {
+ name = erofs;
+ common = fs/erofs.c;
+};
+
module = {
name = ext2;
common = fs/ext2.c;
diff --git a/grub-core/fs/erofs.c b/grub-core/fs/erofs.c
new file mode 100644
index 0..14c86f435
--- /dev/null
+++ b/grub-core/fs/erofs.c
@@ -0,0 +1,1008 @@
+/* erofs.c - Enhanced Read-Only File System */
+/*
+ * GRUB -- GRand Unified Bootloader
+ * Copyright (C) 2024 Free Software Foundation, Inc.
+ *
+ * GRUB is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 3 of the License, or
+ * (at your option) any later version.
+ *
+ * GRUB is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILI
Re: [PATCH v11 2/2] fs/erofs: Add tests for EROFS in grub-fs-tester
On 2024/5/17 05:37, Daniel Kiper wrote: On Fri, May 10, 2024 at 08:52:56AM +0800, Gao Xiang wrote: From: Yifan Zhao In this patch, three tests of EROFS are introduced and they cover compact, extended and chunk-based inodes, respectively. Signed-off-by: Yifan Zhao Reviewed-by: Glenn Washburn Signed-off-by: Gao Xiang Reviewed-by: Daniel Kiper Thanks for the review! Thanks, Gao Xiang Daniel ___ Grub-devel mailing list Grub-devel@gnu.org https://lists.gnu.org/mailman/listinfo/grub-devel
Re: [PATCH v11 1/2] fs/erofs: Add support for EROFS
Hi Daniel,
On 2024/5/17 05:33, Daniel Kiper wrote:
On Fri, May 10, 2024 at 08:52:55AM +0800, Gao Xiang wrote:
...
In general patch LGTM except some nits...
Thanks for the comments. Let me address them.
---
Tested-by Link:
https://lists.gnu.org/archive/html/grub-devel/2024-05/msg1.html
...
+struct grub_erofs_super
+{
+ grub_uint32_t magic;
+ grub_uint32_t checksum;
+ grub_uint32_t feature_compat;
+ grub_uint8_t log2_blksz;
+ grub_uint8_t sb_extslots;
May I ask you to align struct/union members in the following way?
grub_uint32_t magic;
grub_uint32_t checksum;
grub_uint32_t feature_compat;
grub_uint8_t log2_blksz;
grub_uint8_t sb_extslots;
grub_uint16_t root_nid;
grub_uint64_t inos;
grub_uint64_t build_time;
grub_uint32_t build_time_nsec;
...
Sorry, anyway, I'm not quite familiar with GRUB coding
styles.
It seems that using TABs in the middle of names and types.
Is my understanding correct? I will try to update like this.
Please do this for all/most structs and unions definitions below.
...
+
+static grub_err_t
+erofs_read_inode (struct grub_erofs_data *data, grub_fshelp_node_t node)
+{
+ union grub_erofs_inode *di;
+ grub_err_t err;
+ grub_uint16_t i_format;
+ grub_uint64_t addr = erofs_iloc (node);
+
+ di = (union grub_erofs_inode *) &node->inode;
+
+ err = grub_disk_read (data->disk, addr >> GRUB_DISK_SECTOR_BITS,
+ addr & (GRUB_DISK_SECTOR_SIZE - 1),
+ sizeof (struct grub_erofs_inode_compact), &di->c);
+ if (err != GRUB_ERR_NONE)
+return err;
+
+ i_format = grub_le_to_cpu16 (di->c.i_format);
+ node->inode_type = (i_format >> EROFS_I_VERSION_BIT) & EROFS_I_VERSION_MASKS;
+ node->inode_datalayout = (i_format >> EROFS_I_DATALAYOUT_BIT) &
EROFS_I_DATALAYOUT_MASKS;
+
+ switch (node->inode_type)
+{
+case EROFS_INODE_LAYOUT_EXTENDED:
+ addr += sizeof (struct grub_erofs_inode_compact);
+ err = grub_disk_read (
+ data->disk, addr >> GRUB_DISK_SECTOR_BITS,
+ addr & (GRUB_DISK_SECTOR_SIZE - 1),
+ sizeof (struct grub_erofs_inode_extended) - sizeof (struct
grub_erofs_inode_compact),
+ (grub_uint8_t *) di + sizeof (struct grub_erofs_inode_compact));
This function call is unreadable. Please fix it. I am OK with lines
longer than 80 chars. So, first argument for the grub_disk_read() should
be immediately behind "(". Then other arguments may follow below
starting from the same column with the first argument.
grub_disk_read (data->disk, addr >> GRUB_DISK_SECTOR_BITS,
addr & (GRUB_DISK_SECTOR_SIZE - 1),
...
Okay, will fix.
...
+
+ /* file_size is checked by caller and cannot be zero, hence nblocks > 0 */
+ file_size = erofs_inode_file_size (node);
+ if (grub_add (file_size, blocksz - 1, &nblocks))
+return grub_error (GRUB_ERR_OUT_OF_RANGE, "overflow is detected");
You use "overflow is detected" message everywhere. It is not helpful.
I think (almost) every message should be different, e.g. "nblocks overflow"
or "invalid argument: %d..." or ...
Let me fix it as "nblocks overflow".
+ nblocks >>= node->data->sb.log2_blksz;
+ lastblk = nblocks - tailendpacking;
+
+ map->m_flags = EROFS_MAP_MAPPED;
+
+ /* no overflow as (lastblk <= nblocks) && (nblocks * blocksz <= UINT64_MAX -
blocksz + 1) */
+ if (map->m_la < (lastblk * blocksz))
+{
+ if (grub_mul ((grub_uint64_t) grub_le_to_cpu32
(node->inode.e.i_u.raw_blkaddr), blocksz,
+ &map->m_pa) ||
Please move this to the line with grub_mul()...
Okay, will fix as
if (grub_mul ((grub_uint64_t) grub_le_to_cpu32 (node->inode.e.i_u.raw_blkaddr),
blocksz, &map->m_pa) ||
+ grub_add (map->m_pa, map->m_la, &map->m_pa))
+ return grub_error (GRUB_ERR_OUT_OF_RANGE, "overflow is detected");
Again, this message does not help...
Will fix as "m_pa overflow".
+ if (grub_sub (lastblk * blocksz, map->m_la, &map->m_plen))
+ return grub_error (GRUB_ERR_OUT_OF_RANGE, "overflow is detected");
To be precise, this is an underflow...
Will fix as "m_plen underflow".
+}
+ else if (tailendpacking)
+{
+ if (grub_add (erofs_iloc (node), erofs_inode_size (node), &map->m_pa) ||
+ grub_add (map->m_pa, erofs_inode_xattr_ibody_size (node), &map->m_pa)
||
+ grub_add (map->m_pa, map->m_la % blocksz, &map->m_pa))
+ return grub_error (GRUB_ERR_OUT_OF_RANGE, "overflow is detected");
Please fix all these
[PATCH v12 2/2] fs/erofs: Add tests for EROFS in grub-fs-tester
From: Yifan Zhao
In this patch, three tests of EROFS are introduced and they cover
compact, extended and chunk-based inodes, respectively.
Signed-off-by: Yifan Zhao
Reviewed-by: Glenn Washburn
Reviewed-by: Daniel Kiper
Signed-off-by: Gao Xiang
---
.gitignore | 1 +
Makefile.util.def| 6 ++
tests/erofs_test.in | 20
tests/util/grub-fs-tester.in | 32 +---
4 files changed, 52 insertions(+), 7 deletions(-)
create mode 100644 tests/erofs_test.in
diff --git a/.gitignore b/.gitignore
index 11fcecf5c..4c1f91db8 100644
--- a/.gitignore
+++ b/.gitignore
@@ -104,6 +104,7 @@ widthspec.bin
/docs/version-dev.texi
/docs/version.texi
/ehci_test
+/erofs_test
/example_grub_script_test
/example_scripted_test
/example_unit_test
diff --git a/Makefile.util.def b/Makefile.util.def
index 8d3bc107f..0f74a1680 100644
--- a/Makefile.util.def
+++ b/Makefile.util.def
@@ -764,6 +764,12 @@ script = {
dependencies = 'garbage-gen$(BUILD_EXEEXT)';
};
+script = {
+ testcase = native;
+ name = erofs_test;
+ common = tests/erofs_test.in;
+};
+
script = {
testcase = native;
name = ext234_test;
diff --git a/tests/erofs_test.in b/tests/erofs_test.in
new file mode 100644
index 0..5627a
--- /dev/null
+++ b/tests/erofs_test.in
@@ -0,0 +1,20 @@
+#!@BUILD_SHEBANG@
+
+set -e
+
+if [ "x$EUID" = "x" ] ; then
+ EUID=`id -u`
+fi
+
+if [ "$EUID" != 0 ] ; then
+ exit 99
+fi
+
+if ! which mkfs.erofs >/dev/null 2>&1; then
+ echo "mkfs.erofs not installed; cannot test erofs."
+ exit 99
+fi
+
+"@builddir@/grub-fs-tester" erofs_compact
+"@builddir@/grub-fs-tester" erofs_extended
+"@builddir@/grub-fs-tester" erofs_chunk
diff --git a/tests/util/grub-fs-tester.in b/tests/util/grub-fs-tester.in
index ea8b2d1f6..df5dc7542 100644
--- a/tests/util/grub-fs-tester.in
+++ b/tests/util/grub-fs-tester.in
@@ -227,6 +227,10 @@ for LOGSECSIZE in $(range "$MINLOGSECSIZE"
"$MAXLOGSECSIZE" 1); do
xsquash*)
MINBLKSIZE=4096
MAXBLKSIZE=1048576;;
+ x"erofs_"*)
+ MINBLKSIZE=4096
+ MAXBLKSIZE=4096
+ ;;
xxfs|xf2fs)
MINBLKSIZE=$SECSIZE
# OS Limitation: GNU/Linux doesn't accept > 4096
@@ -382,8 +386,8 @@ for LOGSECSIZE in $(range "$MINLOGSECSIZE" "$MAXLOGSECSIZE"
1); do
FSLABEL="g;/_é䏌䐓䏕䎛䎾䏴кит u"
#FSLABEL="g;/_é莭莽😁кит u"
;;
- # FS LIMITATION: reiserfs, extN and jfs label is at most 16
UTF-8 characters
- x"reiserfs_old" | x"reiserfs" | x"ext"* | x"lvm"* | x"luks"* |
x"mdraid"* | x"jfs" | x"jfs_caseins")
+ # FS LIMITATION: reiserfs, extN, jfs and erofs label is at most
16 UTF-8 characters
+ x"reiserfs_old" | x"reiserfs" | x"ext"* | x"lvm"* | x"luks"* |
x"mdraid"* | x"jfs" | x"jfs_caseins" | x"erofs_"*)
FSLABEL="g;/éт 莭😁";;
# FS LIMITATION: No underscore, space, semicolon, slash or
international characters in UFS* in label. Limited to 32 UTF-8 characters
x"ufs1" | x"ufs1_sun" | x"ufs2")
@@ -661,7 +665,7 @@ for LOGSECSIZE in $(range "$MINLOGSECSIZE" "$MAXLOGSECSIZE"
1); do
x"tarfs" | x"cpio_"*| x"ziso9660" | x"romfs" | x"squash4_"*\
| x"iso9660" | xjoliet | xrockridge | xrockridge_joliet \
| x"iso9660_1999" | xjoliet_1999 | xrockridge_1999 \
- | xrockridge_joliet_1999)
+ | xrockridge_joliet_1999 | x"erofs_"*)
MNTPOINTRW="$MASTER"
MNTPOINTRO="$MASTER"
mkdir -p "$MASTER";;
@@ -805,7 +809,7 @@ for LOGSECSIZE in $(range "$MINLOGSECSIZE" "$MAXLOGSECSIZE"
1); do
sleep 1
"zfs" create "$FSLABEL"/"grub fs"
sleep 1;;
- x"tarfs" | x"cpio_"* | x"iso9660" | xjoliet | xrockridge |
xrockridge_joliet | x"iso9660_1999" | xjoliet_1999 | xrockridge_1999 |
xrockridge_joliet_1999 | x"ziso9660" | x"romfs" | x"squash4_"*)
+ x"tarfs" | x"cpio_"* | x"iso9660" | xjoliet | xrockridge |
xrockridge_joliet | x"iso9660_1999" | xjoliet_1999 | xrockridge_1999 |
xrockridge_joliet_1999 | x"
[PATCH v12 1/2] fs/erofs: Add support for EROFS
From: Yifan Zhao
EROFS [1] is a lightweight read-only filesystem designed for performance
which has already been shipped in most Linux distributions as well as widely
used in several scenarios, such as Android system partitions, container
images, and rootfs for embedded devices.
This patch brings EROFS uncompressed support. Now, it's possible to boot
directly through GRUB with an EROFS rootfs.
EROFS compressed files will be supported later since it has more work to
polish.
[1] https://erofs.docs.kernel.org
Signed-off-by: Yifan Zhao
Tested-by: Daniel Axtens # fuzz testing only
Signed-off-by: Gao Xiang
---
INSTALL |8 +-
Makefile.util.def |1 +
docs/grub.texi |3 +-
grub-core/Makefile.core.def |5 +
grub-core/fs/erofs.c| 1002 +++
5 files changed, 1014 insertions(+), 5 deletions(-)
create mode 100644 grub-core/fs/erofs.c
diff --git a/INSTALL b/INSTALL
index 8d9207c84..84030c9f4 100644
--- a/INSTALL
+++ b/INSTALL
@@ -77,15 +77,15 @@ Prerequisites for make-check:
* If running a Linux kernel the following modules must be loaded:
- fuse, loop
- - btrfs, ext4, f2fs, fat, hfs, hfsplus, jfs, mac-roman, minix, nilfs2,
+ - btrfs, erofs, ext4, f2fs, fat, hfs, hfsplus, jfs, mac-roman, minix, nilfs2,
reiserfs, udf, xfs
- On newer kernels, the exfat kernel modules may be used instead of the
exfat FUSE filesystem
* The following are Debian named packages required mostly for the full
suite of filesystem testing (but some are needed by other tests as well):
- - btrfs-progs, dosfstools, e2fsprogs, exfat-utils, f2fs-tools, genromfs,
-hfsprogs, jfsutils, nilfs-tools, ntfs-3g, reiserfsprogs, squashfs-tools,
-reiserfsprogs, udftools, xfsprogs, zfs-fuse
+ - btrfs-progs, dosfstools, e2fsprogs, erofs-utils, exfat-utils, f2fs-tools,
+genromfs, hfsprogs, jfsutils, nilfs-tools, ntfs-3g, reiserfsprogs,
+squashfs-tools, reiserfsprogs, udftools, xfsprogs, zfs-fuse
- exfat-fuse, if not using the exfat kernel module
- gzip, lzop, xz-utils
- attr, cpio, g++, gawk, parted, recode, tar, util-linux
diff --git a/Makefile.util.def b/Makefile.util.def
index 9432365a9..8d3bc107f 100644
--- a/Makefile.util.def
+++ b/Makefile.util.def
@@ -98,6 +98,7 @@ library = {
common = grub-core/fs/cpio_be.c;
common = grub-core/fs/odc.c;
common = grub-core/fs/newc.c;
+ common = grub-core/fs/erofs.c;
common = grub-core/fs/ext2.c;
common = grub-core/fs/fat.c;
common = grub-core/fs/exfat.c;
diff --git a/docs/grub.texi b/docs/grub.texi
index d32266f69..b198d963d 100644
--- a/docs/grub.texi
+++ b/docs/grub.texi
@@ -353,6 +353,7 @@ blocklist notation. The currently supported filesystem
types are @dfn{Amiga
Fast FileSystem (AFFS)}, @dfn{AtheOS fs}, @dfn{BeFS},
@dfn{BtrFS} (including raid0, raid1, raid10, gzip and lzo),
@dfn{cpio} (little- and big-endian bin, odc and newc variants),
+@dfn{EROFS} (only uncompressed support for now),
@dfn{Linux ext2/ext3/ext4}, @dfn{DOS FAT12/FAT16/FAT32},
@dfn{exFAT}, @dfn{F2FS}, @dfn{HFS}, @dfn{HFS+},
@dfn{ISO9660} (including Joliet, Rock-ridge and multi-chunk files),
@@ -6276,7 +6277,7 @@ assumed to be encoded in UTF-8.
NTFS, JFS, UDF, HFS+, exFAT, long filenames in FAT, Joliet part of
ISO9660 are treated as UTF-16 as per specification. AFS and BFS are read
as UTF-8, again according to specification. BtrFS, cpio, tar, squash4, minix,
-minix2, minix3, ROMFS, ReiserFS, XFS, ext2, ext3, ext4, FAT (short names),
+minix2, minix3, ROMFS, ReiserFS, XFS, EROFS, ext2, ext3, ext4, FAT (short
names),
F2FS, RockRidge part of ISO9660, nilfs2, UFS1, UFS2 and ZFS are assumed
to be UTF-8. This might be false on systems configured with legacy charset
but as long as the charset used is superset of ASCII you should be able to
diff --git a/grub-core/Makefile.core.def b/grub-core/Makefile.core.def
index 8e1b1d9f3..7fa9446bd 100644
--- a/grub-core/Makefile.core.def
+++ b/grub-core/Makefile.core.def
@@ -1442,6 +1442,11 @@ module = {
common = fs/odc.c;
};
+module = {
+ name = erofs;
+ common = fs/erofs.c;
+};
+
module = {
name = ext2;
common = fs/ext2.c;
diff --git a/grub-core/fs/erofs.c b/grub-core/fs/erofs.c
new file mode 100644
index 0..630f95e75
--- /dev/null
+++ b/grub-core/fs/erofs.c
@@ -0,0 +1,1002 @@
+/* erofs.c - Enhanced Read-Only File System */
+/*
+ * GRUB -- GRand Unified Bootloader
+ * Copyright (C) 2024 Free Software Foundation, Inc.
+ *
+ * GRUB is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 3 of the License, or
+ * (at your option) any later version.
+ *
+ * GRUB is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for
[PATCH v12 0/2] Introduce EROFS support
Hi folks,
EROFS [1] is a lightweight read-only filesystem designed for performance
which has already been shipped in most Linux distributions as well as widely
used in several scenarios, such as Android system partitions, container
images, and rootfs for embedded devices.
This patch brings EROFS uncompressed support together with related tests.
Now, it's possible to boot directly through GRUB with an EROFS rootfs.
EROFS compressed files will be supported later since it has more work to
polish.
[1] https://erofs.docs.kernel.org
changelog since v11:
- Address nits mentioned by Daniel for better reading:
https://lore.kernel.org/grub-devel/zkz7xw56awybf...@tomti.i.net-space.pl/
- Simplify erofs_read_symlink() since erofs_inode_file_size() is checked
against EROFS_PATH_LEN(4096) first.
Tested-by Link (Commit 1):
https://lists.gnu.org/archive/html/grub-devel/2024-05/msg1.html
Reviewed-by Link (Commit 2):
https://lists.gnu.org/archive/html/grub-devel/2024-04/msg00101.html
Reviewed-by Link (Commit 2):
https://lore.kernel.org/grub-devel/zkz8fdd+zckt+...@tomti.i.net-space.pl/
Yifan Zhao (2):
fs/erofs: Add support for EROFS
fs/erofs: Add tests for EROFS in grub-fs-tester
.gitignore |1 +
INSTALL |8 +-
Makefile.util.def|7 +
docs/grub.texi |3 +-
grub-core/Makefile.core.def |5 +
grub-core/fs/erofs.c | 1002 ++
tests/erofs_test.in | 20 +
tests/util/grub-fs-tester.in | 32 +-
8 files changed, 1066 insertions(+), 12 deletions(-)
create mode 100644 grub-core/fs/erofs.c
create mode 100644 tests/erofs_test.in
Interdiff against v11:
diff --git a/grub-core/fs/erofs.c b/grub-core/fs/erofs.c
index 14c86f435..630f95e75 100644
--- a/grub-core/fs/erofs.c
+++ b/grub-core/fs/erofs.c
@@ -39,37 +39,37 @@ GRUB_MOD_LICENSE ("GPLv3+");
struct grub_erofs_super
{
- grub_uint32_t magic;
- grub_uint32_t checksum;
- grub_uint32_t feature_compat;
- grub_uint8_t log2_blksz;
- grub_uint8_t sb_extslots;
-
- grub_uint16_t root_nid;
- grub_uint64_t inos;
-
- grub_uint64_t build_time;
- grub_uint32_t build_time_nsec;
- grub_uint32_t blocks;
- grub_uint32_t meta_blkaddr;
- grub_uint32_t xattr_blkaddr;
- grub_packed_guid_t uuid;
- grub_uint8_t volume_name[16];
- grub_uint32_t feature_incompat;
+ grub_uint32_tmagic;
+ grub_uint32_tchecksum;
+ grub_uint32_tfeature_compat;
+ grub_uint8_t log2_blksz;
+ grub_uint8_t sb_extslots;
+
+ grub_uint16_troot_nid;
+ grub_uint64_tinos;
+
+ grub_uint64_tbuild_time;
+ grub_uint32_tbuild_time_nsec;
+ grub_uint32_tblocks;
+ grub_uint32_tmeta_blkaddr;
+ grub_uint32_txattr_blkaddr;
+ grub_packed_guid_t uuid;
+ grub_uint8_t volume_name[16];
+ grub_uint32_tfeature_incompat;
union
{
-grub_uint16_t available_compr_algs;
-grub_uint16_t lz4_max_distance;
+grub_uint16_t available_compr_algs;
+grub_uint16_t lz4_max_distance;
} GRUB_PACKED u1;
- grub_uint16_t extra_devices;
- grub_uint16_t devt_slotoff;
- grub_uint8_t log2_dirblksz;
- grub_uint8_t xattr_prefix_count;
- grub_uint32_t xattr_prefix_start;
- grub_uint64_t packed_nid;
- grub_uint8_t reserved2[24];
+ grub_uint16_textra_devices;
+ grub_uint16_tdevt_slotoff;
+ grub_uint8_t log2_dirblksz;
+ grub_uint8_t xattr_prefix_count;
+ grub_uint32_txattr_prefix_start;
+ grub_uint64_tpacked_nid;
+ grub_uint8_t reserved2[24];
} GRUB_PACKED;
#define EROFS_INODE_LAYOUT_COMPACT 0
@@ -89,8 +89,8 @@ struct grub_erofs_super
struct grub_erofs_inode_chunk_info
{
- grub_uint16_t format;
- grub_uint16_t reserved;
+ grub_uint16_tformat;
+ grub_uint16_treserved;
} GRUB_PACKED;
#define EROFS_CHUNK_FORMAT_BLKBITS_MASK0x001F
@@ -107,64 +107,64 @@ struct grub_erofs_inode_chunk_info
struct grub_erofs_inode_chunk_index
{
- grub_uint16_t advise;
- grub_uint16_t device_id;
- grub_uint32_t blkaddr;
+ grub_uint16_tadvise;
+ grub_uint16_tdevice_id;
+ grub_uint32_tblkaddr;
};
union grub_erofs_inode_i_u
{
- grub_uint32_t compressed_blocks;
- grub_uint32_t raw_blkaddr;
+ grub_uint32_tcompressed_blocks;
+ grub_uint32_traw_blkaddr;
- grub_uint32_t rdev;
+ grub_uint32_trdev;
- struct grub_erofs_inode_chunk_info c;
+ struct grub_erofs_inode_chunk_info c;
};
struct grub_erofs_inode_compact
{
- grub_uint16_t i_format;
+ grub_uint16_ti_format;
- grub_uint16_t i_xattr_icount;
- grub_uint16_t i_mode;
- grub_uint16_t
Re: [PATCH v11 1/2] fs/erofs: Add support for EROFS
Hi Daniel, On 2024/5/17 08:34, Gao Xiang wrote: Hi Daniel, On 2024/5/17 05:33, Daniel Kiper wrote: On Fri, May 10, 2024 at 08:52:55AM +0800, Gao Xiang wrote: ... In general patch LGTM except some nits... Thanks for the comments. Let me address them. I've addressed your comments as a try, please kindly take some time at the latest diff and full version... https://lore.kernel.org/grub-devel/20240517044054.2752375-1-hsiang...@linux.alibaba.com Thanks, Gao Xiang Daniel ___ Grub-devel mailing list Grub-devel@gnu.org https://lists.gnu.org/mailman/listinfo/grub-devel
Re: [PATCH v12 1/2] fs/erofs: Add support for EROFS
Hi Daniel,
On 2024/5/17 18:22, Daniel Kiper wrote:
On Fri, May 17, 2024 at 12:40:53PM +0800, Gao Xiang wrote:
From: Yifan Zhao
EROFS [1] is a lightweight read-only filesystem designed for performance
which has already been shipped in most Linux distributions as well as widely
used in several scenarios, such as Android system partitions, container
images, and rootfs for embedded devices.
This patch brings EROFS uncompressed support. Now, it's possible to boot
directly through GRUB with an EROFS rootfs.
EROFS compressed files will be supported later since it has more work to
polish.
[1] https://erofs.docs.kernel.org
Signed-off-by: Yifan Zhao
Tested-by: Daniel Axtens # fuzz testing only
Signed-off-by: Gao Xiang
---
[...]
+static grub_err_t
+erofs_map_blocks_chunkmode (grub_fshelp_node_t node,
+ struct grub_erofs_map_blocks *map)
+{
+ grub_uint16_t chunk_format = grub_le_to_cpu16 (node->inode.e.i_u.c.format);
+ grub_uint64_t unit, pos, chunknr, blkaddr;
+ grub_uint8_t chunkbits;
+ grub_err_t err;
+
+ if (chunk_format & EROFS_CHUNK_FORMAT_INDEXES)
+unit = sizeof (struct grub_erofs_inode_chunk_index);
+ else
+unit = EROFS_BLOCK_MAP_ENTRY_SIZE;
+
+ chunkbits = node->data->sb.log2_blksz + (chunk_format &
EROFS_CHUNK_FORMAT_BLKBITS_MASK);
+ if (chunkbits > 63)
+return grub_error (GRUB_ERR_BAD_FS, "invalid chunkbits %u @ inode %"
PRIuGRUB_UINT64_T,
+ chunkbits, node->ino);
+
+ chunknr = map->m_la >> chunkbits;
+
+ if (grub_add (erofs_iloc (node), erofs_inode_size (node), &pos))
+return grub_error (GRUB_ERR_OUT_OF_RANGE, "chunkmap position overflow when
adding inode size");
+
+ if (grub_add (pos, erofs_inode_xattr_ibody_size (node), &pos))
+return grub_error (GRUB_ERR_OUT_OF_RANGE, "chunkmap position overflow when
adding xattr size");
+
+ /* pos = ALIGN_UP(pos, unit) */
+ if (grub_add (pos, unit - 1, &pos))
+return grub_error (GRUB_ERR_OUT_OF_RANGE, "position overflow when seeking at
the start of chunkmap");
+ pos &= ~(unit - 1);
Please create a macro as I asked you earlier. Be careful with underflows too.
Since I'm not sure how does it looks like, I wrote the
following helper, hopefully it looks good to you:
#define ALIGN_UP_OVF(v, align, res) \
({ \
bool __failed;\
typeof(v) a; \
\
__failed = grub_sub (align, 1, &a); \
if (__failed == false)\
{ \
__failed = grub_add (v, a, res);\
if (__failed == false) \
{ \
*(res) &= ~(a); \
} \
} \
__failed;})
Otherwise patch set LGTM...
Thanks! Let me send the next version soon.
Thanks,
Gao Xiang
Daniel
___
Grub-devel mailing list
Grub-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/grub-devel
[PATCH v13 1/3] safemath: Add ALIGN_UP_OVF() that checks for {over, under}flow
The following EROFS patch will use this helper to handle overflow
ALIGN_UP() cases.
Signed-off-by: Gao Xiang
---
include/grub/safemath.h | 16
1 file changed, 16 insertions(+)
diff --git a/include/grub/safemath.h b/include/grub/safemath.h
index fbd9b5925..baaea0ef4 100644
--- a/include/grub/safemath.h
+++ b/include/grub/safemath.h
@@ -32,6 +32,22 @@
#define grub_cast(a, res) grub_add ((a), 0, (res))
+#define ALIGN_UP_OVF(v, align, res)\
+({ \
+ bool __failed; \
+ typeof(v) a; \
+ \
+ __failed = grub_sub ((typeof(v))(align), 1, &a); \
+ if (__failed == false) \
+{ \
+__failed = grub_add (v, a, res); \
+if (__failed == false) \
+ {\
+*(res) &= ~a; \
+ }\
+} \
+__failed;})
+
#else
#error gcc 5.1 or newer or clang 8.0 or newer is required
#endif
--
2.39.3
___
Grub-devel mailing list
Grub-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/grub-devel
[PATCH v13 0/3] Introduce EROFS support
Hi folks,
EROFS [1] is a lightweight read-only filesystem designed for performance
which has already been shipped in most Linux distributions as well as widely
used in several scenarios, such as Android system partitions, container
images, and rootfs for embedded devices.
This patch brings EROFS uncompressed support together with related tests.
Now, it's possible to boot directly through GRUB with an EROFS rootfs.
EROFS compressed files will be supported later since it has more work to
polish.
[1] https://erofs.docs.kernel.org
v12:
https://lore.kernel.org/grub-devel/20240517044054.2752375-1-hsiang...@linux.alibaba.com
changelog since v12:
- Address ALIGN_UP_OVF() suggested by Daniel:
https://lore.kernel.org/grub-devel/zkcv7g7mjbg8s...@tomti.i.net-space.pl
Tested-by Link (Commit 1):
https://lists.gnu.org/archive/html/grub-devel/2024-05/msg1.html
Reviewed-by Link (Commit 2):
https://lists.gnu.org/archive/html/grub-devel/2024-04/msg00101.html
Reviewed-by Link (Commit 2):
https://lore.kernel.org/grub-devel/zkz8fdd+zckt+...@tomti.i.net-space.pl/
Gao Xiang (1):
safemath: Add ALIGN_UP_OVF() that checks for {over,under}flow
Yifan Zhao (2):
fs/erofs: Add support for EROFS
fs/erofs: Add tests for EROFS in grub-fs-tester
.gitignore |1 +
INSTALL |8 +-
Makefile.util.def|7 +
docs/grub.texi |3 +-
grub-core/Makefile.core.def |5 +
grub-core/fs/erofs.c | 1000 ++
include/grub/safemath.h | 16 +
tests/erofs_test.in | 20 +
tests/util/grub-fs-tester.in | 32 +-
9 files changed, 1080 insertions(+), 12 deletions(-)
create mode 100644 grub-core/fs/erofs.c
create mode 100644 tests/erofs_test.in
Interdiff against v12:
diff --git a/grub-core/fs/erofs.c b/grub-core/fs/erofs.c
index 630f95e75..005d975de 100644
--- a/grub-core/fs/erofs.c
+++ b/grub-core/fs/erofs.c
@@ -405,10 +405,8 @@ erofs_map_blocks_chunkmode (grub_fshelp_node_t node,
if (grub_add (pos, erofs_inode_xattr_ibody_size (node), &pos))
return grub_error (GRUB_ERR_OUT_OF_RANGE, "chunkmap position overflow when
adding xattr size");
- /* pos = ALIGN_UP(pos, unit) */
- if (grub_add (pos, unit - 1, &pos))
+ if (ALIGN_UP_OVF (pos, unit - 1, &pos))
return grub_error (GRUB_ERR_OUT_OF_RANGE, "position overflow when seeking
at the start of chunkmap");
- pos &= ~(unit - 1);
/* No overflow for multiplication as chunkbits >= 9 and sizeof(unit) <= 8. */
if (grub_add (pos, chunknr * unit, &pos))
diff --git a/include/grub/safemath.h b/include/grub/safemath.h
index fbd9b5925..baaea0ef4 100644
--- a/include/grub/safemath.h
+++ b/include/grub/safemath.h
@@ -32,6 +32,22 @@
#define grub_cast(a, res) grub_add ((a), 0, (res))
+#define ALIGN_UP_OVF(v, align, res)\
+({ \
+ bool __failed; \
+ typeof(v) a; \
+ \
+ __failed = grub_sub ((typeof(v))(align), 1, &a); \
+ if (__failed == false) \
+{ \
+__failed = grub_add (v, a, res); \
+if (__failed == false) \
+ {\
+*(res) &= ~a; \
+ }\
+} \
+__failed;})
+
#else
#error gcc 5.1 or newer or clang 8.0 or newer is required
#endif
--
2.39.3
___
Grub-devel mailing list
Grub-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/grub-devel
[PATCH v13 3/3] fs/erofs: Add tests for EROFS in grub-fs-tester
From: Yifan Zhao
In this patch, three tests of EROFS are introduced and they cover
compact, extended and chunk-based inodes, respectively.
Signed-off-by: Yifan Zhao
Reviewed-by: Glenn Washburn
Reviewed-by: Daniel Kiper
Signed-off-by: Gao Xiang
---
.gitignore | 1 +
Makefile.util.def| 6 ++
tests/erofs_test.in | 20
tests/util/grub-fs-tester.in | 32 +---
4 files changed, 52 insertions(+), 7 deletions(-)
create mode 100644 tests/erofs_test.in
diff --git a/.gitignore b/.gitignore
index 11fcecf5c..4c1f91db8 100644
--- a/.gitignore
+++ b/.gitignore
@@ -104,6 +104,7 @@ widthspec.bin
/docs/version-dev.texi
/docs/version.texi
/ehci_test
+/erofs_test
/example_grub_script_test
/example_scripted_test
/example_unit_test
diff --git a/Makefile.util.def b/Makefile.util.def
index 8d3bc107f..0f74a1680 100644
--- a/Makefile.util.def
+++ b/Makefile.util.def
@@ -764,6 +764,12 @@ script = {
dependencies = 'garbage-gen$(BUILD_EXEEXT)';
};
+script = {
+ testcase = native;
+ name = erofs_test;
+ common = tests/erofs_test.in;
+};
+
script = {
testcase = native;
name = ext234_test;
diff --git a/tests/erofs_test.in b/tests/erofs_test.in
new file mode 100644
index 0..5627a
--- /dev/null
+++ b/tests/erofs_test.in
@@ -0,0 +1,20 @@
+#!@BUILD_SHEBANG@
+
+set -e
+
+if [ "x$EUID" = "x" ] ; then
+ EUID=`id -u`
+fi
+
+if [ "$EUID" != 0 ] ; then
+ exit 99
+fi
+
+if ! which mkfs.erofs >/dev/null 2>&1; then
+ echo "mkfs.erofs not installed; cannot test erofs."
+ exit 99
+fi
+
+"@builddir@/grub-fs-tester" erofs_compact
+"@builddir@/grub-fs-tester" erofs_extended
+"@builddir@/grub-fs-tester" erofs_chunk
diff --git a/tests/util/grub-fs-tester.in b/tests/util/grub-fs-tester.in
index ea8b2d1f6..df5dc7542 100644
--- a/tests/util/grub-fs-tester.in
+++ b/tests/util/grub-fs-tester.in
@@ -227,6 +227,10 @@ for LOGSECSIZE in $(range "$MINLOGSECSIZE"
"$MAXLOGSECSIZE" 1); do
xsquash*)
MINBLKSIZE=4096
MAXBLKSIZE=1048576;;
+ x"erofs_"*)
+ MINBLKSIZE=4096
+ MAXBLKSIZE=4096
+ ;;
xxfs|xf2fs)
MINBLKSIZE=$SECSIZE
# OS Limitation: GNU/Linux doesn't accept > 4096
@@ -382,8 +386,8 @@ for LOGSECSIZE in $(range "$MINLOGSECSIZE" "$MAXLOGSECSIZE"
1); do
FSLABEL="g;/_é䏌䐓䏕䎛䎾䏴кит u"
#FSLABEL="g;/_é莭莽😁кит u"
;;
- # FS LIMITATION: reiserfs, extN and jfs label is at most 16
UTF-8 characters
- x"reiserfs_old" | x"reiserfs" | x"ext"* | x"lvm"* | x"luks"* |
x"mdraid"* | x"jfs" | x"jfs_caseins")
+ # FS LIMITATION: reiserfs, extN, jfs and erofs label is at most
16 UTF-8 characters
+ x"reiserfs_old" | x"reiserfs" | x"ext"* | x"lvm"* | x"luks"* |
x"mdraid"* | x"jfs" | x"jfs_caseins" | x"erofs_"*)
FSLABEL="g;/éт 莭😁";;
# FS LIMITATION: No underscore, space, semicolon, slash or
international characters in UFS* in label. Limited to 32 UTF-8 characters
x"ufs1" | x"ufs1_sun" | x"ufs2")
@@ -661,7 +665,7 @@ for LOGSECSIZE in $(range "$MINLOGSECSIZE" "$MAXLOGSECSIZE"
1); do
x"tarfs" | x"cpio_"*| x"ziso9660" | x"romfs" | x"squash4_"*\
| x"iso9660" | xjoliet | xrockridge | xrockridge_joliet \
| x"iso9660_1999" | xjoliet_1999 | xrockridge_1999 \
- | xrockridge_joliet_1999)
+ | xrockridge_joliet_1999 | x"erofs_"*)
MNTPOINTRW="$MASTER"
MNTPOINTRO="$MASTER"
mkdir -p "$MASTER";;
@@ -805,7 +809,7 @@ for LOGSECSIZE in $(range "$MINLOGSECSIZE" "$MAXLOGSECSIZE"
1); do
sleep 1
"zfs" create "$FSLABEL"/"grub fs"
sleep 1;;
- x"tarfs" | x"cpio_"* | x"iso9660" | xjoliet | xrockridge |
xrockridge_joliet | x"iso9660_1999" | xjoliet_1999 | xrockridge_1999 |
xrockridge_joliet_1999 | x"ziso9660" | x"romfs" | x"squash4_"*)
+ x"tarfs" | x"cpio_"* | x"iso9660" | xjoliet | xrockridge |
xrockridge_joliet | x"iso9660_1999" | xjoliet_1999 | xrockridge_1999 |
xrockridge_joliet_1999 | x"
[PATCH v13 2/3] fs/erofs: Add support for EROFS
From: Yifan Zhao
EROFS [1] is a lightweight read-only filesystem designed for performance
which has already been shipped in most Linux distributions as well as widely
used in several scenarios, such as Android system partitions, container
images, and rootfs for embedded devices.
This patch brings EROFS uncompressed support. Now, it's possible to boot
directly through GRUB with an EROFS rootfs.
EROFS compressed files will be supported later since it has more work to
polish.
[1] https://erofs.docs.kernel.org
Signed-off-by: Yifan Zhao
Tested-by: Daniel Axtens # fuzz testing only
Signed-off-by: Gao Xiang
---
INSTALL |8 +-
Makefile.util.def |1 +
docs/grub.texi |3 +-
grub-core/Makefile.core.def |5 +
grub-core/fs/erofs.c| 1000 +++
5 files changed, 1012 insertions(+), 5 deletions(-)
create mode 100644 grub-core/fs/erofs.c
diff --git a/INSTALL b/INSTALL
index 8d9207c84..84030c9f4 100644
--- a/INSTALL
+++ b/INSTALL
@@ -77,15 +77,15 @@ Prerequisites for make-check:
* If running a Linux kernel the following modules must be loaded:
- fuse, loop
- - btrfs, ext4, f2fs, fat, hfs, hfsplus, jfs, mac-roman, minix, nilfs2,
+ - btrfs, erofs, ext4, f2fs, fat, hfs, hfsplus, jfs, mac-roman, minix, nilfs2,
reiserfs, udf, xfs
- On newer kernels, the exfat kernel modules may be used instead of the
exfat FUSE filesystem
* The following are Debian named packages required mostly for the full
suite of filesystem testing (but some are needed by other tests as well):
- - btrfs-progs, dosfstools, e2fsprogs, exfat-utils, f2fs-tools, genromfs,
-hfsprogs, jfsutils, nilfs-tools, ntfs-3g, reiserfsprogs, squashfs-tools,
-reiserfsprogs, udftools, xfsprogs, zfs-fuse
+ - btrfs-progs, dosfstools, e2fsprogs, erofs-utils, exfat-utils, f2fs-tools,
+genromfs, hfsprogs, jfsutils, nilfs-tools, ntfs-3g, reiserfsprogs,
+squashfs-tools, reiserfsprogs, udftools, xfsprogs, zfs-fuse
- exfat-fuse, if not using the exfat kernel module
- gzip, lzop, xz-utils
- attr, cpio, g++, gawk, parted, recode, tar, util-linux
diff --git a/Makefile.util.def b/Makefile.util.def
index 9432365a9..8d3bc107f 100644
--- a/Makefile.util.def
+++ b/Makefile.util.def
@@ -98,6 +98,7 @@ library = {
common = grub-core/fs/cpio_be.c;
common = grub-core/fs/odc.c;
common = grub-core/fs/newc.c;
+ common = grub-core/fs/erofs.c;
common = grub-core/fs/ext2.c;
common = grub-core/fs/fat.c;
common = grub-core/fs/exfat.c;
diff --git a/docs/grub.texi b/docs/grub.texi
index d32266f69..b198d963d 100644
--- a/docs/grub.texi
+++ b/docs/grub.texi
@@ -353,6 +353,7 @@ blocklist notation. The currently supported filesystem
types are @dfn{Amiga
Fast FileSystem (AFFS)}, @dfn{AtheOS fs}, @dfn{BeFS},
@dfn{BtrFS} (including raid0, raid1, raid10, gzip and lzo),
@dfn{cpio} (little- and big-endian bin, odc and newc variants),
+@dfn{EROFS} (only uncompressed support for now),
@dfn{Linux ext2/ext3/ext4}, @dfn{DOS FAT12/FAT16/FAT32},
@dfn{exFAT}, @dfn{F2FS}, @dfn{HFS}, @dfn{HFS+},
@dfn{ISO9660} (including Joliet, Rock-ridge and multi-chunk files),
@@ -6276,7 +6277,7 @@ assumed to be encoded in UTF-8.
NTFS, JFS, UDF, HFS+, exFAT, long filenames in FAT, Joliet part of
ISO9660 are treated as UTF-16 as per specification. AFS and BFS are read
as UTF-8, again according to specification. BtrFS, cpio, tar, squash4, minix,
-minix2, minix3, ROMFS, ReiserFS, XFS, ext2, ext3, ext4, FAT (short names),
+minix2, minix3, ROMFS, ReiserFS, XFS, EROFS, ext2, ext3, ext4, FAT (short
names),
F2FS, RockRidge part of ISO9660, nilfs2, UFS1, UFS2 and ZFS are assumed
to be UTF-8. This might be false on systems configured with legacy charset
but as long as the charset used is superset of ASCII you should be able to
diff --git a/grub-core/Makefile.core.def b/grub-core/Makefile.core.def
index 8e1b1d9f3..7fa9446bd 100644
--- a/grub-core/Makefile.core.def
+++ b/grub-core/Makefile.core.def
@@ -1442,6 +1442,11 @@ module = {
common = fs/odc.c;
};
+module = {
+ name = erofs;
+ common = fs/erofs.c;
+};
+
module = {
name = ext2;
common = fs/ext2.c;
diff --git a/grub-core/fs/erofs.c b/grub-core/fs/erofs.c
new file mode 100644
index 0..005d975de
--- /dev/null
+++ b/grub-core/fs/erofs.c
@@ -0,0 +1,1000 @@
+/* erofs.c - Enhanced Read-Only File System */
+/*
+ * GRUB -- GRand Unified Bootloader
+ * Copyright (C) 2024 Free Software Foundation, Inc.
+ *
+ * GRUB is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 3 of the License, or
+ * (at your option) any later version.
+ *
+ * GRUB is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for
Re: [PATCH v13 0/3] Introduce EROFS support
On 2024/5/17 20:17, Gao Xiang wrote:
Hi folks,
EROFS [1] is a lightweight read-only filesystem designed for performance
which has already been shipped in most Linux distributions as well as widely
used in several scenarios, such as Android system partitions, container
images, and rootfs for embedded devices.
This patch brings EROFS uncompressed support together with related tests.
Now, it's possible to boot directly through GRUB with an EROFS rootfs.
EROFS compressed files will be supported later since it has more work to
polish.
[1] https://erofs.docs.kernel.org
v12:
https://lore.kernel.org/grub-devel/20240517044054.2752375-1-hsiang...@linux.alibaba.com
changelog since v12:
- Address ALIGN_UP_OVF() suggested by Daniel:
https://lore.kernel.org/grub-devel/zkcv7g7mjbg8s...@tomti.i.net-space.pl
Tested-by Link (Commit 1):
https://lists.gnu.org/archive/html/grub-devel/2024-05/msg1.html
Reviewed-by Link (Commit 2):
https://lists.gnu.org/archive/html/grub-devel/2024-04/msg00101.html
Reviewed-by Link (Commit 2):
https://lore.kernel.org/grub-devel/zkz8fdd+zckt+...@tomti.i.net-space.pl/
Gao Xiang (1):
safemath: Add ALIGN_UP_OVF() that checks for {over,under}flow
Yifan Zhao (2):
fs/erofs: Add support for EROFS
fs/erofs: Add tests for EROFS in grub-fs-tester
.gitignore |1 +
INSTALL |8 +-
Makefile.util.def|7 +
docs/grub.texi |3 +-
grub-core/Makefile.core.def |5 +
grub-core/fs/erofs.c | 1000 ++
include/grub/safemath.h | 16 +
tests/erofs_test.in | 20 +
tests/util/grub-fs-tester.in | 32 +-
9 files changed, 1080 insertions(+), 12 deletions(-)
create mode 100644 grub-core/fs/erofs.c
create mode 100644 tests/erofs_test.in
Interdiff against v12:
diff --git a/grub-core/fs/erofs.c b/grub-core/fs/erofs.c
index 630f95e75..005d975de 100644
--- a/grub-core/fs/erofs.c
+++ b/grub-core/fs/erofs.c
@@ -405,10 +405,8 @@ erofs_map_blocks_chunkmode (grub_fshelp_node_t node,
if (grub_add (pos, erofs_inode_xattr_ibody_size (node), &pos))
return grub_error (GRUB_ERR_OUT_OF_RANGE, "chunkmap position overflow when
adding xattr size");
- /* pos = ALIGN_UP(pos, unit) */
- if (grub_add (pos, unit - 1, &pos))
+ if (ALIGN_UP_OVF (pos, unit - 1, &pos))
Sorry this line should be
if (ALIGN_UP_OVF (pos, unit, &pos))
I will send out a new version to fix it, please ignore this version.
Thanks,
Gao Xiang
___
Grub-devel mailing list
Grub-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/grub-devel
[PATCH v14 0/3] Introduce EROFS support
[ This is a RESEND version of v13 due to a mis-fix. ]
Hi folks,
EROFS [1] is a lightweight read-only filesystem designed for performance
which has already been shipped in most Linux distributions as well as widely
used in several scenarios, such as Android system partitions, container
images, and rootfs for embedded devices.
This patch brings EROFS uncompressed support together with related tests.
Now, it's possible to boot directly through GRUB with an EROFS rootfs.
EROFS compressed files will be supported later since it has more work to
polish.
[1] https://erofs.docs.kernel.org
v12:
https://lore.kernel.org/grub-devel/20240517044054.2752375-1-hsiang...@linux.alibaba.com
changelog since v12:
- Address ALIGN_UP_OVF() suggested by Daniel:
https://lore.kernel.org/grub-devel/zkcv7g7mjbg8s...@tomti.i.net-space.pl
Tested-by Link (Commit 1):
https://lists.gnu.org/archive/html/grub-devel/2024-05/msg1.html
Reviewed-by Link (Commit 2):
https://lists.gnu.org/archive/html/grub-devel/2024-04/msg00101.html
Reviewed-by Link (Commit 2):
https://lore.kernel.org/grub-devel/zkz8fdd+zckt+...@tomti.i.net-space.pl/
Gao Xiang (1):
safemath: Add ALIGN_UP_OVF() that checks for {over,under}flow
Yifan Zhao (2):
fs/erofs: Add support for EROFS
fs/erofs: Add tests for EROFS in grub-fs-tester
.gitignore |1 +
INSTALL |8 +-
Makefile.util.def|7 +
docs/grub.texi |3 +-
grub-core/Makefile.core.def |5 +
grub-core/fs/erofs.c | 1000 ++
include/grub/safemath.h | 16 +
tests/erofs_test.in | 20 +
tests/util/grub-fs-tester.in | 32 +-
9 files changed, 1080 insertions(+), 12 deletions(-)
create mode 100644 grub-core/fs/erofs.c
create mode 100644 tests/erofs_test.in
Interdiff against v12:
diff --git a/grub-core/fs/erofs.c b/grub-core/fs/erofs.c
index 630f95e75..46cfc2e5c 100644
--- a/grub-core/fs/erofs.c
+++ b/grub-core/fs/erofs.c
@@ -405,10 +405,8 @@ erofs_map_blocks_chunkmode (grub_fshelp_node_t node,
if (grub_add (pos, erofs_inode_xattr_ibody_size (node), &pos))
return grub_error (GRUB_ERR_OUT_OF_RANGE, "chunkmap position overflow when
adding xattr size");
- /* pos = ALIGN_UP(pos, unit) */
- if (grub_add (pos, unit - 1, &pos))
+ if (ALIGN_UP_OVF (pos, unit, &pos))
return grub_error (GRUB_ERR_OUT_OF_RANGE, "position overflow when seeking
at the start of chunkmap");
- pos &= ~(unit - 1);
/* No overflow for multiplication as chunkbits >= 9 and sizeof(unit) <= 8. */
if (grub_add (pos, chunknr * unit, &pos))
diff --git a/include/grub/safemath.h b/include/grub/safemath.h
index fbd9b5925..baaea0ef4 100644
--- a/include/grub/safemath.h
+++ b/include/grub/safemath.h
@@ -32,6 +32,22 @@
#define grub_cast(a, res) grub_add ((a), 0, (res))
+#define ALIGN_UP_OVF(v, align, res)\
+({ \
+ bool __failed; \
+ typeof(v) a; \
+ \
+ __failed = grub_sub ((typeof(v))(align), 1, &a); \
+ if (__failed == false) \
+{ \
+__failed = grub_add (v, a, res); \
+if (__failed == false) \
+ {\
+*(res) &= ~a; \
+ }\
+} \
+__failed;})
+
#else
#error gcc 5.1 or newer or clang 8.0 or newer is required
#endif
--
2.39.3
___
Grub-devel mailing list
Grub-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/grub-devel
[PATCH v14 3/3] fs/erofs: Add tests for EROFS in grub-fs-tester
From: Yifan Zhao
In this patch, three tests of EROFS are introduced and they cover
compact, extended and chunk-based inodes, respectively.
Signed-off-by: Yifan Zhao
Reviewed-by: Glenn Washburn
Reviewed-by: Daniel Kiper
Signed-off-by: Gao Xiang
---
.gitignore | 1 +
Makefile.util.def| 6 ++
tests/erofs_test.in | 20
tests/util/grub-fs-tester.in | 32 +---
4 files changed, 52 insertions(+), 7 deletions(-)
create mode 100644 tests/erofs_test.in
diff --git a/.gitignore b/.gitignore
index 11fcecf5c..4c1f91db8 100644
--- a/.gitignore
+++ b/.gitignore
@@ -104,6 +104,7 @@ widthspec.bin
/docs/version-dev.texi
/docs/version.texi
/ehci_test
+/erofs_test
/example_grub_script_test
/example_scripted_test
/example_unit_test
diff --git a/Makefile.util.def b/Makefile.util.def
index 8d3bc107f..0f74a1680 100644
--- a/Makefile.util.def
+++ b/Makefile.util.def
@@ -764,6 +764,12 @@ script = {
dependencies = 'garbage-gen$(BUILD_EXEEXT)';
};
+script = {
+ testcase = native;
+ name = erofs_test;
+ common = tests/erofs_test.in;
+};
+
script = {
testcase = native;
name = ext234_test;
diff --git a/tests/erofs_test.in b/tests/erofs_test.in
new file mode 100644
index 0..5627a
--- /dev/null
+++ b/tests/erofs_test.in
@@ -0,0 +1,20 @@
+#!@BUILD_SHEBANG@
+
+set -e
+
+if [ "x$EUID" = "x" ] ; then
+ EUID=`id -u`
+fi
+
+if [ "$EUID" != 0 ] ; then
+ exit 99
+fi
+
+if ! which mkfs.erofs >/dev/null 2>&1; then
+ echo "mkfs.erofs not installed; cannot test erofs."
+ exit 99
+fi
+
+"@builddir@/grub-fs-tester" erofs_compact
+"@builddir@/grub-fs-tester" erofs_extended
+"@builddir@/grub-fs-tester" erofs_chunk
diff --git a/tests/util/grub-fs-tester.in b/tests/util/grub-fs-tester.in
index ea8b2d1f6..df5dc7542 100644
--- a/tests/util/grub-fs-tester.in
+++ b/tests/util/grub-fs-tester.in
@@ -227,6 +227,10 @@ for LOGSECSIZE in $(range "$MINLOGSECSIZE"
"$MAXLOGSECSIZE" 1); do
xsquash*)
MINBLKSIZE=4096
MAXBLKSIZE=1048576;;
+ x"erofs_"*)
+ MINBLKSIZE=4096
+ MAXBLKSIZE=4096
+ ;;
xxfs|xf2fs)
MINBLKSIZE=$SECSIZE
# OS Limitation: GNU/Linux doesn't accept > 4096
@@ -382,8 +386,8 @@ for LOGSECSIZE in $(range "$MINLOGSECSIZE" "$MAXLOGSECSIZE"
1); do
FSLABEL="g;/_é䏌䐓䏕䎛䎾䏴кит u"
#FSLABEL="g;/_é莭莽😁кит u"
;;
- # FS LIMITATION: reiserfs, extN and jfs label is at most 16
UTF-8 characters
- x"reiserfs_old" | x"reiserfs" | x"ext"* | x"lvm"* | x"luks"* |
x"mdraid"* | x"jfs" | x"jfs_caseins")
+ # FS LIMITATION: reiserfs, extN, jfs and erofs label is at most
16 UTF-8 characters
+ x"reiserfs_old" | x"reiserfs" | x"ext"* | x"lvm"* | x"luks"* |
x"mdraid"* | x"jfs" | x"jfs_caseins" | x"erofs_"*)
FSLABEL="g;/éт 莭😁";;
# FS LIMITATION: No underscore, space, semicolon, slash or
international characters in UFS* in label. Limited to 32 UTF-8 characters
x"ufs1" | x"ufs1_sun" | x"ufs2")
@@ -661,7 +665,7 @@ for LOGSECSIZE in $(range "$MINLOGSECSIZE" "$MAXLOGSECSIZE"
1); do
x"tarfs" | x"cpio_"*| x"ziso9660" | x"romfs" | x"squash4_"*\
| x"iso9660" | xjoliet | xrockridge | xrockridge_joliet \
| x"iso9660_1999" | xjoliet_1999 | xrockridge_1999 \
- | xrockridge_joliet_1999)
+ | xrockridge_joliet_1999 | x"erofs_"*)
MNTPOINTRW="$MASTER"
MNTPOINTRO="$MASTER"
mkdir -p "$MASTER";;
@@ -805,7 +809,7 @@ for LOGSECSIZE in $(range "$MINLOGSECSIZE" "$MAXLOGSECSIZE"
1); do
sleep 1
"zfs" create "$FSLABEL"/"grub fs"
sleep 1;;
- x"tarfs" | x"cpio_"* | x"iso9660" | xjoliet | xrockridge |
xrockridge_joliet | x"iso9660_1999" | xjoliet_1999 | xrockridge_1999 |
xrockridge_joliet_1999 | x"ziso9660" | x"romfs" | x"squash4_"*)
+ x"tarfs" | x"cpio_"* | x"iso9660" | xjoliet | xrockridge |
xrockridge_joliet | x"iso9660_1999" | xjoliet_1999 | xrockridge_1999 |
xrockridge_joliet_1999 | x"
[PATCH v14 1/3] safemath: Add ALIGN_UP_OVF() that checks for {over, under}flow
The following EROFS patch will use this helper to handle overflow
ALIGN_UP() cases.
Signed-off-by: Gao Xiang
---
include/grub/safemath.h | 16
1 file changed, 16 insertions(+)
diff --git a/include/grub/safemath.h b/include/grub/safemath.h
index fbd9b5925..baaea0ef4 100644
--- a/include/grub/safemath.h
+++ b/include/grub/safemath.h
@@ -32,6 +32,22 @@
#define grub_cast(a, res) grub_add ((a), 0, (res))
+#define ALIGN_UP_OVF(v, align, res)\
+({ \
+ bool __failed; \
+ typeof(v) a; \
+ \
+ __failed = grub_sub ((typeof(v))(align), 1, &a); \
+ if (__failed == false) \
+{ \
+__failed = grub_add (v, a, res); \
+if (__failed == false) \
+ {\
+*(res) &= ~a; \
+ }\
+} \
+__failed;})
+
#else
#error gcc 5.1 or newer or clang 8.0 or newer is required
#endif
--
2.39.3
___
Grub-devel mailing list
Grub-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/grub-devel
[PATCH v14 2/3] fs/erofs: Add support for EROFS
From: Yifan Zhao
EROFS [1] is a lightweight read-only filesystem designed for performance
which has already been shipped in most Linux distributions as well as widely
used in several scenarios, such as Android system partitions, container
images, and rootfs for embedded devices.
This patch brings EROFS uncompressed support. Now, it's possible to boot
directly through GRUB with an EROFS rootfs.
EROFS compressed files will be supported later since it has more work to
polish.
[1] https://erofs.docs.kernel.org
Signed-off-by: Yifan Zhao
Tested-by: Daniel Axtens # fuzz testing only
Signed-off-by: Gao Xiang
---
INSTALL |8 +-
Makefile.util.def |1 +
docs/grub.texi |3 +-
grub-core/Makefile.core.def |5 +
grub-core/fs/erofs.c| 1000 +++
5 files changed, 1012 insertions(+), 5 deletions(-)
create mode 100644 grub-core/fs/erofs.c
diff --git a/INSTALL b/INSTALL
index 8d9207c84..84030c9f4 100644
--- a/INSTALL
+++ b/INSTALL
@@ -77,15 +77,15 @@ Prerequisites for make-check:
* If running a Linux kernel the following modules must be loaded:
- fuse, loop
- - btrfs, ext4, f2fs, fat, hfs, hfsplus, jfs, mac-roman, minix, nilfs2,
+ - btrfs, erofs, ext4, f2fs, fat, hfs, hfsplus, jfs, mac-roman, minix, nilfs2,
reiserfs, udf, xfs
- On newer kernels, the exfat kernel modules may be used instead of the
exfat FUSE filesystem
* The following are Debian named packages required mostly for the full
suite of filesystem testing (but some are needed by other tests as well):
- - btrfs-progs, dosfstools, e2fsprogs, exfat-utils, f2fs-tools, genromfs,
-hfsprogs, jfsutils, nilfs-tools, ntfs-3g, reiserfsprogs, squashfs-tools,
-reiserfsprogs, udftools, xfsprogs, zfs-fuse
+ - btrfs-progs, dosfstools, e2fsprogs, erofs-utils, exfat-utils, f2fs-tools,
+genromfs, hfsprogs, jfsutils, nilfs-tools, ntfs-3g, reiserfsprogs,
+squashfs-tools, reiserfsprogs, udftools, xfsprogs, zfs-fuse
- exfat-fuse, if not using the exfat kernel module
- gzip, lzop, xz-utils
- attr, cpio, g++, gawk, parted, recode, tar, util-linux
diff --git a/Makefile.util.def b/Makefile.util.def
index 9432365a9..8d3bc107f 100644
--- a/Makefile.util.def
+++ b/Makefile.util.def
@@ -98,6 +98,7 @@ library = {
common = grub-core/fs/cpio_be.c;
common = grub-core/fs/odc.c;
common = grub-core/fs/newc.c;
+ common = grub-core/fs/erofs.c;
common = grub-core/fs/ext2.c;
common = grub-core/fs/fat.c;
common = grub-core/fs/exfat.c;
diff --git a/docs/grub.texi b/docs/grub.texi
index d32266f69..b198d963d 100644
--- a/docs/grub.texi
+++ b/docs/grub.texi
@@ -353,6 +353,7 @@ blocklist notation. The currently supported filesystem
types are @dfn{Amiga
Fast FileSystem (AFFS)}, @dfn{AtheOS fs}, @dfn{BeFS},
@dfn{BtrFS} (including raid0, raid1, raid10, gzip and lzo),
@dfn{cpio} (little- and big-endian bin, odc and newc variants),
+@dfn{EROFS} (only uncompressed support for now),
@dfn{Linux ext2/ext3/ext4}, @dfn{DOS FAT12/FAT16/FAT32},
@dfn{exFAT}, @dfn{F2FS}, @dfn{HFS}, @dfn{HFS+},
@dfn{ISO9660} (including Joliet, Rock-ridge and multi-chunk files),
@@ -6276,7 +6277,7 @@ assumed to be encoded in UTF-8.
NTFS, JFS, UDF, HFS+, exFAT, long filenames in FAT, Joliet part of
ISO9660 are treated as UTF-16 as per specification. AFS and BFS are read
as UTF-8, again according to specification. BtrFS, cpio, tar, squash4, minix,
-minix2, minix3, ROMFS, ReiserFS, XFS, ext2, ext3, ext4, FAT (short names),
+minix2, minix3, ROMFS, ReiserFS, XFS, EROFS, ext2, ext3, ext4, FAT (short
names),
F2FS, RockRidge part of ISO9660, nilfs2, UFS1, UFS2 and ZFS are assumed
to be UTF-8. This might be false on systems configured with legacy charset
but as long as the charset used is superset of ASCII you should be able to
diff --git a/grub-core/Makefile.core.def b/grub-core/Makefile.core.def
index 8e1b1d9f3..7fa9446bd 100644
--- a/grub-core/Makefile.core.def
+++ b/grub-core/Makefile.core.def
@@ -1442,6 +1442,11 @@ module = {
common = fs/odc.c;
};
+module = {
+ name = erofs;
+ common = fs/erofs.c;
+};
+
module = {
name = ext2;
common = fs/ext2.c;
diff --git a/grub-core/fs/erofs.c b/grub-core/fs/erofs.c
new file mode 100644
index 0..46cfc2e5c
--- /dev/null
+++ b/grub-core/fs/erofs.c
@@ -0,0 +1,1000 @@
+/* erofs.c - Enhanced Read-Only File System */
+/*
+ * GRUB -- GRand Unified Bootloader
+ * Copyright (C) 2024 Free Software Foundation, Inc.
+ *
+ * GRUB is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 3 of the License, or
+ * (at your option) any later version.
+ *
+ * GRUB is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for
Re: [PATCH v14 1/3] safemath: Add ALIGN_UP_OVF() that checks for {over,under}flow
Hi Vladimir,
On 2024/5/18 00:38, Vladimir 'phcoder' Serbinenko wrote:
I think that given that align is a non zero const we can trust it.
From the EROFS specific cases, they are always non-zero values,
So I agree with you on this..
Yet Daniel said ".. Be careful with underflows too." in
https://lore.kernel.org/r/zkcv7g7mjbg8s...@tomti.i.net-space.pl
although I'm not quite sure but I guess like this.
Also as a generic helper, I think `align` could be zero if it's
a variable..
Anyway, either way works for EROFS. I just try my best to do
what I could do for this first series for upstreaming and we
could move forward to the next step..
Thanks,
Gao Xiang
Le ven. 17 mai 2024, 15:56, Gao Xiang mailto:hsiang...@linux.alibaba.com>> a écrit :
The following EROFS patch will use this helper to handle overflow
ALIGN_UP() cases.
Signed-off-by: Gao Xiang mailto:hsiang...@linux.alibaba.com>>
---
include/grub/safemath.h | 16
1 file changed, 16 insertions(+)
diff --git a/include/grub/safemath.h b/include/grub/safemath.h
index fbd9b5925..baaea0ef4 100644
--- a/include/grub/safemath.h
+++ b/include/grub/safemath.h
@@ -32,6 +32,22 @@
#define grub_cast(a, res) grub_add ((a), 0, (res))
+#define ALIGN_UP_OVF(v, align, res) \
+({ \
+ bool __failed; \
+ typeof(v) a; \
+ \
+ __failed = grub_sub ((typeof(v))(align), 1, &a); \
+ if (__failed == false) \
+ { \
+ __failed = grub_add (v, a, res); \
+ if (__failed == false) \
+ { \
+ *(res) &= ~a; \
+ } \
+ } \
+__failed;})
+
#else
#error gcc 5.1 or newer or clang 8.0 or newer is required
#endif
--
2.39.3
___
Grub-devel mailing list
Grub-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/grub-devel
Re: [PATCH v14 1/3] safemath: Add ALIGN_UP_OVF() that checks for {over,under}flow
On 2024/5/18 17:20, Vladimir 'phcoder' Serbinenko wrote: Le ven. 17 mai 2024, 21:26, Gao Xiang mailto:hsiang...@linux.alibaba.com>> a écrit : Hi Vladimir, On 2024/5/18 00:38, Vladimir 'phcoder' Serbinenko wrote: > I think that given that align is a non zero const we can trust it. From the EROFS specific cases, they are always non-zero values, So I agree with you on this.. Yet Daniel said ".. Be careful with underflows too." in https://lore.kernel.org/r/zkcv7g7mjbg8s...@tomti.i.net-space.pl <https://lore.kernel.org/r/zkcv7g7mjbg8s...@tomti.i.net-space.pl> although I'm not quite sure but I guess like this. Also as a generic helper, I think `align` could be zero if it's a variable.. It's rare for align to be a variable and then additional checks are needed that align is a power of 2. And power of 2 is never zero. This check is better done on the caller side. You just need a comment specify that it's caller responsibility to check it. I'd like to know Daniel's comment about this first, there are already too many comments out of nits. I'm fine with updating such nits but I would like to confirm _which comment_ I should follow for upstreaming. Thanks, Gao Xiang Anyway, either way works for EROFS. I just try my best to do what I could do for this first series for upstreaming and we could move forward to the next step.. Thanks, Gao Xiang ___ Grub-devel mailing list Grub-devel@gnu.org https://lists.gnu.org/mailman/listinfo/grub-devel
Re: [PATCH v14 1/3] safemath: Add ALIGN_UP_OVF() that checks for {over,under}flow
Hi Daniel, There is too many comments on this minor stuff, would you mind giving some hints of your perference so I could have a chance to finish this stuff? https://lore.kernel.org/grub-devel/CAEaD8JNwa8W=3lvavedg0hbyz10+xhedwcsv8tqe5md_13s...@mail.gmail.com/ https://lore.kernel.org/grub-devel/8f2512f9-e851-467c-bbb8-450bf0219...@gmail.com/ Thanks in advance. Thanks, Gao Xiang On 2024/5/19 11:21, Gao Xiang wrote: On 2024/5/18 17:20, Vladimir 'phcoder' Serbinenko wrote: Le ven. 17 mai 2024, 21:26, Gao Xiang mailto:hsiang...@linux.alibaba.com>> a écrit : Hi Vladimir, On 2024/5/18 00:38, Vladimir 'phcoder' Serbinenko wrote: > I think that given that align is a non zero const we can trust it. From the EROFS specific cases, they are always non-zero values, So I agree with you on this.. Yet Daniel said ".. Be careful with underflows too." in https://lore.kernel.org/r/zkcv7g7mjbg8s...@tomti.i.net-space.pl <https://lore.kernel.org/r/zkcv7g7mjbg8s...@tomti.i.net-space.pl> although I'm not quite sure but I guess like this. Also as a generic helper, I think `align` could be zero if it's a variable.. It's rare for align to be a variable and then additional checks are needed that align is a power of 2. And power of 2 is never zero. This check is better done on the caller side. You just need a comment specify that it's caller responsibility to check it. I'd like to know Daniel's comment about this first, there are already too many comments out of nits. I'm fine with updating such nits but I would like to confirm _which comment_ I should follow for upstreaming. Thanks, Gao Xiang Anyway, either way works for EROFS. I just try my best to do what I could do for this first series for upstreaming and we could move forward to the next step.. Thanks, Gao Xiang ___ Grub-devel mailing list Grub-devel@gnu.org https://lists.gnu.org/mailman/listinfo/grub-devel
[PATCH v14 NEW ALT 1/3] safemath: Add ALIGN_UP_OVF() that checks for {over, under}flow
The following EROFS patch will use this helper to handle overflow
ALIGN_UP() cases.
Signed-off-by: Gao Xiang
---
Hi Daniel,
I've also made another patch to conform Vladimir's and
Boyang's comments as much as I can.
Thanks,
Gao Xiang
include/grub/safemath.h | 16
1 file changed, 16 insertions(+)
diff --git a/include/grub/safemath.h b/include/grub/safemath.h
index fbd9b5925..bb8a5b39c 100644
--- a/include/grub/safemath.h
+++ b/include/grub/safemath.h
@@ -32,6 +32,22 @@
#define grub_cast(a, res) grub_add ((a), 0, (res))
+/*
+ * It's caller's responsibility that `align` should not equal to 0 and
+ * it must be a power of 2.
+ */
+#define ALIGN_UP_OVF(v, align, res)\
+({ \
+ bool __failed; \
+ typeof(v) __a = ((typeof(v))(align) - 1);\
+ \
+ __failed = grub_add (v, __a, res); \
+ if (__failed == false) \
+{ \
+ *(res) &= ~__a; \
+} \
+__failed;})
+
#else
#error gcc 5.1 or newer or clang 8.0 or newer is required
#endif
--
2.39.3
___
Grub-devel mailing list
Grub-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/grub-devel
Re: [PATCH v14 NEW ALT 1/3] safemath: Add ALIGN_UP_OVF() that checks for {over,under}flow
Hi Daniel and all, On 2024/5/21 00:53, Daniel Kiper wrote: On Sun, May 19, 2024 at 11:48:24AM +0800, Gao Xiang wrote: The following EROFS patch will use this helper to handle overflow ALIGN_UP() cases. Signed-off-by: Gao Xiang I think Vladimir is right. Thanks for your confirmation! :-) Reviewed-by: Daniel Kiper Please post v15 an I will merge it this week. Okay, let me sort out a new version soon. Thank you for adding EROFS support! Thank you all for taking so much time on this stuffs too, thanks again! Thanks, Gao Xiang Daniel ___ Grub-devel mailing list Grub-devel@gnu.org https://lists.gnu.org/mailman/listinfo/grub-devel
[PATCH v15 0/3] Introduce EROFS support
Hi folks,
EROFS [1] is a lightweight read-only filesystem designed for performance
which has already been shipped in most Linux distributions as well as widely
used in several scenarios, such as Android system partitions, container
images, and rootfs for embedded devices.
This patch brings EROFS uncompressed support together with related tests.
Now, it's possible to boot directly through GRUB with an EROFS rootfs.
EROFS compressed files will be supported later since it has more work to
polish.
[1] https://erofs.docs.kernel.org
v14:
https://lore.kernel.org/grub-devel/20240517125641.3346171-1-hsiang...@linux.alibaba.com
changelog since v12:
- drop unnecessary check of `align` in ALIGN_UP_OVF() sicne it is mostly
a non zero const that we can trust suggested by Vladimir and Boyang:
https://lore.kernel.org/grub-devel/CAEaD8JNwa8W=3lvavedg0hbyz10+xhedwcsv8tqe5md_13s...@mail.gmail.com
Reviewed-by Link (Commit 1):
https://lore.kernel.org/grub-devel/zkuadi5z+e%2fmk...@tomti.i.net-space.pl
Tested-by Link (Commit 2):
https://lists.gnu.org/archive/html/grub-devel/2024-05/msg1.html
Reviewed-by Link (Commit 3):
https://lists.gnu.org/archive/html/grub-devel/2024-04/msg00101.html
Reviewed-by Link (Commit 3):
https://lore.kernel.org/grub-devel/zkz8fdd+zckt+...@tomti.i.net-space.pl/
Gao Xiang (1):
safemath: Add ALIGN_UP_OVF() that checks for {over,under}flow
Yifan Zhao (2):
fs/erofs: Add support for EROFS
fs/erofs: Add tests for EROFS in grub-fs-tester
.gitignore |1 +
INSTALL |8 +-
Makefile.util.def|7 +
docs/grub.texi |3 +-
grub-core/Makefile.core.def |5 +
grub-core/fs/erofs.c | 1000 ++
include/grub/safemath.h | 16 +
tests/erofs_test.in | 20 +
tests/util/grub-fs-tester.in | 32 +-
9 files changed, 1080 insertions(+), 12 deletions(-)
create mode 100644 grub-core/fs/erofs.c
create mode 100644 tests/erofs_test.in
Interdiff against v14:
diff --git a/include/grub/safemath.h b/include/grub/safemath.h
index baaea0ef4..bb8a5b39c 100644
--- a/include/grub/safemath.h
+++ b/include/grub/safemath.h
@@ -32,19 +32,19 @@
#define grub_cast(a, res) grub_add ((a), 0, (res))
+/*
+ * It's caller's responsibility that `align` should not equal to 0 and
+ * it must be a power of 2.
+ */
#define ALIGN_UP_OVF(v, align, res)\
({ \
bool __failed; \
- typeof(v) a; \
+ typeof(v) __a = ((typeof(v))(align) - 1);\
\
- __failed = grub_sub ((typeof(v))(align), 1, &a); \
+ __failed = grub_add (v, __a, res); \
if (__failed == false) \
{ \
-__failed = grub_add (v, a, res); \
-if (__failed == false) \
- {\
-*(res) &= ~a; \
- }\
+ *(res) &= ~__a; \
} \
__failed;})
--
2.39.3
___
Grub-devel mailing list
Grub-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/grub-devel
[PATCH v15 1/3] safemath: Add ALIGN_UP_OVF() that checks for {over, under}flow
The following EROFS patch will use this helper to handle overflow
ALIGN_UP() cases.
Reviewed-by: Daniel Kiper
Signed-off-by: Gao Xiang
---
include/grub/safemath.h | 16
1 file changed, 16 insertions(+)
diff --git a/include/grub/safemath.h b/include/grub/safemath.h
index fbd9b5925..bb8a5b39c 100644
--- a/include/grub/safemath.h
+++ b/include/grub/safemath.h
@@ -32,6 +32,22 @@
#define grub_cast(a, res) grub_add ((a), 0, (res))
+/*
+ * It's caller's responsibility that `align` should not equal to 0 and
+ * it must be a power of 2.
+ */
+#define ALIGN_UP_OVF(v, align, res)\
+({ \
+ bool __failed; \
+ typeof(v) __a = ((typeof(v))(align) - 1);\
+ \
+ __failed = grub_add (v, __a, res); \
+ if (__failed == false) \
+{ \
+ *(res) &= ~__a; \
+} \
+__failed;})
+
#else
#error gcc 5.1 or newer or clang 8.0 or newer is required
#endif
--
2.39.3
___
Grub-devel mailing list
Grub-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/grub-devel
[PATCH v15 3/3] fs/erofs: Add tests for EROFS in grub-fs-tester
From: Yifan Zhao
In this patch, three tests of EROFS are introduced and they cover
compact, extended and chunk-based inodes, respectively.
Signed-off-by: Yifan Zhao
Reviewed-by: Glenn Washburn
Reviewed-by: Daniel Kiper
Signed-off-by: Gao Xiang
---
.gitignore | 1 +
Makefile.util.def| 6 ++
tests/erofs_test.in | 20
tests/util/grub-fs-tester.in | 32 +---
4 files changed, 52 insertions(+), 7 deletions(-)
create mode 100644 tests/erofs_test.in
diff --git a/.gitignore b/.gitignore
index 11fcecf5c..4c1f91db8 100644
--- a/.gitignore
+++ b/.gitignore
@@ -104,6 +104,7 @@ widthspec.bin
/docs/version-dev.texi
/docs/version.texi
/ehci_test
+/erofs_test
/example_grub_script_test
/example_scripted_test
/example_unit_test
diff --git a/Makefile.util.def b/Makefile.util.def
index 8d3bc107f..0f74a1680 100644
--- a/Makefile.util.def
+++ b/Makefile.util.def
@@ -764,6 +764,12 @@ script = {
dependencies = 'garbage-gen$(BUILD_EXEEXT)';
};
+script = {
+ testcase = native;
+ name = erofs_test;
+ common = tests/erofs_test.in;
+};
+
script = {
testcase = native;
name = ext234_test;
diff --git a/tests/erofs_test.in b/tests/erofs_test.in
new file mode 100644
index 0..5627a
--- /dev/null
+++ b/tests/erofs_test.in
@@ -0,0 +1,20 @@
+#!@BUILD_SHEBANG@
+
+set -e
+
+if [ "x$EUID" = "x" ] ; then
+ EUID=`id -u`
+fi
+
+if [ "$EUID" != 0 ] ; then
+ exit 99
+fi
+
+if ! which mkfs.erofs >/dev/null 2>&1; then
+ echo "mkfs.erofs not installed; cannot test erofs."
+ exit 99
+fi
+
+"@builddir@/grub-fs-tester" erofs_compact
+"@builddir@/grub-fs-tester" erofs_extended
+"@builddir@/grub-fs-tester" erofs_chunk
diff --git a/tests/util/grub-fs-tester.in b/tests/util/grub-fs-tester.in
index ea8b2d1f6..df5dc7542 100644
--- a/tests/util/grub-fs-tester.in
+++ b/tests/util/grub-fs-tester.in
@@ -227,6 +227,10 @@ for LOGSECSIZE in $(range "$MINLOGSECSIZE"
"$MAXLOGSECSIZE" 1); do
xsquash*)
MINBLKSIZE=4096
MAXBLKSIZE=1048576;;
+ x"erofs_"*)
+ MINBLKSIZE=4096
+ MAXBLKSIZE=4096
+ ;;
xxfs|xf2fs)
MINBLKSIZE=$SECSIZE
# OS Limitation: GNU/Linux doesn't accept > 4096
@@ -382,8 +386,8 @@ for LOGSECSIZE in $(range "$MINLOGSECSIZE" "$MAXLOGSECSIZE"
1); do
FSLABEL="g;/_é䏌䐓䏕䎛䎾䏴кит u"
#FSLABEL="g;/_é莭莽😁кит u"
;;
- # FS LIMITATION: reiserfs, extN and jfs label is at most 16
UTF-8 characters
- x"reiserfs_old" | x"reiserfs" | x"ext"* | x"lvm"* | x"luks"* |
x"mdraid"* | x"jfs" | x"jfs_caseins")
+ # FS LIMITATION: reiserfs, extN, jfs and erofs label is at most
16 UTF-8 characters
+ x"reiserfs_old" | x"reiserfs" | x"ext"* | x"lvm"* | x"luks"* |
x"mdraid"* | x"jfs" | x"jfs_caseins" | x"erofs_"*)
FSLABEL="g;/éт 莭😁";;
# FS LIMITATION: No underscore, space, semicolon, slash or
international characters in UFS* in label. Limited to 32 UTF-8 characters
x"ufs1" | x"ufs1_sun" | x"ufs2")
@@ -661,7 +665,7 @@ for LOGSECSIZE in $(range "$MINLOGSECSIZE" "$MAXLOGSECSIZE"
1); do
x"tarfs" | x"cpio_"*| x"ziso9660" | x"romfs" | x"squash4_"*\
| x"iso9660" | xjoliet | xrockridge | xrockridge_joliet \
| x"iso9660_1999" | xjoliet_1999 | xrockridge_1999 \
- | xrockridge_joliet_1999)
+ | xrockridge_joliet_1999 | x"erofs_"*)
MNTPOINTRW="$MASTER"
MNTPOINTRO="$MASTER"
mkdir -p "$MASTER";;
@@ -805,7 +809,7 @@ for LOGSECSIZE in $(range "$MINLOGSECSIZE" "$MAXLOGSECSIZE"
1); do
sleep 1
"zfs" create "$FSLABEL"/"grub fs"
sleep 1;;
- x"tarfs" | x"cpio_"* | x"iso9660" | xjoliet | xrockridge |
xrockridge_joliet | x"iso9660_1999" | xjoliet_1999 | xrockridge_1999 |
xrockridge_joliet_1999 | x"ziso9660" | x"romfs" | x"squash4_"*)
+ x"tarfs" | x"cpio_"* | x"iso9660" | xjoliet | xrockridge |
xrockridge_joliet | x"iso9660_1999" | xjoliet_1999 | xrockridge_1999 |
xrockridge_joliet_1999 | x"
[PATCH v15 2/3] fs/erofs: Add support for EROFS
From: Yifan Zhao
EROFS [1] is a lightweight read-only filesystem designed for performance
which has already been shipped in most Linux distributions as well as widely
used in several scenarios, such as Android system partitions, container
images, and rootfs for embedded devices.
This patch brings EROFS uncompressed support. Now, it's possible to boot
directly through GRUB with an EROFS rootfs.
EROFS compressed files will be supported later since it has more work to
polish.
[1] https://erofs.docs.kernel.org
Signed-off-by: Yifan Zhao
Tested-by: Daniel Axtens # fuzz testing only
Signed-off-by: Gao Xiang
---
INSTALL |8 +-
Makefile.util.def |1 +
docs/grub.texi |3 +-
grub-core/Makefile.core.def |5 +
grub-core/fs/erofs.c| 1000 +++
5 files changed, 1012 insertions(+), 5 deletions(-)
create mode 100644 grub-core/fs/erofs.c
diff --git a/INSTALL b/INSTALL
index 8d9207c84..84030c9f4 100644
--- a/INSTALL
+++ b/INSTALL
@@ -77,15 +77,15 @@ Prerequisites for make-check:
* If running a Linux kernel the following modules must be loaded:
- fuse, loop
- - btrfs, ext4, f2fs, fat, hfs, hfsplus, jfs, mac-roman, minix, nilfs2,
+ - btrfs, erofs, ext4, f2fs, fat, hfs, hfsplus, jfs, mac-roman, minix, nilfs2,
reiserfs, udf, xfs
- On newer kernels, the exfat kernel modules may be used instead of the
exfat FUSE filesystem
* The following are Debian named packages required mostly for the full
suite of filesystem testing (but some are needed by other tests as well):
- - btrfs-progs, dosfstools, e2fsprogs, exfat-utils, f2fs-tools, genromfs,
-hfsprogs, jfsutils, nilfs-tools, ntfs-3g, reiserfsprogs, squashfs-tools,
-reiserfsprogs, udftools, xfsprogs, zfs-fuse
+ - btrfs-progs, dosfstools, e2fsprogs, erofs-utils, exfat-utils, f2fs-tools,
+genromfs, hfsprogs, jfsutils, nilfs-tools, ntfs-3g, reiserfsprogs,
+squashfs-tools, reiserfsprogs, udftools, xfsprogs, zfs-fuse
- exfat-fuse, if not using the exfat kernel module
- gzip, lzop, xz-utils
- attr, cpio, g++, gawk, parted, recode, tar, util-linux
diff --git a/Makefile.util.def b/Makefile.util.def
index 9432365a9..8d3bc107f 100644
--- a/Makefile.util.def
+++ b/Makefile.util.def
@@ -98,6 +98,7 @@ library = {
common = grub-core/fs/cpio_be.c;
common = grub-core/fs/odc.c;
common = grub-core/fs/newc.c;
+ common = grub-core/fs/erofs.c;
common = grub-core/fs/ext2.c;
common = grub-core/fs/fat.c;
common = grub-core/fs/exfat.c;
diff --git a/docs/grub.texi b/docs/grub.texi
index d32266f69..b198d963d 100644
--- a/docs/grub.texi
+++ b/docs/grub.texi
@@ -353,6 +353,7 @@ blocklist notation. The currently supported filesystem
types are @dfn{Amiga
Fast FileSystem (AFFS)}, @dfn{AtheOS fs}, @dfn{BeFS},
@dfn{BtrFS} (including raid0, raid1, raid10, gzip and lzo),
@dfn{cpio} (little- and big-endian bin, odc and newc variants),
+@dfn{EROFS} (only uncompressed support for now),
@dfn{Linux ext2/ext3/ext4}, @dfn{DOS FAT12/FAT16/FAT32},
@dfn{exFAT}, @dfn{F2FS}, @dfn{HFS}, @dfn{HFS+},
@dfn{ISO9660} (including Joliet, Rock-ridge and multi-chunk files),
@@ -6276,7 +6277,7 @@ assumed to be encoded in UTF-8.
NTFS, JFS, UDF, HFS+, exFAT, long filenames in FAT, Joliet part of
ISO9660 are treated as UTF-16 as per specification. AFS and BFS are read
as UTF-8, again according to specification. BtrFS, cpio, tar, squash4, minix,
-minix2, minix3, ROMFS, ReiserFS, XFS, ext2, ext3, ext4, FAT (short names),
+minix2, minix3, ROMFS, ReiserFS, XFS, EROFS, ext2, ext3, ext4, FAT (short
names),
F2FS, RockRidge part of ISO9660, nilfs2, UFS1, UFS2 and ZFS are assumed
to be UTF-8. This might be false on systems configured with legacy charset
but as long as the charset used is superset of ASCII you should be able to
diff --git a/grub-core/Makefile.core.def b/grub-core/Makefile.core.def
index 8e1b1d9f3..7fa9446bd 100644
--- a/grub-core/Makefile.core.def
+++ b/grub-core/Makefile.core.def
@@ -1442,6 +1442,11 @@ module = {
common = fs/odc.c;
};
+module = {
+ name = erofs;
+ common = fs/erofs.c;
+};
+
module = {
name = ext2;
common = fs/ext2.c;
diff --git a/grub-core/fs/erofs.c b/grub-core/fs/erofs.c
new file mode 100644
index 0..46cfc2e5c
--- /dev/null
+++ b/grub-core/fs/erofs.c
@@ -0,0 +1,1000 @@
+/* erofs.c - Enhanced Read-Only File System */
+/*
+ * GRUB -- GRand Unified Bootloader
+ * Copyright (C) 2024 Free Software Foundation, Inc.
+ *
+ * GRUB is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 3 of the License, or
+ * (at your option) any later version.
+ *
+ * GRUB is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for
Re: [PATCH 2/6] erofs: Replace 64-bit modulo with bitwise operations
On 2024/9/2 08:22, Vladimir Serbinenko wrote: Otherwise depending on compiler we end up with umoddi3 reference and failed module dependency resolution Signed-off-by: Vladimir Serbinenko Reviewed-by: Gao Xiang Thanks, Gao Xiang ___ Grub-devel mailing list Grub-devel@gnu.org https://lists.gnu.org/mailman/listinfo/grub-devel
Re: [PATCH 1/2] fs/erofs: Add support for EROFS
Hi Glenn, On 2023/7/17 11:51, Glenn Washburn wrote: On Mon, 17 Jul 2023 00:03:08 +0800 Yifan Zhao wrote: EROFS is a lightweight read-only filesystem designed for performance which has already been widely used in several scenarios, such as Android system partitions, container images, and rootfs for embedded devices. This patch brings EROFS uncompressed support. Now, it's possible to boot directly through GRUB with an EROFS rootfs. EROFS compressed support will be developed later since it has more work to polish. I would expect that a very small minority are using EROFS without compression. Does that also seem true to you? This is relevant because I believe GRUB is in a feature freeze, but if there were a compelling reason to include this in the upcoming release, Daniel might make an exception (just guessing). On the one hand, in favor of including the current code without compression in the next release is that its simple enough and doesn't touch existing code, so a bug should have minor impact on the rest of GRUB. On the other hand, if there is a low probability of its use, perhaps its not worth it. As it stands, it Thanks for the reply! EROFS can have per-file compression, which means some files could be compressed, the others are not. Therefore, the majority of rootfs can still be compressed, but leave the real kernel image uncompressed though, so we could still boot systems from EROFS filesystems with EROFS rootfs images. One barrier to support EROFS compression is that there is no public LZ4 library in grub (Yifan said there is one in the ZFS filesystem, but no generic one.) Personally I'd suggest support uncompressed EROFS for now since it removes another filesystem dependency of EROFS rootfs so we don't need to add another partition for this. seems that GRUB is being released every couple years. Ok, I think if we missed this release, we will wait for another two years. Thanks, Gao Xiang When you create an updated patch series, it would be much appreciated if you use --range-diff or --interdiff so that reviewers (me) can verify more easily the changes. Signed-off-by: Yifan Zhao ___ Grub-devel mailing list Grub-devel@gnu.org https://lists.gnu.org/mailman/listinfo/grub-devel
Re: [PATCH v3 1/2] fs/erofs: Add support for EROFS
clude
+#include
+
+GRUB_MOD_LICENSE ("GPLv3+");
+
+#define EROFS_SUPER_OFFSET (1024)
+#define EROFS_MAGIC 0xE0F5E1E2
+#define EROFS_ISLOTBITS (5)
+
+#define EROFS_FEATURE_INCOMPAT_CHUNKED_FILE 0x0004
+#define EROFS_ALL_FEATURE_INCOMPAT (EROFS_FEATURE_INCOMPAT_CHUNKED_FILE)
+
+struct grub_erofs_super
+{
+ grub_uint32_t magic;
+ grub_uint32_t checksum;
+ grub_uint32_t feature_compat;
+ grub_uint8_t log2_blksz;
+ grub_uint8_t sb_extslots;
+
+ grub_uint16_t root_nid;
+ grub_uint64_t inos;
+
+ grub_uint64_t build_time;
+ grub_uint32_t build_time_nsec;
+ grub_uint32_t blocks;
+ grub_uint32_t meta_blkaddr;
+ grub_uint32_t xattr_blkaddr;
+ grub_uint8_t uuid[16];
+ grub_uint8_t volume_name[16];
+ grub_uint32_t feature_incompat;
+
+ union
+ {
+grub_uint16_t available_compr_algs;
+grub_uint16_t lz4_max_distance;
+ } GRUB_PACKED u1;
+
+ grub_uint16_t extra_devices;
+ grub_uint16_t devt_slotoff;
+ grub_uint8_t log2_dirblksz;
+ grub_uint8_t xattr_prefix_count;
+ grub_uint32_t xattr_prefix_start;
+ grub_uint64_t packed_nid;
+ grub_uint8_t reserved2[24];
+} GRUB_PACKED;
+
+#define EROFS_INODE_LAYOUT_COMPACT 0
+#define EROFS_INODE_LAYOUT_EXTENDED 1
+
+enum
+{
+ EROFS_INODE_FLAT_PLAIN = 0,
+ EROFS_INODE_COMPRESSED_FULL = 1,
+ EROFS_INODE_FLAT_INLINE = 2,
+ EROFS_INODE_COMPRESSED_COMPACT = 3,
+ EROFS_INODE_CHUNK_BASED = 4,
+ EROFS_INODE_DATALAYOUT_MAX
+};
+
+#define EROFS_I_VERSION_MASKS 0x01
+#define EROFS_I_DATALAYOUT_MASKS 0x07
+
+#define EROFS_I_VERSION_BIT 0
+#define EROFS_I_DATALAYOUT_BIT 1
+
+struct grub_erofs_inode_chunk_info
+{
+ grub_uint16_t format;
+ grub_uint16_t reserved;
+} GRUB_PACKED;
+
+#define EROFS_CHUNK_FORMAT_BLKBITS_MASK 0x001F
+#define EROFS_CHUNK_FORMAT_INDEXES 0x0020
+
+#define EROFS_BLOCK_MAP_ENTRY_SIZE 4
+
+#define EROFS_NULL_ADDR -1
+
+struct grub_erofs_inode_chunk_index
+{
+ grub_uint16_t advise;
+ grub_uint16_t device_id;
+ grub_uint32_t blkaddr;
+};
+
+union grub_erofs_inode_i_u
+{
+ grub_uint32_t compressed_blocks;
+ grub_uint32_t raw_blkaddr;
+
+ grub_uint32_t rdev;
+
+ struct grub_erofs_inode_chunk_info c;
+};
+
+struct grub_erofs_inode_compact
+{
+ grub_uint16_t i_format;
+
+ grub_uint16_t i_xattr_icount;
+ grub_uint16_t i_mode;
+ grub_uint16_t i_nlink;
+ grub_uint32_t i_size;
+ grub_uint32_t i_reserved;
+
+ union grub_erofs_inode_i_u i_u;
+
+ grub_uint32_t i_ino;
+ grub_uint16_t i_uid;
+ grub_uint16_t i_gid;
+ grub_uint32_t i_reserved2;
+} GRUB_PACKED;
+
+struct grub_erofs_inode_extended
+{
+ grub_uint16_t i_format;
+
+ grub_uint16_t i_xattr_icount;
+ grub_uint16_t i_mode;
+ grub_uint16_t i_reserved;
+ grub_uint64_t i_size;
+
+ union grub_erofs_inode_i_u i_u;
+
+ grub_uint32_t i_ino;
+
+ grub_uint32_t i_uid;
+ grub_uint32_t i_gid;
+ grub_uint64_t i_mtime;
+ grub_uint32_t i_mtime_nsec;
+ grub_uint32_t i_nlink;
+ grub_uint8_t i_reserved2[16];
+} GRUB_PACKED;
+
+enum
+{
+ EROFS_FT_UNKNOWN,
+ EROFS_FT_REG_FILE,
+ EROFS_FT_DIR,
+ EROFS_FT_CHRDEV,
+ EROFS_FT_BLKDEV,
+ EROFS_FT_FIFO,
+ EROFS_FT_SOCK,
+ EROFS_FT_SYMLINK,
+ EROFS_FT_MAX
+};
+
+#define EROFS_NAME_LEN 255
+
+struct grub_erofs_dirent
+{
+ grub_uint64_t nid;
+ grub_uint16_t nameoff;
+ grub_uint8_t file_type;
+ grub_uint8_t reserved;
+} GRUB_PACKED;
+
+enum
+{
+ BH_Meta,
+ BH_Mapped,
+};
Why need those (BH_xxx) for a bootloader? you could always use
#define EROFS_MAP_META 0x01
#define EROFS_MAP_MAPPED0x02
Otherwise it looks good to me too, please fix the nits above:
Reviewed-by: Gao Xiang
Thanks,
Gao Xiang
___
Grub-devel mailing list
Grub-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/grub-devel
Re: [PATCH 1/2] fs/erofs: Add support for EROFS
Hi Daniel, On 2023/7/21 01:37, Glenn Washburn wrote: On Thu, 20 Jul 2023 21:14:22 +0800 wrote: Thank you for the reply! As Xiang pointed out, we believe that uncompressed support for EROFS is still valuable, so I would be very grateful if this support could be merged into the GRUB mainline. Yes, I believe this will get merged into master. The question is "when", after or before release. Now that I know that the compression is on a per-file basis and so this version can still be widely used (with the condition that the kernel and initrd are uncompressed by EROFS), I am in support of having this included in the upcoming release. Ultimately the decision is Daniel's though. ping.. May I ask what's your opinion of EROFS uncompressed support for GRUB. I assume compressed EROFS support needs more work to do, so could we support uncompressed support as the first step since EROFS supports per-file compression configuration? The v4 version is already finished with Glenn's RVB: https://lists.gnu.org/archive/html/grub-devel/2023-07/msg00102.html By the way, we expect to bring support for the EROFS compression part within a few weeks. However, AFAIK, GRUB lacks generic support for lz4 compression algorithms. Would it be more appropriate to introduce a dedicated lz4 library for EROFS, like zfs, or generic lz4 support for grub? I have not looked into ZFS's usage of its internal library, nor how hard it would be to make it a separate module. But I prefer the separate module approach. Perhaps Daniel can speak to his preferred> method. same here. Thanks, Gao Xiang Glenn ___ Grub-devel mailing list Grub-devel@gnu.org https://lists.gnu.org/mailman/listinfo/grub-devel
Re: [PATCH 1/2] fs/erofs: Add support for EROFS
Hi, On 2023/9/8 03:57, Daniel Kiper wrote: Hi Gao, On Wed, Sep 06, 2023 at 01:46:25PM +0800, Gao Xiang wrote: Hi Daniel, On 2023/7/21 01:37, Glenn Washburn wrote: On Thu, 20 Jul 2023 21:14:22 +0800 wrote: Thank you for the reply! As Xiang pointed out, we believe that uncompressed support for EROFS is still valuable, so I would be very grateful if this support could be merged into the GRUB mainline. Yes, I believe this will get merged into master. The question is "when", after or before release. Now that I know that the compression is on a per-file basis and so this version can still be widely used (with the condition that the kernel and initrd are uncompressed by EROFS), I am in support of having this included in the upcoming release. Ultimately the decision is Daniel's though. ping.. May I ask what's your opinion of EROFS uncompressed support for GRUB. I assume compressed EROFS support needs more work to do, so could we support uncompressed support as the first step since EROFS supports per-file compression configuration? The v4 version is already finished with Glenn's RVB: https://lists.gnu.org/archive/html/grub-devel/2023-07/msg00102.html We are in the GRUB 2.12 release process and busy with it. I will take a look at the patch set after the release. Sorry for delay. Okay, thanks. By the way, we expect to bring support for the EROFS compression part within a few weeks. However, AFAIK, GRUB lacks generic support for lz4 compression algorithms. Would it be more appropriate to introduce a dedicated lz4 library for EROFS, like zfs, or generic lz4 support for grub? I have not looked into ZFS's usage of its internal library, nor how hard it would be to make it a separate module. But I prefer the separate module approach. Perhaps Daniel can speak to his preferred> method. I agree with Glenn, please put/move lz4 compression code to separate module. Thanks for the confirmation on this stuff! Thanks, Gao Xiang Daniel ___ Grub-devel mailing list Grub-devel@gnu.org https://lists.gnu.org/mailman/listinfo/grub-devel
Re: [PATCH 1/2] fs/erofs: Add support for EROFS
Hi Daniel, On Fri, Sep 08, 2023 at 10:04:53AM +0800, Gao Xiang wrote: > Hi, > > On 2023/9/8 03:57, Daniel Kiper wrote: > > Hi Gao, > > > > On Wed, Sep 06, 2023 at 01:46:25PM +0800, Gao Xiang wrote: > > > Hi Daniel, > > > > > > On 2023/7/21 01:37, Glenn Washburn wrote: > > > > On Thu, 20 Jul 2023 21:14:22 +0800 > > > > wrote: > > > > > > > > > Thank you for the reply! > > > > > > > > > > As Xiang pointed out, we believe that uncompressed support for > > > > > EROFS is still valuable, so I would be very grateful if this > > > > > support could be merged into the GRUB mainline. > > > > > > > > Yes, I believe this will get merged into master. The question is > > > > "when", after or before release. Now that I know that the compression is > > > > on a per-file basis and so this version can still be widely used (with > > > > the condition that the kernel and initrd are uncompressed by EROFS), I > > > > am in support of having this included in the upcoming release. > > > > Ultimately the decision is Daniel's though. > > > > > > ping.. May I ask what's your opinion of EROFS uncompressed support > > > for GRUB. I assume compressed EROFS support needs more work to do, > > > so could we support uncompressed support as the first step since > > > EROFS supports per-file compression configuration? > > > > > > The v4 version is already finished with Glenn's RVB: > > > https://lists.gnu.org/archive/html/grub-devel/2023-07/msg00102.html > > > > We are in the GRUB 2.12 release process and busy with it. I will take > > a look at the patch set after the release. Sorry for delay. > > Okay, thanks. It seems that GRUB 2.12 has been always released. Is it possible to consider this to the upstream? Also EROFS has some more information on the webpages: https://erofs.docs.kernel.org > > > ... > > > > > > > > I have not looked into ZFS's usage of its internal library, nor how > > > > hard it would be to make it a separate module. But I prefer the > > > > separate module approach. Perhaps Daniel can speak to his preferred> > > > > method. > > > > I agree with Glenn, please put/move lz4 compression code to separate module. > > Thanks for the confirmation on this stuff! For EROFS support for compressed files, it still needs some work/review (mainly seperate modules) to do anyway. IMHO, we could address them step-by-step. Thanks, Gao Xiang > > Thanks, > Gao Xiang > > > > > Daniel ___ Grub-devel mailing list Grub-devel@gnu.org https://lists.gnu.org/mailman/listinfo/grub-devel
Re: [PATCH 1/2] fs/erofs: Add support for EROFS
On 2023/12/27 21:10, Daniel Kiper wrote: Hi Gao, On Mon, Dec 25, 2023 at 03:26:28PM +0800, Gao Xiang wrote: Hi Daniel, [...] It seems that GRUB 2.12 has been always released. Is it possible to consider this to the upstream? Sure thing. Though there is a holiday season here. So, expect reviews in a few weeks. Thanks. That would be great helpful! Thanks, Gao Xiang ___ Grub-devel mailing list Grub-devel@gnu.org https://lists.gnu.org/mailman/listinfo/grub-devel
