Hi,
Following up on [1], I propose a different way of handling sysupgrade backups
that would no longer rely on a filesystem stub (jffs2, ubifs or otherwise).
Note: the following elements describe squashfs with jffs2 overlay images, but I
suspect they apply to more cases (ubifs…).
Current status:
- When sysupgrade is asked to preserve the device configuration over reflash,
the process will:
- prepare a tarball of the config data onto tmpfs
- embed it into a filesystem stub appended after the squashfs data
- append a specific marker (deadc0de) to designate the end of the valid
jffs2 segment
- When the new image boots:
- the squashfs mtdsplitter locates the end of the squashfs image and
places the beginning of the rootfs_data partition after it
- the jffs2 stub is now located at the beginning of the rootfs_data
partition (the writeable overlay)
- a LEDE-specific patch to the jffs2 driver clears the nodes located
after the marker
- the overlay is then mounted and the data it contains is recovered.
This has several drawbacks, as [1] illustrates: in particular it makes it
complicated, if not impossible, to change the filesystem used on the overlay or
any of its parameters - such as sector size in the case of jffs2 - without
losing the backup data.
Proposal:
Use a plain archive on the mtd.
Implementation detail:
- When sysupgrade is called to preserve the device configuration over reflash,
the process will:
- prepare a well formatted (see below) archive of the configuration
data onto tmpfs
- append that data after the squashfs image on the mtd
- When the new image boots:
- the squashfs splitter locates the end of the squashfs image and
places the beginning of the rootfs_data partition after it (step unchanged)
- the archive is now located at the beginning of the rootfs_data
partition
- a hook to the preinit process will attempt to unpack that archive
into tmpfs
- if it is successful (archive found and valid), the
rootfs_data partition will be formatted
- if it is unsuccessful (archive found but invalid), the
rootfs_data partition will be formatted
- if it is unsuccessful (archive not found), the rootfs_data
will not be formatted (regular boot)
- the overlay will then be mounted and if present the data from the
archive will be applied (see below)
Archive format:
For the sake of simplicity and futureproofness, I propose:
- that the archive data be plain, uncompressed tar (tar has low overhead, has
checksumming and was designed to operate in the absence of a filesystem)
- that the archive content be prepared as follows:
- sysupgrade.tgz (the current compressed backup tarball that is stored
in the jffs2 stub)
- a ‘payload’ file, which would be a script containing two mandatory
elements:
- a “REVISION” variable to store the current version/revision
of the system that generated the backup
- a “restore()” function that would be called to restore the
content of the backup
The ‘payload’ file would be the last element in the tar archive
Goals:
- Preserve the ability to use various compression schemes in the archive data
without imposing ex-ante knowledge of it from the new firmware image
(decompression is handled by the ‘payload’ script itself).
- Ensure that if the archive is truncated or damaged, the ‘payload' script will
not be created and thus not executed (the payload script could perform final
checksumming of the data and of itself too, if necessary).
- Preserve future flexibility in the content of the archive data, enabling
other (platform-specific?) hooks to be added down the road without breaking
backward compatibility (by building into the ‘payload’ script) and not
enforcing any specific data structure other than the presence of the ‘payload’
script.
- Enable the new firwmare to easily detect what version the archive data comes
from and make use of that information if necessary (e.g. apply special
treatment or discard the backup).
Benefits:
- No longer tie backup data to a specific filesystem, enabling seamless change
to the underlying overlay filesystem.
- Full control over rootfs_data segment: ensure overlay integrity after flash
by performing a true format.
- Get rid of all out-of-tree, LEDE-specific hacks to jffs2 etc, making it
possible to use vanilla code instead.
- Flexible, futureproof system.
Notes:
- The proposed system does put more strain on the RAM usage during upgrade than
the existing system.
- The proposed system mtd space requirements are similar to the existing system
(same compression level, one extra file, overhead of tar can be lower than
jffs2)
- In the case of factory images, the deadc0de marker could simply be replaced
by an archive containing a single ‘payload’ script whose sole purpose would be
to trigger the overlay formatting.
The proposed system should be possible to implement alongside the existing one
if a transition is desirable (if no archive is found in rootfs_data, fall back
to the current system of looking for the deadc0de marker).
Would there be interest in such a system? Have I missed something?
Comments welcome :)
T.
[1] http://lists.infradead.org/pipermail/lede-dev/2017-November/009863.html
_______________________________________________
Lede-dev mailing list
Lede-dev@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/lede-dev
