Hi,
this sounds interesting. I am running into a similar problem.
Generally we have immutable distributions. The core system and the applications
are in their own, isolated sandboxes, which could be Snaps, Docker containers,
Flatpaks, ... With this, each application brings its own libraries and they
often are of mismatching versions, and also any system-provided, centrally
stored scanner drivers (SANE backends) cannot be dlopened as the core system's
or a driver container's file system is not visible inside the application's sandbox.
IP connections can usually be easily defined between sandboxes/containers
though.
So my idea, mainly with the all-Snap Ubuntu Core Desktop in mind, but applying
to any immutable distro, is to use the same IP protocol as driverless network
scanners and printer/scanner multi-function devices already use. These protocols
are well-defined, established in the industry, and (therefore) stable across
different software versions and implementations.
In my design (derived from driverless IPP printing) the applications come with
only the sane-airscan (written by Alexander Pevzner, CCed) backend, the backend
which supports driverless scanning, currently eSCL and WSD, later perhaps also
IPP Scan. All modern driverless printer/scanner multi-function devices
understand either eSCL or WSD (and IPP for printing). There are also some
driverless standalone scanners around which support these standard protocols.
This way the applications talk directly to the scanners, without need of any
centralized scanner support in the operating system.
Also USB devices can be driverless. Here the IPP-over-USB standard comes to play
which maps the USB device onto localhost, port 60000 and following (should be
actually HTTP-over-USB, as it supports all HTTP protocols: IPP, eSCL, WSD, web
admin interface).
Using industry-standard protocols also allows connections across operating
systems and does not limit to only SANE (we did it more than a decade ago with
CUPS, replacing CUPS' own broadcasting/browsing protocol by DNS-SD).
For scanners which need a driver, as they have a non-standard communication
protocol, we introduce emulations of eSCL scanners, the so-called Scanner
Applications. At the front they appear as eSCL (driverless) scanners and at the
back they are talking with the actual scanner device, converting the protocols
internally. Like a physical network device they advertise the presence of the
scanner via DNS-SD (Avahi under Linux) and the eSCL protocol allows clients to
poll the full set of capabilities (resolutions, scan sizes, ADF, ...) from the
Scanner Application and also to conduct the actual scan.
One Scanner Application can contain any amount of scanner drivers and there can
be any amount of Scanner Applications installed on the system. So we can have
one Scanner Application retro-fitting the whole sane-backends package and other
Scanner Applications issued by scanner manufacturers to support their devices.
Internally Scanner Applications do not need to use SANE, they can accomplish
their job with whatever the developer considers best. So we will have native
Scanner Applications not using SANE internally and retro-fitting Scanner
Applications using SANE.
Applications which scan and scanner drivers (Scanner Applications) can be in
different sandboxes and so be installed independently on immutable (and also
classic) operating system distributions, so especially one can add scanner
drivers to an immutable distribution this way, and as each sandboxed package
brings their own libraries, they can work with different libraries. So
especially also the situation of packages using different libraries in Nix OS
gets solved this way.
For the implementation, on the client side there is not much to do. One only
needs to have a SANE-supporting application and ship it together with the
sane-airscan backend.
On the server/driver/device side is more to do but we are well into it. As we
come from the situation of printer/scanner multi-function devices, out emulation
is actually also an emulation of such a multi-function device if we want to
support a non-driverless multi-function device.
Several years back we (at OpenPrinting) have already started with printing. All
modern printers are driverless. they do driverless IPP printing. AirPrint,
Mopria, IPP Everywhere, and Wi-Fi Direct Print are all just flavors of
driverless IPP printing. Here the printer always advertises itself by DNS-SD.
Then clients can find the printer and poll the printer's capabilities from the
printer via an IPP request and then print the job in a standard data format
(PDF, but also some raster formats for cheaper printers), so there is no need
for a driver (aka device-model-specific software or data, like CUPS filters
and/or PPD files) for printing.
For non-driverless legacy or specialty printers we have introduced Printer
Applications, emulations of driverless IPP printers. To create such Printer
Applications there is a standard library, PAPPL (Printer APPlication Library)
written by Michael Sweet, author of CUPS. It contains everything which every
Printer Application needs: Daemon, handlers for DNS-SD and IPP, web admin
interface, ... and Michael also ha written two native (no CUPS filters and PPD
files hidden inside) Printer Applications, hp-printer-app (for legacy PCL
lasers) and LPrint (for label printers).
I have written pappl-retrofit, an additional library for PAPPL with which one
can easily wrap classic CUPS printer drivers and PostScript PPD files into
Printer Applications (retro-fitting).
Currently Akarshan Kapoor (CCed), GSoC contributor for OpenPrinting last year
and this year, is adding scanner support to PAPPL, so that with PAPPL one can
make both Printer and Scanner Applications, even support multi-function printers
with one single application. He is adding eSCL support to PAPPL (which before
only supported IPP printing) and SANE support to pappl-retrofit.
This way we get a universal Printer/Scanner driver format which allows for
distribution-independent binary package and for adding drivers not only to
classically installed but also to immutable operating systems.
We are snapping these Printer and Scanner Applications and putting them into the
Snap Store. Currently you find all free software printer drivers which usually
come with Linux distributions in 4 retro-fitting Printer Applications and the 2
native Printer Applications from Michael Sweet. Scanner Applications will
hopefully come soon.
Printer Applications and Scanner Applications are not required to get
distributed as Snaps, one can also just install them classically or put them
into other containers.
At OpenPrinting we have this year also a GSoC project on OCI (Docker, ROCKs,
podman, ...) containerization to cover most immutable distros which are around.
For the issues you mentioned, network scanners are available, those on modern
network multi-function devices usually directly, without need of a Scanner
Application. SANE-shared scanners we can support by the planned
sane-backends-retro-fitting Scanner Applications.
Scanner Applications can be configured to share scanners in the local network or
to restrict them for only local use.
The eSCL being an industry standard protocol it will be supported by any version
of sane-airscan and PAPPL once it has scanning support. No changes between
software versions are expected.
Links:
Printer Applications:
https://openprinting.github.io/achievements/#all-free-drivers-in-a-ppd-less-world---or---all-free-drivers-in-snaps
Scanner Applications:
https://openprinting.github.io/current/#scanner-applications
PAPPL: https://github.com/michaelrsweet/pappl/
pappl-retrofit: https://github.com/OpenPrinting/pappl-retrofit/
LPrint: https://github.com/michaelrsweet/lprint/
OpenPrinting in the Snap Store: https://snapcraft.io/publisher/openprinting
LPrint in the Snap Store: https://snapcraft.io/lprint
Scanning support for PAPPL:
https://dev.to/kappuccino111/sandboxing-scanners-a-leap-into-the-driverless-realm-gsoc-23-report-3eci
https://www.youtube.com/watch?v=AAeUseU35Cc
Till
On 09/05/2024 17:39, Guillaume Girol via sane-devel wrote:
Hello,
I'm a downstream maintainer of SANE on NixOS, a linux distribution. I
consider modifying NixOS to make it so all application using libsane
would actually only use saned, and never load the backends to directly
access scanners themselves. I send this message to the mailing list to
know if I missed reasons it could be a bad idea.
Rationale
=========
NixOS is quite different from typical linux distributions in that there
is no globally installed library. /usr/lib does not exist and without
special packaging care, dlopen() never finds anything. Each library is
installed in a unique prefix /nix/store/unique-id/lib/foo.so so several
versions of the same library can coexist. Exeecutable find the
libraries they depend on via explicit RPATH. As a result I can have an
old version of python in /nix/store/some-id/bin/python using an old
version of glibc in /nix/store/other-id/lib/libc.so via RPATH and the
rest of the system using a newer version of glibc in /nix/store/yet-
another-id/lib/libc.so also via RPATH.
This does not work for SANE as it uses dlopen() to find backends
depending on a single, global configuration file. For now we have an
exception to the "no lib installed globally" design for SANE. However
this is not perfect: if I use an old version of, say, simple-scan, it
may dlopen() more recent version of some SANE backend, which depends on
a more recent glibc than the version that old simple-scan brings with
its RPATH. Thus dlopen() fails. In practice this means that software
installed from the past or the future often fails to scan, whereas this
usually works quite well for other functionnality.
Solution considered
===================
To solve this, I want applications using SANE to stop using dlopen() to
access the scanner, but to use a network protocol instead. This is a
bit like glibc can talk to nscd over a socket instead of dlopening nss
modules.
The solution I'm considering consists in linking all normal
applications to a stripped-down variant of libsane which only has the
network backend and is configured to connect to a saned instance on
localhost (instead of obeying what you would find in /etc/sane.d on
typical distros). Each application comes with its own copy of this
libsane, it's not global anymore. (Don't mind the disk space cost, all
of NixOS works this way, this is part of the tradeoff). When the user
configures their system for scanner support, a global instance of saned
is setup which is linked against a full-fledged copy of libsane obeying
the full configuration files you would typically find in /etc/sane.d.
When an old application attempts to scan, it uses its own, old version
of libsane to talk to a recent saned which dlopen()s sane backends of
matching version, so all works fine.
Possible issues?
================
Any issues with this design I missed? In my light testing, it appears
to work quite fine. The problems I found:
- all users can use the scanners instead of just users in the lp or
scanner group previously. I suppose this can be solved with a firewall
(firewalls can filter the uid of local sockets).
- network scanners are not available. I propose a fix in
https://gitlab.com/sane-project/backends/-/merge_requests/834
- is the network protocol that saned uses stable across versions? In my
testing, it is, but maybe it's by sheer luck.
- maybe stuff I have not considered.
Cheers,
Guillaume Girol
