[ANN] In-tree helper crates for Rust XPCOM components
Hi all,
Last year, Nika Layzell landed support for implementing XPCOM components in
Rust [1]. Since then, folks on different teams have been adding components,
and working out some common patterns. There are now several in-tree helper
crates that provide more idiomatic wrappers for these patterns, and I
thought I'd take the time to summarize them here.
TL;DR: If you're building a new XPCOM component, please give Rust a try!
[2] The ergonomics get better with every release, and there are building
blocks to help you.
* The `xpcom` crate (`xpcom/rust/xpcom` in the tree) is the bread and
butter for a component implementation. It provides functions for creating
instances and services, and the familiar `RefPtr` for managing
reference-counted pointers.
`xpcom` also includes a set of `derive` macros for declaring your interface
implementations. Check out its docs [3] for more details and examples.
* `nserror` (`xpcom/rust/nserror`) reflects `nsresult` codes into Rust.
You'll almost always want to import this. 😊
* `nsstring` (`xpcom/rust/nsstring`) exposes bindings for XPCOM string
types. You can use the same `ns{A,C}String` types as C++ for owned
strings—there's also `ns{A, C}Str` for dependent or borrowed strings—and
pass them back and forth over the boundary.
* `moz_task` (`xpcom/rust/moz_task`) wraps XPCOM's threading functions.
There are helpers for getting and creating threads, dispatching async
runnables, and thread-safe handles. The goal of this crate is to make it
easy and safeâ„¢ to write threaded code.
* `storage` (`storage/rust`) is an interface to mozStorage, our wrapper for
SQLite. It can wrap an existing storage connection, and prepare and execute
statements. `storage` wraps the synchronous connection API, and lets you
execute statements asynchronously via `moz_task`.
* `storage_variant` (`storage/variant`) is a crate for working with
variants, and also provides a `HashPropertyBag` type that's useful for
passing hash maps over XPCOM to JS.
If you're looking for examples of how to build components, check out:
* `kvstore` (`toolkit/components/kvstore`), which exposes the LMDB
key-value store (via the Rkv library [4]). The API is asynchronous, using
`moz_task` to schedule all I/O on a background thread, and supports
getting, setting, and iterating over keys.
* `cert_storage` (`security/manager/ssl/cert_storage`), which stores lists
of revoked intermediate certificates [5].
* `bookmark_sync` (`toolkit/components/places/bookmark_sync`), which merges
[6] bookmarks from Firefox Sync with bookmarks in the Places database.
Please feel free to file bugs, extend these crates, or add your own helpers
to make it even easier to build Rust components! 🦀
Cheers,
~ lina
[1]: https://bugzilla.mozilla.org/show_bug.cgi?id=1293362
[2]: https://firefox-source-docs.mozilla.org/build/buildsystem/rust.html
[3]:
https://searchfox.org/mozilla-central/rev/2c912888e3b7ae4baf161d98d7a01434f31830aa/xpcom/rust/xpcom/xpcom_macros/src/lib.rs
[4]: https://docs.rs/rkv
[5]:
https://blog.mozilla.org/security/2015/03/03/revoking-intermediate-certificates-introducing-onecrl/
[6]: https://mozilla.github.io/dogear
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform
Re: Intent to unship: system SQLite support on Linux (--enable-system-sqlite)
Thank you very much for sending this out, Marco! I completely agree with your analysis: allowing system SQLite complicates things for us, and introduces many sharp edges without benefitting our users. With packaging conventions changing, it makes even less sense to keep around. I fully support its removal. Cheers, ~ lina On Mon, Feb 10, 2020 at 6:22 AM Marco Bonardo wrote: > The Storage support team intends to remove support for building > Mozilla-based applications using the system provided SQLite library on > Linux (--enable-system-sqlite). > > > > When: After the next mozilla-central merge to Nightly 75, February 10th. We > plan to unship support during the 75 cycle. > > > > Where: https://bugzilla.mozilla.org/show_bug.cgi?id=1611386 > > Why: > >1. > >We have limited resources available to us. Developing new features that >touch SQLite low level APIs have often required additional resources to >ensure system SQLite keeps working; we are at a point where we can't > afford >those development costs. >2. > >We had bugs due to it and the most recent of which, tracked in >https://bugzilla.mozilla.org/show_bug.cgi?id=1607902, is due to a >mistake we made in relying on some Sqlite internal details. The result > of >that, due to supporting system Sqlite, is that we requested the Sqlite > team >to undo some code improvements in a .1 release, and, in spite of that, >older Firefox versions on a specific new Sqlite version will crash > forever. >We don't want to be in this situation where the Sqlite team can't > improve >their product freely. >3. > >There is risk of losing users due to the kind of crashing bugs this can >cause, because not everyone would be able to connect the misbehavior > with >the update of an external library, that may have been updated along with >other packages updates. And downgrading Sqlite may not be an option if >another software requires the new version instead. >4. > >For quite some time we ended up enforcing our Sqlite preferences onto >the system library (things like ‘secure_delete’ or ‘fts’, for example); >addressing those cases requires time and resources. We don't want to >enforce our specific choices. >5. > >Our Web Storage team will soon start working with Sqlite encryption, in >a way that may not be compatible with system Sqlite. Again, having to > cope >with the system library would mean a lot of additional effort, and more >requirements for system Sqlite that may not be well digested. >6. > >We have our own custom memory allocator, and hooking it up has >complications we'd prefer to avoid, if possible. >7. > >The benefits of using system SQLite so far have been minimal. We wanted >to be good citizens by sharing as much as possible with the system, as > it >was common practice. Today the historical packaging strategies on Linux >systems are evolving with the introduction of new packaging systems > (Snap, >Flatpak, for example) that changed the relationship with dependencies by >packing them up with the main application. >The remaining benefits (e.g. package size, memory) are negligible on >today's hardware. >The security benefits we get because the system Sqlite may be more >up-to-date are minimal, because we don't expose Sqlite to Web content, > nor >to add-ons. Additionally, we actively track and test every new Sqlite >release and apply them as soon as possible. > > > > Thanks, > > Marco > ___ > dev-platform mailing list > dev-platform@lists.mozilla.org > https://lists.mozilla.org/listinfo/dev-platform > ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Testing Rust code in tree
Oooh, this question comes at a perfect time, the Glean team has been looking at this recently in bug 1627024, too! Nick's quote from the Oxidation wiki alludes to this, but, if your Rust code is pulling in Gecko crates like `nsstring`, `xpcom`, and such, I don't think `RUST_TESTS` will work—you'd have to write a GTest. There's been some more discussion about that in bug 1628074 and bug 1373878 that came down to "because of complicated reasons, you can't write Rust tests that link against Gecko symbols". (Please correct me if that's no longer the case, though!) https://searchfox.org/mozilla-central/source/toolkit/library/gtest/rust has a list of Rust GTests, and https://searchfox.org/mozilla-central/source/xpcom/rust/gtest/nsstring is a nice example of the glue needed to set one up. Hope this helps! Cheers, ~ lina On Mon, May 11, 2020 at 3:54 PM Mike Hommey wrote: > On Mon, May 11, 2020 at 03:37:07PM -0700, Dave Townsend wrote: > > Do we have any standard way to test in-tree Rust code? > > > > Context: We're building a standalone binary in Rust that in the future > will > > be distributed with Firefox and of course we want to test it. It lives > > in-tree and while we could use something like xpcshell to drive the > > produced executable and verify its effects it would be much nicer to be > > able to use Rust tests themselves. But I don't see a standard way to do > > that right now. > > > > Is there something, should we build something? > > > https://searchfox.org/mozilla-central/rev/446160560bf32ebf4cb7c4e25d7386ee22667255/python/mozbuild/mozbuild/frontend/context.py#1393 > > Mike > ___ > dev-platform mailing list > dev-platform@lists.mozilla.org > https://lists.mozilla.org/listinfo/dev-platform > ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
