Add wasm32-unknown-unknown compilation target support for matrix_sdk_sqlite by NoManColonies · Pull Request #6329 · matrix-org/matrix-rust-sdk

NoManColonies · 2026-03-20T17:23:43Z

Before starting, I apologize for any inconvenience caused by my non-native English.

This PR adds support for compiling matrix_sdk_sqlite to the wasm32-unknown-unknown target, enabling use in browser and similar WASM environments.

Currently, matrix_sdk in WASM is limited to:

in-memory state store/event cache, or
IndexedDB-backed state store with in-memory event cache

This means persistent event/media caching is not available without a custom implementation.
This PR introduces SQLite-backed persistence for WASM as an additional store option.

Summary of changes

Bump rusqlite to 0.38 (adds wasm32-unknown-unknown support via sqlite-wasm-rs)
- cache feature now required for prepared/cached statements
- fallible_uint feature required for unsigned integer casting
Add wasm32-wasi-vfs feature for WASM targets to enable a browser-compatible VFS
- Default backend: OPFS
- Other VFS implementation is split into a separate PR
Add web-time dependency for media retention policy serialization
Gate Tokio fs feature and deadpool-sync behind non-WASM targets
Replace Send with SendOutsideWasm where required for non-Send WASM environments
Use #[cfg_attr(...)] instead of unconditional #[async_trait] for target-specific behavior
Replace SyncWrapper<T> with RefCell<T> in the WASM deadpool adapter
- SyncWrapper<T> requires Send bound
- RefCell<T> provides equivalent interior mutability without Send bound

Notes for reviewers

Default VFS backend is OPFS; other VFS implementation is split into a separate PR
No public API changes beyond documented feature additions
- js feature is needed in order to compile to WASM target
Changes are largely conditional on target_family = "wasm"

I've documented the public API Changes in the appropriate CHANGELOG.md files
This PR was made with the help of AI
- This PR description was machine translated/assisted

Signed-off-by: Geng Kongpop geng@finnomena.com

codecov · 2026-03-20T18:15:49Z

Codecov Report

❌ Patch coverage is 75.75758% with 8 lines in your changes missing coverage. Please review.
✅ Project coverage is 89.83%. Comparing base (8b762b0) to head (5a38eff).
⚠️ Report is 2 commits behind head on main.
✅ All tests successful. No failed tests found.

Files with missing lines	Patch %	Lines
crates/matrix-sdk-sqlite/src/test_utils.rs	0.00%	2 Missing and 1 partial ⚠️
crates/matrix-sdk-sqlite/src/lib.rs	66.66%	0 Missing and 2 partials ⚠️
crates/matrix-sdk-sqlite/src/connection/default.rs	66.66%	0 Missing and 1 partial ⚠️
crates/matrix-sdk-sqlite/src/crypto_store.rs	90.90%	0 Missing and 1 partial ⚠️
crates/matrix-sdk-sqlite/src/state_store.rs	50.00%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #6329      +/-   ##
==========================================
+ Coverage   89.82%   89.83%   +0.01%     
==========================================
  Files         378      379       +1     
  Lines      104337   104330       -7     
  Branches   104337   104330       -7     
==========================================
+ Hits        93719    93729      +10     
+ Misses       7016     7007       -9     
+ Partials     3602     3594       -8

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

codspeed-hq · 2026-03-20T18:28:52Z

Merging this PR will degrade performance by 49.4%

❌ 1 regressed benchmark
✅ 49 untouched benchmarks

⚠️ Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

	Mode	Benchmark	`BASE`	`HEAD`	Efficiency
❌	Simulation	`Restore session [memory store]`	112.8 ms	222.9 ms	-49.4%

_{Comparing NoManColonies:main (5a38eff) with main (dab87b7)}

richvdh

Thanks very much for submitting this. We're quite excited by it, as it opens the door to things like being able to use the wasm bindings for matrix-sdk-crypto in node.js (matrix-org/matrix-sdk-crypto-wasm#168), and maybe even dropping support for indexeddb altogether.

Some questions, other than those below:

How were you able to test/build this? I tried things like cargo check -p matrix-sdk-sqlite --target wasm32-unknown-unknown and wasm-pack test --chrome --headless crates/matrix-sdk-sqlite, and got build errors.

Would you be able to add support to cargo xtask ci for checking and testing matrix-sdk-sqlite with the wasm target? I think you'll need to add a new entry to WasmFeatureSet.
Would you be able to break the changes down into smaller commits, to make them easier to review?

richvdh · 2026-03-26T07:52:37Z

@NoManColonies I assume you're still working on this. Once it's ready for review, please rebase your changes on main and ping me.

NoManColonies · 2026-03-28T17:17:48Z

@richvdh

Thanks very much for submitting this. We're quite excited by it, as it opens the door to things like being able to use the wasm bindings for matrix-sdk-crypto in node.js (matrix-org/matrix-sdk-crypto-wasm#168), and maybe even dropping support for indexeddb altogether.

Some questions, other than those below:

How were you able to test/build this? I tried things like cargo check -p matrix-sdk-sqlite --target wasm32-unknown-unknown and wasm-pack test --chrome --headless crates/matrix-sdk-sqlite, and got build errors.
Would you be able to add support to cargo xtask ci for checking and testing matrix-sdk-sqlite with the wasm target? I think you'll need to add a new entry to WasmFeatureSet.

Would you be able to break the changes down into smaller commits, to make them easier to review?

Thank you very much for taking your time to review this PR and apologize for the late reply.
To address your questions:

How were you able to test/build this? I tried things like cargo check -p matrix-sdk-sqlite --target wasm32-unknown-unknown and wasm-pack test --chrome --headless crates/matrix-sdk-sqlite, and got build errors.

I initially tested the prototype in a temporary downstream project and I seem to have forgotten to properly configure the feature requirements in the upstream crate, my apologies. You should be able to build and test with the following commands:

Cargo check: cargo check -p matrix-sdk-sqlite --target wasm32-unknown-unknown --features js
Wasm-pack test: wasm-pack test --chrome --headless crates/matrix-sdk-sqlite --features js --release
- Currently, the OPFS backend has a harmless debug assertion that triggers at the end of test. So you may need to run the test under release profile

Would you be able to add support to cargo xtask ci for checking and testing matrix-sdk-sqlite with the wasm target? I think you'll need to add a new entry to WasmFeatureSet.

I’ve added tests as requested, but current VFS backend (OPFS) require dedicated worker context and is therefore not supported in a Node.js environment without polyfill. I have made a separate PR adding alternative backends that should be usable in Node.js.

Would you be able to break the changes down into smaller commits, to make them easier to review?

Apologies for the large initial commit, it was initially composed of many small commit that didn’t compile. I saw a note in CONTRIBUTING.md that requires all commits to compile successfully, and all initial changes were required in order to compile under WASM targets. If you'd prefer, I can break the initial commit down into a smaller one.

Since your previous comments, I have made the following changes:

I have refactored associated type of Manager trait in src/connection.rs to use a wrapper type around RefCell<T> and keeping the interface similar to SyncWrapper<T> in order to avoid most of conditional compilation. If you prefer a WASM connection manager entirely in a separate module, please let me know. I initially did not go this route, as it requires a lot of duplicate code in the connection manager that is largely the same as non-WASM targets WASM specific connection manager implementation has also been moved to a separate module.
I have created a utility function setup_db_fs in src/utils.rs for setting up filesystem under different compilation targets in order to avoid duplicate code in each module.
I have added tests for cargo xtask ci to cover the matrix-sdk-sqlite crate when compiled for wasm targets.
- The provided test vector isn’t compatible with SQLite compiled for wasm, so I dumped the database contents and converted them using online tool. The wasm tests now use this test vector instead of the previously provided one.
I have refactored VFS registration to avoid global installation which is harder to test and could limit downstream usage of SQLite.

If you have additional requests, please let know.

chore: address breaking changes from rusqlite feat: support alternative v/fs for wasm environment feat: support non-send future for wasm environment fix: solve incorrect error transformation feat: WIP support connection pool in wasm environment fix: use full-path method call for wasm environment + other fixes for wasm fix: more wasm related fixes feat: use RefCell to simulate interior-mutability in wasm environment fix: use implicit borrow instead of explicit call to `as_ref()` fix: more wasm related fixes fix: resolve ambiguity between lock guard and ref cell implicit reference + more wasm fixes fix: resolve wasm related lifetime issues chore: turn on serde feature for web time when compiled as wasm32 target chore: ran cargo check to verify Cargo.lock file chore: fix comment formatting

…g conditional compilation

richvdh

Thanks very much for the updates. I've taken a more detailed look. Generally it looks great, but I have some comments.

richvdh · 2026-03-30T20:41:03Z

+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+//! An implementation of `deadpool` for `rusqlite` for usage


It doesn't really implement the whole of deadpool, it's just a manager.

Suggested change

//! An implementation of `deadpool` for `rusqlite` for usage

//! An implementation of `deadpool::managed::Manager` for `rusqlite` for usage

richvdh · 2026-03-30T20:41:18Z

+// limitations under the License.
+
+//! An implementation of `deadpool` for `rusqlite` for usage
+//! in WASM environment.


Suggested change

//! in WASM environment.

//! in WASM environments.

richvdh · 2026-03-30T20:48:39Z

+//! Similar to the one implemented in `crate::connection::default`,
+//! we do not implement connection recycling here. Mostly due to
+//! [`managed::Manager`] trait expecting a future output with `Send`
+//! bound which is not available in WASM environment.


I don't really understand this comment. What does "do not implement connection recycling" mean? In what sense does Manager expect a future with Send?

richvdh · 2026-03-30T20:49:12Z

+/// The default runtime used by `matrix-sdk-sqlite` for `deadpool`.
+pub const RUNTIME: Runtime = Runtime::Tokio1;
+
+deadpool::managed_reexports!(
+    "matrix-sdk-sqlite",
+    Manager,
+    managed::Object<Manager>,
+    rusqlite::Error,
+    Infallible
+);
+
+/// Type representing a connection to SQLite from the [`Pool`].
+pub type Connection = Object;


Since this is the same in both implementations, maybe it could live in mod.rs?`

richvdh · 2026-03-30T20:54:06Z

+        }
+        .await
+    }
+}


I think this should probably live in connection::wasm

(and it probably needs a clearer name. I think probably just ConnectionWrapper would be fine, given it will be inside the connection::wasm module)

richvdh · 2026-03-30T23:11:26Z

+    if let Some(WasmFeatureSet::Sqlite) = cmd {
+        // Current VFS backend is not supported in Node.JS just yet.
+        // We force them to run in browser only for now.
+        run_wasm_pack_tests(Some(WasmFeatureSet::SqliteAllFeatures), WasmTestRunner::Chrome)?;
+        run_wasm_pack_tests(Some(WasmFeatureSet::SqliteAllFeatures), WasmTestRunner::Firefox)?;
+        run_wasm_pack_tests(Some(WasmFeatureSet::SqliteCache), WasmTestRunner::Chrome)?;
+        run_wasm_pack_tests(Some(WasmFeatureSet::SqliteCache), WasmTestRunner::Firefox)?;
+        run_wasm_pack_tests(Some(WasmFeatureSet::SqliteState), WasmTestRunner::Chrome)?;
+        run_wasm_pack_tests(Some(WasmFeatureSet::SqliteState), WasmTestRunner::Firefox)?;
+        run_wasm_pack_tests(Some(WasmFeatureSet::SqliteCrypto), WasmTestRunner::Chrome)?;
+        run_wasm_pack_tests(Some(WasmFeatureSet::SqliteCrypto), WasmTestRunner::Firefox)?;
+        return Ok(());
+    }


can you put this next to the indexeddb equivalent?

richvdh · 2026-03-30T23:12:28Z

+          # [m]-sqlite is not supported on Node.js yet, due to
+          # dedicated worker requirement for OPFS


I don't think this matters here.

Suggested change

# [m]-sqlite is not supported on Node.js yet, due to

# dedicated worker requirement for OPFS

richvdh · 2026-03-30T23:48:39Z

+        // SQLite WASM test suites has to be ran in release mode due to
+        // a harmless debug assertion when closing database.
+        //
+        // Ref: https://github.com/Spxg/sqlite-wasm-rs/blob/master/crates/sqlite-wasm-vfs/src/sahpool.rs#L672


why does this happen? The assertion is "close without open", but doesn't the database file get opened?

I’m not entirely sure why this happens. I don’t have deep insight into SQLite internals or the VFS implementation, but testing in a real browser environment seems to work as expected. Creating and opening a database in OPFS behaves correctly.

I’m not sure why this assertion ("close without open") is being triggered in this case. It may be worth checking with the author of https://github.com/Spxg/sqlite-wasm-rs for more insight.

richvdh · 2026-03-31T00:00:18Z

+      # Required in order to compile SQLite to WebAssembly
+      - name: Install emscripten toolchains
+        run: |
+          git clone https://github.com/emscripten-core/emsdk.git
+          cd emsdk
+          ./emsdk install latest
+          ./emsdk activate latest


This doesn't seem to be necessary when I run this locally. Are you sure it's needed?

richvdh · 2026-04-13T10:40:30Z

@NoManColonies are you still working on this?

NoManColonies · 2026-04-13T15:28:53Z

@NoManColonies are you still working on this?

@richvdh Apologies for the delay. I got a bit caught up with some IRL stuff. I've made the following changes as requested:

Updated doc comments for better clarity
Moved the shared re-exports inside the connection module to mod.rs, and refactored manager initialization to avoid having to call setup_db_fs in multiple places
Moved wasm-specific code from utils.rs to wasm.rs, and renamed the wrapper type as suggested
Removed the forced runner in the wasm CI, as it turned out not to be necessary
The unit tests already configure the wasm-bindgen test runner, so there’s no need to set it again in CI
Added test utilities for managing temporary directories, reducing the need for conditional compilation within unit tests

richvdh · 2026-04-14T12:31:13Z

@NoManColonies no need to apologise for the delay: I am happy to know you are still working on this!

However, I see you have force-pushed since my previous review, which means there is no way for me to see what has changed. As https://github.com/matrix-org/matrix-rust-sdk/blob/main/CONTRIBUTING.md says: "Do not force push after a review has started."

In order for me to see what has changed since 4bd5f07, please:

Reset your branch to 4bd5f07, and force-push once more.
Optionally, merge the latest main into your branch.
Make your new changes
Push your new changes to github.

…_up_fs multiple times

NoManColonies · 2026-04-14T14:20:40Z

@NoManColonies no need to apologise for the delay: I am happy to know you are still working on this!

However, I see you have force-pushed since my previous review, which means there is no way for me to see what has changed. As https://github.com/matrix-org/matrix-rust-sdk/blob/main/CONTRIBUTING.md says: "Do not force push after a review has started."

In order for me to see what has changed since 4bd5f07, please:

Reset your branch to 4bd5f07, and force-push once more.

Optionally, merge the latest main into your branch.

Make your new changes

Push your new changes to github.

Apologies for that. I’ve reset the branch back to 4bd5f07 and re-applied the changes. Please let me know if anything looks off.

NoManColonies changed the title ~~Pull changes from upstream~~ Added wasm32-unknown-unknown compilation support target for matrix_sdk_sqlite Mar 20, 2026

NoManColonies changed the title ~~Added wasm32-unknown-unknown compilation support target for matrix_sdk_sqlite~~ Added wasm32-unknown-unknown compilation target support for matrix_sdk_sqlite Mar 20, 2026

NoManColonies force-pushed the main branch from ad25eab to 44967f3 Compare March 23, 2026 04:09

NoManColonies marked this pull request as ready for review March 23, 2026 10:06

NoManColonies requested a review from a team as a code owner March 23, 2026 10:06

NoManColonies requested review from poljar and removed request for a team March 23, 2026 10:06

poljar mentioned this pull request Mar 23, 2026

Support Sqlite storage type matrix-org/matrix-sdk-crypto-wasm#168

Open

richvdh self-requested a review March 25, 2026 12:12

richvdh requested changes Mar 25, 2026

View reviewed changes

Comment thread crates/matrix-sdk-sqlite/src/connection.rs Outdated

Comment thread crates/matrix-sdk-sqlite/src/connection/default.rs

Comment thread crates/matrix-sdk-sqlite/src/crypto_store.rs Outdated

richvdh changed the title ~~Added wasm32-unknown-unknown compilation target support for matrix_sdk_sqlite~~ Add wasm32-unknown-unknown compilation target support for matrix_sdk_sqlite Mar 25, 2026

NoManColonies requested a review from a team as a code owner March 26, 2026 04:20

NoManColonies force-pushed the main branch 3 times, most recently from 5e409c1 to bf489aa Compare March 26, 2026 10:28

NoManColonies marked this pull request as draft March 27, 2026 19:58

NoManColonies force-pushed the main branch from 36759ed to ad559d8 Compare March 28, 2026 11:26

NoManColonies mentioned this pull request Mar 28, 2026

Configurable VFS for matrix_sdk_sqlite compiled to WASM targets #6365

Draft

3 tasks

NoManColonies marked this pull request as ready for review March 28, 2026 17:18

richvdh self-requested a review March 30, 2026 10:23

Geng Kongpop added 6 commits March 31, 2026 00:54

chore(changelogs): update change log

7bfe5a1

fix(sqlite): add js feature to support compiling sqlite to WASM target

746b2e8

chore(sqlite): fix lint warning under WASM target

5325493

refactor(sqlite): create SyncWrapper replacement struct for avoidin…

bb04434

…g conditional compilation

refactor(sqlite): move db fs setup to utils module

7d698fd

Geng Kongpop added 6 commits March 31, 2026 00:55

fix(ci): force SQLite wasm test to run in browser only

4ec20bf

chore(ci): allow dead code when certain features in disabled

281aca3

feat(ci): run matrix sdk wasm test with sqlite features enabled

4e1b849

fix(ci): missing testing feature

ec87f20

fix(test): incorrect database name passed when clear_data is true

fc742f3

chore(changelog): update sdk change log

7ed7f48

NoManColonies force-pushed the main branch from 675dd89 to 9d27165 Compare March 30, 2026 17:56

Geng Kongpop added 2 commits March 31, 2026 01:05

refactor(connection): split wasm implementation into a separate module

42108b1

chore: remove unintented changes from rebase conflict

4bd5f07

NoManColonies force-pushed the main branch from d92729b to 4bd5f07 Compare March 30, 2026 18:05

richvdh requested changes Mar 31, 2026

View reviewed changes

richvdh mentioned this pull request Apr 13, 2026

Crypto store does not work on nodejs matrix-org/matrix-js-sdk#4769

Open

NoManColonies force-pushed the main branch 2 times, most recently from 257fa8e to a7cde3b Compare April 13, 2026 15:24

richvdh self-requested a review April 14, 2026 10:45

Geng Kongpop added 9 commits April 14, 2026 20:30

Merge remote-tracking branch 'upstream/main'

fc2b76e

chore(ci-workflow): remove emsdk installation

c4b68d9

chore(ci): group sqlite feature among other sdk features

3ff2df7

chore(ci): remove forced runner

a62dc9c

refactor(test): use common sleep method instead of platform specific one

0f3d612

refactor(connection): refactor connection module to avoid calling set…

30c9dbc

…_up_fs multiple times

feat(test): create shared test utils

750eeed

refactor(test): use temp dir creation utils for unit tests

8f405c0

chore: remove wasm target os check

93758da

NoManColonies force-pushed the main branch from b4537c4 to 93758da Compare April 14, 2026 14:04

chore: clean up git diff

5a38eff

	//! An implementation of `deadpool` for `rusqlite` for usage
	//! An implementation of `deadpool::managed::Manager` for `rusqlite` for usage

		# [m]-sqlite is not supported on Node.js yet, due to
		# dedicated worker requirement for OPFS

Conversation

NoManColonies commented Mar 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary of changes

Notes for reviewers

Uh oh!

codecov bot commented Mar 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

codspeed-hq bot commented Mar 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merging this PR will degrade performance by 49.4%

Performance Changes

Uh oh!

richvdh left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

richvdh commented Mar 26, 2026

Uh oh!

NoManColonies commented Mar 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

richvdh left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

richvdh commented Apr 13, 2026

Uh oh!

NoManColonies commented Apr 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

richvdh commented Apr 14, 2026

Uh oh!

NoManColonies commented Apr 14, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

NoManColonies commented Mar 20, 2026 •

edited

Loading

codecov bot commented Mar 20, 2026 •

edited

Loading

codspeed-hq bot commented Mar 20, 2026 •

edited

Loading

richvdh left a comment •

edited

Loading

NoManColonies commented Mar 28, 2026 •

edited

Loading

NoManColonies commented Apr 13, 2026 •

edited

Loading