As builders we're often focused on what comes next, but it's also important to reflect on what we've shipped. This edition's feature story is about the upcoming Sync Streams — a major update to our dynamic partial replication system. Skip to "Feature story" for that.
Product updates shipped in August:
- Database adapters
- Our MySQL support has been promoted to Beta. This means it’s now considered stable for production use, and is fully covered under our support SLAs (although resources like docs and demo apps may still be somewhat on the thin side — that’s for V1).
- Client SDKs
- Our Node.js client SDK has also been promoted to Beta.
- It’s currently possible to batch client mutations for uploads using %%getCrudBatch()%%, but this does not respect transaction boundaries. So we added %%getCrudTransactions()%% to batch transactions when uploading mutations. See Dart, Kotlin and JS PRs.
- In our July update, I wrote about incremental watched queries for JS that can avoid re-renders when dealing with large datasets. This was essentially patching the underlying behavior — SQLite queries would still re-run on any table change with the diff being calculated in memory on top of that. Some use cases need the same end result, but more efficient SQLite performance, so we shipped high-performance diffs in our JavaScript SDKs that use SQLite triggers.
- We added a bunch of stuff to our Swift support: enabled SwiftUI Previews, strict concurrency checking for Swift 6, and a new full-text search demo app.
- PowerSync Cloud
- You will now get an email when your usage reaches (or exceeds) what’s included in your PowerSync Cloud plan. We announced this on Discord. In the future we’ll be adding budgeting tools to cap spending, but that’s a tricky problem since we don’t just want to turn off actively syncing instances.
- PowerSync Service
- We released v1.15 of the service, which has a solid number of improvements and additions:
- We added automatic configuration for Supabase asymmetric JWT signing keys (JWKS) when using Supabase Auth and a Supabase-hosted database (PR #311)
- We enabled transport compression for both HTTP Streams and WebSockets, reducing initial sync download size by a whopping 80%. Next up: client-side storage compression.
- Support for Sync Streams. See below.
Behind the scenes
- We have a strong culture of writing internal proposals for new features at PowerSync, and we started publishing those publicly now. See Conrad’s announcement on Discord for our first batch of proposals.
- Community contributions:
- %%omnicpie%% fixed a nasty bug in the official MongoDB Node.js driver used by the PowerSync Service (OK OK, technically not a community contribution to PowerSync, but this was amazing to see and we can undo our workaround fixes now).
- %%whygee-dev%% got the ball rolling for the batched transaction uploads I mentioned above.
- %%Radiokot%% fixed some logging issues in our Kotlin SDK.
- %%ecstatic-morse%% added a note to our docs explaining the consequences of ID aliases when uploading.
- A few creative apps using PowerSync were shared on our #showcase channel. Check out Frontiers Market, PuffinPilot and Whelper.
- Christiaan wrote a post on what synced SQLite brings to Expo Apps that was featured on Expo’s blog.
- We worked with Matthew Weidner on using PowerSync as an experimental testbed for his work on Collaborative Text Editing without CRDTs or OT.
Feature story: Sync Streams
I usually keep the content in these emails to things we’ve shipped, but Sync Streams will be such a fundamental change to the product that I wanted to take some time to tell you about what’s coming.
Sync Streams is an evolution of our Sync Rules system, and it's significant enough to warrant a name change. Our Sync Rules system was initially designed for offline-first use cases where you want to “sync everything upfront” when the client connects. By contrast, Sync Streams is a configurable system that’s optimized for the best of both worlds: it caters for dynamic on-demand syncing, as well as “sync everything upfront” that’s needed for offline-first.
A Sync Stream is a parameterized subset of data that clients are able to subscribe to just-in-time. You define your streams upfront, and a client can subscribe to them one or more times with different parameters. Imagine a web app use case where users open up multiple projects in different tabs. Sync Streams are perfect for this kind of use case where your users are mostly online, and they only need to sync contextual data for the specific things they’re looking at — each tab subscribes just-in-time to just the specific project the user is viewing. It naturally works great for multiple tabs, since multiple subscriptions across tabs are automatically handled. Each subscription includes a configurable %%TTL%% that keeps data active after the client unsubscribes, acting as a warm cache for recently-accessed data.
If you want “sync everything upfront” behavior, that’s easy too: you can configure Sync Streams to be auto-subscribed by the client on connecting.
As a side-effect of all this work, we’ve also simplified the syntax and mental model compared to Sync Rules, delivering significant DX improvements across the entire stack.
It should go without saying, but we will offer a smooth migration path as well as a proper LTS plan for customers using Sync Rules, and we’ll never compromise on our offline-first support.
That’s it for this issue!