Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: electric-sql/pglite
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: @electric-sql/pglite@0.2.14
Choose a base ref
...
head repository: electric-sql/pglite
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: @electric-sql/pglite@0.2.15
Choose a head ref
  • 7 commits
  • 19 files changed
  • 4 contributors

Commits on Nov 27, 2024

  1. fix(pglite/tools): add node imports to the package.json browser exclu…

    …des (#444)
    samwillis authored Nov 27, 2024

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature.
    Copy the full SHA
    17c9875 View commit details
  2. Publish new versions (#445)

    Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
    github-actions[bot] and github-actions[bot] authored Nov 27, 2024

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature.
    Copy the full SHA
    bfddaaa View commit details

Commits on Dec 8, 2024

  1. doc: export return type for syncShapeToTable (#452)

    * doc: export return type for syncShapeToTable
    
    * feat: expose stream object as of for useShape hook
    
    * Changeset
    
    ---------
    
    Co-authored-by: Sam Willis <sam.willis@gmail.com>
    yacineb and samwillis authored Dec 8, 2024

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature.
    Copy the full SHA
    c3d98d5 View commit details
  2. fix(pglite/live): Remove a debug console.log from the live query plug…

    …in (#457)
    samwillis authored Dec 8, 2024

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature.
    Copy the full SHA
    fa13714 View commit details
  3. chore(pglite/sync): Bump the version of the Electric sync client (#459)

    * Bump the version of the Electric sync client
    
    * Fix types
    samwillis authored Dec 8, 2024

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature.
    Copy the full SHA
    515a155 View commit details

Commits on Dec 9, 2024

  1. feat(pglite/sync): Add commitGranularity and onInitialSync parame…

    …ters to `syncShapeToTable` (#456)
    
    * Add `commitGranularity` parameter in the `syncShapeToTable`
    
    * Fix tests
    
    * Docs
    
    * add tests for commitGranularity
    
    * Add test of the commitThrottle option
    
    * Fix onInitialSync and add test
    
    * doc: export return type for syncShapeToTable (#452)
    
    * doc: export return type for syncShapeToTable
    
    * feat: expose stream object as of for useShape hook
    
    * Changeset
    
    ---------
    
    Co-authored-by: Sam Willis <sam.willis@gmail.com>
    
    * Remove part of test thats flaky on CI
    
    * formatting
    
    * Fix types
    
    * Remove transaction commit granularity until Electric has stabilised on LSN metadata
    
    * Post review changes
    
    ---------
    
    Co-authored-by: Yacine <ybsoft2011@live.com>
    samwillis and yacineb authored Dec 9, 2024

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature.
    Copy the full SHA
    46c102c View commit details
  2. Publish new versions (#458)

    Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
    github-actions[bot] and github-actions[bot] authored Dec 9, 2024

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature.
    Copy the full SHA
    9aff673 View commit details
57 changes: 24 additions & 33 deletions docs/docs/sync.md
Original file line number Diff line number Diff line change
@@ -38,7 +38,12 @@ You can then use the `syncShapeToTable` method to sync a table from Electric:

```ts
const shape = await pg.electric.syncShapeToTable({
shape: { url: 'http://localhost:3000/v1/shape', table: 'todo' },
shape: {
url: 'http://localhost:3000/v1/shape',
params: {
table: 'todo',
},
},
table: 'todo',
primaryKey: ['id'],
})
@@ -69,7 +74,7 @@ The `syncShapeToTable` is a relatively thin wrapper around the Electric [ShapeSt
It takes the following options as an object:

- `shape: ShapeStreamOptions`<br>
The shape stream specification to sync, described by [`ShapeStreamOptions`](#shapestreamoptions).
The shape stream specification to sync, described by the Electric [ShapeStream API](https://electric-sql.com/docs/api/clients/typescript#shapestream) options, see the [ShapeStream API](https://electric-sql.com/docs/api/clients/typescript#options) for more details.

- `table: string`<br>
The name of the table to sync into.
@@ -89,6 +94,21 @@ It takes the following options as an object:
- `useCopy: boolean`<br>
Whether to use the `COPY FROM` command to insert the initial data, defaults to `false`. This process may be faster than inserting row by row as it combines the inserts into a CSV to be passed to Postgres.

- `commitGranularity: CommitGranularity`<br>
The granularity of the commit operation, defaults to `"up-to-date"`. Note that a commit will always be performed immediately on the `up-to-date` message.
Options:

- `"up-to-date"`: Commit all messages when the `up-to-date` message is received.
<!-- - `"transaction"`: Commit all messages within transactions as they were applied to the source Postgres. -->
- `"operation"`: Commit each message in its own transaction.
- `number`: Commit every N messages.

- `commitThrottle: number`<br>
The number of milliseconds to wait between commits, defaults to `0`.

- `onInitialSync: () => void`<br>
A callback that is called when the initial sync is complete.

The returned `shape` object from the `syncShapeToTable` call has the following methods:

- `isUpToDate: boolean`<br>
@@ -100,40 +120,11 @@ The returned `shape` object from the `syncShapeToTable` call has the following m
- `subscribe(cb: () => void, error: (err: FetchError | Error) => void)`<br>
A callback to indicate that the shape caught up to the main Postgres.

- `subscribeMustRefresh(cb: () => void)`<br>
A callback that is called when the stream emits a `must-refresh` message.

- `unsubscribeMustRefresh(cb: () => void)`<br>
Unsubscribe from the `mustRefresh` notification.

- `unsubscribe()`<br>
Unsubscribe from the shape. Note that this does not clear the state that has been synced into the table.

### `ShapeStreamOptions`

- `url: string`<br>
The full URL to where the Shape is hosted. This can either be the Electric server directly, or a proxy. E.g. for a local Electric instance, you might set `http://localhost:3000/v1/shape`

- `table: string`<br>
The name of the table in the remote database to sync from

- `where?: string`<br>
Where clauses for the shape.

- `offset?: Offset`<br>
The "offset" on the shape log. This is typically not set as the ShapeStream will handle this automatically. A common scenario where you might pass an offset is if you're maintaining a local cache of the log. If you've gone offline and are re-starting a ShapeStream to catch-up to the latest state of the Shape, you'd pass in the last offset and shapeId you'd seen from the Electric server so it knows at what point in the shape to catch you up from.

- `shapeId?: string`<br>
The server side `shapeId`, similar to `offset`, this isn't typically used unless you're maintaining a cache of the shape log.

- `backoffOptions`<br>
Options to configure the backoff rules on failure

- `subscribe?: boolean`<br>
Automatically fetch updates to the Shape. If you just want to sync the current shape and stop, pass false.

- `signal?: AbortSignal`<br>
A `AbortSignal` instance to use to abort the sync.
- `stream: ShapeStream`<br>
The underlying `ShapeStream` instance, see the [ShapeStream API](https://electric-sql.com/docs/api/clients/typescript#shapestream) for more details.

## Limitations

7 changes: 7 additions & 0 deletions packages/benchmark/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,12 @@
# benchmark

## 0.0.15

### Patch Changes

- Updated dependencies [fa13714]
- @electric-sql/pglite@0.2.15

## 0.0.14

### Patch Changes
2 changes: 1 addition & 1 deletion packages/benchmark/package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "benchmark",
"version": "0.0.14",
"version": "0.0.15",
"description": "",
"main": "index.js",
"type": "module",
7 changes: 7 additions & 0 deletions packages/pglite-react/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,12 @@
# @electric-sql/pglite-react

## 0.2.15

### Patch Changes

- Updated dependencies [fa13714]
- @electric-sql/pglite@0.2.15

## 0.2.14

### Patch Changes
2 changes: 1 addition & 1 deletion packages/pglite-react/package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "@electric-sql/pglite-react",
"version": "0.2.14",
"version": "0.2.15",
"description": "Hooks for using PGlite",
"type": "module",
"private": false,
8 changes: 8 additions & 0 deletions packages/pglite-repl/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,13 @@
# @electric-sql/pglite-repl

## 0.2.15

### Patch Changes

- Updated dependencies [fa13714]
- @electric-sql/pglite@0.2.15
- @electric-sql/pglite-react@0.2.15

## 0.2.14

### Patch Changes
2 changes: 1 addition & 1 deletion packages/pglite-repl/package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "@electric-sql/pglite-repl",
"version": "0.2.14",
"version": "0.2.15",
"author": "Electric DB Limited",
"homepage": "https://pglite.dev",
"license": "Apache-2.0",
10 changes: 10 additions & 0 deletions packages/pglite-sync/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,15 @@
# @electric-sql/pglite-sync

## 0.2.17

### Patch Changes

- 46c102c: Add options for the `commitGranularity` parameter in the `syncShapeToTable` function, enabling the user to choose how often the sync should commit.
- c3d98d5: Export return type of `syncShapeToTable`
- 515a155: Bump the version of the Electric sync client
- Updated dependencies [fa13714]
- @electric-sql/pglite@0.2.15

## 0.2.16

### Patch Changes
4 changes: 2 additions & 2 deletions packages/pglite-sync/package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "@electric-sql/pglite-sync",
"version": "0.2.16",
"version": "0.2.17",
"description": "ElectricSQL Sync for PGlite",
"type": "module",
"private": false,
@@ -45,7 +45,7 @@
"dist"
],
"dependencies": {
"@electric-sql/client": "~0.8.0"
"@electric-sql/client": "~0.9.0"
},
"devDependencies": {
"@electric-sql/pglite": "workspace:*",
Loading