|
|
@@ -150,6 +150,67 @@ def run_upgrade( |
|
|
|
... |
|
|
|
``` |
|
|
|
|
|
|
|
## Background updates |
|
|
|
|
|
|
|
It is sometimes appropriate to perform database migrations as part of a background |
|
|
|
process (instead of blocking Synapse until the migration is done). In particular, |
|
|
|
this is useful for migrating data when adding new columns or tables. |
|
|
|
|
|
|
|
Pending background updates stored in the `background_updates` table and are denoted |
|
|
|
by a unique name, the current status (stored in JSON), and some dependency information: |
|
|
|
|
|
|
|
* Whether the update requires a previous update to be complete. |
|
|
|
* A rough ordering for which to complete updates. |
|
|
|
|
|
|
|
A new background updates needs to be added to the `background_updates` table: |
|
|
|
|
|
|
|
```sql |
|
|
|
INSERT INTO background_updates (ordering, update_name, depends_on, progress_json) VALUES |
|
|
|
(7706, 'my_background_update', 'a_previous_background_update' '{}'); |
|
|
|
``` |
|
|
|
|
|
|
|
And then needs an associated handler in the appropriate datastore: |
|
|
|
|
|
|
|
```python |
|
|
|
self.db_pool.updates.register_background_update_handler( |
|
|
|
"my_background_update", |
|
|
|
update_handler=self._my_background_update, |
|
|
|
) |
|
|
|
``` |
|
|
|
|
|
|
|
There are a few types of updates that can be performed, see the `BackgroundUpdater`: |
|
|
|
|
|
|
|
* `register_background_update_handler`: A generic handler for custom SQL |
|
|
|
* `register_background_index_update`: Create an index in the background |
|
|
|
* `register_background_validate_constraint`: Validate a constraint in the background |
|
|
|
(PostgreSQL-only) |
|
|
|
* `register_background_validate_constraint_and_delete_rows`: Similar to |
|
|
|
`register_background_validate_constraint`, but deletes rows which don't fit |
|
|
|
the constraint. |
|
|
|
|
|
|
|
For `register_background_update_handler`, the generic handler must track progress |
|
|
|
and then finalize the background update: |
|
|
|
|
|
|
|
```python |
|
|
|
async def _my_background_update(self, progress: JsonDict, batch_size: int) -> int: |
|
|
|
def _do_something(txn: LoggingTransaction) -> int: |
|
|
|
... |
|
|
|
self.db_pool.updates._background_update_progress_txn( |
|
|
|
txn, "my_background_update", {"last_processed": last_processed} |
|
|
|
) |
|
|
|
return last_processed - prev_last_processed |
|
|
|
|
|
|
|
num_processed = await self.db_pool.runInteraction("_do_something", _do_something) |
|
|
|
await self.db_pool.updates._end_background_update("my_background_update") |
|
|
|
|
|
|
|
return num_processed |
|
|
|
``` |
|
|
|
|
|
|
|
Synapse will attempt to rate-limit how often background updates are run via the |
|
|
|
given batch-size and the returned number of processed entries (and how long the |
|
|
|
function took to run). See |
|
|
|
[background update controller callbacks](../modules/background_update_controller_callbacks.md). |
|
|
|
|
|
|
|
## Boolean columns |
|
|
|
|
|
|
|
Boolean columns require special treatment, since SQLite treats booleans the |
|
|
|