Change Log¶
All notable changes to this project will be documented in this file.
The format is based on Keep a Changelog and this project adheres to Semantic Versioning.
SynchDB 1.0 - 2024-12-24¶
This release focuses on bug fixes and performance enhancements following the v1.0 Beta1 release that generally makes it more usable under moderate to high data loads. A lot more Debezium tuning related parameters have also been exposed as PostgreSQL GUCs, allowing user to test with different parameters.
Added¶
- added a data cache in DML parsing stage to prevent frequent access to PostgreSQL's catalog to obtain a table's tuple descriptor structure.
- added a variant of
synchdb_start_engine_bgw(name, mode)
that takes a second argument to indicate a custom snapshot mode to start the connector with. More detail here. - added several new GUCs that can be adjusted to tune the performance of Debezium Runner. Refer to here for complete list.
- added a debug SQL function
synchdb_log_jvm_meminfo(name)
that causes specified connector to output current JVM heap memory usage summary in PostgreSQL log file. - added a new VIEW
synchdb_stats_view
that prints statistic information for all connectors. More detail here. - added a new SQL function
synchdb_reset_stats(name)
to clear statistic information of specified connector. More detail here. - added a mess creation script to quickly generate test tables and data on MySQL database type.
Changed¶
- synchdb_state_view(): added a new field called
stage
that indicates the current stage of a connector (value can either beInitial Snapshot
orChange Data Capture
). - synchdb_state_view(): will only show state of valid connectors.
- removed sending "partial batch completion" notification to Debezium Runner in case of error, because a batch is now handled by one PostgreSQL transaction, and partial completion is not allowed.
- SSL related parameters per connector can now be specified in the rule file.
- the maximum heap memory to allocate to JVM that runs the Debezium Runner can now be configured via GUC.
- max number of connector background worker is now configurable instead of hardcoded 30.
Fixed¶
- fixed rapid memory buildups in Debezium runner in JVM by adding a throttle control in the receiving of change events.
- resolved majority of memory leak in both SynchDB and Debezium runner components.
- corrected the use of memory context in SynchDB such that heap memory can be correctly freed at the end of each change event processing.
- significantly increased the processing speed of SynchDB by processing a batch within a single PostgreSQL transaction rather than multiple.
- corrected SQLServer's default data type size mapping for
char
type from 0 to -1. - resolved high memory usage during DML processing using SPI.
SynchDB 1.0 Beta1 - 2024-10-23¶
The first SynchDB beta software release that lays a robust foundation for seamless replication from heterogeneous databases to PostgreSQL.
Added¶
- logical replication from heterogeneous databases: (MySQL and SQLServer).
- DDL replication (CREATE TABLE, DROP TABLE, ALTER TABLE ADD COLUMN, ALTER TABLE DROP COLUMN, ALTER TABLE ALTER COLUMN).
- DML replication (INSERT, UPDATE, DELETE).
- max 30 concurrent connector workers.
- automatic connector launcher at PostgreSQL startup.
- global connector state and last error message views.
- selective databases and tables replication.
- change events in batches.
- connector restarts in different snapshot modes.
- offset management interfaces to select custom replication resume point.
- default data type and object name transform rules for supported heterogeneous databases.
- JSON rule file to define custom: (data type, column name, table name and data expression transform rules).
- 2 data apply modes (SPI, HeapAM API).
- several utility functions to perform connector operations: (start, stop, pause, resume).
Changed¶
N/A
Fixed¶
N/A