Change data capture, usually called CDC, is a pattern for observing changes in a source system and applying them elsewhere. It can reduce the need to reread an entire database when only a small part of the information has changed.
This explainer uses PostgreSQL documentation to clarify the concepts behind logical change capture. It describes a general engineering pattern rather than Covorg’s internal technology choices. A production design must account for the source database, permissions, transaction behaviour and destination requirements.
Distinguish the recovery log from the analytical event
PostgreSQL’s Write Ahead Logging records changes before the corresponding modified data pages are persisted. Its primary role is database durability and recovery. A recovery log is therefore not the same thing as a complete business event model.
Logical replication represents changes to data objects using replication identity, usually a primary key. An analytical consumer still needs to interpret the affected table, operation and record in business terms. A row update might mean an order correction, a status transition or an unrelated administrative edit.
Bridge the initial snapshot and subsequent changes
A new analytical copy needs an initial state before it can apply ongoing changes. PostgreSQL describes a typical logical replication start as a table snapshot followed by changes made since that initial copy. Coordinating these boundaries avoids missing changes that occur while the initial data is being copied.
The destination must also define its publication boundary. Users should not mistake a partially populated snapshot for a complete operational view. Record progress and expose a readiness state until the required source scope has been captured and reconciled.
| Concept | Purpose | Question for design |
|---|---|---|
| Snapshot boundary | Establish the starting state | Which changes belong after the snapshot? |
| Record identity | Locate the affected entity | Can updates and deletions identify a row? |
| Source position | Track progress through changes | When is the checkpoint safe to advance? |
| Replay policy | Recover after interruption | Will repeated input create duplicate effects? |
| Schema version | Interpret changed record structure | How are incompatible changes reviewed? |
Choose the destination’s historical meaning
A latest state model updates the current representation of each record. A history model retains successive versions or change events. The first is useful for current balances and open work, while the second supports transition analysis and the reconstruction of earlier observations.
Choose how to handle deletions, source corrections and transaction boundaries. A deletion may remove an entity from a current view while leaving a governed historical record where retention rules permit it. A business process spanning several source systems may require reconciliation beyond the consistency guarantees of any one replication stream.
Understand the limits of replay
A checkpoint should identify progress that the destination can recover. If processing fails after an update but before the checkpoint advances, the same input may be read again. Use a duplicate safe update strategy and test interruption around each persistence boundary.
Change history is not necessarily retained indefinitely. Monitor consumer progress and agree what happens if the required source position is no longer available. Recovery may require a new snapshot and reconciliation. CDC also does not remove the need for business validation, access control or explicit metric definitions.
Change capture lifecycle
- 01
Establish state
Capture an initial snapshot with a coordinated change boundary.
- 02
Read changes
Receive identified source operations in the supported ordering scope.
- 03
Apply changes
Update the agreed current or historical destination model.
- 04
Advance safely
Persist a recoverable checkpoint and monitor remaining backlog.
Design considerations
- Verify source identity requirements for updates and deletions.
- Evaluate transaction guarantees within their documented source and subscription scope.
- Plan a complete rebuild path when retained change history cannot support recovery.
Sources
The following sources explain the underlying technical patterns.
