Commit Graph

10 Commits

Author SHA1 Message Date
74abf12dcf feat: add exponential backoff retry strategy
Introduce a retry policy with exponential backoff and jitter for extractor and loader errors, with configurable max attempts and delay caps.
2026-04-13 14:00:00 -05:00
7f3d2b8cc4 refactor: rename Batch to Partition across pipeline
Rename internal types and channels from Batch to Partition for consistency with the data model.
2026-04-11 18:00:00 -05:00
1f8689cdfc feat: add configuration parsing and job definitions
Add a YAML config schema for migration jobs (source/target schema, table, primary key, options) and parse it at startup.
2026-04-10 14:00:00 -05:00
4f181269a4 feat: add extract query builders for both dialects
Add QueryFromObject helpers to build select queries from a table schema, with limits and geometry support.
2026-04-09 16:00:00 -05:00
c5987fa7f8 feat: add type transformations (uuid, wkb, datetime)
Apply row-level transformations for special mssql types: uniqueidentifier, geometry/geography (WKB to EWKB) and datetime (UTC).
2026-04-08 11:00:00 -05:00
92f010a9a6 feat: add logging and migration runner skeleton
Wire a structured logger and a top-level migration job that drives extract, transform and load stages.
2026-04-06 13:00:00 -05:00
f7b8dd73d4 refactor: unify database url configuration
Consolidate source/target connection handling around a single URL-based configuration model.
2026-04-03 14:00:00 -05:00
d36b089916 feat: add dynamic query building and type casting
Build SQL queries from column metadata and apply type casts for special postgres types (uuid, jsonb, geometry, etc.).
2026-04-02 11:00:00 -05:00
f6bd1da7ef feat: add data extraction from postgres
Implement extractData to stream rows from a postgres source table.
2026-04-01 13:00:00 -05:00
3e1db7b9e4 chore: init project with go.mod and base layout
Bootstrap the go-migrate project:
- go.mod (Go 1.26)
- cmd/ entrypoint skeleton
- base .gitignore and config files
2026-03-31 09:00:00 -05:00