8 Commits

Author SHA1 Message Date
5ee132a1ce feat: add validation feature for row count comparison
Add a validate command that compares row counts between source and target tables after a migration.
2026-05-12 17:00:00 -05:00
6f8a332719 refactor: enhance config handling and unify main entrypoint
Unify configuration parsing logic across entrypoints and clean up legacy script-side config paths.
2026-05-09 12:00:00 -05:00
bf15979c00 refactor: consolidate error handling and retry configuration
Centralize the error channel size, retry policy and max-failed batches/partition limits in the job config.
2026-05-05 16:00:00 -05:00
933a7572d5 feat: integrate azure storage handling in migration pipeline
Stream transformer output to an Azure blob container when a job declares ToStorage, with per-job prefix and credential configuration.
2026-04-21 15:00:00 -05:00
f0caa55a21 feat: add source and target database type fields
Add source_db_type and target_db_type to the migration config so the same binary can drive any supported direction.
2026-04-17 13:00:00 -05:00
ecf0937a53 feat: add database wrapper interfaces for mssql and postgres
Introduce a DB dialect interface and per-dialect wrappers that expose connection, query and column type operations with pre/post SQL hooks.
2026-04-15 19:00:00 -05:00
74abf12dcf feat: add exponential backoff retry strategy
Introduce a retry policy with exponential backoff and jitter for extractor and loader errors, with configurable max attempts and delay caps.
2026-04-13 14:00:00 -05:00
7f3d2b8cc4 refactor: rename Batch to Partition across pipeline
Rename internal types and channels from Batch to Partition for consistency with the data model.
2026-04-11 18:00:00 -05:00