4ce79687b3
feat: add dry run and individual db config parameters
...
Add a dry-run mode that validates connections and counts source rows without writing, and accept per-connection host/port/user/password/options alongside URLs.
2026-05-15 13:00:00 -05:00
5ee132a1ce
feat: add validation feature for row count comparison
...
Add a validate command that compares row counts between source and target tables after a migration.
2026-05-12 17:00:00 -05:00
6f8a332719
refactor: enhance config handling and unify main entrypoint
...
Unify configuration parsing logic across entrypoints and clean up legacy script-side config paths.
2026-05-09 12:00:00 -05:00
bf15979c00
refactor: consolidate error handling and retry configuration
...
Centralize the error channel size, retry policy and max-failed batches/partition limits in the job config.
2026-05-05 16:00:00 -05:00
70ce71f59b
refactor: replace per-dialect extractors with generic extractor
...
Collapse the mssql- and postgres-specific extractors into a single generic implementation driven by the DB dialect interface.
2026-04-28 18:00:00 -05:00
f8e6c19431
refactor: switch config to cleanenv and remove unused dependencies
...
Migrate config loading to cleanenv and drop unused dependencies and legacy config plumbing.
2026-04-25 11:00:00 -05:00
933a7572d5
feat: integrate azure storage handling in migration pipeline
...
Stream transformer output to an Azure blob container when a job declares ToStorage, with per-job prefix and credential configuration.
2026-04-21 15:00:00 -05:00
f0caa55a21
feat: add source and target database type fields
...
Add source_db_type and target_db_type to the migration config so the same binary can drive any supported direction.
2026-04-17 13:00:00 -05:00
ecf0937a53
feat: add database wrapper interfaces for mssql and postgres
...
Introduce a DB dialect interface and per-dialect wrappers that expose connection, query and column type operations with pre/post SQL hooks.
2026-04-15 19:00:00 -05:00
74abf12dcf
feat: add exponential backoff retry strategy
...
Introduce a retry policy with exponential backoff and jitter for extractor and loader errors, with configurable max attempts and delay caps.
2026-04-13 14:00:00 -05:00
7f3d2b8cc4
refactor: rename Batch to Partition across pipeline
...
Rename internal types and channels from Batch to Partition for consistency with the data model.
2026-04-11 18:00:00 -05:00
1f8689cdfc
feat: add configuration parsing and job definitions
...
Add a YAML config schema for migration jobs (source/target schema, table, primary key, options) and parse it at startup.
2026-04-10 14:00:00 -05:00
4f181269a4
feat: add extract query builders for both dialects
...
Add QueryFromObject helpers to build select queries from a table schema, with limits and geometry support.
2026-04-09 16:00:00 -05:00
c5987fa7f8
feat: add type transformations (uuid, wkb, datetime)
...
Apply row-level transformations for special mssql types: uniqueidentifier, geometry/geography (WKB to EWKB) and datetime (UTC).
2026-04-08 11:00:00 -05:00
92f010a9a6
feat: add logging and migration runner skeleton
...
Wire a structured logger and a top-level migration job that drives extract, transform and load stages.
2026-04-06 13:00:00 -05:00
f7b8dd73d4
refactor: unify database url configuration
...
Consolidate source/target connection handling around a single URL-based configuration model.
2026-04-03 14:00:00 -05:00
d36b089916
feat: add dynamic query building and type casting
...
Build SQL queries from column metadata and apply type casts for special postgres types (uuid, jsonb, geometry, etc.).
2026-04-02 11:00:00 -05:00
f6bd1da7ef
feat: add data extraction from postgres
...
Implement extractData to stream rows from a postgres source table.
2026-04-01 13:00:00 -05:00
3e1db7b9e4
chore: init project with go.mod and base layout
...
Bootstrap the go-migrate project:
- go.mod (Go 1.26)
- cmd/ entrypoint skeleton
- base .gitignore and config files
2026-03-31 09:00:00 -05:00
