DiegoAlessandroMotta
bfe335c933
Remove obsolete MSSQL and PostgreSQL scripts and related types
...
- Deleted `seed-manzana.go`, `site-holder-attach.go`, `types.go`, `utils.go`, `main.go` files from both `mssql-info-test` and `mssql-test` directories.
- Removed `pg-info-test/main.go` and `test-etl/main.go` files.
- Eliminated `wkb-to-ewkb/main.go` for unused WKB to EWKB conversion functionality.
- Cleaned up associated functions and types that are no longer needed.
2026-07-16 20:23:58 -05:00
86258718d8
refactor: update benchmark results and configuration for improved performance in go-migrate
2026-05-29 14:38:25 -05:00
13cd02a824
refactor: add reverse configuration and enhance transformation logic for PostgreSQL to SQL Server migration
2026-05-29 13:23:39 -05:00
f844004942
refactor: add comprehensive README documentation for go-migrate tool
2026-05-29 13:16:13 -05:00
4ba26092a9
refactor: implement bidirectional transformation support with PostgreSQL integration
2026-05-29 12:17:16 -05:00
537b7fbd28
refactor: remove obsolete skill registry documentation
2026-05-29 12:01:33 -05:00
d534314cff
feat: add Docker Compose configuration for database services and storage
2026-05-29 11:32:43 -05:00
b386965bb8
refactor: enhance database configuration handling with individual parameters and URL resolution methods
2026-05-15 09:59:42 -05:00
961fa48025
refactor: implement dry run feature to validate connections and count source rows without migrating
2026-05-15 08:38:12 -05:00
837fdc7abb
refactor: add validation feature to compare row counts between source and target databases
2026-05-12 11:07:55 -05:00
9c7662d5cb
refactor: add logging for successful database connections in main function
2026-05-12 10:57:05 -05:00
13bbd4e82c
refactor: update SOURCE_DB_URL in .env.example to include connection timeout settings
2026-05-12 08:33:38 -05:00
b3e1979bfb
refactor: implement expiry check to manage database connection interruptions
2026-05-12 08:28:41 -05:00
b98c998820
refactor: update MockTableAnalyzer methods to improve parameter handling and capture range constraints
2026-05-11 11:26:27 -05:00
fe35d2a34c
refactor: update RangeConfig to use pointers for min and max; adjust partition calculation logic to handle nil values
2026-05-11 11:21:59 -05:00
0784458106
refactor: add prefix support for storage column configuration and update blob path generation
2026-05-11 11:04:36 -05:00
6f2e3e28f1
refactor: enhance error handling in ProcessPartition by tracking last row values
2026-05-11 10:02:58 -05:00
604702ef43
refactor: add unit tests for loaderAccumulator and consume functions; enhance error handling and batch processing logic
2026-05-11 08:36:54 -05:00
34a3122e5b
refactor: add comprehensive tests for batch processing and error handling in Consume
2026-05-11 08:22:35 -05:00
68220e4c41
refactor: implement loaderAccumulator for batch processing; streamline error handling in Consume
2026-05-11 01:32:21 -05:00
ab9a2d8694
refactor: simplify batch processing by removing partition dependency and introducing batch accumulator
2026-05-11 00:38:42 -05:00
16217f6ee2
refactor: update debug log to display partition count instead of partitions
2026-05-09 12:16:48 -05:00
723041ff2c
refactor: add partition calculation strategy and implement estimation logic; enhance table analyzers for max/min column queries
2026-05-09 10:35:36 -05:00
8f8d2d11a4
refactor: enhance config handling in main; unify config parsing logic across scripts
2026-05-09 08:36:30 -05:00
0c59d06af6
refactor: adjust configuration parameters for extractors and transformers; optimize batch processing settings
2026-05-09 02:07:09 -05:00
6fa9b21b1c
refactor: update totalRows and chunkSize constants for improved performance
2026-05-09 01:49:39 -05:00
a8be31c18b
refactor: adjust configuration parameters for extractors and loaders; enhance logging messages for clarity
2026-05-09 01:48:36 -05:00
5a8bce7701
refactor: update totalRows constant and add siteHolderAttach data generation logic; enhance row generation and loading process
2026-05-09 01:33:12 -05:00
b690e580c5
refactor: enhance logging and batch processing in migration; adjust configuration parameters for improved performance
2026-05-09 01:16:34 -05:00
68d983ea57
refactor: remove unused Extractor and Loader interfaces from types.go; streamline code structure
2026-05-09 00:32:32 -05:00
1bc7b67643
refactor: replace Exec with Consume method in MssqlTransformer; enhance retry handling and streamline transformation logic
2026-05-09 00:30:49 -05:00
d124da8b20
refactor: enhance error messages for max retries in partition and batch processing
2026-05-08 23:36:59 -05:00
b5fd6d0534
refactor: remove unused LoaderError type and associated file; streamline error handling structure
2026-05-08 23:32:07 -05:00
85d7d69da9
refactor: streamline error handling in migration process; consolidate failed partitions and batches tracking
2026-05-08 23:22:53 -05:00
d54108d5e5
refactor: move max failed batches configuration to retry section; clean up unused error handling code
2026-05-08 23:00:23 -05:00
212d3663e2
refactor: remove unused error channels and enhance job configuration; add max failed batches load parameter
2026-05-08 22:28:31 -05:00
c4e233401b
refactor: update GenericLoader to use ProcessBatchWithRetries; enhance error handling and retry logic
2026-05-08 07:50:54 -05:00
a216a8016f
refactor: update extractor methods to support FromJsonColumns; enhance data processing capabilities
2026-05-07 08:17:25 -05:00
46ddd0d6b7
refactor: enhance extraction query handling; add support for JSON column extraction and wildcard patterns
2026-05-07 07:40:21 -05:00
80babf24f2
refactor: implement Consume method in GenericLoader; enhance error handling in ProcessBatch
2026-05-06 18:56:50 -05:00
f12937a1c3
refactor: standardize job error channel size; update batch size for transformed batches
2026-05-05 23:20:34 -05:00
2a5f703f3c
refactor: enhance retry handling in extractor processes; unify backoff delay computation
2026-05-05 23:16:13 -05:00
7cb959a103
refactor: update migration job configuration parameters; rename queue and batch size fields for clarity
2026-05-05 23:04:27 -05:00
6414943cf3
refactor: streamline error handling and processing in GenericExtractor; implement partition processing with retries
2026-04-27 01:11:45 -05:00
00459e42e6
refactor: update column field names in ExtractionQuery struct; enhance logging in consume and process methods
2026-04-27 00:25:48 -05:00
52fe083ab7
refactor: add consume and process methods for GenericExtractor; streamline data extraction logic
2026-04-26 19:39:14 -05:00
9a00d6af04
refactor: replace specific extractor implementations with a generic extractor; remove mssql and postgres extractor files
2026-04-26 19:33:59 -05:00
33af391986
refactor: correct import path for db_dialects package in mssql and postgres files; add db_dialects package
2026-04-26 19:25:16 -05:00
2b2d740d2e
feat: implement QueryFromObject method for mssql and postgres wrappers; enhance query building with limits and geometry support
2026-04-26 19:24:22 -05:00
fbe17b3842
refactor: unify db dialect definition in dbdialects package
2026-04-26 01:33:34 -05:00
