Commit Graph

Select branches

Hide Pull Requests

feat/mssql-to-mssql

main

refactor/extractor-retry-logic

refactor/retry-backoff

Mono Color

• 6ad25e5889 feat: update job configuration for infraestructura_site_holder__attach; enhance storage handling and retry logic main Kylesoda 2026-04-22 15:23:19 -05:00
• 0ac5f01b65 feat: update configuration handling to use cleanenv; remove unused dependencies and improve error logging Kylesoda 2026-04-22 12:31:31 -05:00
• 3b1371a270 feat: integrate Azure storage handling in migration process; update transformers and job processing logic Kylesoda 2026-04-21 14:15:20 -05:00
• bb7b35619a feat: add ToStorage configuration to JobConfig for enhanced data handling Kylesoda 2026-04-21 13:43:03 -05:00
• cd2efb8692 feat: add new job configuration for red_terminal__attach with source and target definitions Kylesoda 2026-04-21 13:38:11 -05:00
• 9964ef819b feat: refactor migration job to use preSQL and postSQL from TargetTable; update config structure for job definitions Kylesoda 2026-04-21 12:30:40 -05:00
• bd51223855 feat: add truncate query handling in migration process; update job configuration to remove commented truncate SQL Kylesoda 2026-04-21 11:40:02 -05:00
• aa71eeb5c1 feat: enhance range handling in MSSQL and Postgres extractors; update partition range generator logic Kylesoda 2026-04-21 11:32:52 -05:00
• 9eb8800864 feat: add range configuration to job and update extractors for inclusive range handling Kylesoda 2026-04-21 11:29:34 -05:00
• 09bd364976 feat: implement Azure Blob Storage client and refactor configuration structure Kylesoda 2026-04-21 11:03:56 -05:00
• f2e6edd8fa feat: add caarlos0/env package for environment variable management and refactor appConfig structure Kylesoda 2026-04-21 10:41:38 -05:00
• c5a18f6d95 chore: update .env.example to set LOG_LEVEL to INFO and add Azure storage configuration variables Kylesoda 2026-04-21 10:15:01 -05:00
• 8217b13d08 feat: add azblob client implementation for file uploads Kylesoda 2026-04-21 10:01:42 -05:00
• 16c232762e feat: refactor extractor interface and implement Consume function for ETL process Kylesoda 2026-04-20 00:19:41 -05:00
• b4a846575d Update .env.example with new variables Kylesoda 2026-04-19 23:08:09 -05:00
• 5bd730f026 feat: update .gitignore to include .vscode directory Kylesoda 2026-04-19 19:37:35 -05:00
• 7bd80d4180 feat: enhance logging configuration to use dynamic log level from environment variable Kylesoda 2026-04-19 19:32:27 -05:00
• 63cf26e1ab feat: update target database type to sqlserver and clean up unused code in mssql.go feat/mssql-to-mssql Kylesoda 2026-04-17 16:29:32 -05:00
• 846a49d40c feat: implement GenericLoader for batch processing and utility functions Kylesoda 2026-04-17 15:58:08 -05:00
• ec96532d04 refactor: optimize row handling in mssql extractor and transformer refactor/extractor-retry-logic Kylesoda 2026-04-17 01:10:45 -05:00
• 46597c4ffd refactor: implement extractor retry logic and streamline extractor interface Kylesoda 2026-04-17 00:33:49 -05:00
• 15d1b96849 refactor: streamline error handling and remove redundant code in mssql extractor Kylesoda 2026-04-17 00:23:01 -05:00
• 73b65e2a3f refactor: remove extractor error channel and simplify retry logic in mssql and postgres extractors Kylesoda 2026-04-17 00:07:51 -05:00
• 1c3db39b21 update extractor interface DiegoAlessandroMotta 2026-04-16 23:49:23 -05:00
• f65e67e02f feat: add backoff package for retry logic and implement example usage in new backoff-test script refactor/retry-backoff Kylesoda 2026-04-13 22:26:53 -05:00
• 93b302db8e feat: refactor job result handling and remove unused files Kylesoda 2026-04-16 16:47:35 -05:00
• 39c0d99502 feat: extend context timeout to 1 minute for database queries in mssql.go and postgres.go Kylesoda 2026-04-16 13:19:46 -05:00
• b418ded78b feat: update SQL query to filter out specific column names in mssql.go Kylesoda 2026-04-16 13:14:39 -05:00
• 0d0511716f feat: add row count mismatch error handling in processMigrationJob and update SQL query to exclude additional graph-related columns Kylesoda 2026-04-16 12:46:55 -05:00
• 67fb0148ae feat: add Makefile for building binaries across platforms Kylesoda 2026-04-16 11:51:05 -05:00
• 098cf36e3c feat: comment out TRUNCATE statements in pre_sql for MANZANA and PUERTO jobs Kylesoda 2026-04-16 11:46:24 -05:00
• 5484716b81 feat: add source and target database type fields to MigrationConfig Kylesoda 2026-04-16 09:08:21 -05:00
• df4c3bc390 feat: refactor db handling to use db-wrapper package; enhance connection management and result handling for MSSQL and Postgres Kylesoda 2026-04-16 08:48:29 -05:00
• ea41a7c218 feat: register MSSQL and Postgres drivers in db-wrapper for improved factory pattern support Kylesoda 2026-04-15 23:09:56 -05:00
• f09284ecdc feat: enhance db-wrapper with improved MSSQL and Postgres implementations; add row result handling and dialect support Kylesoda 2026-04-15 22:55:14 -05:00
• 0384d5423f feat: add range configuration to job settings for enhanced data processing control Kylesoda 2026-04-15 20:23:21 -05:00
• 1ce3d9e153 refactor: update partition handling to use Range struct for better clarity and consistency Kylesoda 2026-04-13 22:36:07 -05:00
• ed889b740a add db-wrapper package types DiegoAlessandroMotta 2026-04-15 20:22:23 -05:00
• 803f8988b8 refactor: update extractor interfaces to return row counts instead of using pointers for rows read Kylesoda 2026-04-13 19:25:18 -05:00
• 33c9cd9c3e feat: implement database wrapper interfaces for MSSQL and Postgres; enhance migration job processing with pre and post SQL execution Kylesoda 2026-04-13 07:57:18 -05:00
• 85074da2ec feat: add max partition and chunk error limits to extractor and loader error handlers Kylesoda 2026-04-12 20:57:31 -05:00
• f126d5bbd0 feat: implement exponential backoff strategy for error handling in extractor and loader processes; enhance retry configuration options Kylesoda 2026-04-12 20:35:29 -05:00
• 5633dc98d0 fix: enhance error handling in extractor and loader processes; ensure proper job error propagation and logging Kylesoda 2026-04-12 20:11:19 -05:00
• 01780b4b02 refactor: remove unused ColumnType and inspect-columns files; update migration job to use separate table analyzers for source and target databases Kylesoda 2026-04-12 19:16:14 -05:00
• aded502ee4 feat: implement Postgres table analyzer with column type querying and metadata retrieval Kylesoda 2026-04-12 13:38:36 -05:00
• 4d3cd6e4cf feat: add MSSQL table analyzer and integrate partition range generation for improved data migration Kylesoda 2026-04-11 01:23:13 -05:00
• 7830ae862d refactor: rename batch-related variables and functions for consistency and clarity Kylesoda 2026-04-11 00:44:12 -05:00
• 955bc65ce9 refactor: rename Batch to Partition in error handling and processing functions for consistency Kylesoda 2026-04-11 00:30:54 -05:00
• 9eb9821daf refactor: rename Batch to Partition and update related types and channels for consistency Kylesoda 2026-04-11 00:09:28 -05:00
• cd0e53b1d2 feat: implement MSSQL extractor, transformer, and Postgres loader for enhanced data migration Kylesoda 2026-04-10 23:39:37 -05:00
• 1be7018ba3 feat: refactor configuration to include source and target database types Kylesoda 2026-04-10 22:58:57 -05:00
• a5b5a04feb feat: update extractor and transformer constructors to return Extractor interface Kylesoda 2026-04-10 20:42:16 -05:00
• c1bae79f98 feat: implement Postgres loader and refactor migration job processing Kylesoda 2026-04-10 20:40:01 -05:00
• 053e6bd673 feat: add MSSQL extractor and transformer implementations for improved data migration Kylesoda 2026-04-10 19:59:44 -05:00
• eb3c3bbfce feat: refactor error handling to use custom_errors.LoaderError for improved error management Kylesoda 2026-04-10 19:35:38 -05:00
• 9493a2d32f feat: refactor error handling to accept max retry attempts as a parameter for improved flexibility Kylesoda 2026-04-10 19:32:12 -05:00
• d228a048b8 feat: update extractor error handling to use models.UnknownRowValues for improved type consistency Kylesoda 2026-04-10 19:29:07 -05:00
• ca621352c9 feat: refactor models to improve type handling and enhance error management across migration processes Kylesoda 2026-04-10 19:27:27 -05:00
• c2ea84bfcf feat: implement extractor error handling and batch processing for MSSQL and Postgres Kylesoda 2026-04-10 19:06:41 -05:00
• 6345a0d694 feat: enhance migration job processing with detailed metrics and error handling Kylesoda 2026-04-09 21:55:19 -05:00
• 1db35c796c feat: enhance migration job processing with parallel execution and improved logging Kylesoda 2026-04-09 20:02:04 -05:00
• 0d9f955b2f feat: enhance batch processing by adding rowsPerBatch parameter and improving logging messages Kylesoda 2026-04-09 19:46:45 -05:00
• 524d892a60 feat: refactor migration job structure to use SourceTableInfo and TargetTableInfo for improved configuration handling Kylesoda 2026-04-09 19:20:50 -05:00
• e8ace6ecf9 feat: implement job configuration structure and YAML parsing for migration jobs Kylesoda 2026-04-09 18:12:11 -05:00
• adbc962464 feat: add configuration parsing and job management in YAML format Kylesoda 2026-04-09 17:47:58 -05:00
• a0b51f40c1 feat: add context support to migration job processing for improved cancellation and error handling Kylesoda 2026-04-09 00:43:11 -05:00
• b64a76ca45 feat: improve error handling and job cancellation in migration process Kylesoda 2026-04-09 00:38:16 -05:00
• 51480015ba feat: enhance concurrency management by adding WaitGroup support in extractors and loaders Kylesoda 2026-04-09 00:22:30 -05:00
• dc632361e5 feat: implement loader error handling and refactor chunk processing in migration job Kylesoda 2026-04-08 23:42:31 -05:00
• 0ee5d9032c feat: add context support to error handlers for improved cancellation and error management Kylesoda 2026-04-08 23:07:41 -05:00
• d3a3b26bb3 feat: enhance error handling and context management in MSSQL extraction process Kylesoda 2026-04-08 22:39:07 -05:00
• 554618daad feat: refactor transformation logic in MSSQL processing to use context and improve error handling Kylesoda 2026-04-08 21:47:24 -05:00
• 7924dd3af7 feat: refactor chunk handling in loader and transformer for improved data processing Kylesoda 2026-04-08 21:11:26 -05:00
• f6dfcd390f feat: refactor chunk handling in extractor and transformer for improved data processing Kylesoda 2026-04-08 21:09:26 -05:00
• 853be4a5a6 feat: update chunk size for MSSQL processing and enhance error handling in transformation functions Kylesoda 2026-04-08 20:48:36 -05:00
• eeef3bc813 fix: correct variable name for job error in MSSQL extraction Kylesoda 2026-04-08 20:39:16 -05:00
• e158986947 feat: enhance error handling with JobError struct and update extractor logic Kylesoda 2026-04-08 20:21:58 -05:00
• bc6f9a6a70 feat: implement batch processing for MSSQL with improved structure and logging Kylesoda 2026-04-08 17:35:12 -05:00
• 51d83661a4 fix: update globalID generation for consistency in UUID handling Kylesoda 2026-04-08 14:12:33 -05:00
• 75b04d4b2e feat: add MANZANA migration job and update related processing logic Kylesoda 2026-04-08 10:16:27 -05:00
• 3765e8adb3 feat: implement PUERTO data generation and loading with improved structure and logging Kylesoda 2026-04-08 10:08:17 -05:00
• 4434054b21 feat: implement data generation and loading for MANZANA with improved structure and logging Kylesoda 2026-04-08 09:57:11 -05:00
• 1e2a37e59f fix: adjust migration parameters for improved performance and consistency Kylesoda 2026-04-07 23:52:49 -05:00
• 8afdb45318 feat: implement batch processing for MSSQL extraction and transformation with range handling Kylesoda 2026-04-07 23:39:55 -05:00
• 8903a04f4d fix: standardize variable naming and adjust loader sleep duration for consistency Kylesoda 2026-04-07 20:22:00 -05:00
• c318428563 feat: update chunk size for MSSQL data loading and add utility functions for database operations Kylesoda 2026-04-07 17:57:08 -05:00
• 8738ef92b7 feat: implement MSSQL bulk loading functionality with transaction handling Kylesoda 2026-04-07 16:21:10 -05:00
• 270a66dbbf feat: enhance logging and performance metrics for migration processes Kylesoda 2026-04-07 15:53:17 -05:00
• 7bb67ddfcf feat: refactor data extraction and loading functions for improved context handling Kylesoda 2026-04-07 12:55:05 -05:00
• 6da321c4bb feat: implement MSSQL row transformation and loading functions Kylesoda 2026-04-07 11:47:27 -05:00
• 2f8263d332 feat: add ensureUTC function and apply it to datetime transformations in MSSQL processing Kylesoda 2026-04-07 11:37:04 -05:00
• aea310a3dd feat: add WKB to EWKB conversion with SRID handling and integrate into MSSQL transformation Kylesoda 2026-04-07 10:05:45 -05:00
• f305ddec0b feat: add MSSQL UUID transformation and update migration process for channel handling Kylesoda 2026-04-06 21:53:09 -05:00
• 8de4838e00 feat: streamline data extraction from MSSQL and Postgres, add UUID transformation Kylesoda 2026-04-06 21:37:17 -05:00
• d4786ed920 feat: implement PostgreSQL data extraction and transformation process Kylesoda 2026-04-06 21:04:17 -05:00
• f589664320 feat: implement MSSQL data extraction and transformation process Kylesoda 2026-04-06 20:36:11 -05:00
• 382c2099f7 feat: add ORDER BY clause to SQL extract queries for consistent result ordering Kylesoda 2026-04-06 19:13:14 -05:00
• dfc76eecd0 feat: implement ETL process with extractors and loaders Kylesoda 2026-04-06 13:50:36 -05:00
• c0e9bcf88c feat: add extract query builders Kylesoda 2026-04-06 11:39:27 -05:00
• c49ff65b1d feat: remove unused column type logging and streamline migration job processing Kylesoda 2026-04-06 10:48:56 -05:00