Extral Documentation

Welcome to Extral, a versatile ETL (Extract, Transform, Load) application designed to move data from a source database to a destination database.

Contents:

• Getting Started
  • Installation
  • Basic Usage
  • Example Usage
  • Testing
  • Next Steps
• Configuration
  • Configuration File Structure
  • Global Configuration
  • Pipeline Configuration
  • Source and Destination Connectors
  • Table Configuration
  • Complete Example
  • Legacy Configuration Format
• API Reference
  • Configuration Module
  • Main Module
  • Connectors
  • Core Modules
  • Utility Modules

Features

Extral supports the following connectors:

• MySQL / MariaDB - Both source and destination

• PostgreSQL - Both source and destination

• File Sources - CSV and JSON file support

Key capabilities:

• Parallel Processing - Configure multiple worker threads for simultaneous table processing

• Incremental Loading - Track changes using cursor fields to load only new/updated data

• Multiple Load Strategies - Append, replace, or merge data based on your requirements

• Flexible Configuration - YAML-based configuration for easy setup and maintenance

Quick Start

Install Extral:

pip install extral

Create a configuration file (see Configuration for details):

source:
  type: mysql
  host: localhost
  user: root
  password: example_password
  database: source_db

destination:
  type: postgresql
  host: localhost
  user: loader
  password: example_password
  database: dest_db

tables:
  - name: customers
    strategy: merge
    merge_key: id

Run the ETL process:

extral --config config.yaml

Indices and tables

• Index

• Module Index

• Search Page