ACH-ARKIVO-ImportMedia/.github/copilot-instructions.md

ACH Server Media Import - Agent Instructions

Guidelines and standards for the ACH Media Import project.

Project Overview

This project is a Python-based utility that imports media files from an S3-compatible bucket into a PostgreSQL database, enforcing specific naming conventions and metadata validation.

Technical Stack

• Language: Python 3.8+
• Database: PostgreSQL (via psycopg2)
• Cloud Storage: AWS S3/S3-compatible storage (via boto3)
• Containerization: Docker & Docker Compose
• Environment: Managed via .env and config.py

Architecture & Modular Design

The project uses a utility-based modular architecture orchestrated by main.py.

• main.py: Entry point and workflow orchestrator.
• s3_utils.py: S3 client operations and bucket listing.
• db_utils.py: Database connectivity and SQL execution.
• validation_utils.py: Pattern matching and business logic validation.
• logging_config.py: Centralized logging configuration.
• error_handler.py: Error handling and notifications.
• email_utils.py: SMTP integration for alerts.

Domain Logic: Inventory Codes

The core validation revolves around "Inventory Codes" which MUST follow a strict 12-character format:

• ^[VA][OC]-[A-Z0-9]{3}-\d{5}$
• Examples: VA-C01-12345, OC-A99-67890.
• Files not matching this pattern in S3 are logged but skipped.

Development Workflows

Environment Setup

• Windows: Use . .venv\Scripts\Activate.ps1
• Linux/macOS: Use source .venv/bin/activate
• Dependency installation: pip install -r requirements.txt

Local Execution

• Run script: python main.py
• Verify Configuration: Ensure .env is populated with DB_, AWS_, and SMTP_ variables.

Docker Operations

• Build/Up: docker compose up -d --build
• Logs: docker compose logs -f app
• Stop: docker compose stop

Coding Standards & Conventions

Logging

• Use the custom logger from logging_config.py.
• Log Levels: Use logging.INFO, logging.WARNING, and the custom CUSTOM_ERROR_LEVEL (35) via error_handler.py.
• Logs are rotated and stored in the logs/ directory.

Error Handling

• Wrap critical operations that should trigger notifications in try-except blocks that call error_handler.notify_error().
• Avoid silent failures; ensure errors are logged to the appropriate file sync.

Configuration

• Access settings exclusively via the config.py module's dictionaries: db_config, aws_config, ach_config.
• Never hardcode credentials or endpoints.

Related Files

• query-sql.md: Reference for database schema and SQL logic.
• requirements.txt: Project dependencies.
• docker-compose.yml: Deployment configuration.