diff --git a/.gitea/copilot-instructions.md b/.gitea/copilot-instructions.md new file mode 100644 index 0000000..9877b96 --- /dev/null +++ b/.gitea/copilot-instructions.md @@ -0,0 +1,93 @@ +# ACH Server Media Import - Agent Instructions + +Guidelines and standards for the ACH Media Import project. + +## Project Overview +This project is a Python-based utility that imports media files from an S3-compatible bucket into a PostgreSQL database, enforcing specific naming conventions and metadata validation. + +## Technical Stack +- **Language**: Python 3.8+ +- **Database**: PostgreSQL (via `psycopg2`) +- **Cloud Storage**: AWS S3/S3-compatible storage (via `boto3`) +- **Containerization**: Docker & Docker Compose +- **Environment**: Managed via `.env` and `config.py` + +## Architecture & Modular Design +The project uses a utility-based modular architecture orchestrated by `main.py`. +- [main.py](main.py): Entry point and workflow orchestrator. +- [s3_utils.py](s3_utils.py): S3 client operations and bucket listing. +- [db_utils.py](db_utils.py): Database connectivity and SQL execution. +- [validation_utils.py](validation_utils.py): Pattern matching and business logic validation. +- [logging_config.py](logging_config.py): Centralized logging configuration. +- [error_handler.py](error_handler.py): Error handling and notifications. +- [email_utils.py](email_utils.py): SMTP integration for alerts. + +## Domain Logic: Inventory Codes +The core validation revolves around "Inventory Codes" which MUST follow a strict 12-character format: +- `^[VA][OC]-[A-Z0-9]{3}-\d{5}$` +- Examples: `VA-C01-12345`, `OC-A99-67890`. +- Files not matching this pattern in S3 are logged but skipped. + +## Development Workflows + +### Environment Setup +- **Windows**: Use `. .venv\Scripts\Activate.ps1` +- **Linux/macOS**: Use `source .venv/bin/activate` +- **Dependency installation**: `pip install -r requirements.txt` + +### Local Execution +- **Run script**: `python main.py` +- **Verify Configuration**: Ensure `.env` is populated with `DB_`, `AWS_`, and `SMTP_` variables. + +### Docker Operations +- **Build/Up**: `docker compose up -d --build` +- **Logs**: `docker compose logs -f app` +- **Stop**: `docker compose stop` + +## Coding Standards & Conventions + +### Logging +- Use the custom logger from `logging_config.py`. +- **Log Levels**: Use `logging.INFO`, `logging.WARNING`, and the custom `CUSTOM_ERROR_LEVEL` (35) via `error_handler.py`. +- Logs are rotated and stored in the `logs/` directory. + +### Error Handling +- Wrap critical operations that should trigger notifications in try-except blocks that call `error_handler.notify_error()`. +- Avoid silent failures; ensure errors are logged to the appropriate file sync. + +### Configuration +- Access settings exclusively via the `config.py` module's dictionaries: `db_config`, `aws_config`, `ach_config`. +- Never hardcode credentials or endpoints. + +## Copilot / Agent Behavior + +This repository is used with an AI assistant. When interacting with the assistant, follow these principles: + +- **Do not modify code unless explicitly requested.** The assistant should not change files unless given a clear instruction to do so. +- **Ask before acting.** If a change is needed, the assistant should describe the required modification and confirm before applying it. +- **Prefer explanation over edits.** When debugging or answering questions, provide guidance and analysis rather than directly editing source files. +- **Keep changes minimal.** If a code change is approved, apply the smallest possible edit that resolves the issue. + +## Code Style & Maintainability + +When generating or modifying code, prioritize **maintainability and clarity over optimization**. + +This is **development-stage code**, so it must remain easy to read, understand, and modify by humans. + +Guidelines: + +- Prefer **clear, explicit implementations** rather than clever or overly compact solutions. +- Avoid **micro-optimizations** or complex patterns that reduce readability. +- Do **not introduce obscure algorithms or creative tricks** that make the code difficult to understand. +- Write code that a developer unfamiliar with the project can quickly follow. +- Use **meaningful variable and function names**. +- Add **thoughtful comments** explaining non-obvious logic, assumptions, and decisions. +- Favor **simple and conventional approaches** instead of experimental or highly abstract ones. +- Maintain a **consistent structure and formatting**. + +The goal is **clean, maintainable, well-documented code**, not maximum performance or cleverness. + +## Related Files +- [query-sql.md](query-sql.md): Reference for database schema and SQL logic. +- [requirements.txt](requirements.txt): Project dependencies. +- [docker-compose.yml](docker-compose.yml): Deployment configuration. diff --git a/.gitea/workflows/docker-build.yml b/.gitea/workflows/docker-build.yml new file mode 100644 index 0000000..d09f4ad --- /dev/null +++ b/.gitea/workflows/docker-build.yml @@ -0,0 +1,27 @@ +name: Build Docker Image + +on: + push: + branches: [ main ] + +jobs: + build: + name: Build Docker image + runs-on: ubuntu-latest + + steps: + - name: Checkout repository + uses: actions/checkout@v4 + + - name: Set up Docker Buildx + uses: docker/setup-buildx-action@v3 + + - name: Build Docker image + uses: docker/build-push-action@v5 + with: + context: . + file: ./Dockerfile + push: false + load: true + tags: | + ach-server-import-media02:latest diff --git a/.gitignore b/.gitignore index 1d68424..24a1f82 100644 --- a/.gitignore +++ b/.gitignore @@ -9,6 +9,9 @@ logs/ *.log output/ +input/ +doc/ +docs/ TODO-mime.md s3_cache.json diff --git a/README.md b/README.md index 6bbb64c..88cb684 100644 --- a/README.md +++ b/README.md @@ -104,7 +104,7 @@ If you want to execute the importer manually inside the running container (for d docker compose exec app python /app/main.py # Or using docker exec with the container name -docker exec -it ACH_server_media_importer python /app/main.py +docker exec -it ACH_server_media_importer02 python /app/main.py ``` ### Stop