Skip to main content

Architecture

The extraction service (backend/app/services/extraction/real.py) runs entirely locally — no external API calls, no OpenAI dependency.

Pipeline Steps

1. File Retrieval

The service downloads the file from MinIO using the file_key returned during upload.

2. Text Extraction

PDF files: Uses pdfplumber.open() to extract text from each page. Falls back to raw UTF-8 decode for non-standard PDFs (scanned docs, malformed headers). DOCX files: Uses python-docx to iterate paragraphs.

3. Field Extraction

Each field uses a specialized extraction function:

4. Output

The raw_text is sanitized to remove control characters that would break JSON serialization.

Known Brands (pattern matching)

Nike, Adidas, Under Armour, Gatorade, Beats by Dre, Oakley, Red Bull, Fanatics, EA Sports, Topps, State Farm, Chick-fil-A, Raising Cane’s, Barstool Sports, BODYARMOR, New Balance, Puma, Coca-Cola, Pepsi, Jordan, Reebok, Monster Energy

Date Formats Supported

  • 2025-08-01 (ISO)
  • 08/01/2025 (US)
  • August 1, 2025 (long)
  • 1 August 2025 (international)

Extending the Extractor

To add a new known brand, add it to KNOWN_BRANDS in real.py. To add a new deal type, add a regex pattern to DEAL_TYPE_PATTERNS. To integrate an LLM (e.g., OpenAI) for higher accuracy, implement a new class extending BaseExtractionService and swap it in upload.py:get_extraction_service().