Sunbelt Computer Software

PL/B Language Development and Support

History for processing/process_document.py - Chunkwise/chunkwise · GitHub

Commits

History for

Commits on Dec 6, 2025

Batch documents up to 10 MB to reduce number of jobs. Refactored processing service to handle multiple documents. Use Manifest S3 file to store file names instead of environment variable due to siz…
tylermcgraw
committed

Commits on Nov 21, 2025

Separated config, services, and utils into isolated files
tylermcgraw
committed
Added automatic retry with backup (for rate limiting). Map chunks to embeddings to ensure they are the same length.
tylermcgraw
committed

Commits on Nov 19, 2025

Added document normalization. Updated database to use a single table.
tylermcgraw
committed
Added local testing and benchmarks. Added batch embedding for larger documents
tylermcgraw
committed
Fixed errors. Added local testing.
tylermcgraw
committed

Commits on Nov 18, 2025

Created service for processing documents. Needs to be tested locally
tylermcgraw
committed

Loading