Compare commits
4 Commits
9d7109b60f
...
c17bb57a3c
| Author | SHA1 | Date | |
|---|---|---|---|
| c17bb57a3c | |||
| 0bcd7821f2 | |||
| 71d8f0fab5 | |||
| c76d533783 |
13
.env.prod
Normal file
@@ -0,0 +1,13 @@
|
|||||||
|
# Database Configuration
|
||||||
|
# Connecting to existing 'postgres-db' container in 'arbit-app_arbit-network'
|
||||||
|
DB_USER=postgres
|
||||||
|
DB_PASSWORD='M@tr!x#149@dm!N'
|
||||||
|
DB_NAME=ocr
|
||||||
|
DB_HOST=postgres-db
|
||||||
|
DB_PORT=5432
|
||||||
|
|
||||||
|
# Mail Configuration (IMAP)
|
||||||
|
MAIL_SERVER=imap.gmail.com
|
||||||
|
MAIL_USERNAME=matrixinfotech.it@gmail.com
|
||||||
|
MAIL_PASSWORD=qtxsthbxbisgcmqu
|
||||||
|
MAIL_PORT=993
|
||||||
95
DEPLOYMENT.md
Normal file
@@ -0,0 +1,95 @@
|
|||||||
|
# OCR Application Deployment Guide
|
||||||
|
|
||||||
|
This guide describes how to deploy the OCR application on a Linux host (e.g., Ubuntu/Debian).
|
||||||
|
|
||||||
|
## Architecture
|
||||||
|
- **Backend**: Containerized (FastAPI, Python 3.9).
|
||||||
|
- **Database**: Containerized (PostgreSQL 15).
|
||||||
|
- **Frontend**: Static files (Angular) served by Host Nginx.
|
||||||
|
- **Reverse Proxy**: Host Nginx proxies requests to Frontend (Static) and Backend (API).
|
||||||
|
|
||||||
|
## Prerequisites
|
||||||
|
- **Deployment Host**: Docker & Docker Compose, Nginx.
|
||||||
|
- **Build Machine**: Node.js & NPM (to run the package script).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 1. Full Stack Deployment (Docker)
|
||||||
|
|
||||||
|
The app connects to your **existing Postgres container** (`postgres-db`) in the `arbit-app_arbit-network`.
|
||||||
|
|
||||||
|
1. **Create the Database**:
|
||||||
|
Since we are using an existing postgres instance, we must manually create the `ocr_db`.
|
||||||
|
```bash
|
||||||
|
# Run this on your host to create the DB inside the existing container
|
||||||
|
docker exec -it postgres-db psql -U postgres -c "CREATE DATABASE ocr_db;"
|
||||||
|
```
|
||||||
|
|
||||||
|
2. **Navigate to the packaged directory**:
|
||||||
|
```bash
|
||||||
|
cd ocr_build
|
||||||
|
```
|
||||||
|
|
||||||
|
3. **Verify Environment**:
|
||||||
|
Ensure `.env.prod` exists and points to `DB_HOST=postgres-db`.
|
||||||
|
**Also configure your Email credentials** in `.env.prod` if you want the Mailbox feature to work (Gmail requires an App Password).
|
||||||
|
```bash
|
||||||
|
cat .env.prod
|
||||||
|
```
|
||||||
|
|
||||||
|
4. **Start App Containers**:
|
||||||
|
This will start `ocr_backend` and `ocr_frontend`.
|
||||||
|
```bash
|
||||||
|
# Build and start in detached mode
|
||||||
|
docker-compose up -d --build
|
||||||
|
```
|
||||||
|
|
||||||
|
5. **Verify Status**:
|
||||||
|
```bash
|
||||||
|
docker-compose ps
|
||||||
|
```
|
||||||
|
You should see `ocr_backend` and `ocr_frontend` running.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2. Nginx Configuration (Host Reverse Proxy)
|
||||||
|
|
||||||
|
Since the Frontend is now running in a container on port 8080 (serving `/ocrf/`), we configure the Host Nginx to proxy traffic to it.
|
||||||
|
|
||||||
|
1. **Create Config File**:
|
||||||
|
Copy the provided config to `/etc/nginx/sites-available/ocr`.
|
||||||
|
```bash
|
||||||
|
sudo cp nginx_host.conf /etc/nginx/sites-available/ocr
|
||||||
|
```
|
||||||
|
|
||||||
|
2. **Enable Site**:
|
||||||
|
```bash
|
||||||
|
sudo ln -s /etc/nginx/sites-available/ocr /etc/nginx/sites-enabled/
|
||||||
|
```
|
||||||
|
|
||||||
|
3. **Test & Reload**:
|
||||||
|
```bash
|
||||||
|
sudo nginx -t
|
||||||
|
sudo systemctl reload nginx
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 3. Verification
|
||||||
|
|
||||||
|
- **Url**: `http://app.technobeesolutions.in/ocrf/` (Proxies to Frontend Container)
|
||||||
|
- **API**: `http://app.technobeesolutions.in/ocrb/` (Proxies to Backend Container)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 4. Troubleshooting
|
||||||
|
|
||||||
|
- **Logs**:
|
||||||
|
```bash
|
||||||
|
docker-compose logs -f backend
|
||||||
|
```
|
||||||
|
- **Database**:
|
||||||
|
Connect via the existing container:
|
||||||
|
```bash
|
||||||
|
docker exec -it postgres-db psql -U postgres -d ocr_db
|
||||||
|
```
|
||||||
30
backend/Dockerfile
Normal file
@@ -0,0 +1,30 @@
|
|||||||
|
# Use official lightweight Python image
|
||||||
|
FROM python:3.9-slim
|
||||||
|
|
||||||
|
# Install system dependencies
|
||||||
|
# tesseract-ocr: for pytesseract
|
||||||
|
# poppler-utils: for pdf2image
|
||||||
|
# libtesseract-dev: development headers
|
||||||
|
RUN apt-get update && apt-get install -y \
|
||||||
|
tesseract-ocr \
|
||||||
|
poppler-utils \
|
||||||
|
libtesseract-dev \
|
||||||
|
&& rm -rf /var/lib/apt/lists/*
|
||||||
|
|
||||||
|
# Set working directory
|
||||||
|
WORKDIR /app
|
||||||
|
|
||||||
|
# Copy requirements first to leverage Docker cache
|
||||||
|
COPY requirements.txt .
|
||||||
|
|
||||||
|
# Install Python dependencies
|
||||||
|
RUN pip install --no-cache-dir -r requirements.txt
|
||||||
|
|
||||||
|
# Copy the rest of the application code
|
||||||
|
COPY . .
|
||||||
|
|
||||||
|
# Expose port (default for Uvicorn)
|
||||||
|
EXPOSE 8000
|
||||||
|
|
||||||
|
# Run the application
|
||||||
|
CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000"]
|
||||||
@@ -1,5 +1,7 @@
|
|||||||
import os
|
import os
|
||||||
from sqlalchemy import create_engine, Column, Integer, String, Text, DateTime, Boolean, ForeignKey, LargeBinary
|
import urllib.parse
|
||||||
|
from sqlalchemy import create_engine, Column, Integer, String, Text, DateTime, Boolean, ForeignKey, LargeBinary, Enum as SqlEnum
|
||||||
|
from sqlalchemy.dialects.postgresql import JSONB
|
||||||
from sqlalchemy.ext.declarative import declarative_base
|
from sqlalchemy.ext.declarative import declarative_base
|
||||||
from sqlalchemy.orm import sessionmaker, relationship
|
from sqlalchemy.orm import sessionmaker, relationship
|
||||||
from dotenv import load_dotenv
|
from dotenv import load_dotenv
|
||||||
@@ -7,22 +9,11 @@ from dotenv import load_dotenv
|
|||||||
# Load environment variables
|
# Load environment variables
|
||||||
load_dotenv()
|
load_dotenv()
|
||||||
|
|
||||||
DB_USER = os.getenv("DB_USER")
|
DB_USER = os.getenv("DB_USER", "postgres")
|
||||||
DB_PASSWORD = os.getenv("DB_PASSWORD")
|
DB_PASSWORD = os.getenv("DB_PASSWORD", "password")
|
||||||
DB_HOST = os.getenv("DB_HOST")
|
DB_HOST = os.getenv("DB_HOST", "localhost")
|
||||||
DB_PORT = os.getenv("DB_PORT")
|
DB_PORT = os.getenv("DB_PORT", "5432")
|
||||||
import urllib.parse
|
DB_NAME = os.getenv("DB_NAME", "ocr_db")
|
||||||
|
|
||||||
# ... (imports)
|
|
||||||
|
|
||||||
# Load environment variables
|
|
||||||
load_dotenv()
|
|
||||||
|
|
||||||
DB_USER = os.getenv("DB_USER")
|
|
||||||
DB_PASSWORD = os.getenv("DB_PASSWORD")
|
|
||||||
DB_HOST = os.getenv("DB_HOST")
|
|
||||||
DB_PORT = os.getenv("DB_PORT")
|
|
||||||
DB_NAME = os.getenv("DB_NAME")
|
|
||||||
|
|
||||||
encoded_user = urllib.parse.quote_plus(DB_USER)
|
encoded_user = urllib.parse.quote_plus(DB_USER)
|
||||||
encoded_password = urllib.parse.quote_plus(DB_PASSWORD)
|
encoded_password = urllib.parse.quote_plus(DB_PASSWORD)
|
||||||
@@ -35,6 +26,26 @@ SessionLocal = sessionmaker(autocommit=False, autoflush=False, bind=engine)
|
|||||||
|
|
||||||
Base = declarative_base()
|
Base = declarative_base()
|
||||||
|
|
||||||
|
class Vendor(Base):
|
||||||
|
__tablename__ = "vendors"
|
||||||
|
|
||||||
|
id = Column(Integer, primary_key=True, index=True)
|
||||||
|
name = Column(String, unique=True, index=True)
|
||||||
|
default_model = Column(String) # 'text' or 'vision'
|
||||||
|
created_at = Column(DateTime)
|
||||||
|
|
||||||
|
class Document(Base):
|
||||||
|
__tablename__ = "documents"
|
||||||
|
|
||||||
|
id = Column(Integer, primary_key=True, index=True)
|
||||||
|
vendor_id = Column(Integer, ForeignKey("vendors.id"), nullable=True)
|
||||||
|
filename = Column(String)
|
||||||
|
upload_date = Column(DateTime)
|
||||||
|
status = Column(String) # 'pending', 'verified'
|
||||||
|
processed_data = Column(JSONB) # Store the verified extraction results
|
||||||
|
|
||||||
|
vendor = relationship("Vendor")
|
||||||
|
|
||||||
class Email(Base):
|
class Email(Base):
|
||||||
__tablename__ = "emails"
|
__tablename__ = "emails"
|
||||||
|
|
||||||
@@ -54,7 +65,7 @@ class Attachment(Base):
|
|||||||
email_id = Column(Integer, ForeignKey("emails.id"))
|
email_id = Column(Integer, ForeignKey("emails.id"))
|
||||||
filename = Column(String)
|
filename = Column(String)
|
||||||
content_type = Column(String)
|
content_type = Column(String)
|
||||||
file_content = Column(LargeBinary) # Storing content directly in DB as requested
|
file_content = Column(LargeBinary)
|
||||||
|
|
||||||
email = relationship("Email", back_populates="attachments")
|
email = relationship("Email", back_populates="attachments")
|
||||||
|
|
||||||
|
|||||||
@@ -85,6 +85,12 @@ class LoginResponse(BaseModel):
|
|||||||
token: str
|
token: str
|
||||||
message: str
|
message: str
|
||||||
|
|
||||||
|
@app.post("/api/login", response_model=LoginResponse)
|
||||||
|
def login(request: LoginRequest):
|
||||||
|
if request.username == "admin" and request.password == "admin":
|
||||||
|
return LoginResponse(token="fake-super-secret-token", message="Success")
|
||||||
|
raise HTTPException(status_code=401, detail="Invalid username or password")
|
||||||
|
|
||||||
class NERResponse(BaseModel):
|
class NERResponse(BaseModel):
|
||||||
text: str
|
text: str
|
||||||
|
|
||||||
|
|||||||
@@ -10,3 +10,4 @@ imap-tools
|
|||||||
apscheduler
|
apscheduler
|
||||||
python-dotenv
|
python-dotenv
|
||||||
pdfplumber
|
pdfplumber
|
||||||
|
pdf2image
|
||||||
|
|||||||
22
deployment/nginx.conf
Normal file
@@ -0,0 +1,22 @@
|
|||||||
|
server {
|
||||||
|
listen 80;
|
||||||
|
server_name app.technobeesolutions.in;
|
||||||
|
|
||||||
|
# Proxy Angular Frontend Container
|
||||||
|
location /ocrf/ {
|
||||||
|
proxy_pass http://localhost:8080/;
|
||||||
|
proxy_set_header Host $host;
|
||||||
|
proxy_set_header X-Real-IP $remote_addr;
|
||||||
|
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||||
|
proxy_set_header X-Forwarded-Proto $scheme;
|
||||||
|
}
|
||||||
|
|
||||||
|
# Proxy API requests to the Docker Backend
|
||||||
|
location /ocrb/ {
|
||||||
|
proxy_pass http://localhost:8000/;
|
||||||
|
proxy_set_header Host $host;
|
||||||
|
proxy_set_header X-Real-IP $remote_addr;
|
||||||
|
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||||
|
proxy_set_header X-Forwarded-Proto $scheme;
|
||||||
|
}
|
||||||
|
}
|
||||||
BIN
docengine/.DS_Store
vendored
Normal file
59
docengine/.env
Normal file
@@ -0,0 +1,59 @@
|
|||||||
|
# Application
|
||||||
|
APP_NAME=DocEngine
|
||||||
|
APP_VERSION=1.0.0
|
||||||
|
APP_ENV=development
|
||||||
|
APP_DEBUG=true
|
||||||
|
APP_HOST=0.0.0.0
|
||||||
|
APP_PORT=7989
|
||||||
|
APP_WORKERS=4
|
||||||
|
|
||||||
|
# Database
|
||||||
|
DB_HOST=103.125.129.116
|
||||||
|
DB_PORT=5432
|
||||||
|
DB_NAME=ocr
|
||||||
|
DB_USER=postgres
|
||||||
|
DB_PASSWORD=M@triXPostgr3s@6202
|
||||||
|
DB_SCHEMA=admin
|
||||||
|
DB_POOL_SIZE=20
|
||||||
|
DB_MAX_OVERFLOW=10
|
||||||
|
DB_ECHO=false
|
||||||
|
|
||||||
|
# Redis
|
||||||
|
REDIS_HOST=103.125.129.116
|
||||||
|
REDIS_PORT=7901
|
||||||
|
REDIS_DB=0
|
||||||
|
REDIS_PASSWORD=M@triXR3d1s@6202
|
||||||
|
|
||||||
|
# Celery
|
||||||
|
CELERY_BROKER_URL=redis://:M%40triXR3d1s%406202@103.125.129.116:7901/0
|
||||||
|
CELERY_RESULT_BACKEND=redis://:M%40triXR3d1s%406202@103.125.129.116:7901/1
|
||||||
|
|
||||||
|
# JWT
|
||||||
|
JWT_SECRET_KEY=a7f3c9e1d4b8f2a6c0e5d7b3a9f1c4e8d2b6a0f5c3e7d1b9a4f8c2e6d0b5a3
|
||||||
|
JWT_ALGORITHM=HS256
|
||||||
|
JWT_ACCESS_TOKEN_EXPIRE_MINUTES=30
|
||||||
|
JWT_REFRESH_TOKEN_EXPIRE_DAYS=7
|
||||||
|
|
||||||
|
# Storage
|
||||||
|
STORAGE_PROVIDER=local
|
||||||
|
STORAGE_LOCAL_PATH=./storage
|
||||||
|
STORAGE_MAX_FILE_SIZE_MB=100
|
||||||
|
|
||||||
|
# OCR
|
||||||
|
OCR_LANGUAGE=en
|
||||||
|
OCR_USE_GPU=false
|
||||||
|
|
||||||
|
# Logging
|
||||||
|
LOG_LEVEL=INFO
|
||||||
|
LOG_FORMAT=json
|
||||||
|
|
||||||
|
# CORS
|
||||||
|
CORS_ORIGINS=["http://localhost:3000","http://localhost:8080","http://localhost:4200"]
|
||||||
|
CORS_ALLOW_CREDENTIALS=true
|
||||||
|
|
||||||
|
# Rate Limiting
|
||||||
|
RATE_LIMIT_REQUESTS=100
|
||||||
|
RATE_LIMIT_WINDOW_SECONDS=60
|
||||||
|
|
||||||
|
# Prometheus
|
||||||
|
PROMETHEUS_ENABLED=true
|
||||||
BIN
docengine/alembic/__pycache__/env.cpython-313.pyc
Normal file
BIN
docengine/app/__pycache__/__init__.cpython-313.pyc
Normal file
BIN
docengine/app/__pycache__/main.cpython-313.pyc
Normal file
BIN
docengine/app/api/__pycache__/__init__.cpython-313.pyc
Normal file
BIN
docengine/app/api/__pycache__/router.cpython-313.pyc
Normal file
BIN
docengine/app/api/v1/__pycache__/__init__.cpython-313.pyc
Normal file
BIN
docengine/app/api/v1/__pycache__/auth.cpython-313.pyc
Normal file
BIN
docengine/app/api/v1/__pycache__/documents.cpython-313.pyc
Normal file
BIN
docengine/app/api/v1/__pycache__/health.cpython-313.pyc
Normal file
BIN
docengine/app/api/v1/__pycache__/templates.cpython-313.pyc
Normal file
BIN
docengine/app/core/__pycache__/__init__.cpython-313.pyc
Normal file
BIN
docengine/app/core/__pycache__/aes.cpython-313.pyc
Normal file
BIN
docengine/app/core/__pycache__/config.cpython-313.pyc
Normal file
BIN
docengine/app/core/__pycache__/database.cpython-313.pyc
Normal file
BIN
docengine/app/core/__pycache__/dependencies.cpython-313.pyc
Normal file
BIN
docengine/app/core/__pycache__/exceptions.cpython-313.pyc
Normal file
BIN
docengine/app/core/__pycache__/logging_config.cpython-313.pyc
Normal file
BIN
docengine/app/core/__pycache__/security.cpython-313.pyc
Normal file
BIN
docengine/app/events/__pycache__/__init__.cpython-313.pyc
Normal file
BIN
docengine/app/events/__pycache__/handlers.cpython-313.pyc
Normal file
BIN
docengine/app/middleware/__pycache__/__init__.cpython-313.pyc
Normal file
BIN
docengine/app/middleware/__pycache__/audit.cpython-313.pyc
Normal file
BIN
docengine/app/middleware/__pycache__/cors.cpython-313.pyc
Normal file
BIN
docengine/app/middleware/__pycache__/metrics.cpython-313.pyc
Normal file
BIN
docengine/app/middleware/__pycache__/rate_limit.cpython-313.pyc
Normal file
BIN
docengine/app/models/__pycache__/__init__.cpython-313.pyc
Normal file
BIN
docengine/app/models/__pycache__/base.cpython-313.pyc
Normal file
BIN
docengine/app/models/__pycache__/document.cpython-313.pyc
Normal file
BIN
docengine/app/models/__pycache__/template.cpython-313.pyc
Normal file
BIN
docengine/app/models/__pycache__/user.cpython-313.pyc
Normal file
BIN
docengine/app/repositories/__pycache__/__init__.cpython-313.pyc
Normal file
BIN
docengine/app/repositories/__pycache__/base.cpython-313.pyc
Normal file
BIN
docengine/app/schemas/__pycache__/__init__.cpython-313.pyc
Normal file
BIN
docengine/app/schemas/__pycache__/auth.cpython-313.pyc
Normal file
BIN
docengine/app/schemas/__pycache__/common.cpython-313.pyc
Normal file
BIN
docengine/app/schemas/__pycache__/document.cpython-313.pyc
Normal file
BIN
docengine/app/schemas/__pycache__/template.cpython-313.pyc
Normal file
BIN
docengine/app/schemas/__pycache__/user.cpython-313.pyc
Normal file
BIN
docengine/app/services/__pycache__/__init__.cpython-313.pyc
Normal file
BIN
docengine/app/services/__pycache__/ocr_service.cpython-313.pyc
Normal file
BIN
docengine/app/services/__pycache__/pdf_service.cpython-313.pyc
Normal file
0
docengine/app/storage/__init__.py
Normal file
BIN
docengine/app/storage/__pycache__/__init__.cpython-313.pyc
Normal file
BIN
docengine/app/storage/__pycache__/provider.cpython-313.pyc
Normal file
163
docengine/app/storage/provider.py
Normal file
@@ -0,0 +1,163 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import hashlib
|
||||||
|
import os
|
||||||
|
import shutil
|
||||||
|
import uuid
|
||||||
|
from abc import ABC, abstractmethod
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
from app.core.config import settings
|
||||||
|
from app.core.exceptions import StorageError
|
||||||
|
from app.core.logging_config import get_logger
|
||||||
|
|
||||||
|
logger = get_logger(__name__)
|
||||||
|
|
||||||
|
|
||||||
|
class StorageProvider(ABC):
|
||||||
|
"""Abstract base class for storage providers."""
|
||||||
|
|
||||||
|
@abstractmethod
|
||||||
|
def save_file(self, file_data: bytes, directory: str, filename: str | None = None) -> str:
|
||||||
|
"""Save file data and return the storage path."""
|
||||||
|
...
|
||||||
|
|
||||||
|
@abstractmethod
|
||||||
|
def read_file(self, storage_path: str) -> bytes:
|
||||||
|
"""Read file data from storage."""
|
||||||
|
...
|
||||||
|
|
||||||
|
@abstractmethod
|
||||||
|
def delete_file(self, storage_path: str) -> bool:
|
||||||
|
"""Delete a file from storage. Returns True if successful."""
|
||||||
|
...
|
||||||
|
|
||||||
|
@abstractmethod
|
||||||
|
def file_exists(self, storage_path: str) -> bool:
|
||||||
|
"""Check if a file exists in storage."""
|
||||||
|
...
|
||||||
|
|
||||||
|
@abstractmethod
|
||||||
|
def get_file_size(self, storage_path: str) -> int:
|
||||||
|
"""Get the file size in bytes."""
|
||||||
|
...
|
||||||
|
|
||||||
|
@abstractmethod
|
||||||
|
def get_absolute_path(self, storage_path: str) -> str:
|
||||||
|
"""Get the absolute filesystem path for a storage path."""
|
||||||
|
...
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def compute_checksum(data: bytes, algorithm: str = "sha256") -> str:
|
||||||
|
"""Compute checksum of file data."""
|
||||||
|
hasher = hashlib.new(algorithm)
|
||||||
|
hasher.update(data)
|
||||||
|
return hasher.hexdigest()
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def generate_filename(original_filename: str) -> str:
|
||||||
|
"""Generate a unique filename preserving the original extension."""
|
||||||
|
ext = Path(original_filename).suffix.lower()
|
||||||
|
return f"{uuid.uuid4().hex}{ext}"
|
||||||
|
|
||||||
|
|
||||||
|
class LocalStorageProvider(StorageProvider):
|
||||||
|
"""Local filesystem storage provider."""
|
||||||
|
|
||||||
|
def __init__(self, base_path: str | None = None) -> None:
|
||||||
|
self.base_path = Path(base_path or settings.storage_local_path).resolve()
|
||||||
|
self._ensure_directories()
|
||||||
|
|
||||||
|
def _ensure_directories(self) -> None:
|
||||||
|
"""Create required storage directories."""
|
||||||
|
for subdir in ("documents", "templates", "images", "temp", "rendered"):
|
||||||
|
(self.base_path / subdir).mkdir(parents=True, exist_ok=True)
|
||||||
|
|
||||||
|
def _resolve_path(self, storage_path: str) -> Path:
|
||||||
|
"""Resolve a storage path to an absolute path."""
|
||||||
|
resolved = (self.base_path / storage_path).resolve()
|
||||||
|
if not str(resolved).startswith(str(self.base_path)):
|
||||||
|
raise StorageError(f"Path traversal detected: {storage_path}")
|
||||||
|
return resolved
|
||||||
|
|
||||||
|
def save_file(self, file_data: bytes, directory: str, filename: str | None = None) -> str:
|
||||||
|
"""Save file data to local storage."""
|
||||||
|
if filename is None:
|
||||||
|
filename = f"{uuid.uuid4().hex}.bin"
|
||||||
|
|
||||||
|
dir_path = self.base_path / directory
|
||||||
|
dir_path.mkdir(parents=True, exist_ok=True)
|
||||||
|
|
||||||
|
file_path = dir_path / filename
|
||||||
|
try:
|
||||||
|
file_path.write_bytes(file_data)
|
||||||
|
storage_path = str(file_path.relative_to(self.base_path))
|
||||||
|
logger.info("file_saved", storage_path=storage_path, size=len(file_data))
|
||||||
|
return storage_path
|
||||||
|
except OSError as e:
|
||||||
|
raise StorageError(f"Failed to save file: {e}") from e
|
||||||
|
|
||||||
|
def read_file(self, storage_path: str) -> bytes:
|
||||||
|
"""Read file data from local storage."""
|
||||||
|
file_path = self._resolve_path(storage_path)
|
||||||
|
if not file_path.exists():
|
||||||
|
raise StorageError(f"File not found: {storage_path}")
|
||||||
|
try:
|
||||||
|
return file_path.read_bytes()
|
||||||
|
except OSError as e:
|
||||||
|
raise StorageError(f"Failed to read file: {e}") from e
|
||||||
|
|
||||||
|
def delete_file(self, storage_path: str) -> bool:
|
||||||
|
"""Delete a file from local storage."""
|
||||||
|
file_path = self._resolve_path(storage_path)
|
||||||
|
if not file_path.exists():
|
||||||
|
return False
|
||||||
|
try:
|
||||||
|
file_path.unlink()
|
||||||
|
logger.info("file_deleted", storage_path=storage_path)
|
||||||
|
return True
|
||||||
|
except OSError as e:
|
||||||
|
logger.error("file_delete_failed", storage_path=storage_path, error=str(e))
|
||||||
|
raise StorageError(f"Failed to delete file: {e}") from e
|
||||||
|
|
||||||
|
def file_exists(self, storage_path: str) -> bool:
|
||||||
|
"""Check if a file exists in local storage."""
|
||||||
|
file_path = self._resolve_path(storage_path)
|
||||||
|
return file_path.exists()
|
||||||
|
|
||||||
|
def get_file_size(self, storage_path: str) -> int:
|
||||||
|
"""Get the file size in bytes."""
|
||||||
|
file_path = self._resolve_path(storage_path)
|
||||||
|
if not file_path.exists():
|
||||||
|
raise StorageError(f"File not found: {storage_path}")
|
||||||
|
return file_path.stat().st_size
|
||||||
|
|
||||||
|
def get_absolute_path(self, storage_path: str) -> str:
|
||||||
|
"""Get the absolute filesystem path."""
|
||||||
|
return str(self._resolve_path(storage_path))
|
||||||
|
|
||||||
|
def save_temp_file(self, file_data: bytes, filename: str) -> str:
|
||||||
|
"""Save a temporary file."""
|
||||||
|
return self.save_file(file_data, "temp", filename)
|
||||||
|
|
||||||
|
def cleanup_temp(self) -> int:
|
||||||
|
"""Remove all files in the temp directory."""
|
||||||
|
temp_dir = self.base_path / "temp"
|
||||||
|
count = 0
|
||||||
|
if temp_dir.exists():
|
||||||
|
for item in temp_dir.iterdir():
|
||||||
|
if item.is_file():
|
||||||
|
item.unlink()
|
||||||
|
count += 1
|
||||||
|
elif item.is_dir():
|
||||||
|
shutil.rmtree(item)
|
||||||
|
count += 1
|
||||||
|
logger.info("temp_cleanup", files_removed=count)
|
||||||
|
return count
|
||||||
|
|
||||||
|
|
||||||
|
def get_storage_provider() -> StorageProvider:
|
||||||
|
"""Factory function to get the configured storage provider."""
|
||||||
|
if settings.storage_provider == "local":
|
||||||
|
return LocalStorageProvider()
|
||||||
|
raise StorageError(f"Unknown storage provider: {settings.storage_provider}")
|
||||||
BIN
docengine/app/tasks/__pycache__/__init__.cpython-313.pyc
Normal file
BIN
docengine/app/tasks/__pycache__/document_tasks.cpython-313.pyc
Normal file
BIN
docengine/app/workers/__pycache__/__init__.cpython-313.pyc
Normal file
BIN
docengine/app/workers/__pycache__/celery_app.cpython-313.pyc
Normal file
BIN
docengine/storage/documents/132cc8646df7415cb619809de75dbf3f.pdf
Normal file
BIN
docengine/storage/documents/1a2895b54df943c1bc2d715394ad58d0.pdf
Normal file
BIN
docengine/storage/documents/240fc5973ed14397997da20d547e141f.pdf
Normal file
BIN
docengine/storage/documents/588cc01bb78b46b785ff9cfef9ad5f04.pdf
Normal file
BIN
docengine/storage/documents/6b01bfc43bf44909a24e7d9950e193e7.pdf
Normal file
BIN
docengine/storage/documents/7eaabc95fa8d407490e07a5e49feb5dc.pdf
Normal file
BIN
docengine/storage/documents/9364f5c4c1a34cc69a729b9ab573801e.pdf
Normal file
BIN
docengine/storage/documents/968b0d7dc1d84b5994b4afe5a98f4d60.pdf
Normal file
BIN
docengine/storage/documents/a82e16443a7640d4b8830cee83f41fba.png
Normal file
|
After Width: | Height: | Size: 73 KiB |
BIN
docengine/storage/documents/b97ac6eff5634b159a778ab0a75fcedb.pdf
Normal file
BIN
docengine/storage/documents/de0a62d8bd524f53a894178b4863346f.pdf
Normal file
|
After Width: | Height: | Size: 241 KiB |
|
After Width: | Height: | Size: 258 KiB |
|
After Width: | Height: | Size: 7.2 KiB |
|
After Width: | Height: | Size: 16 KiB |
|
After Width: | Height: | Size: 198 KiB |
|
After Width: | Height: | Size: 185 KiB |
|
After Width: | Height: | Size: 258 KiB |
|
After Width: | Height: | Size: 5.7 KiB |
|
After Width: | Height: | Size: 307 KiB |
41
docker-compose.yaml
Normal file
@@ -0,0 +1,41 @@
|
|||||||
|
services:
|
||||||
|
# Backend API
|
||||||
|
backend:
|
||||||
|
build:
|
||||||
|
context: ./backend
|
||||||
|
dockerfile: Dockerfile
|
||||||
|
container_name: ocr_backend
|
||||||
|
restart: always
|
||||||
|
environment:
|
||||||
|
DB_USER: ${DB_USER}
|
||||||
|
DB_PASSWORD: ${DB_PASSWORD}
|
||||||
|
DB_HOST: ${DB_HOST}
|
||||||
|
DB_PORT: ${DB_PORT}
|
||||||
|
DB_NAME: ${DB_NAME}
|
||||||
|
|
||||||
|
# Mail Config
|
||||||
|
MAIL_SERVER: ${MAIL_SERVER}
|
||||||
|
MAIL_PORT: ${MAIL_PORT}
|
||||||
|
MAIL_USERNAME: ${MAIL_USERNAME}
|
||||||
|
MAIL_PASSWORD: ${MAIL_PASSWORD}
|
||||||
|
ports:
|
||||||
|
- "8000:8000"
|
||||||
|
networks:
|
||||||
|
- arbit-network
|
||||||
|
|
||||||
|
# Frontend Angular App
|
||||||
|
frontend:
|
||||||
|
build:
|
||||||
|
context: ./frontend
|
||||||
|
dockerfile: Dockerfile
|
||||||
|
container_name: ocr_frontend
|
||||||
|
restart: always
|
||||||
|
ports:
|
||||||
|
- "8075:80"
|
||||||
|
networks:
|
||||||
|
- arbit-network
|
||||||
|
|
||||||
|
networks:
|
||||||
|
arbit-network:
|
||||||
|
external: true
|
||||||
|
name: arbit-app_arbit-network
|
||||||
36
frontend/Dockerfile
Normal file
@@ -0,0 +1,36 @@
|
|||||||
|
# Stage 1: Build the Angular application
|
||||||
|
FROM node:18 as build
|
||||||
|
|
||||||
|
WORKDIR /app
|
||||||
|
|
||||||
|
# Install dependencies first (better caching)
|
||||||
|
COPY package*.json ./
|
||||||
|
RUN npm install
|
||||||
|
|
||||||
|
# Copy source code
|
||||||
|
COPY . .
|
||||||
|
|
||||||
|
# Build for production with specific base-href
|
||||||
|
RUN npm run build -- --configuration production --base-href /ocrf/
|
||||||
|
|
||||||
|
# Stage 2: Serve via Nginx
|
||||||
|
FROM nginx:alpine
|
||||||
|
|
||||||
|
# Remove default nginx static assets
|
||||||
|
RUN rm -rf /usr/share/nginx/html/*
|
||||||
|
|
||||||
|
# Create target directory (since we use /ocrf/ base-href)
|
||||||
|
RUN mkdir -p /usr/share/nginx/html/ocrf
|
||||||
|
|
||||||
|
# Copy built assets from builder stage
|
||||||
|
# Note: Adjust 'dist/frontend/browser' based on your angular.json output path.
|
||||||
|
# Angular 17+ creates 'browser' folder.
|
||||||
|
COPY --from=build /app/dist/frontend/browser /usr/share/nginx/html/ocrf
|
||||||
|
|
||||||
|
# Copy custom nginx config
|
||||||
|
COPY nginx.conf /etc/nginx/conf.d/default.conf
|
||||||
|
|
||||||
|
# Expose port 80
|
||||||
|
EXPOSE 80
|
||||||
|
|
||||||
|
CMD ["nginx", "-g", "daemon off;"]
|
||||||
@@ -55,7 +55,8 @@
|
|||||||
}
|
}
|
||||||
],
|
],
|
||||||
"styles": [
|
"styles": [
|
||||||
"src/styles.scss"
|
"src/styles.scss",
|
||||||
|
"node_modules/primeflex/primeflex.css"
|
||||||
]
|
]
|
||||||
},
|
},
|
||||||
"configurations": {
|
"configurations": {
|
||||||
@@ -63,8 +64,8 @@
|
|||||||
"budgets": [
|
"budgets": [
|
||||||
{
|
{
|
||||||
"type": "initial",
|
"type": "initial",
|
||||||
"maximumWarning": "500kB",
|
"maximumWarning": "1MB",
|
||||||
"maximumError": "1MB"
|
"maximumError": "2MB"
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
"type": "anyComponentStyle",
|
"type": "anyComponentStyle",
|
||||||
|
|||||||
16
frontend/nginx.conf
Normal file
@@ -0,0 +1,16 @@
|
|||||||
|
server {
|
||||||
|
listen 80;
|
||||||
|
server_name localhost;
|
||||||
|
root /usr/share/nginx/html/ocrf;
|
||||||
|
index index.html;
|
||||||
|
|
||||||
|
location / {
|
||||||
|
try_files $uri $uri/ /index.html;
|
||||||
|
}
|
||||||
|
|
||||||
|
# Optional: Cache static assets
|
||||||
|
location ~* \.(js|css|png|jpg|jpeg|gif|ico)$ {
|
||||||
|
expires 1y;
|
||||||
|
add_header Cache-Control "public, no-transform";
|
||||||
|
}
|
||||||
|
}
|
||||||
7
frontend/package-lock.json
generated
@@ -16,6 +16,7 @@
|
|||||||
"@angular/forms": "^21.1.0",
|
"@angular/forms": "^21.1.0",
|
||||||
"@angular/platform-browser": "^21.1.0",
|
"@angular/platform-browser": "^21.1.0",
|
||||||
"@angular/router": "^21.1.0",
|
"@angular/router": "^21.1.0",
|
||||||
|
"primeflex": "^4.0.0",
|
||||||
"primeicons": "^7.0.0",
|
"primeicons": "^7.0.0",
|
||||||
"primeng": "^17.18.0",
|
"primeng": "^17.18.0",
|
||||||
"quill": "^2.0.3",
|
"quill": "^2.0.3",
|
||||||
@@ -6570,6 +6571,12 @@
|
|||||||
"dev": true,
|
"dev": true,
|
||||||
"license": "MIT"
|
"license": "MIT"
|
||||||
},
|
},
|
||||||
|
"node_modules/primeflex": {
|
||||||
|
"version": "4.0.0",
|
||||||
|
"resolved": "https://registry.npmjs.org/primeflex/-/primeflex-4.0.0.tgz",
|
||||||
|
"integrity": "sha512-UOEZCRjR36+sm5bUpDhS1xbA068l9VC6y1aTNVqQPtXuKIdPTqAWHRUxj3mKAoPrQ9W373ooJJMgNVXfiaw04g==",
|
||||||
|
"license": "MIT"
|
||||||
|
},
|
||||||
"node_modules/primeicons": {
|
"node_modules/primeicons": {
|
||||||
"version": "7.0.0",
|
"version": "7.0.0",
|
||||||
"resolved": "https://registry.npmjs.org/primeicons/-/primeicons-7.0.0.tgz",
|
"resolved": "https://registry.npmjs.org/primeicons/-/primeicons-7.0.0.tgz",
|
||||||
|
|||||||
@@ -3,7 +3,7 @@
|
|||||||
"version": "0.0.0",
|
"version": "0.0.0",
|
||||||
"scripts": {
|
"scripts": {
|
||||||
"ng": "ng",
|
"ng": "ng",
|
||||||
"start": "ng serve",
|
"start": "ng serve --proxy-config proxy.conf.json",
|
||||||
"build": "ng build",
|
"build": "ng build",
|
||||||
"watch": "ng build --watch --configuration development",
|
"watch": "ng build --watch --configuration development",
|
||||||
"test": "ng test"
|
"test": "ng test"
|
||||||
@@ -31,6 +31,7 @@
|
|||||||
"@angular/forms": "^21.1.0",
|
"@angular/forms": "^21.1.0",
|
||||||
"@angular/platform-browser": "^21.1.0",
|
"@angular/platform-browser": "^21.1.0",
|
||||||
"@angular/router": "^21.1.0",
|
"@angular/router": "^21.1.0",
|
||||||
|
"primeflex": "^4.0.0",
|
||||||
"primeicons": "^7.0.0",
|
"primeicons": "^7.0.0",
|
||||||
"primeng": "^17.18.0",
|
"primeng": "^17.18.0",
|
||||||
"quill": "^2.0.3",
|
"quill": "^2.0.3",
|
||||||
|
|||||||
10
frontend/proxy.conf.json
Normal file
@@ -0,0 +1,10 @@
|
|||||||
|
{
|
||||||
|
"/ocrb": {
|
||||||
|
"target": "http://localhost:8000",
|
||||||
|
"secure": false,
|
||||||
|
"changeOrigin": true,
|
||||||
|
"pathRewrite": {
|
||||||
|
"^/ocrb": ""
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
93
frontend/public/assets/fonts/poppins/OFL.txt
Normal file
@@ -0,0 +1,93 @@
|
|||||||
|
Copyright 2020 The Poppins Project Authors (https://github.com/itfoundry/Poppins)
|
||||||
|
|
||||||
|
This Font Software is licensed under the SIL Open Font License, Version 1.1.
|
||||||
|
This license is copied below, and is also available with a FAQ at:
|
||||||
|
https://openfontlicense.org
|
||||||
|
|
||||||
|
|
||||||
|
-----------------------------------------------------------
|
||||||
|
SIL OPEN FONT LICENSE Version 1.1 - 26 February 2007
|
||||||
|
-----------------------------------------------------------
|
||||||
|
|
||||||
|
PREAMBLE
|
||||||
|
The goals of the Open Font License (OFL) are to stimulate worldwide
|
||||||
|
development of collaborative font projects, to support the font creation
|
||||||
|
efforts of academic and linguistic communities, and to provide a free and
|
||||||
|
open framework in which fonts may be shared and improved in partnership
|
||||||
|
with others.
|
||||||
|
|
||||||
|
The OFL allows the licensed fonts to be used, studied, modified and
|
||||||
|
redistributed freely as long as they are not sold by themselves. The
|
||||||
|
fonts, including any derivative works, can be bundled, embedded,
|
||||||
|
redistributed and/or sold with any software provided that any reserved
|
||||||
|
names are not used by derivative works. The fonts and derivatives,
|
||||||
|
however, cannot be released under any other type of license. The
|
||||||
|
requirement for fonts to remain under this license does not apply
|
||||||
|
to any document created using the fonts or their derivatives.
|
||||||
|
|
||||||
|
DEFINITIONS
|
||||||
|
"Font Software" refers to the set of files released by the Copyright
|
||||||
|
Holder(s) under this license and clearly marked as such. This may
|
||||||
|
include source files, build scripts and documentation.
|
||||||
|
|
||||||
|
"Reserved Font Name" refers to any names specified as such after the
|
||||||
|
copyright statement(s).
|
||||||
|
|
||||||
|
"Original Version" refers to the collection of Font Software components as
|
||||||
|
distributed by the Copyright Holder(s).
|
||||||
|
|
||||||
|
"Modified Version" refers to any derivative made by adding to, deleting,
|
||||||
|
or substituting -- in part or in whole -- any of the components of the
|
||||||
|
Original Version, by changing formats or by porting the Font Software to a
|
||||||
|
new environment.
|
||||||
|
|
||||||
|
"Author" refers to any designer, engineer, programmer, technical
|
||||||
|
writer or other person who contributed to the Font Software.
|
||||||
|
|
||||||
|
PERMISSION & CONDITIONS
|
||||||
|
Permission is hereby granted, free of charge, to any person obtaining
|
||||||
|
a copy of the Font Software, to use, study, copy, merge, embed, modify,
|
||||||
|
redistribute, and sell modified and unmodified copies of the Font
|
||||||
|
Software, subject to the following conditions:
|
||||||
|
|
||||||
|
1) Neither the Font Software nor any of its individual components,
|
||||||
|
in Original or Modified Versions, may be sold by itself.
|
||||||
|
|
||||||
|
2) Original or Modified Versions of the Font Software may be bundled,
|
||||||
|
redistributed and/or sold with any software, provided that each copy
|
||||||
|
contains the above copyright notice and this license. These can be
|
||||||
|
included either as stand-alone text files, human-readable headers or
|
||||||
|
in the appropriate machine-readable metadata fields within text or
|
||||||
|
binary files as long as those fields can be easily viewed by the user.
|
||||||
|
|
||||||
|
3) No Modified Version of the Font Software may use the Reserved Font
|
||||||
|
Name(s) unless explicit written permission is granted by the corresponding
|
||||||
|
Copyright Holder. This restriction only applies to the primary font name as
|
||||||
|
presented to the users.
|
||||||
|
|
||||||
|
4) The name(s) of the Copyright Holder(s) or the Author(s) of the Font
|
||||||
|
Software shall not be used to promote, endorse or advertise any
|
||||||
|
Modified Version, except to acknowledge the contribution(s) of the
|
||||||
|
Copyright Holder(s) and the Author(s) or with their explicit written
|
||||||
|
permission.
|
||||||
|
|
||||||
|
5) The Font Software, modified or unmodified, in part or in whole,
|
||||||
|
must be distributed entirely under this license, and must not be
|
||||||
|
distributed under any other license. The requirement for fonts to
|
||||||
|
remain under this license does not apply to any document created
|
||||||
|
using the Font Software.
|
||||||
|
|
||||||
|
TERMINATION
|
||||||
|
This license becomes null and void if any of the above conditions are
|
||||||
|
not met.
|
||||||
|
|
||||||
|
DISCLAIMER
|
||||||
|
THE FONT SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
|
||||||
|
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTIES OF
|
||||||
|
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT
|
||||||
|
OF COPYRIGHT, PATENT, TRADEMARK, OR OTHER RIGHT. IN NO EVENT SHALL THE
|
||||||
|
COPYRIGHT HOLDER BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
|
||||||
|
INCLUDING ANY GENERAL, SPECIAL, INDIRECT, INCIDENTAL, OR CONSEQUENTIAL
|
||||||
|
DAMAGES, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
|
||||||
|
FROM, OUT OF THE USE OR INABILITY TO USE THE FONT SOFTWARE OR FROM
|
||||||
|
OTHER DEALINGS IN THE FONT SOFTWARE.
|
||||||