โ† Back to Home
React FastAPI Flask SQLite Docker Full-Stack ยท Production

โš™๏ธ Multilingual Exam Review Agent

exam_checker_agent โ€” Auto question splitting + per-question routed detection + four-stage image-text review + three-way plagiarism + measurable evaluation; production batch-processing rewrite

Overview

A production-grade rewrite of exam_smart_checker, upgrading the single-user Streamlit tool into a multi-user concurrent web platform with batch task management, a 5-step automated review pipeline, and result download capabilities.

Use Cases: Multi-institution concurrent review, background batch task management, SaaS deployment
๐Ÿ“– Full design docs (desensitized archive) Architecture, prompt engineering, the four-stage image-text review pipeline, dedup/aggregation and metrics (with Mermaid flowcharts) live in a standalone docs repo โ€” no runnable source or exam data: Read on GitHub โ†’

Tech Stack

Python 3.12 FastAPI (primary API) Flask (legacy WSGI, in-process) React + Vite (frontend) SQLite (task persistence) Docker + Dockerfile Multilingual LLM detection

Architecture

โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”‚ Frontend (React + Vite) โ”‚ โ”‚ Batch mgmt / Task list / Results / File download โ”‚ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ”‚ HTTP port 8777 โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ–ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”‚ FastAPI Main Service (backend/main.py) โ”‚ โ”‚ /api/v2/* โ†’ FastAPI routers โ”‚ โ”‚ /api/v1/* โ†’ LegacySmartCheckerService.dispatch() โ”‚ โ”‚ (in-process Flask WSGI bridge) โ”‚ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ”‚ โ”‚ โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ–ผโ”€โ”€โ”€โ”€โ”€โ”€โ” โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ–ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”‚ Task State โ”‚ โ”‚ Flask API Server โ”‚ โ”‚ Machine โ”‚ โ”‚ (api_server.py) โ”‚ โ”‚ parse โ”‚ โ”‚ ~1356 lines โ”‚ โ”‚ โ†’ review โ”‚ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ”‚ โ†’ int_plag โ”‚ โ”‚ โ†’ crs_plag โ”‚ โ”‚ โ†’ his_plag โ”‚ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
  • backend/app/routers/ โ€” FastAPI routing layer
  • backend/app/services/ โ€” business logic layer
  • backend/app/repositories/ โ€” data access layer (SQLite)
  • backend/app/domain/ โ€” task state machine definition
  • backend/api_server.py โ€” legacy Flask service (WSGI in-process)
  • frontend/ โ€” React frontend
  • run.py โ€” unified entry point, port 8777
  • Dockerfile / deploy.sh

Core Algorithms / Key Design

1. Dual Web Framework WSGI In-Process Fusion

FastAPI serves as the primary server. LegacySmartCheckerService.dispatch() routes /api/v1/* to the Flask WSGI app in-process, achieving a smooth migration that preserves legacy capabilities with zero extra network hops.

2. 5-Step Task State Machine

Each review task progresses through five serial stages: parse โ†’ review โ†’ internal_plag โ†’ cross_plag โ†’ history_plag. Optimistic locking prevents concurrent conflicts. A heartbeat mechanism automatically detects and recovers stale tasks (running tasks that miss heartbeat deadlines).

3. AppRuntime Dependency Injection Container

A unified AppRuntime singleton manages the lifecycle of all service instances. FastAPI's DI system retrieves Repositories, Services etc. through it, eliminating global state.

4. X-Client-Id Multi-Tenant Isolation

Every request carries a X-Client-Id header. Task queries and result reads are scoped to the corresponding client owner, preventing cross-institution data leakage.

5. Dynamic Multilingual Prompt Selection

A language detection module identifies the exam language (zh/ja/de/fr) and dynamically loads the corresponding LLM review prompt, enabling multilingual support within a single system.

Innovations / My Contributions

  • Designed the overall batch platform architecture (FastAPI + SQLite + React full-stack)
  • Implemented the FastAPI โ†” Flask WSGI in-process bridge, preserving legacy capabilities at zero migration cost
  • Designed the 5-step task state machine with optimistic locking and heartbeat zombie recovery
  • Built the AppRuntime DI container for unified service lifecycle management
  • Implemented X-Client-Id per-client data isolation
  • Wrote Docker image and one-command deployment scripts

โ˜… Image Review: Four-Stage Multimodal Pipeline

Image-text consistency (a figure whose labels contradict the question, or the wrong figure entirely) is the hardest, highest-value part of review. VLMs often "describe the image fluently yet miss the key anchor". The pipeline:

โ‘ 
Classify + Formula OCR
VLM splits formula/diagram (borderline โ†’ diagram to avoid misses); formula images OCR'd back into the stem.
โ‘ก
Multi-image Detection
All diagrams + enriched stem in one call; forced "image observation checklist + claim reconciliation table" to enumerate facts, not impressions.
โ‘ข
Text-only Re-review
Triggers when Step3 finds no error: no image, only the VLM's own checklist vs. the stem; falseโ†’true flips only to protect precision.
โ‘ฃ
Engineering
Global base64 cache (8-thread prefetch), SVGโ†’PNG, min-size upscale, zip+xlsx image bridging with cross-row placeholder renumbering.
Quantified gain (mmdataset-v3 benchmark) Refactoring "one-shot image to VLM" into this four-stage pipeline + forced-reconciliation prompt + text-only re-review, image-text inconsistency detection reaches recall 0.84 (qwen3.6-27B local) ~ 0.88 (plus cloud), F1 0.89~0.92, precision 0.95+, FPRโ‰ˆ0.10 (the weak implementation reached only ~0.26 recall), while "re-review flips falseโ†’true only" preserved precision. Only vision-capable multimodal models are compared.
๐Ÿ”ฌ Prompt/pipeline ablation (same dataset) On one dataset (973 samples) and one judge, comparing "minimal prompt / single strong prompt / four-stage full pipeline ร— qwen3.6-plus / 27b" (6 configs): model capability sets the ceiling; the prompt is a precisionโ†”recall dial; the four-stage scaffolding only matters for the weaker model (27b) โ€” for the strong model (plus) a one-line prompt already nears the full pipeline. Full table + charts + detailed conclusions: ablation doc โ†’

Agent or Workflow? (an honest take)

It is a deterministic multi-stage pipeline with two constrained LLM decision points: routing (which checks to run per question; missing key = false, fail-closed) and error aggregation (keep / merge / delete; error_type locked, failures fall back without dropping errors). The LLM never drives control flow or self-replans โ€” the philosophy is "use reproducible rules wherever possible, reserve the LLM for genuine semantic judgment, and constrain its output space to the minimum". Positioned honestly as a reliable LLM-in-the-loop workflow, with a clear path toward a more agentic form.

Screenshots

New batch task
New batch task โ€” upload exam, 4-step wizard
Task dashboard
Task dashboard โ€” batch progress and pipeline status
Case library
Few-shot case library โ€” Excel import, embedding cache

Links

๐Ÿ“– Full Design Docs GitHub Profile View README