# 🚀 DOING: Regression Test — Test Suite Storage + AI Judge

## 📁 Folders & Files liên quan
```
backend/
├── api/
│   ├── regression_test/
│   │   ├── regression_test_route.py   ← MODIFY (thêm suite + judge endpoints)
│   │   └── regression_db.py           ← 🆕 NEW (SQLite helpers)
│   └── prompt_optimizer/
│       └── prompt_optimizer_route.py   ← OK (tái dùng judge logic, không đổi)
├── common/
│   └── llm_factory.py                 ← OK (dùng create_llm cho judge, không đổi)
└── database/
    └── regression.db                   ← 🆕 NEW (test_suites, test_cases, test_runs, test_results)
```

## 📌 Context
- **Status:** 🚧 In Progress

## 🎯 Goal
Nâng cấp Regression Test từ 74-line "fire-and-forget" thành hệ thống test tự động: lưu test suites vào SQLite, tích hợp AI Judge chấm điểm, lưu run history để detect regression.

## ⚠️ Active Trade-offs
- Storage: SQLite `database/regression.db`
- AI Judge: tái dùng logic từ `prompt_optimizer/judge` (đã có)
- Không thay đổi existing endpoint — chỉ thêm mới

---

## 🔍 Hiện trạng

### ✅ Đã có:
- `api/regression_test/regression_test_route.py` — 74 lines, 1 endpoint `POST /run`
- `api/prompt_optimizer/prompt_optimizer_route.py` — `/judge` endpoint đánh giá 5 tiêu chí

### ❌ Thiếu:
1. **Không có test suite library** — mỗi lần phải paste JSON test cases thủ công
2. **Không có expected answers** — không biết response đúng hay sai
3. **Không có AI Judge** — human phải đọc từng response
4. **Không có history** — test #5 tuần trước pass, tuần này fail → không ai biết
5. **Không connect với pipeline #17** — auto-improve xong không tự verify

---

## 🛠 Impact & Files Touched
| File | Symbol | Change | Blast Radius |
|------|--------|--------|--------------|
| NEW: `api/regression_test/regression_db.py` | `init_db`, CRUD | ADD | regression_test_route |
| `api/regression_test/regression_test_route.py` | endpoints | MODIFY (add suite + judge) | Low |

**Risk:** Low

## 📋 Execution Checklist

### Phase 1: SQLite Test Suite Storage (~25m)
- [x] Task 1.1: Tạo `api/regression_test/regression_db.py` với `database/regression.db`:
  - Bảng `test_suites`: `id, name, description, target_url, created_at, updated_at, test_count`
  - Bảng `test_cases`: `id, suite_id (FK), question, expected_keywords, expected_behavior, category, difficulty`
  - Bảng `test_runs`: `run_id, suite_id, timestamp, target_url, total, passed, failed, avg_score, model_name, prompt_version`
  - Bảng `test_results`: `id, run_id (FK), case_id (FK), question, response, status, score, judge_reasoning, latency_ms`
- [x] Task 1.2: CRUD helpers:
  - `create_suite(name, target_url)` → return suite
  - `add_test_case(suite_id, question, expected_keywords, expected_behavior)`
  - `get_suite(suite_id)` → suite + test cases
  - `list_suites()` → all suites
  - `delete_suite(suite_id)`

### Phase 2: API Endpoints — Suite Management (~15m)
- [x] Task 2.1: `POST /api/regression-test/suites` — tạo test suite mới
- [x] Task 2.2: `GET /api/regression-test/suites` — list all suites
- [x] Task 2.3: `GET /api/regression-test/suites/{id}` — chi tiết suite + test cases
- [x] Task 2.4: `POST /api/regression-test/suites/{id}/cases` — thêm test case
- [x] Task 2.5: `DELETE /api/regression-test/suites/{id}` — xóa suite
- [x] Task 2.6: `POST /api/regression-test/suites/{id}/import` — bulk import test cases từ JSON

### Phase 3: AI Judge Integration (~20m)
- [x] Task 3.1: Tạo `_judge_response(question, response, expected_keywords, expected_behavior)`:
  - Dùng `create_llm(DEFAULT_MODEL)` để đánh giá response
  - Score: 0-10 (factual accuracy, relevance, tone, helpfulness)
  - Pass nếu score ≥ 7
- [x] Task 3.2: Upgrade `POST /api/regression-test/run` → sau khi có response, tự chạy judge cho mỗi case
- [x] Task 3.3: Save full results (response + score + reasoning) vào `test_results` table

### Phase 4: History + Comparison (~15m)
- [x] Task 4.1: `GET /api/regression-test/runs` — list all test runs (newest first)
- [x] Task 4.2: `GET /api/regression-test/runs/{run_id}` — chi tiết 1 run + all case results
- [x] Task 4.3: `GET /api/regression-test/runs/{run_id}/regressions` — compare with previous run of same suite → highlight cases that went from pass → fail
- [x] Task 4.4: Auto-detect regression: flag cases where score dropped ≥ 2 points

### Phase 5: Seed Default Suite (~10m)
- [x] Task 5.1: Tạo default test suite "Canifa Core" với 20 test cases:
  - 5 cases: Product search (áo phông nam, váy nữ, đồ trẻ em...)
  - 5 cases: Policy/Knowledge (đổi trả, hotline, cửa hàng...)
  - 5 cases: Edge cases (hỏi ngoài scope, spam, tiếng Anh...)
  - 5 cases: Sales behavior (upsell, combo, urgency...)

### Phase 6: Testing (~10m)
- [x] Task 6.1: Test create suite + add cases
- [x] Task 6.2: Test run with AI judge → verify scores saved
- [x] Task 6.3: Test regression detection between 2 runs

## 🔄 Sub-Doings
_None yet_

## ✅ Completion Gate
- [x] All tasks [x] | [x] Sub-Doings resolved | [x] Smoke test passed | [x] Trade-offs confirmed

---
_Started: 2026-05-10 | Updated: 2026-05-10_
