# 6. Testing & Troubleshooting

## Quick Test Script

Save as `test_autotrain.py`:

```python
import requests
import json

BASE = "http://localhost:8000/autotrain"

def test_feedback_submit():
    print("\n=== 1. Submit Feedback ===")
    resp = requests.post(f"{BASE}/feedback", json={
        "query": "tìm áo khoác cho trời 35 độ",
        "ai_response": "Mình gợi ý áo khoác nỉ này rất ấm...",
        "feedback_text": "Trời nóng quá, AI còn tư vấn áo khoác",
        "feedback_type": "bug",
        "user_id": "test-user-001"
    })
    data = resp.json()
    print(f"Feedback ID: {data['feedback_id']}")
    return data['feedback_id']

def test_run_all(feedback_id):
    print("\n=== 2. Run All Pending ===")
    resp = requests.post(f"{BASE}/run-all")
    data = resp.json()
    print(f"Processed: {data['processed']}")
    for r in data['results']:
        if 'rule_id' in r:
            print(f"  ✓ Generated rule: {r['rule_id']}")
    return data

def test_list_rules():
    print("\n=== 3. List Generated Rules ===")
    resp = requests.get(f"{BASE}/rules?status=pending")
    rules = resp.json()
    for r in rules:
        print(f"  {r['rule_id']}: {r['new_rule'][:50]}... (conf: {r['confidence']})")
    return rules

def test_approve_rule(rule_id):
    print(f"\n=== 4. Approve Rule {rule_id} ===")
    resp = requests.post(f"{BASE}/rules/{rule_id}/approve?reviewed_by=admin")
    print(f"Status: {resp.json()['status']}")

def test_deploy_rule(rule_id):
    print(f"\n=== 5. Deploy Rule {rule_id} ===")
    resp = requests.post(f"{BASE}/rules/{rule_id}/deploy?deployed_by=admin")
    data = resp.json()
    print(f"Status: {data['status']}")
    print(f"Note: {data['note']}")

def test_status():
    print("\n=== 6. System Status ===")
    resp = requests.get(f"{BASE}/status")
    s = resp.json()
    print(f"  Total feedbacks: {s['total_feedbacks']}")
    print(f"  Pending feedbacks: {s['pending_feedbacks']}")
    print(f"  Total rules: {s['total_rules']}")
    print(f"  Pending rules: {s['pending_rules']}")
    print(f"  Deployed rules: {s['deployed_rules']}")

if __name__ == "__main__":
    # Clear previous test data by restarting DB? Skip for now
    
    # Run flow
    fb_id = test_feedback_submit()
    test_run_all([fb_id])
    rules = test_list_rules()
    
    if rules:
        test_approve_rule(rules[0]['rule_id'])
        test_deploy_rule(rules[0]['rule_id'])
    
    test_status()
    
    print("\n✅ Test complete!")
```

Run with: `python test_autotrain.py`

---

## Manual cURL Test

```bash
# 1. Submit feedback
curl -X POST http://localhost:8000/autotrain/feedback \
  -H "Content-Type: application/json" \
  -d '{
    "user_id": "CNF-98453",
    "query": "tìm áo khoác cho trời 35 độ",
    "ai_response": "Mình gợi ý áo khoác nỉ này rất ấm, phù hợp cho thời tiết lạnh.",
    "feedback_text": "Trời nóng quá, AI còn tư vấn áo khoác",
    "feedback_type": "bug"
  }'

# Response:
# {"id":1,"feedback_id":"FB-1001","user_id":"CNF-98453","query":"...","feedback_text":"...","status":"pending"}

# 2. Check status
curl http://localhost:8000/autotrain/status

# 3. Generate rule manually (optional - run-all does this automatically)
curl -X POST http://localhost:8000/autotrain/generate/FB-1001

# 4. List rules
curl http://localhost:8000/autotrain/rules

# 5. Approve rule
curl -X POST http://localhost:8000/autotrain/rules/RUL-1001/approve?reviewed_by=admin

# 6. Deploy rule
curl -X POST http://localhost:8000/autotrain/rules/RUL-1001/deploy?deployed_by=admin

# 7. Verify deployment in DB
curl http://localhost:8000/autotrain/status
```

---

## Database Verification

```bash
# Check SQLite directly
sqlite3 backend/database/autotrain.db

# Inside sqlite3:
.tables
SELECT * FROM feedbacks;
SELECT * FROM generated_rules;
SELECT * FROM deployments;
```

Expected output after full flow:

```
feedbacks:
| id | feedback_id | status    |
|----|-------------|-----------|
| 1  | FB-1001     | processed|

generated_rules:
| id | rule_id  | feedback_id | status   |
|----|----------|-------------|----------|
| 1  | RUL-1001 | FB-1001     | deployed |

deployments:
| id | rule_id  | deployed_by |
|----|----------|-------------|
| 1  | RUL-1001 | admin       |
```

---

## Common Issues & Solutions

### Issue 1: "Feedback not found" (404)

**Cause:** `feedback_id` doesn't exist or was already deleted.

**Fix:** Check with `GET /autotrain/feedback` to verify the ID exists.

---

### Issue 2: "Rule not found or not approved" (400 on deploy)

**Cause:** Rule status is `pending`, not `approved`.

**Fix:** First run `POST /autotrain/rules/{rule_id}/approve`.

---

### Issue 3: No rules generated after run-all

**Cause:** Mock AI pattern didn't match any known patterns.

**Fix:** The fallback should still generate a rule with confidence 0.5. If not, check logs for errors. Add debug logging:

```python
logger.debug(f"[AutoTrain] Feedback text: {feedback_text}, Pattern matched: {matched_pattern}")
```

---

### Issue 4: SQLite database not initialized

**Cause:** `init_db()` failed or DB path is wrong.

**Fix:** Check DB path:

```python
DB_PATH = Path(__file__).parent.parent.parent / "database" / "autotrain.db"
print(f"DB Path: {DB_PATH}")
print(f"Exists: {DB_PATH.exists()}")
```

Manually initialize:

```bash
python -c "from api.autotrain.autotrain_route import init_db; init_db()"
```

---

### Issue 5: Duplicate feedback_id error

**Cause:** Manually providing `feedback_id` that already exists.

**Fix:** Omit `feedback_id` to auto-generate, or ensure unique.

---

### Issue 6: Unicode/Garbled characters in database

**Cause:** SQLite default encoding issues.

**Fix:** Ensure UTF-8 encoding when connecting:

```python
conn = sqlite3.connect(DB_PATH, detect_types=sqlite3.PARSE_DECLTYPES)
conn.execute("PRAGMA encoding='UTF-8'")
```

---

## Debugging Tips

### Enable detailed logging

```python
import logging
logging.basicConfig(level=logging.DEBUG)
```

### Check API logs

```bash
# Tail FastAPI logs
tail -f logs/uvicorn.log | grep autotrain
```

### Inspect rule generation

```python
# Add to autotrain_route.py temporarily
logger.info(f"[AutoTrain] Generating rule for feedback: {feedback_id}")
logger.info(f"[AutoTrain] Feedback text: {feedback['feedback_text']}")
logger.info(f"[AutoTrain] Rule generated: {rule_data}")
```

---

## Expected Behavior Checklist

- [ ] Submit feedback → Returns `feedback_id`, status `pending`
- [ ] `GET /status` shows increased `total_feedbacks` and `pending_feedbacks`
- [ ] `POST /run-all` processes all pending → generates rules
- [ ] `GET /rules` shows new rules with status `pending`
- [ ] `POST /approve` changes status to `approved`
- [ ] `POST /deploy` changes status to `deployed` and creates deployment log
- [ ] `GET /status` shows updated counts

If any step fails, check:
1. Database connection
2. Table schema matches expected
3. FastAPI server logs for stack traces

---

## Load Testing

```bash
# Using ApacheBench
ab -n 100 -p feedback_sample.json -T application/json -X POST http://localhost:8000/autotrain/feedback

# Using wrk
wrk -t12 -c100 -d30s --latency -s post.lua http://localhost:8000/autotrain/feedback
```

`post.lua`:
```lua
request = function()
    return wrk.format("POST", "/autotrain/feedback", {
        ["Content-Type"] = "application/json"
    }, json_body)
end
```

---

## Performance Optimization

Current bottlenecks:
- **SQLite writes** are serialized → Use connection pooling or switch to PostgreSQL
- **Mock AI** is CPU-bound pattern matching → Already fast
- **No caching** → Every request hits DB → Add Redis cache for status endpoint

Optimization targets:
- `POST /feedback`: < 50ms
- `GET /status`: < 20ms (with caching)
- `POST /run-all` (100 feedbacks): < 5 seconds

---

## Integration Testing

```python
# test_integration.py
def test_full_pipeline():
    """Test complete feedback → rule → deploy flow."""
    # 1. Submit
    fb_resp = client.post("/autotrain/feedback", json=sample_feedback)
    assert fb_resp.status_code == 200
    fb_id = fb_resp.json()['feedback_id']
    
    # 2. Generate
    gen_resp = client.post(f"/autotrain/generate/{fb_id}")
    assert gen_resp.status_code == 200
    rule_id = gen_resp.json()['rule_id']
    
    # 3. Approve
    app_resp = client.post(f"/autotrain/rules/{rule_id}/approve")
    assert app_resp.json()['status'] == 'approved'
    
    # 4. Deploy
    dep_resp = client.post(f"/autotrain/rules/{rule_id}/deploy")
    assert dep_resp.json()['status'] == 'deployed'
    
    print("✅ Full pipeline test passed!")
```

Run with: `pytest test_integration.py -v`

---

## Troubleshooting Flowchart

```mermaid
graph TD
    A[Test Failed] --> B{Which step?}
    B -->|Submit| C1[Check JSON body]
    B -->|Generate| C2[Check feedback exists]
    B -->|Approve| C3[Check rule status]
    B -->|Deploy| C4[Check rule approved]
    
    C1 --> D1[Required fields: query, ai_response, feedback_text]
    C2 --> D2[DB connection + table schema]
    C3 --> D3[Should be 'pending' before approve]
    C4 --> D4[Run approve first]
    
    D1 --> E[Fix JSON, retry]
    D2 --> E
    D3 --> E
    D4 --> E
    
    E --> F{Work?}
    F -->|Yes| G[✅ Success]
    F -->|No| H[Check server logs]
    H --> I[Enable DEBUG logging]
    I --> J[Identify error]
    J --> K[Fix code]
    K --> E
```

---

## Log Locations

By default, FastAPI/uvicorn logs to stdout. To enable file logging:

```python
# In main.py or autotrain_route.py
import logging
from logging.handlers import RotatingFileHandler

handler = RotatingFileHandler(
    'logs/autotrain.log',
    maxBytes=1000000,
    backupCount=5
)
handler.setLevel(logging.DEBUG)
logging.getLogger('api.autotrain').addHandler(handler)
```

---

## Test Data Cleanup

To reset between tests:

```python
# Clear all tables
def cleanup_test_data():
    conn = get_db()
    cur = conn.cursor()
    cur.execute("DELETE FROM deployments")
    cur.execute("DELETE FROM generated_rules")
    cur.execute("DELETE FROM feedbacks")
    # Reset sequences
    cur.execute("DELETE FROM sqlite_sequence WHERE name='feedbacks'")
    cur.execute("DELETE FROM sqlite_sequence WHERE name='generated_rules'")
    cur.execute("DELETE FROM sqlite_sequence WHERE name='deployments'")
    conn.commit()
    conn.close()
```

Or just delete the DB file and re-initialize:

```bash
rm backend/database/autotrain.db
python -c "from api.autotrain.autotrain_route import init_db; init_db()"
```

---

## Monitoring Alerts

Set up alerts for:

| Condition | Threshold | Action |
|-----------|-----------|--------|
| Pending feedbacks > 100 | Warning | Slack notification |
| Rule generation error rate > 5% | Critical | PagerDuty |
| Deployments per hour > 50 | Info | Log only |
| Approval time > 24h | Warning | Email admin |
| Deployment failures > 3/hour | Critical | PagerDuty |

---

## Support

If issues persist:
1. Check this troubleshooting guide
2. Review cookbook sections on Architecture and API
3. Inspect `backend/database/autotrain.db` directly
4. Enable DEBUG logging and review logs
5. Open issue with full error details and reproduction steps
