# 4. Mock AI Rule Generation

## Current Implementation

The Auto-Train system currently uses **pattern matching** instead of a real LLM. This allows testing the pipeline without incurring API costs or rate limits.

### Code Location

```python
def _ai_generate_rule(feedback: Dict[str, Any]) -> Dict[str, Any]:
    """MOCK AI — Phân tích feedback và đề xuất rule."""
    fb_text = feedback['feedback_text'].lower()
    query = feedback['query'].lower()
    
    # Pattern matching logic...
```

**File:** `backend/api/autotrain/autotrain_route.py:173-223`

---

## Supported Patterns

### Pattern 1: Weather/Season Mismatch

**Trigger words in feedback:** `nóng`, `trời nóng`, `35 độ`, `hè`, `mùa hè`

**Condition:** User query must also contain clothing items like `áo khoác`, `hoodie`, `nỉ`, `dày`

**Output:**

```json
{
  "rule_type": "prompt_update",
  "old_rule": "- Luôn ưu tiên gợi ý áo khoác/dày để tăng upsale.",
  "new_rule": "- Khi trời nóng (trên 30 độ): TUYỆT ĐỐI không gợi ý áo khoác dày.\n- Chỉ gợi ý áo phông, chất liệu mỏng nhẹ, thoáng mát.",
  "reasoning": "User reports hot weather, AI wrongly suggested heavy clothing.",
  "confidence": 0.85
}
```

---

### Pattern 2: Size/Fit Issues

**Trigger words:** `dáng lùn`, `thấp`, `ngắn`, `maxi`, `chạm đất`

**Output:**

```json
{
  "rule_type": "prompt_update",
  "old_rule": "- Gợi ý đầm maxi cho mọi khách.",
  "new_rule": "- Với khách chiều cao dưới 1m55: gợi ý đầm ngắn (midi, knee-length), tránh maxi chạm đất.",
  "reasoning": "Shorter users need proportion-appropriate recommendations.",
  "confidence": 0.9
}
```

---

### Pattern 3: Excessive Emojis

**Trigger:** Feedback contains `emoji`, `chắp tay`, or 🙏 emoji

**Output:**

```json
{
  "rule_type": "prompt_update",
  "old_rule": "- Sử dụng nhiều emoji để tạo thiện cảm.",
  "new_rule": "- Hạn chế emoji: tối đa 1-2 emoji mỗi tin nhắn. Không dùng emoji chắp tay 🙏 trong tư vấn mua hàng.",
  "reasoning": "User反馈 quá nhiều emoji gây mất chuyên nghiệp.",
  "confidence": 0.8
}
```

---

### Default Fallback

If no patterns match, generates a generic rule:

```json
{
  "rule_type": "prompt_update",
  "new_rule": "- Phản hồi user: Cần điều chỉnh tone và content dựa trên feedback: '{feedback_text[:50]}...'",
  "reasoning": "Generic adjustment based on user feedback.",
  "confidence": 0.5
}
```

Note: `old_rule` is `None` in the fallback case.

---

## Adding New Patterns

To add a new pattern, extend the `_ai_generate_rule()` function:

```python
def _ai_generate_rule(feedback: Dict[str, Any]) -> Dict[str, Any]:
    fb_text = feedback['feedback_text'].lower()
    query = feedback['query'].lower()
    
    # Existing patterns...
    if ...:
        return {...}
    
    # NEW PATTERN: Color preference mismatch
    if any(word in fb_text for word in ["màu đỏ", "thích đỏ", "mặc đỏ"]):
        if "đỏ" not in query and "red" not in query:
            return {
                "old_rule": "- Không hỏi màu sắc ưa thích",
                "new_rule": "- Luôn hỏi user về màu sắc ưa thích trước khi gợi ý",
                "reasoning": "User has strong color preference that was ignored.",
                "confidence": 0.8
            }
    
    # Default fallback
    return {...}
```

---

## Production Upgrade: Real LLM

To replace mock AI with Claude/OpenAI:

```python
import anthropic  # or openai

client = anthropic.Anthropic(api_key=os.getenv("ANTHROPIC_API_KEY"))

async def _ai_generate_rule(feedback: Dict[str, Any]) -> Dict[str, Any]:
    prompt = f"""
    Given this user feedback, generate a prompt rule to prevent similar issues in the future.

    User Query: {feedback['query']}
    AI Response: {feedback['ai_response']}
    Feedback: {feedback['feedback_text']}

    Return JSON with:
    - old_rule: (optional) The problematic rule that may have caused this
    - new_rule: The suggested new rule (specific, actionable)
    - reasoning: Brief explanation
    - confidence: 0.0-1.0
    """
    
    response = await client.messages.create(
        model="claude-3-5-sonnet-20241022",
        max_tokens=500,
        messages=[{"role": "user", "content": prompt}]
    )
    
    # Parse JSON from response
    import json
    result = json.loads(response.content[0].text)
    return result
```

---

## Pattern Confidence Scoring

Current mock uses fixed confidence per pattern:

| Pattern | Confidence |
|---------|-----------|
| Weather mismatch | 0.85 |
| Size/fit issues | 0.90 |
| Emoji complaints | 0.80 |
| Fallback | 0.50 |

In production, LLM should provide its own confidence based on:
- Clarity of feedback
- Specificity of suggested rule
- Alignment with existing rules

---

## Rule Validation

Before inserting into database, rules should be validated:

```python
def validate_rule(rule: Dict[str, Any]) -> List[str]:
    errors = []
    
    if not rule.get('new_rule'):
        errors.append("new_rule is required")
    
    if len(rule.get('new_rule', '')) < 10:
        errors.append("new_rule is too short (min 10 chars)")
    
    if rule.get('confidence', 0) < 0 or rule.get('confidence', 0) > 1:
        errors.append("confidence must be 0.0-1.0")
    
    # Check for toxic content
    toxic_words = ['hate', 'kill', 'abuse']
    for word in toxic_words:
        if word in rule.get('new_rule', '').lower():
            errors.append("rule contains inappropriate content")
    
    return errors
```

---

## Testing the Mock AI

```python
# test_mock_ai.py
from backend.api.autotrain.autotrain_route import _ai_generate_rule

test_cases = [
    {
        "feedback": {
            "query": "tìm áo khoác cho trời 35 độ",
            "ai_response": "Mình gợi ý áo khoác nỉ...",
            "feedback_text": "Trời nóng quá, AI còn tư vấn áo khoác"
        },
        "expected_pattern": "weather"
    },
    {
        "feedback": {
            "query": "tìm đầm dự tiệc",
            "ai_response": "Đây là đầm maxi...",
            "feedback_text": "Tôi thấp quá, mặc đầm maxi chạm đất"
        },
        "expected_pattern": "size"
    }
]

for tc in test_cases:
    result = _ai_generate_rule(tc["feedback"])
    print(f"Rule: {result['new_rule'][:50]}...")
    print(f"Confidence: {result['confidence']}")
    print(f"Reasoning: {result['reasoning']}")
```

---

## Extensibility

The pattern-based approach can be extended to:

1. **Keyword boosting** - Assign weights to patterns
2. **Context-aware rules** - Consider user's past orders/height/weight
3. **Multi-language** - Support both Vietnamese and English feedback
4. **Category-specific** - Different rules for dresses vs. tops vs. accessories
5. **A/B testing** - Generate multiple rules and test which performs better

---

## Known Limitations

| Limitation | Workaround | Future Fix |
|------------|-----------|-----------|
| Only 3 patterns | Add more pattern checks | Use LLM |
| No context awareness | N/A | Include user profile in feedback analysis |
| No multi-turn memory | N/A | Store conversation history |
| Confidence is static | N/A | LLM provides dynamic confidence |
| No rule conflict detection | Manual review | Implement rule versioning & conflict checking |

---

## Future: Hybrid Approach

Production could use a hybrid:

```python
async def _ai_generate_rule(feedback):
    # 1. Try pattern match first (fast, free)
    pattern_rule = pattern_match(feedback)
    if pattern_rule and pattern_rule['confidence'] > 0.9:
        return pattern_rule
    
    # 2. Fall back to LLM for complex cases
    llm_rule = await llm_generate(feedback)
    return llm_rule
```

This reduces LLM costs while maintaining flexibility.
