🤖 Auto-Train System

1. Overview

Auto-Train là hệ thống tự động cập nhật prompt rules từ user feedback. Đây là pipeline end-to-end:

2. Architecture Flow

Diagram not found. Please regenerate with generate_autotrain_diagram.py

'">

3. Database Schema

feedbacks table

generated_rules table

deployments table

4. API Endpoints

5. Mock AI Rule Generation

Current implementation uses pattern matching on feedback text:

Python Implementation

def _ai_generate_rule(feedback: Dict[str, Any]) -> Dict[str, Any]:
    fb_text = feedback['feedback_text'].lower()
    query = feedback['query'].lower()

    # Pattern: Weather mismatch
    if "nóng" in fb_text and "áo khoác" in query:
        return {
            "old_rule": "- Luôn ưu tiên áo khoác/dày",
            "new_rule": "- Khi trời nóng (>30°): KHÔNG gợi ý áo khoác dày",
            "reasoning": "Weather mismatch correction",
            "confidence": 0.85
        }

    # Pattern: Size/fit issues
    if any(word in fb_text for word in ["dáng lùn", "thấp"]):
        return {
            "old_rule": "- Gợi ý đầm maxi cho mọi khách",
            "new_rule": "- Dưới 1m55: gợi ý đầm ngắn (midi, knee-length)",
            "reasoning": "Proportion-appropriate recommendations",
            "confidence": 0.9
        }

    return {
        "new_rule": f"- Adjust based on: {feedback['feedback_text'][:50]}...",
        "reasoning": "Generic adjustment",
        "confidence": 0.5
    }

6. Deployment Flow (Production)

Currently deploy_rule() only logs to DB. Production implementation would:

1. Fetch the approved rule from DB
2. Load target file (fashion_rules.json or prompt_v2.txt)
3. Apply the rule update (append or replace)
4. Write back with backup
5. Reload chatbot configuration (hot-reload if supported)
6. Log deployment in deployments table

Production Deploy Code Template

async def deploy_rule(rule_id: str):
    rule = await get_rule(rule_id)
    target_path = CONFIG_DIR / rule['module_name']

    # Load existing rules
    content = json.load(open(target_path))

    # Apply update
    content['rules'].append({
        "rule": rule['new_rule'],
        "added_from": rule['feedback_id'],
        "confidence": rule['confidence'],
        "deployed_at": datetime.now().isoformat()
    })

    # Backup and write
    backup(target_path)
    json.dump(content, open(target_path, 'w'), indent=2)

    # Hot-reload chatbot config
    await chatbot_reload()

    await log_deployment(rule_id)
    return {"status": "deployed"}

7. Testing Checklist

• Submit feedback → DB stores correctly
• Generate rule → Mock AI produces relevant rule
• Approve rule → Status changes to 'approved'
• Deploy → Status changes to 'deployed', log created
• run-all → Processes all pending feedbacks
• Concurrent deploy → No race conditions
• Invalid rule_id → 404 error response

8. Extending the System

Add new AI pattern

# In _ai_generate_rule():
if "pattern_keyword" in fb_text:
    return {
        "rule_type": "prompt_update",
        "new_rule": "- Your rule here",
        "reasoning": "Your reasoning",
        "confidence": 0.85
    }

Add new module type

ALLOWED_MODULES = [
    "fashion_rules.json",
    "prompt_v2.txt",
    "stylist_guidelines.md"
]

Hook to real LLM

async def _ai_generate_rule(feedback):
    prompt = f"""Given user feedback:
    Query: {feedback['query']}
    AI Response: {feedback['ai_response']}
    Feedback: {feedback['feedback_text']}

    Suggest a prompt rule to prevent this issue:"""

    resp = await anthropic_client.messages.create(
        model="claude-3-5-sonnet-20241022",
        max_tokens=1000,
        messages=[{"role": "user", "content": prompt}]
    )
    return parse_llm_response(resp.content[0].text)

9. Monitoring & Alerting

10. Security Considerations

• Authentication: Add JWT auth to all endpoints (currently open)
• Authorization: Only admin can approve/deploy
• Input validation: Pydantic validates; add profanity filter for feedback_text
• Rate limiting: Limit to 10 feedbacks per user/hour
• Audit log: Track who deployed what (already in deployments table)

11. Future Enhancements

• A/B Testing: Deploy rules to 10% of users first
• Rollback: One-click revert to previous rule version
• Auto-approve: Confidence threshold > 0.9 auto-approves
• Feedback Clustering: Group similar feedback before generation
• Webhook Notifications: Notify admin on new rule suggestion
• Version Control: Store rule history in Git

12. Quick Start

Start server

Submit feedback

Generate rule

Approve rule

Deploy rule

Check status