# 5. Deployment & Production

## Current State (Mock)

The current `deploy_rule()` endpoint only **logs** the deployment to the database. It does **not** actually update production files.

```python
# Mock deployment - just logging
cur.execute("""
    INSERT INTO deployments (rule_id, deployed_by, version_before, version_after)
    VALUES (?, ?, ?, ?)
""", (rule_id, deployed_by, rule_dict['old_rule'][:100], rule_dict['new_rule'][:100]))

cur.execute("UPDATE generated_rules SET status = 'deployed' WHERE rule_id = ?", (rule_id,))
```

**File:** `backend/api/autotrain/autotrain_route.py:466-477`

---

## Production Implementation

To make deployment real, you need to:

### 1. Define Target Files

The `module_name` field in `generated_rules` determines which file to update:

| module_name | Target File | Format |
|-------------|-------------|--------|
| `fashion_rules.json` | `backend/agent/fashion_rules.json` | JSON array of rules |
| `prompt_v2.txt` | `backend/agent/prompt_v2.txt` | Plain text prompt |
| `stylist_guidelines.md` | `backend/agent/stylist_guidelines.md` | Markdown |

### 2. Implement File Update Logic

```python
import json
import shutil
from datetime import datetime
from pathlib import Path

CONFIG_DIR = Path(__file__).parent.parent.parent / "agent"

async def deploy_rule(rule_id: str):
    """Production deployment: copy rule to config file."""
    conn = get_db()
    cur = conn.cursor()
    
    # Get approved rule
    cur.execute("SELECT * FROM generated_rules WHERE rule_id = ? AND status = 'approved'", (rule_id,))
    rule = cur.fetchone()
    
    if not rule:
        raise HTTPException(400, "Rule not found or not approved")
    
    rule_dict = dict(rule)
    target_path = CONFIG_DIR / rule_dict['module_name']
    
    # Load existing content
    if target_path.suffix == '.json':
        with open(target_path, 'r', encoding='utf-8') as f:
            content = json.load(f)
        
        # Append new rule
        if isinstance(content, list):
            content.append({
                "rule": rule_dict['new_rule'],
                "added_from": rule_dict['feedback_id'],
                "confidence": rule_dict['confidence'],
                "deployed_at": datetime.now().isoformat()
            })
        elif isinstance(content, dict) and 'rules' in content:
            content['rules'].append({
                "rule": rule_dict['new_rule'],
                "added_from": rule_dict['feedback_id'],
                "confidence": rule_dict['confidence'],
                "deployed_at": datetime.now().isoformat()
            })
        
        # Backup before write
        backup_path = target_path.with_suffix(f'.backup.{datetime.now().strftime("%Y%m%d_%H%M%S")}.json')
        shutil.copy2(target_path, backup_path)
        
        # Write updated
        with open(target_path, 'w', encoding='utf-8') as f:
            json.dump(content, f, indent=2, ensure_ascii=False)
    
    else:  # .txt or .md
        # Read existing
        with open(target_path, 'r', encoding='utf-8') as f:
            existing = f.read()
        
        # Append new rule section
        backup_path = target_path.with_suffix(f'.backup.{datetime.now().strftime("%Y%m%d_%H%M%S")}')
        shutil.copy2(target_path, backup_path)
        
        with open(target_path, 'a', encoding='utf-8') as f:
            f.write(f"\n\n# Auto-Train Rule: {rule_id}\n")
            f.write(f"- Added from feedback: {rule_dict['feedback_id']}\n")
            f.write(f"- Confidence: {rule_dict['confidence']}\n")
            f.write(f"- Deployed at: {datetime.now().isoformat()}\n\n")
            f.write(rule_dict['new_rule'])
    
    # Log deployment
    cur.execute("""
        INSERT INTO deployments (rule_id, deployed_by, version_before, version_after, rollback_of)
        VALUES (?, ?, ?, ?, ?)
    """, (
        rule_id,
        deployed_by or "admin",
        rule_dict['old_rule'][:200] if rule_dict['old_rule'] else None,
        rule_dict['new_rule'][:200],
        None
    ))
    
    cur.execute("UPDATE generated_rules SET status = 'deployed' WHERE rule_id = ?", (rule_id,))
    conn.commit()
    conn.close()
    
    # TODO: Hot-reload chatbot config if supported
    # await reload_chatbot_config()
    
    return {"status": "deployed", "rule_id": rule_id, "target_file": str(target_path)}
```

---

## Hot Reload

After deployment, the chatbot needs to pick up the new rules. Options:

### Option A: Polling (Current)
Chatbot re-reads config file on each query (simple, no restart needed).

**Pros:** Instant effect, zero downtime  
**Cons:** Small I/O overhead per query

### Option B: Signal-based Reload
Send SIGHUP or Unix socket signal to reload config.

```python
import os
os.kill(chatbot_pid, signal.SIGHUP)
```

**Pros:** Cleaner, no repeated file reads  
**Cons:** Requires tracking PID, more complex

### Option C: In-memory Config Service
Store rules in Redis/memory, chatbot subscribes to change notifications.

```python
import redis
r = redis.Redis()
r.publish('config_update', json.dumps({'rule_id': rule_id}))
```

**Pros:** Instant propagation, no file I/O  
**Cons:** Adds Redis dependency

---

## Rollback Strategy

### Manual Rollback

```python
# 1. Find problematic deployment
SELECT * FROM deployments WHERE rule_id = 'RUL-1001';

# 2. Restore backup file
shutil.copy2(backup_path, target_path)

# 3. Mark rule as rejected (or create rollback record)
cur.execute("""
    INSERT INTO deployments (rule_id, deployed_by, rollback_of, version_before, version_after)
    VALUES (?, ?, ?, ?, ?)
""", (rollback_id, "admin", original_deployment_id, new_rule, old_rule))
```

### Automatic Rollback

If chatbot error rate increases > X% within Y minutes after deployment:

```python
if error_rate_spike():
    # Auto-rollback latest deployment
    latest = get_latest_deployment()
    restore_backup(latest['backup_path'])
    create_rollback_record(latest['id'])
    alert_admin(f"Auto-rollback: {latest['rule_id']}")
```

---

## Backup Strategy

Automated backups:

```python
# Daily backup of all config files
BACKUP_DIR = Path("/backups/autotrain")
for file in ['fashion_rules.json', 'prompt_v2.txt']:
    src = CONFIG_DIR / file
    dst = BACKUP_DIR / f"{file}.{datetime.now().strftime('%Y%m%d')}"
    shutil.copy2(src, dst)

# Keep last 30 days
# Compress older backups
```

---

## Monitoring Deployments

Add metrics:

```python
# Prometheus metrics
DEPLOYMENTS_TOTAL = Counter('autotrain_deployments_total', 'Total deployments')
DEPLOYMENT_DURATION = Histogram('autotrain_deployment_seconds', 'Deployment duration')
DEPLOYMENT_ERRORS = Counter('autotrain_deployment_errors_total', 'Deployment failures')

@DEPLOYMENT_DURATION.time()
def deploy():
    try:
        # deployment logic
        DEPLOYMENTS_TOTAL.inc()
    except Exception as e:
        DEPLOYMENT_ERRORS.inc()
        raise
```

Dashboard should show:
- Deployments per hour/day
- Success vs failure rate
- Time from approval to deployment
- Rollback frequency

---

## A/B Testing Deployment

Instead of deploying to 100% immediately:

```python
# Deploy to 10% of users first
SAMPLE_SIZE = 0.10

# Store variant in database with rollout percentage
cur.execute("""
    INSERT INTO rule_variants (rule_id, rollout_percentage, variant_type)
    VALUES (?, ?, ?)
""", (rule_id, 10, 'new'))

# Chatbot checks variant before applying rule
def should_use_new_rule(user_id, rule_id):
    rollout = get_rollout(rule_id)
    hash_val = hash(f"{user_id}:{rule_id}") % 100
    return hash_val < rollout['rollout_percentage']
```

---

## Safety Checklist Before Deploy

- [ ] Rule has been reviewed and approved by at least 1 admin
- [ ] Confidence score > 0.7 (or human override)
- [ ] No conflicts with existing deployed rules (run conflict detection)
- [ ] Backup of target file exists
- [ ] Rule syntax validated (can be parsed by chatbot)
- [ ] No toxic/inappropriate content (run profanity filter)
- [ ] Test in staging environment first (if available)
- [ ] Notify team in #deployments channel

---

## Rollback Runbook

### If deployment causes issues:

1. **Immediate:** `curl -X POST /autotrain/rollback/{rule_id}` (need to implement)
2. **Manual:** Copy `.backup` file back to original location
3. **Verify:** Check chatbot logs for error reduction
4. **Post-mortem:** Analyze why rule was bad, improve AI/validation

---

## File Structure in Production

```
backend/
├── agent/
│   ├── fashion_rules.json          # Main rules file
│   ├── fashion_rules.json.20240520_143022.backup  # Auto-backup
│   ├── prompt_v2.txt               # System prompt
│   ├── prompt_v2.txt.20240520_143022.backup
│   └── stylist_guidelines.md
├── api/
│   └── autotrain/
│       └── autotrain_route.py      # Deployment logic
└── database/
    └── autotrain.db                # SQLite tracking
```

---

## Next Steps

- [ ] Implement actual file deployment in `deploy_rule()`
- [ ] Add hot-reload mechanism
- [ ] Create admin UI for deployments
- [ ] Add backup/restore endpoints
- [ ] Implement rollback endpoint
- [ ] Add deployment metrics to dashboard
- [ ] Set up alerting for failed deployments
