# Usage Scenarios for Model Manager Skill

This document provides detailed examples and guidance for using the Model Manager skill in different scenarios.

## Scenario 1: List All Available Models

**Purpose**: Get a comprehensive list of all trained models in the system with their basic performance metrics.

**Parameters**:
- No required parameters
- Optional: `--sort_by` (default: training_date), `--limit`

**Example Usage**:

```bash
python list_models.py
```

**Output**: A table showing all models with columns for model ID, station name, training date, R², RMSE, MAE, and MAPE.

**Common Use Cases**:
- Getting an overview of all available models
- Checking how many models have been trained
- Finding the latest trained model

---

## Scenario 2: List Models for a Specific Station

**Purpose**: Filter models by station name to see all models trained for a particular solar power station.

**Parameters**:
- `--station_name`: The station name to filter by (required for this scenario)
- Optional: `--sort_by`, `--limit`

**Example Usage**:

```bash
python list_models.py --station_name P2113
```

**Output**: A filtered table showing only models trained for station P2113, sorted by training date (newest first).

**Common Use Cases**:
- Checking all models available for a specific station
- Comparing different versions of models for the same station
- Finding the best performing model for a station

---

## Scenario 3: List Models Sorted by Performance

**Purpose**: Sort models by performance metrics to identify the best performing models.

**Parameters**:
- `--sort_by`: The metric to sort by (r2, rmse, mae, mape)
- Optional: `--station_name`, `--limit`

**Example Usage**:

```bash
# Sort by R² (highest first)
python list_models.py --sort_by r2 --limit 10

# Sort by RMSE (lowest first)
python list_models.py --sort_by rmse --limit 5
```

**Output**: Models sorted by the specified metric, showing the best performing models first.

**Common Use Cases**:
- Finding the best model overall
- Identifying models with lowest prediction errors
- Comparing model performance across different metrics

---

## Scenario 4: Get Detailed Information About a Specific Model

**Purpose**: Retrieve comprehensive information about a specific model, including architecture, training parameters, and detailed performance metrics.

**Parameters**:
- `--model_id`: The model identifier or filename (required)
- Optional: `--model_path`

**Example Usage**:

```bash
python get_model_details.py --model_id model_20250326_P2113_v1
```

**Output**: Detailed information including:
- Basic info (model ID, filename, station, training date, size)
- Training data range
- Model architecture details
- Performance metrics (R², RMSE, MAE, MAPE, daytime MAPE)
- Training information (epochs, batch size, learning rate, etc.)

**Common Use Cases**:
- Understanding model architecture and configuration
- Checking detailed performance metrics
- Verifying training parameters
- Comparing model specifications

---

## Scenario 5: Compare Multiple Models

**Purpose**: Compare performance metrics across multiple models to identify the best one for a specific use case.

**Parameters**:
- `--model_ids`: Comma-separated list of model IDs (required)
- Optional: `--metrics` (specific metrics to compare)

**Example Usage**:

```bash
# Compare all metrics for three models
python compare_models.py --model_ids model_v1,model_v2,model_v3

# Compare only R² and RMSE for two models
python compare_models.py --model_ids model_v1,model_v2 --metrics r2,rmse
```

**Output**: A comparison table showing:
- Model ID, station, and training date
- Selected performance metrics
- Ranking by R² (highest first)

**Common Use Cases**:
- Choosing between different model versions
- Evaluating impact of training data changes
- Comparing models trained at different times
- Selecting the best model for production

---

## Scenario 6: Find Best Model for a Station

**Purpose**: Identify the best performing model for a specific station.

**Parameters**:
- `--station_name`: The station name
- `--sort_by`: Set to "r2" for best performance

**Example Usage**:

```bash
python list_models.py --station_name P2113 --sort_by r2 --limit 1
```

**Output**: The single best performing model for station P2113.

**Common Use Cases**:
- Selecting a model for production deployment
- Finding the optimal model for a specific station
- Verifying model performance before deployment

---

## Performance Metrics Reference

### R² (Coefficient of Determination)
- **Range**: 0 to 1
- **Interpretation**: Measures how well the model explains variance in the data
- **Better**: Higher values (closer to 1) are better
- **Typical Range**: 0.7-0.95 for solar power forecasting

### RMSE (Root Mean Square Error)
- **Unit**: Same as the target variable (typically kW or MW)
- **Interpretation**: Average prediction error magnitude
- **Better**: Lower values are better
- **Use Case**: Penalizes larger errors more heavily

### MAE (Mean Absolute Error)
- **Unit**: Same as the target variable
- **Interpretation**: Average absolute difference between predicted and actual values
- **Better**: Lower values are better
- **Use Case**: More robust to outliers than RMSE

### MAPE (Mean Absolute Percentage Error)
- **Unit**: Percentage (%)
- **Interpretation**: Average percentage error relative to actual values
- **Better**: Lower values are better
- **Use Case**: Useful for comparing models across different scales

### Daytime MAPE
- **Unit**: Percentage (%)
- **Interpretation**: MAPE calculated only for daytime hours (when solar generation is significant)
- **Better**: Lower values are better
- **Use Case**: More relevant for solar power forecasting since nighttime has zero generation

---

## Tips and Best Practices

### Selecting a Model

1. **For General Use**: Sort by R² to find the overall best performing model
2. **For Error-Sensitive Applications**: Sort by RMSE or MAE
3. **For Percentage-Based Comparison**: Sort by MAPE
4. **For Solar-Specific Use**: Consider daytime MAPE as it's more relevant

### Interpreting Results

1. **High R² but High RMSE**: Model fits well but has large absolute errors
2. **Low MAPE but High RMSE**: Good percentage accuracy but large absolute errors
3. **Consistent Metrics**: All metrics pointing to the same model indicates a robust choice

### Model Selection Workflow

1. List all models for your station
2. Sort by R² to identify top candidates
3. Get detailed information on top 2-3 models
4. Compare them using compare_models.py
5. Check daytime MAPE for solar-specific performance
6. Select based on your specific requirements

---

## Common Queries and Solutions

### Q: How do I find the latest model?
**A**: Use `list_models.py --sort_by training_date` (default sorting)

### Q: How do I find the best model overall?
**A**: Use `list_models.py --sort_by r2 --limit 1`

### Q: How do I compare models from different stations?
**A**: Use `compare_models.py --model_ids model1,model2,model3` (no station filtering)

### Q: How do I check if a model is suitable for production?
**A**: Use `get_model_details.py --model_id <model_id>` and check:
- R² > 0.85
- MAPE < 15%
- Daytime MAPE < 10%

### Q: How do I track model improvements over time?
**A**: Use `list_models.py --station_name <station> --sort_by training_date` to see historical progression

---

## Error Handling

### Connection Error
- **Message**: "无法连接到 API 服务器"
- **Solution**: Verify the API server is running at http://47.96.16.84:5001
- **Check**: `curl http://47.96.16.84:5001/api/unified/models/list`

### Timeout Error
- **Message**: "请求超时（30秒）"
- **Solution**: The API server is slow or overloaded
- **Action**: Wait a moment and retry

### Model Not Found
- **Message**: "未找到匹配的模型"
- **Solution**: Verify the model_id is correct
- **Action**: Use `list_models.py` to see available models

### Invalid Parameters
- **Message**: "缺少必需参数" or similar
- **Solution**: Check that all required parameters are provided
- **Action**: Refer to the usage examples above

---

## Integration with Other Skills

The Model Manager skill can be used in conjunction with:

1. **PV Power Forecast Skill**: Use model information to select the best model for predictions
2. **Report Generator Skill**: Include model performance metrics in reports
3. **Energy Suggestion Executor**: Use model performance to justify recommendations

---

## Future Enhancements

Planned features for future versions:

1. Model export functionality
2. Model versioning and rollback
3. Automated model selection based on criteria
4. Model performance trending and alerts
5. Integration with model training workflows