# Quick Start Guide: Creating Custom DPE Runtime

This guide walks you through creating a custom MaxFrame DPE runtime image with support for Python 3.7-3.12.

## Quick Start

### Step 1: Run the Skill

Invoke the skill to generate Dockerfile:

```bash
cd /path/to/skill/scripts

python3 generate_dockerfile.py \
    --packages transformers torch pandas \
    --output-dir ./my-runtime
```

The skill will **interactively prompt** you to select Python versions.

### Step 2: Select Python Versions

You'll see:

```
Select Python Versions for Custom MaxFrame DPE Runtime
======================================================================

Available Python versions: 3.7, 3.8, 3.9, 3.10, 3.11, 3.12

Default: ALL versions (3.7-3.12)

Options:
  - Press Enter to include all versions (recommended)
  - Specify versions: e.g., '3.10,3.11,3.12'
  - Single version: e.g., '3.10'

Enter Python versions (or press Enter for all):
```

**Press Enter** for all versions, or specify your desired versions.

### Step 3: Review Generated Files

The skill creates:
- `Dockerfile` - Docker image definition
- `README.md` - Build and usage instructions
- `requirements.txt` - Package list
- `.dockerignore` - Build optimization

### Step 4: Build the Image

```bash
cd ./my-runtime
docker build -t my-custom-runtime:v1 .
```

### Step 5: Test the Image

```bash
# Test Python version
docker run --rm my-custom-runtime:v1 conda run -n py310 python --version

# Test MaxFrame SDK
docker run --rm my-custom-runtime:v1 conda run -n py310 python -c "import maxframe; print(maxframe.__version__)"

# Test user packages
docker run --rm my-custom-runtime:v1 conda run -n py310 python -c "import transformers; print(transformers.__version__)"

# List all Python environments
docker run --rm my-custom-runtime:v1 conda env list
```

### Step 6: Push to Registry

```bash
# Tag for your registry
docker tag my-custom-runtime:v1 your-registry.com/my-custom-runtime:v1

# Push
docker push your-registry.com/my-custom-runtime:v1
```

### Step 7: Use in MaxFrame

```python
from maxframe.session import new_session

session = new_session(
    odps_connection,
    image="your-registry.com/my-custom-runtime:v1"
)

# Your MaxFrame code here
```

## Common Scenarios

### Scenario 1: All Python Versions (Default)

```bash
# Interactive - just press Enter
python3 scripts/generate_dockerfile.py \
    --packages transformers torch \
    --output-dir ./my-runtime

# Or specify explicitly
python3 scripts/generate_dockerfile.py \
    --packages transformers torch \
    --python-versions 3.7,3.8,3.9,3.10,3.11,3.12 \
    --output-dir ./my-runtime
```

**Use when:**
- Need maximum compatibility
- Want flexibility for different jobs
- Image size is not a concern

### Scenario 2: Specific Python Versions

```bash
# Interactive - type "3.10,3.11,3.12"
python3 scripts/generate_dockerfile.py \
    --packages transformers torch \
    --output-dir ./my-runtime

# Or specify explicitly
python3 scripts/generate_dockerfile.py \
    --packages transformers torch \
    --python-versions 3.10,3.11,3.12 \
    --output-dir ./my-runtime
```

**Use when:**
- Only need recent Python versions
- Want smaller image size
- Have specific version requirements

### Scenario 3: Single Python Version

```bash
# Interactive - type "3.10"
python3 scripts/generate_dockerfile.py \
    --packages transformers torch \
    --output-dir ./my-runtime

# Or specify explicitly
python3 scripts/generate_dockerfile.py \
    --packages transformers torch \
    --python-versions 3.10 \
    --output-dir ./my-runtime
```

**Use when:**
- Only need one Python version
- Want minimal image size
- Production deployment with fixed version

### Scenario 4: ML/AI Development

```bash
python3 scripts/generate_dockerfile.py \
    --packages pytorch torchvision transformers datasets accelerate \
    --python-versions 3.10,3.11 \
    --image-tag my-ml-runtime:v1 \
    --output-dir ./ml-runtime
```

### Scenario 5: Data Processing

```bash
python3 scripts/generate_dockerfile.py \
    --packages dask polars pyarrow fastparquet \
    --python-versions 3.10 \
    --image-tag my-data-runtime:v1 \
    --output-dir ./data-runtime
```

## Decision Points

### When to Use All Python Versions

**Use all versions (3.7-3.12) when:**
- Building general-purpose runtime
- Don't know specific version requirements
- Want maximum flexibility
- Testing across multiple versions

**When NOT to use all versions:**
- Need minimal image size
- Specific version requirements known
- Production deployment

### When to Use Specific Versions

**Use specific versions when:**
- Know exact version requirements
- Want to reduce image size
- Production deployment
- Limited storage/bandwidth

### Image Size vs Flexibility Trade-off

| Python Versions | Image Size | Flexibility | Build Time |
|----------------|------------|-------------|------------|
| All (6 versions) | 3-5 GB | Maximum | 10-20 min |
| 3 versions | 1.5-2.5 GB | Medium | 5-10 min |
| 1 version | 0.8-1.2 GB | Minimal | 2-5 min |

**Recommendation:**
- **Development**: Use all versions for flexibility
- **Production**: Use specific versions for efficiency

## Testing Your Custom Image

### Basic Tests

```bash
# Test all Python environments
for env in py37 py38 py39 py310 py311 py312; do
    echo "Testing $env..."
    docker run --rm my-custom:v1 conda run -n $env python --version
done

# Test package in all environments
for env in py37 py38 py39 py310 py311 py312; do
    docker run --rm my-custom:v1 conda run -n $env python -c "import transformers; print('$env:', transformers.__version__)"
done
```

### Integration Test

```python
# test_maxframe.py
from maxframe.session import new_session
import maxframe.dataframe as md

session = new_session(image="my-custom:v1")

# Test basic operation
df = md.read_odps_table("test_table")
print(df.head())

session.destroy()
```

## Troubleshooting

### Issue: Build fails with conda errors

**Solution:**
1. Check package availability: `conda search -c conda-forge package-name`
2. Try without version constraints first
3. Check conda-forge channel is accessible

### Issue: Large image size

**Solution:**
1. Use fewer Python versions: `--python-versions 3.10,3.11`
2. Reduce number of packages
3. Ensure `conda clean -afy` is in Dockerfile

### Issue: Import errors in specific environment

**Solution:**
1. Verify package installed: `docker run --rm image conda list -n py310 | grep package`
2. Test import: `docker run --rm image conda run -n py310 python -c "import package"`
3. Rebuild if necessary

### Issue: Package not found in conda-forge

**Solution:**
Use pip fallback in generated Dockerfile:

```dockerfile
RUN for env in py37 py38 py39 py310 py311 py312; do \
        conda run -n $env pip install package-name; \
    done
```

## Advanced Usage

### Environment Variables

```bash
python3 scripts/generate_dockerfile.py \
    --packages transformers torch \
    --env-vars '{"TRANSFORMERS_CACHE": "/tmp/transformers", "HF_HOME": "/tmp/huggingface"}' \
    --output-dir ./my-runtime
```

### Custom Image Tag

```bash
python3 scripts/generate_dockerfile.py \
    --packages transformers \
    --image-tag my-company/maxframe-custom:v2.1 \
    --output-dir ./my-runtime
```

### Dry Run (Preview)

```bash
python3 scripts/generate_dockerfile.py \
    --packages transformers torch \
    --python-versions 3.10 \
    --dry-run
```

## Next Steps

After building your custom image:

1. **Test thoroughly** - All Python environments and packages
2. **Push to registry** - Make it available to your team
3. **Document** - Update team documentation with image details
4. **Monitor** - Track usage and performance
5. **Update** - Rebuild when new packages needed

## Getting Help

- **Generated README.md** - Detailed instructions specific to your image
- **references/base_image_details.md** - Architecture documentation
- **references/conda_best_practices.md** - Conda usage guidelines
- **MaxFrame documentation** - Official MaxFrame docs

---

**Quick Reference:**

```bash
# Generate Dockerfile (interactive)
python3 scripts/generate_dockerfile.py --packages pkg1 pkg2 --output-dir ./output

# Generate Dockerfile (explicit versions)
python3 scripts/generate_dockerfile.py --packages pkg1 pkg2 --python-versions 3.10,3.11 --output-dir ./output

# Build
docker build -t my-image:v1 ./output

# Test
docker run --rm my-image:v1 conda run -n py310 python --version

# Push
docker push my-image:v1
```