165 lines
4.6 KiB
Markdown
165 lines
4.6 KiB
Markdown
# OpenAI Integration Plan for Tyndale AI Service
|
|
|
|
## Summary
|
|
Replace the LocalAdapter stub with an OpenAI API client, using FastAPI's `Depends()` for dependency injection and keeping configuration minimal (API key + model only).
|
|
|
|
---
|
|
|
|
## Files to Modify
|
|
|
|
| File | Action |
|
|
|------|--------|
|
|
| `requirements.txt` | Add `openai>=1.0.0` |
|
|
| `app/config.py` | Add `openai_api_key`, `openai_model` settings |
|
|
| `app/llm/exceptions.py` | **Create**: Custom exception hierarchy |
|
|
| `app/llm/adapter.py` | Add `OpenAIAdapter` class, refactor `get_adapter()` for DI |
|
|
| `app/llm/__init__.py` | Export new components |
|
|
| `app/schemas.py` | Add `"openai"` to mode literal |
|
|
| `app/main.py` | Use `Depends()` for adapter injection |
|
|
| `.env.example` | Add OpenAI env vars |
|
|
| `Dockerfile` | Add `OPENAI_MODEL` default env |
|
|
|
|
---
|
|
|
|
## Implementation Steps
|
|
|
|
### 1. Add OpenAI dependency
|
|
**File**: `requirements.txt`
|
|
```
|
|
openai>=1.0.0
|
|
```
|
|
|
|
### 2. Extend configuration
|
|
**File**: `app/config.py`
|
|
- Add `llm_mode: Literal["local", "remote", "openai"]`
|
|
- Add `openai_api_key: str = ""`
|
|
- Add `openai_model: str = "gpt-4o-mini"`
|
|
|
|
### 3. Create exception hierarchy
|
|
**File**: `app/llm/exceptions.py` (new)
|
|
- `LLMError` (base)
|
|
- `LLMAuthenticationError` (401)
|
|
- `LLMRateLimitError` (429)
|
|
- `LLMConnectionError` (503)
|
|
- `LLMConfigurationError` (500)
|
|
- `llm_exception_to_http()` helper
|
|
|
|
### 4. Add OpenAIAdapter class
|
|
**File**: `app/llm/adapter.py`
|
|
|
|
```python
|
|
class OpenAIAdapter(LLMAdapter):
|
|
def __init__(self, api_key: str, model: str = "gpt-4o-mini"):
|
|
self.client = OpenAI(api_key=api_key)
|
|
self.model = model
|
|
|
|
async def generate(self, conversation_id: str, message: str) -> str:
|
|
# Use asyncio.to_thread() since OpenAI SDK is synchronous
|
|
response = await asyncio.to_thread(
|
|
self.client.chat.completions.create,
|
|
model=self.model,
|
|
messages=[{"role": "user", "content": message}],
|
|
)
|
|
return response.choices[0].message.content
|
|
```
|
|
|
|
### 5. Refactor get_adapter() for dependency injection
|
|
**File**: `app/llm/adapter.py`
|
|
|
|
```python
|
|
from functools import lru_cache
|
|
from typing import Annotated
|
|
from fastapi import Depends
|
|
|
|
@lru_cache()
|
|
def get_settings() -> Settings:
|
|
return settings
|
|
|
|
def get_adapter(settings: Annotated[Settings, Depends(get_settings)]) -> LLMAdapter:
|
|
if settings.llm_mode == "openai":
|
|
return OpenAIAdapter(api_key=settings.openai_api_key, model=settings.openai_model)
|
|
if settings.llm_mode == "remote":
|
|
return RemoteAdapter(url=settings.llm_remote_url, token=settings.llm_remote_token)
|
|
return LocalAdapter()
|
|
|
|
# Type alias for clean injection
|
|
AdapterDependency = Annotated[LLMAdapter, Depends(get_adapter)]
|
|
```
|
|
|
|
### 6. Update /chat endpoint
|
|
**File**: `app/main.py`
|
|
|
|
```python
|
|
from app.llm.adapter import AdapterDependency
|
|
from app.llm.exceptions import LLMError, llm_exception_to_http
|
|
|
|
@app.post("/chat", response_model=ChatResponse)
|
|
async def chat(request: ChatRequest, adapter: AdapterDependency) -> ChatResponse:
|
|
# ... validation ...
|
|
try:
|
|
response_text = await adapter.generate(conversation_id, request.message)
|
|
except LLMError as e:
|
|
raise llm_exception_to_http(e)
|
|
# ... return response ...
|
|
```
|
|
|
|
### 7. Update mode type
|
|
**File**: `app/schemas.py`
|
|
- Change `mode: Literal["local", "remote"]` to `mode: Literal["local", "remote", "openai"]`
|
|
|
|
### 8. Update environment examples
|
|
**File**: `.env.example`
|
|
```
|
|
LLM_MODE=openai
|
|
OPENAI_API_KEY=sk-your-key-here
|
|
OPENAI_MODEL=gpt-4o-mini
|
|
```
|
|
|
|
### 9. Update Dockerfile
|
|
**File**: `Dockerfile`
|
|
- Add `ENV OPENAI_MODEL=gpt-4o-mini` (API key passed at runtime for security)
|
|
|
|
---
|
|
|
|
## Error Handling
|
|
|
|
| OpenAI Exception | Custom Exception | HTTP Status |
|
|
|------------------|------------------|-------------|
|
|
| `AuthenticationError` | `LLMAuthenticationError` | 401 |
|
|
| `RateLimitError` | `LLMRateLimitError` | 429 |
|
|
| `APIConnectionError` | `LLMConnectionError` | 503 |
|
|
| `APIError` | `LLMError` | varies |
|
|
|
|
---
|
|
|
|
## Verification Steps
|
|
|
|
1. **Install dependencies**:
|
|
```bash
|
|
pip install -r requirements.txt
|
|
```
|
|
|
|
2. **Run locally with OpenAI**:
|
|
```bash
|
|
export LLM_MODE=openai
|
|
export OPENAI_API_KEY=sk-your-key
|
|
uvicorn app.main:app --reload
|
|
```
|
|
|
|
3. **Test the /chat endpoint**:
|
|
```bash
|
|
curl -X POST http://localhost:8000/chat \
|
|
-H "Content-Type: application/json" \
|
|
-d '{"message": "Hello, what is 2+2?"}'
|
|
```
|
|
|
|
4. **Verify response contains OpenAI response and mode="openai"**
|
|
|
|
5. **Test Docker build**:
|
|
```bash
|
|
docker build -t tyndale-ai .
|
|
docker run -p 8080:8080 -e LLM_MODE=openai -e OPENAI_API_KEY=sk-your-key tyndale-ai
|
|
```
|
|
|
|
6. **Test error handling** (optional): Use invalid API key to verify 401 response
|