4.6 KiB

Raw Blame History

OpenAI Integration Plan for Tyndale AI Service

Summary

Replace the LocalAdapter stub with an OpenAI API client, using FastAPI's Depends() for dependency injection and keeping configuration minimal (API key + model only).

Files to Modify

File	Action
`requirements.txt`	Add `openai>=1.0.0`
`app/config.py`	Add `openai_api_key`, `openai_model` settings
`app/llm/exceptions.py`	Create: Custom exception hierarchy
`app/llm/adapter.py`	Add `OpenAIAdapter` class, refactor `get_adapter()` for DI
`app/llm/__init__.py`	Export new components
`app/schemas.py`	Add `"openai"` to mode literal
`app/main.py`	Use `Depends()` for adapter injection
`.env.example`	Add OpenAI env vars
`Dockerfile`	Add `OPENAI_MODEL` default env

Implementation Steps

1. Add OpenAI dependency

File: requirements.txt

openai>=1.0.0

2. Extend configuration

File: app/config.py

Add llm_mode: Literal["local", "remote", "openai"]
Add openai_api_key: str = ""
Add openai_model: str = "gpt-4o-mini"

3. Create exception hierarchy

File: app/llm/exceptions.py (new)

LLMError (base)
LLMAuthenticationError (401)
LLMRateLimitError (429)
LLMConnectionError (503)
LLMConfigurationError (500)
llm_exception_to_http() helper

4. Add OpenAIAdapter class

File: app/llm/adapter.py

class OpenAIAdapter(LLMAdapter):
    def __init__(self, api_key: str, model: str = "gpt-4o-mini"):
        self.client = OpenAI(api_key=api_key)
        self.model = model

    async def generate(self, conversation_id: str, message: str) -> str:
        # Use asyncio.to_thread() since OpenAI SDK is synchronous
        response = await asyncio.to_thread(
            self.client.chat.completions.create,
            model=self.model,
            messages=[{"role": "user", "content": message}],
        )
        return response.choices[0].message.content

5. Refactor get_adapter() for dependency injection

File: app/llm/adapter.py

from functools import lru_cache
from typing import Annotated
from fastapi import Depends

@lru_cache()
def get_settings() -> Settings:
    return settings

def get_adapter(settings: Annotated[Settings, Depends(get_settings)]) -> LLMAdapter:
    if settings.llm_mode == "openai":
        return OpenAIAdapter(api_key=settings.openai_api_key, model=settings.openai_model)
    if settings.llm_mode == "remote":
        return RemoteAdapter(url=settings.llm_remote_url, token=settings.llm_remote_token)
    return LocalAdapter()

# Type alias for clean injection
AdapterDependency = Annotated[LLMAdapter, Depends(get_adapter)]

6. Update /chat endpoint

File: app/main.py

from app.llm.adapter import AdapterDependency
from app.llm.exceptions import LLMError, llm_exception_to_http

@app.post("/chat", response_model=ChatResponse)
async def chat(request: ChatRequest, adapter: AdapterDependency) -> ChatResponse:
    # ... validation ...
    try:
        response_text = await adapter.generate(conversation_id, request.message)
    except LLMError as e:
        raise llm_exception_to_http(e)
    # ... return response ...

7. Update mode type

File: app/schemas.py

Change mode: Literal["local", "remote"] to mode: Literal["local", "remote", "openai"]

8. Update environment examples

File: .env.example

LLM_MODE=openai
OPENAI_API_KEY=sk-your-key-here
OPENAI_MODEL=gpt-4o-mini

9. Update Dockerfile

File: Dockerfile

Add ENV OPENAI_MODEL=gpt-4o-mini (API key passed at runtime for security)

Error Handling

OpenAI Exception	Custom Exception	HTTP Status
`AuthenticationError`	`LLMAuthenticationError`	401
`RateLimitError`	`LLMRateLimitError`	429
`APIConnectionError`	`LLMConnectionError`	503
`APIError`	`LLMError`	varies

Verification Steps

Install dependencies:
```
pip install -r requirements.txt
```

Run locally with OpenAI:

export LLM_MODE=openai
export OPENAI_API_KEY=sk-your-key
uvicorn app.main:app --reload

Test the /chat endpoint:

curl -X POST http://localhost:8000/chat \
  -H "Content-Type: application/json" \
  -d '{"message": "Hello, what is 2+2?"}'

Verify response contains OpenAI response and mode="openai"

Test Docker build:

docker build -t tyndale-ai .
docker run -p 8080:8080 -e LLM_MODE=openai -e OPENAI_API_KEY=sk-your-key tyndale-ai

Test error handling (optional): Use invalid API key to verify 401 response

4.6 KiB Raw Blame History