feat: add AskSage LLM provider integration
Implement AskSageAdapter using the official asksageclient SDK to support AskSage as an LLM provider option. This enables users to leverage AskSage's API with configurable email, API key, and model settings. - Add AskSageAdapter class with async support via thread pool - Update Settings to include asksage_email, asksage_api_key, asksage_model - Extend llm_mode literal to include "asksage" option - Update dependency injection to instantiate AskSageAdapter when configured - Remove completed OPENAI_INTEGRATION_PLAN.md - Update requirements.txt with full dependency list including asksageclient
This commit is contained in:
Danny
parent
ad5f8ef798
commit
4c084d7668
|
|
@ -1,4 +1,4 @@
|
|||
# LLM Mode: "local", "remote", or "openai"
|
||||
# LLM Mode: "local", "remote", "openai", or "asksage"
|
||||
LLM_MODE=local
|
||||
|
||||
# Remote LLM Configuration (required if LLM_MODE=remote)
|
||||
|
|
@ -8,3 +8,8 @@ LLM_REMOTE_TOKEN=
|
|||
# OpenAI Configuration (required if LLM_MODE=openai)
|
||||
OPENAI_API_KEY=sk-your-api-key-here
|
||||
OPENAI_MODEL=gpt-4o-mini
|
||||
|
||||
# AskSage Configuration (required if LLM_MODE=asksage)
|
||||
ASKSAGE_EMAIL=your.email@example.com
|
||||
ASKSAGE_API_KEY=your-api-key-here
|
||||
ASKSAGE_MODEL=gpt-4o
|
||||
|
|
|
|||
|
|
@ -1,164 +0,0 @@
|
|||
# OpenAI Integration Plan for Tyndale AI Service
|
||||
|
||||
## Summary
|
||||
Replace the LocalAdapter stub with an OpenAI API client, using FastAPI's `Depends()` for dependency injection and keeping configuration minimal (API key + model only).
|
||||
|
||||
---
|
||||
|
||||
## Files to Modify
|
||||
|
||||
| File | Action |
|
||||
|------|--------|
|
||||
| `requirements.txt` | Add `openai>=1.0.0` |
|
||||
| `app/config.py` | Add `openai_api_key`, `openai_model` settings |
|
||||
| `app/llm/exceptions.py` | **Create**: Custom exception hierarchy |
|
||||
| `app/llm/adapter.py` | Add `OpenAIAdapter` class, refactor `get_adapter()` for DI |
|
||||
| `app/llm/__init__.py` | Export new components |
|
||||
| `app/schemas.py` | Add `"openai"` to mode literal |
|
||||
| `app/main.py` | Use `Depends()` for adapter injection |
|
||||
| `.env.example` | Add OpenAI env vars |
|
||||
| `Dockerfile` | Add `OPENAI_MODEL` default env |
|
||||
|
||||
---
|
||||
|
||||
## Implementation Steps
|
||||
|
||||
### 1. Add OpenAI dependency
|
||||
**File**: `requirements.txt`
|
||||
```
|
||||
openai>=1.0.0
|
||||
```
|
||||
|
||||
### 2. Extend configuration
|
||||
**File**: `app/config.py`
|
||||
- Add `llm_mode: Literal["local", "remote", "openai"]`
|
||||
- Add `openai_api_key: str = ""`
|
||||
- Add `openai_model: str = "gpt-4o-mini"`
|
||||
|
||||
### 3. Create exception hierarchy
|
||||
**File**: `app/llm/exceptions.py` (new)
|
||||
- `LLMError` (base)
|
||||
- `LLMAuthenticationError` (401)
|
||||
- `LLMRateLimitError` (429)
|
||||
- `LLMConnectionError` (503)
|
||||
- `LLMConfigurationError` (500)
|
||||
- `llm_exception_to_http()` helper
|
||||
|
||||
### 4. Add OpenAIAdapter class
|
||||
**File**: `app/llm/adapter.py`
|
||||
|
||||
```python
|
||||
class OpenAIAdapter(LLMAdapter):
|
||||
def __init__(self, api_key: str, model: str = "gpt-4o-mini"):
|
||||
self.client = OpenAI(api_key=api_key)
|
||||
self.model = model
|
||||
|
||||
async def generate(self, conversation_id: str, message: str) -> str:
|
||||
# Use asyncio.to_thread() since OpenAI SDK is synchronous
|
||||
response = await asyncio.to_thread(
|
||||
self.client.chat.completions.create,
|
||||
model=self.model,
|
||||
messages=[{"role": "user", "content": message}],
|
||||
)
|
||||
return response.choices[0].message.content
|
||||
```
|
||||
|
||||
### 5. Refactor get_adapter() for dependency injection
|
||||
**File**: `app/llm/adapter.py`
|
||||
|
||||
```python
|
||||
from functools import lru_cache
|
||||
from typing import Annotated
|
||||
from fastapi import Depends
|
||||
|
||||
@lru_cache()
|
||||
def get_settings() -> Settings:
|
||||
return settings
|
||||
|
||||
def get_adapter(settings: Annotated[Settings, Depends(get_settings)]) -> LLMAdapter:
|
||||
if settings.llm_mode == "openai":
|
||||
return OpenAIAdapter(api_key=settings.openai_api_key, model=settings.openai_model)
|
||||
if settings.llm_mode == "remote":
|
||||
return RemoteAdapter(url=settings.llm_remote_url, token=settings.llm_remote_token)
|
||||
return LocalAdapter()
|
||||
|
||||
# Type alias for clean injection
|
||||
AdapterDependency = Annotated[LLMAdapter, Depends(get_adapter)]
|
||||
```
|
||||
|
||||
### 6. Update /chat endpoint
|
||||
**File**: `app/main.py`
|
||||
|
||||
```python
|
||||
from app.llm.adapter import AdapterDependency
|
||||
from app.llm.exceptions import LLMError, llm_exception_to_http
|
||||
|
||||
@app.post("/chat", response_model=ChatResponse)
|
||||
async def chat(request: ChatRequest, adapter: AdapterDependency) -> ChatResponse:
|
||||
# ... validation ...
|
||||
try:
|
||||
response_text = await adapter.generate(conversation_id, request.message)
|
||||
except LLMError as e:
|
||||
raise llm_exception_to_http(e)
|
||||
# ... return response ...
|
||||
```
|
||||
|
||||
### 7. Update mode type
|
||||
**File**: `app/schemas.py`
|
||||
- Change `mode: Literal["local", "remote"]` to `mode: Literal["local", "remote", "openai"]`
|
||||
|
||||
### 8. Update environment examples
|
||||
**File**: `.env.example`
|
||||
```
|
||||
LLM_MODE=openai
|
||||
OPENAI_API_KEY=sk-your-key-here
|
||||
OPENAI_MODEL=gpt-4o-mini
|
||||
```
|
||||
|
||||
### 9. Update Dockerfile
|
||||
**File**: `Dockerfile`
|
||||
- Add `ENV OPENAI_MODEL=gpt-4o-mini` (API key passed at runtime for security)
|
||||
|
||||
---
|
||||
|
||||
## Error Handling
|
||||
|
||||
| OpenAI Exception | Custom Exception | HTTP Status |
|
||||
|------------------|------------------|-------------|
|
||||
| `AuthenticationError` | `LLMAuthenticationError` | 401 |
|
||||
| `RateLimitError` | `LLMRateLimitError` | 429 |
|
||||
| `APIConnectionError` | `LLMConnectionError` | 503 |
|
||||
| `APIError` | `LLMError` | varies |
|
||||
|
||||
---
|
||||
|
||||
## Verification Steps
|
||||
|
||||
1. **Install dependencies**:
|
||||
```bash
|
||||
pip install -r requirements.txt
|
||||
```
|
||||
|
||||
2. **Run locally with OpenAI**:
|
||||
```bash
|
||||
export LLM_MODE=openai
|
||||
export OPENAI_API_KEY=sk-your-key
|
||||
uvicorn app.main:app --reload
|
||||
```
|
||||
|
||||
3. **Test the /chat endpoint**:
|
||||
```bash
|
||||
curl -X POST http://localhost:8000/chat \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"message": "Hello, what is 2+2?"}'
|
||||
```
|
||||
|
||||
4. **Verify response contains OpenAI response and mode="openai"**
|
||||
|
||||
5. **Test Docker build**:
|
||||
```bash
|
||||
docker build -t tyndale-ai .
|
||||
docker run -p 8080:8080 -e LLM_MODE=openai -e OPENAI_API_KEY=sk-your-key tyndale-ai
|
||||
```
|
||||
|
||||
6. **Test error handling** (optional): Use invalid API key to verify 401 response
|
||||
|
|
@ -6,7 +6,7 @@ from pydantic_settings import BaseSettings
|
|||
class Settings(BaseSettings):
|
||||
"""Application configuration loaded from environment variables."""
|
||||
|
||||
llm_mode: Literal["local", "remote", "openai"] = "local"
|
||||
llm_mode: Literal["local", "remote", "openai", "asksage"] = "local"
|
||||
llm_remote_url: str = ""
|
||||
llm_remote_token: str = ""
|
||||
|
||||
|
|
@ -14,6 +14,11 @@ class Settings(BaseSettings):
|
|||
openai_api_key: str = ""
|
||||
openai_model: str = "gpt-4o-mini"
|
||||
|
||||
# AskSage configuration
|
||||
asksage_email: str = ""
|
||||
asksage_api_key: str = ""
|
||||
asksage_model: str = "gpt-4o"
|
||||
|
||||
class Config:
|
||||
env_file = ".env"
|
||||
env_file_encoding = "utf-8"
|
||||
|
|
|
|||
|
|
@ -1,10 +1,12 @@
|
|||
"""LLM adapter implementations with FastAPI dependency injection support."""
|
||||
|
||||
import asyncio
|
||||
from abc import ABC, abstractmethod
|
||||
from functools import lru_cache
|
||||
from typing import Annotated
|
||||
|
||||
import httpx
|
||||
from asksageclient import AskSageClient
|
||||
from fastapi import Depends
|
||||
from openai import (
|
||||
AsyncOpenAI,
|
||||
|
|
@ -182,6 +184,63 @@ class OpenAIAdapter(LLMAdapter):
|
|||
)
|
||||
|
||||
|
||||
class AskSageAdapter(LLMAdapter):
|
||||
"""AskSage API adapter using the official asksageclient SDK."""
|
||||
|
||||
def __init__(self, email: str, api_key: str, model: str = "gpt-4o"):
|
||||
"""Initialize the AskSage adapter.
|
||||
|
||||
Args:
|
||||
email: AskSage account email
|
||||
api_key: AskSage API key
|
||||
model: Model identifier (e.g., "gpt-4o", "claude-3-opus")
|
||||
"""
|
||||
self.client = AskSageClient(email=email, api_key=api_key)
|
||||
self.model = model
|
||||
|
||||
async def generate(self, conversation_id: str, message: str) -> str:
|
||||
"""Generate a response using the AskSage API.
|
||||
|
||||
Args:
|
||||
conversation_id: The conversation identifier (for future use)
|
||||
message: The user's message
|
||||
|
||||
Returns:
|
||||
The generated response string
|
||||
|
||||
Raises:
|
||||
LLMAuthenticationError: If credentials are invalid
|
||||
LLMConnectionError: If connection fails
|
||||
LLMResponseError: If response content is empty or invalid
|
||||
LLMError: For other API errors
|
||||
"""
|
||||
try:
|
||||
# AskSageClient is synchronous, run in thread pool to avoid blocking
|
||||
response = await asyncio.to_thread(
|
||||
self.client.query,
|
||||
message=message,
|
||||
model=self.model,
|
||||
)
|
||||
|
||||
if not isinstance(response, dict):
|
||||
raise LLMResponseError("AskSage returned invalid response format")
|
||||
|
||||
content = response.get("response")
|
||||
if content is None:
|
||||
raise LLMResponseError("AskSage returned empty response content")
|
||||
return content
|
||||
|
||||
except LLMError:
|
||||
raise
|
||||
except Exception as e:
|
||||
error_msg = str(e).lower()
|
||||
if "auth" in error_msg or "401" in error_msg or "unauthorized" in error_msg:
|
||||
raise LLMAuthenticationError(f"AskSage authentication failed: {e}")
|
||||
if "connect" in error_msg or "timeout" in error_msg:
|
||||
raise LLMConnectionError(f"Could not connect to AskSage: {e}")
|
||||
raise LLMError(f"AskSage API error: {e}")
|
||||
|
||||
|
||||
# --- Dependency Injection Support ---
|
||||
|
||||
|
||||
|
|
@ -225,6 +284,17 @@ def get_adapter(settings: Annotated[Settings, Depends(get_settings)]) -> LLMAdap
|
|||
token=settings.llm_remote_token or None,
|
||||
)
|
||||
|
||||
if settings.llm_mode == "asksage":
|
||||
if not settings.asksage_email or not settings.asksage_api_key:
|
||||
raise LLMConfigurationError(
|
||||
"ASKSAGE_EMAIL and ASKSAGE_API_KEY must be set when LLM_MODE is 'asksage'"
|
||||
)
|
||||
return AskSageAdapter(
|
||||
email=settings.asksage_email,
|
||||
api_key=settings.asksage_api_key,
|
||||
model=settings.asksage_model,
|
||||
)
|
||||
|
||||
return LocalAdapter()
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -17,7 +17,7 @@ class ChatResponse(BaseModel):
|
|||
|
||||
conversation_id: str = Field(..., description="Conversation ID (generated if not provided)")
|
||||
response: str = Field(..., description="The LLM's response")
|
||||
mode: Literal["local", "remote", "openai"] = Field(..., description="Which adapter was used")
|
||||
mode: Literal["local", "remote", "openai", "asksage"] = Field(..., description="Which adapter was used")
|
||||
sources: list = Field(default_factory=list, description="Source references (empty for now)")
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -1,6 +1,33 @@
|
|||
fastapi>=0.109.0
|
||||
uvicorn[standard]>=0.27.0
|
||||
pydantic>=2.5.0
|
||||
pydantic-settings>=2.1.0
|
||||
httpx>=0.26.0
|
||||
openai>=1.0.0
|
||||
annotated-doc==0.0.4
|
||||
annotated-types==0.7.0
|
||||
anyio==4.12.1
|
||||
asksageclient==1.42
|
||||
certifi==2026.1.4
|
||||
charset_normalizer==3.4.4
|
||||
click==8.3.1
|
||||
colorama==0.4.6
|
||||
distro==1.9.0
|
||||
fastapi==0.128.0
|
||||
h11==0.16.0
|
||||
httpcore==1.0.9
|
||||
httptools==0.7.1
|
||||
httpx==0.28.1
|
||||
idna==3.11
|
||||
jiter==0.12.0
|
||||
openai==2.15.0
|
||||
pip_system_certs==5.3
|
||||
pydantic==2.12.5
|
||||
pydantic-settings==2.12.0
|
||||
pydantic_core==2.41.5
|
||||
python-dotenv==1.2.1
|
||||
requests==2.32.5
|
||||
PyYAML==6.0.3
|
||||
sniffio==1.3.1
|
||||
starlette==0.50.0
|
||||
tqdm==4.67.1
|
||||
typing-inspection==0.4.2
|
||||
typing_extensions==4.15.0
|
||||
urllib3==2.6.3
|
||||
uvicorn==0.40.0
|
||||
watchfiles==1.1.1
|
||||
websockets==16.0
|
||||
|
|
|
|||
Loading…
Reference in New Issue