118 lines
2.4 KiB
Markdown
118 lines
2.4 KiB
Markdown
# Tyndale AI Service
|
|
|
|
LLM Chat Service for algorithmic trading support - codebase Q&A, P&L summarization, and strategy enhancement suggestions.
|
|
|
|
## Quick Start
|
|
|
|
### Local Development
|
|
|
|
```bash
|
|
# Install dependencies
|
|
pip install -r requirements.txt
|
|
|
|
# Run the server
|
|
uvicorn app.main:app --reload --port 8080
|
|
```
|
|
|
|
### Docker
|
|
|
|
```bash
|
|
# Build
|
|
docker build -t tyndale-ai-service .
|
|
|
|
# Run
|
|
docker run -p 8080:8080 -e LLM_MODE=local tyndale-ai-service
|
|
```
|
|
|
|
## API Endpoints
|
|
|
|
### Health Check
|
|
|
|
```bash
|
|
curl http://localhost:8080/health
|
|
```
|
|
|
|
Response:
|
|
```json
|
|
{"status": "ok"}
|
|
```
|
|
|
|
### Chat
|
|
|
|
```bash
|
|
curl -X POST http://localhost:8080/chat \
|
|
-H "Content-Type: application/json" \
|
|
-d '{"message": "Hello, how are you?"}'
|
|
```
|
|
|
|
Response:
|
|
```json
|
|
{
|
|
"conversation_id": "uuid-generated-if-not-provided",
|
|
"response": "...",
|
|
"mode": "local",
|
|
"sources": []
|
|
}
|
|
```
|
|
|
|
With conversation ID:
|
|
```bash
|
|
curl -X POST http://localhost:8080/chat \
|
|
-H "Content-Type: application/json" \
|
|
-d '{"message": "Follow up question", "conversation_id": "my-conversation-123"}'
|
|
```
|
|
|
|
## Environment Variables
|
|
|
|
| Variable | Description | Default |
|
|
|----------|-------------|---------|
|
|
| `LLM_MODE` | `local` or `remote` | `local` |
|
|
| `LLM_REMOTE_URL` | Remote LLM endpoint URL | (empty) |
|
|
| `LLM_REMOTE_TOKEN` | Bearer token for remote LLM | (empty) |
|
|
|
|
### Remote Mode Setup
|
|
|
|
```bash
|
|
export LLM_MODE=remote
|
|
export LLM_REMOTE_URL=https://your-llm-service.com/generate
|
|
export LLM_REMOTE_TOKEN=your-api-token
|
|
|
|
uvicorn app.main:app --reload --port 8080
|
|
```
|
|
|
|
The remote adapter expects the LLM service to accept:
|
|
```json
|
|
{"conversation_id": "...", "message": "..."}
|
|
```
|
|
|
|
And return:
|
|
```json
|
|
{"response": "..."}
|
|
```
|
|
|
|
## Project Structure
|
|
|
|
```
|
|
tyndale-ai-service/
|
|
├── app/
|
|
│ ├── __init__.py
|
|
│ ├── main.py # FastAPI app + routes
|
|
│ ├── schemas.py # Pydantic models
|
|
│ ├── config.py # Environment config
|
|
│ └── llm/
|
|
│ ├── __init__.py
|
|
│ └── adapter.py # LLM adapter interface + implementations
|
|
├── requirements.txt
|
|
├── Dockerfile
|
|
├── .env.example
|
|
└── README.md
|
|
```
|
|
|
|
## Features
|
|
|
|
- **Dual mode operation**: Local stub or remote LLM
|
|
- **Conversation tracking**: UUID generation for new conversations
|
|
- **Security**: 10,000 character message limit, no content logging
|
|
- **Cloud Run ready**: Port 8080, stateless design
|
|
- **Async**: Full async/await support with httpx
|