Vetrag

Lightweight veterinary visit reasoning helper with LLM-assisted keyword extraction and disambiguation.

Features

• Switch seamlessly between local Ollama and OpenRouter (OpenAI-compatible) LLM backends by changing environment variables only.
• Structured JSON outputs enforced using provider-supported response formats (Ollama format, OpenAI/OpenRouter response_format: { type: json_object }).
• Integration tests using mock LLM & DB (no network dependency).
• GitHub Actions CI (vet, test, build).

Quick Start

1. Clone & build

git clone <repo-url>
cd vetrag
go build ./...

2. Prepare data

Ensure config.yaml and maindb.yaml / db.yaml exist as provided. Visit data is loaded at runtime (see models.go / db.go).

3. Run with Ollama (local)

Pull or have a model available (example: ollama pull qwen2.5):

export OPENAI_BASE_URL=http://localhost:11434/api/chat
export OPENAI_MODEL=qwen2.5:latest
# API key not required for Ollama
export OPENAI_API_KEY=

go run .

4. Run with OpenRouter

Sign up at https://openrouter.ai and get an API key.

export OPENAI_BASE_URL=https://openrouter.ai/api/v1/chat/completions
export OPENAI_API_KEY=sk-or-XXXXXXXXXXXXXXXX
export OPENAI_MODEL=meta-llama/llama-3.1-70b-instruct  # or any supported model

go run .

Open http://localhost:8080/ in your browser.

5. Health & Chat

curl -s http://localhost:8080/health
curl -s -X POST http://localhost:8080/chat -H 'Content-Type: application/json' -d '{"message":"my dog has diarrhea"}' | jq

Environment Variables

Variable	Purpose	Default (if empty)
OPENAI_BASE_URL	LLM endpoint (Ollama chat or OpenRouter chat completions)	http://localhost:11434/api/chat
OPENAI_API_KEY	Bearer token for OpenRouter/OpenAI-style APIs	(unused if empty)
OPENAI_MODEL	Model identifier (Ollama model tag or OpenRouter model slug)	none (must set for remote)

How Backend Selection Works

llm.go auto-detects the style:

• If the base URL contains openrouter.ai or /v1/ it uses OpenAI-style request & parses choices[0].message.content.
• Otherwise it assumes Ollama and posts to /api/chat with format for structured JSON.

Structured Output

We define a JSON Schema-like map internally and:

• Ollama: send as format (native structured output extension).
• OpenRouter/OpenAI: send response_format: { type: "json_object" } plus a system instruction describing the expected keys.

Prompts

Prompts in config.yaml have been adjusted to explicitly demand JSON only. This reduces hallucinated prose and plays well with both backends.

Testing

Run:

go test ./...

All tests mock the LLM so no network is required.

CI

GitHub Actions workflow at .github/workflows/ci.yml runs vet, tests, build on push/PR.

Troubleshooting

Symptom	Cause	Fix
Provider error referencing response_format and json_schema	Some providers reject json_schema	We now default to json_object; ensure you pulled latest changes.
Empty response	Model returned non-JSON or empty content	Enable debug logs (see below) and inspect raw response.
Non-JSON content (code fences)	Model ignored instruction	Try a stricter system message or switch to a model with better JSON adherence.

Enable Debug Logging

Temporarily edit main.go:

logrus.SetLevel(logrus.DebugLevel)

(You can also refactor later to read a LOG_LEVEL env var.)

Sanitizing Output (Optional Future Improvement)

If some models wrap JSON in text, a post-processor could strip code fences and re-parse. Not implemented yet to keep logic strict.

Next Ideas

• Add retry with exponential backoff for transient 5xx.
• Add optional json fallback if a provider rejects json_object.
• Add streaming support.
• Add integration test with recorded OpenRouter fixture.

License

(Choose and add a LICENSE file if planning to open source.)