Install and configure Optimal Engine

This page covers a self-hosted Engine. A managed Engine on MIOSA is set up for you; see Optimal Engine on MIOSA.

Requirements

Without an embedding model the Engine still works: search is full-text only, and a backfill embeds everything once a model is set up. Without a media tool, a file of that kind is stored with its metadata and a warning instead of its text.

Install

From a checkout

git clone https://github.com/Miosa-osa/OptimalEngine.git
cd OptimalEngine
mix deps.get
mix optimal.auth mint --name "root" --workspace '*'
OPTIMAL_API_ENABLED=true OPTIMAL_AUTH_REQUIRED=true mix run --no-halt

make bootstrap (or mix optimal.bootstrap) compiles, runs migrations, and loads the bundled sample-workspace/ so you have data to try.

With Docker

deploy/Dockerfile.engine builds an OTP release on Elixir 1.19.5 and OTP 28.5.

Set OPTIMAL_ENGINE_API_KEY on the container, or the health check runs without a key and the container reports unhealthy. On Linux, host.docker.internal does not resolve by default; point OLLAMA_HOST at the host’s address.

With Docker Compose

cd deploy
cp env.example .env
# Set OPTIMAL_BOOTSTRAP_KEY and OPTIMAL_ENGINE_API_KEY to the same oe_ key.
docker compose up --build

Compose publishes the Engine on 127.0.0.1:4200 and keeps data in the optimal_data volume. For a public server, add the production overlay, which puts Caddy with automatic TLS in front and keeps the Engine port private:

docker compose -f docker-compose.yml -f docker-compose.prod.yml up -d

Set DOMAIN and LETSENCRYPT_EMAIL in .env first.

As a release

MIX_ENV=prod mix release optimal
OPTIMAL_API_ENABLED=true OPTIMAL_AUTH_REQUIRED=true _build/prod/rel/optimal/bin/optimal start

bin/optimal maintenance <task> runs maintenance with the API off; see Operations.

Settings

The Engine reads its settings from environment variables at boot. An invalid value stops the boot with a message that names the variable.

Server and keys

Paths and storage

Secrets

Each is 64 hex characters; generate one with openssl rand -hex 32.

Embeddings

Behavior

deploy/env.example in the repository lists every variable with a comment.

Where data lives

SQLite is the store. It holds full-text search, vectors, and every record, so one file plus the folders around it is the whole Engine.

<OPTIMAL_ENGINE_ROOT>/          workspace folders: nodes/, .wiki/, assets/
.optimal/index.db               the database (with -wal and -shm files)
.optimal/cache/                 cache and stored embeddings
.optimal/backups/               local backups
.optimal/knowledge-rocksdb/     graph projection, when the rocksdb backend is on

GET /api/stores lists the 13 logical stores and which provider backs each one.

API keys and scopes

Send a key as Authorization: Bearer <key> or X-API-Key: <key>. Keys look like oe_<id>_<secret>; the Engine keeps only a bcrypt hash of the secret.

Mint keys from the checkout:

mix optimal.auth mint --name "agent" --workspace default:notes --scope read --scope write
mix optimal.auth list
mix optimal.auth revoke <id>

Or over HTTP with an admin key: POST /api/auth/keys with {"name", "scopes", "workspace_scope", "expires_at"}. A key can only mint keys narrower than itself. GET /api/auth/whoami shows a key’s own scopes and workspaces.