Skip to content

Large Language Model (LLM) Assistants#

Kestrel hosts two locally-served LLM-based assistants for coding and HPC-support workflows. Both run open-weight models on Kestrel's own GPU nodes, so no code or data leaves NLR: OnField Assistant (ofa), a retrieval-augmented assistant built and maintained in-house and tuned for OpenFOAM/AMReX/MARBLES/VASP/HPC-support workflows, and OpenCode, a general-purpose terminal coding agent that can also reach NLR's centrally-hosted ServeAI Gateway models.

OnField Assistant (ofa)#

OnField Assistant (ofa) is a locally-hosted, retrieval-augmented-generation (RAG) LLM assistant for HPC and scientific-computing workflows. It runs Gemma 4 (31B, Google, Apache 2.0) via Ollama on a Kestrel GPU node, and layers RAG over Kestrel's documentation plus OpenFOAM, AMReX, MARBLES, VASP, ReFrame, and quantum-computing source/paper corpora. It currently ships eight specialized modes — --code (default), --openfoam, --hpc, --amrex, --marbles, --quantum-computing, --vasp, and --rhel9_reframe — each swapping in a different system prompt and RAG index.

module load assistant

puts ofa on $PATH (and the ofa_client Python module on $PYTHONPATH). A GPU is allocated automatically via SLURM the first time you run ofa (override the account/partition/walltime with the OFA_ACCOUNT / OFA_PARTITION / OFA_WALLTIME environment variables).

Note

Full installation notes, the RAG-maintenance playbook, and an in-depth technical writeup live in the project's own docs: nileshsawant.github.io/onfield-assistant.

There are five ways to use ofa:

1. Command line#

$ ofa                            # interactive chat, default --code mode
$ ofa "explain this SLURM error"
$ ofa --openfoam --save ./mycase # OpenFOAM case generator, writes files to ./mycase
$ ofa --hpc                      # Kestrel HPC / Slurm documentation assistant
$ ofa --resume                   # resume the previous session
$ ofa --list-models              # show the model registry

Type quit to leave interactive mode, or /help for the full list of slash commands. Run ofa --help to see every flag, including --save, --fast, --model, --no-rag, and the --serve* flags used below.

2. VS Code Chat (BYOK)#

ofa --serve starts an OpenAI-compatible HTTP server (/v1/chat/completions) on your allocation. Paired with the OnField Assistant VS Code extension (one-click SLURM allocation + login-node port bridge) or a manual ssh -L tunnel, this registers every ofa mode as a "Bring Your Own Key" model in VS Code Copilot Chat's model picker. Full walkthrough: Use ofa from VS Code Chat (the OnField Assistant extension).

3. Python (ofa_client)#

A stdlib-only Python client talks to a running ofa --serve over HTTP — no extra packages needed:

from ofa_client import ask
text = ask("what is a good turbulence model for cavity flow at Re=1e4?")

It also supports attaching files/images, multi-turn Session() objects for client-side conversation history, and auto-detects the server URL/token from $OFA_SCRATCH. This is useful for having a running simulation ask ofa to summarize a plot or diagnose a crash mid-run — see the ofa_client docs.

4. Custom / third-party agents#

Because ofa --serve speaks the standard OpenAI /v1/chat/completions API, any agent framework that supports pointing at a custom base_url + api_key can use ofa as its backend LLM. For example, AMReX Agent has a litellm provider for exactly this; pointed at ofa:

module load assistant
ofa --serve --serve-enable-tools
export LITELLM_BASE_URL="http://localhost:$(cat $OFA_SCRATCH/.ofa_serve_port)/v1"
export LITELLM_API_KEY="$(cat $OFA_SCRATCH/.ofa_api_key)"
export LITELLM_MODEL="ofa-code"

See ofa's Bring your own agent section for the full recipe and other integrations.

OpenCode#

OpenCode is a terminal-native, open-source AI coding agent (also available as a desktop app / IDE extension). NLR's build is enabled to communicate with two kinds of models:

  • ServeAI Gateway: NLR-managed HALO models hosted centrally on OpenStack, with no personal GPU allocation required. OpenCode currently supports access to Devstral 2 123B, GPT-OSS 120B, Gemma 4 31B, Nemotron 3 Super 120B, and Nemotron 3 Nano 30B, all with tool-calling enabled.
  • Local Node Model: models you run yourself via Ollama on a GPU allocation (gpt-oss:120b, gemma4:31b, muse-glimmer:30b), reached at $OLLAMA_HOST (which dynamically sets the access port by default). These models are each able to easily fit within the memory constraints of a single 80GB H100 GPU for local inference.

Note

OpenCode's Kestrel rules instruct it to warn you if it is about to run a local-Ollama model from a login node — that workload needs a GPU job (salloc/sbatch), not the shared login node itself. Only ServeAI Gateway models can be used through OpenCode on login nodes.

Running OpenCode on Kestrel#

After loading the module, run opencode models <provider> on a login node to see the LLMs available to use in OpenCode, where <provider> can be one of local-ollama (which can be directly run on a Kestrel GPU compute node by any user) or serveai (one of the NLR-internal HALO models).

Each time you load the module, it generates a per-user config at ~/.cache/opencode/opencode.json with both providers above pre-wired and a conservative default permission policy. This configuration file is generated upon each launch so that the user is always provided the most up-to-date list of supported LLMs to access. Additionally, when running node-local models, when OpenCode is launched, the configuration file is automatically updated with an available port through which Ollama may use for serving.

Ongoing work aims to integrate the OpenCode module with ofa as a provider, which would give you the entire ofa mode family alongside OpenCode's built-in providers in the same model picker.

Login node access: HALO models only#

module load opencode is only available on Kestrel's GPU login node that runs RHEL9 (i.e., kl5). OpenCode is not available on CPU nodes or any RHEL8 nodes:

ssh kl5.hpc.nlr.gov
module load opencode
opencode

After launching opencode, users may type /models in the interactive prompt to see the available models. Although Ollama (i.e., node-local) models may appear in the "Recent" list, attempting to connect to them on a login node will fail with an error. Only ServeAI Gateway provider models are usable when launching OpenCode from a login node.

Note

Users are encouraged to use OpenCode on compute nodes instead of login nodes whenever possible.

Compute node access: Node-local model serving and HALO models#

This example reflects requesting an interactive debug job for one hour to launch the muse-glimmer:30b model via OpenCode on a single H100 device:

ssh kl5.hpc.nlr.gov
salloc -A <project-handle> -p debug -t 01:00:00 -c 32 -n 1 --mem=85G --gres=gpu:1
module load opencode
opencode --model muse-glimmer:30b

Safety and permissions considerations#

The OpenCode module on Kestrel is designed to prevent the agent from mistakenly moving, deleting, or otherwise changing files on the user's behalf without the user's knowledge. To that end, the OpenCode module is deployed as a container and is given read-only access to Kestrel's filesystem except for the directory from which the agent is launched, which also has write access. Users are encouraged to keep the agent focused on the current working directory (as opposed to multiple /projects folders, for example), as it will only be able to write or modify files there. Additionally, git push, chmod/chown, and rm require confirmation from the user in each session or are denied outright.