HPC cluster - llm command line tool

Purpose: HPC cluster users can use LLM models in terminal to fix issues and get answers. Tool is available for all HPC user with zero configuration and without logging.

This setup is based on llm command line tool. llm - is a CLI tool and Python library for interacting with Large Language Models

Web site: https://llm.datasette.io/en/stable/index.html

Git repo: https://github.com/simonw/LLM
Example setup on Rocky Linux 9.8.

Hide API key from users

Set one server as Nginx forward proxy. Server name chat.

dnf install -y nginx

Create file with API key /etc/nginx/llm-hpc-key.conf

proxy_set_header Authorization "Bearer sk-A...............w";

Limit access

chmod 600 /etc/nginx/llm-hpc-key.conf

Create nginx config /etc/nginx/conf.d/llm-gateway.conf

server {
    listen 8081;
    server_name chat;
    location /v1/ {
        allow all;
        proxy_pass https://llm-server.example.com;
        proxy_http_version 1.1;
        proxy_ssl_server_name on;
        proxy_set_header Host llm-server.example.com;
        include /etc/nginx/llm-hpc-key.conf;
        proxy_buffering off;
        proxy_read_timeout 600s;
    }
}

Test config, enable and start service

nginx -t
systemctl enable nginx
systemctl start nginx

Install and configure llm CLI

dnf -y install python3.12 python3.12-pip
pip3.12 install llm
mkdir /etc/llm/
mv /usr/local/bin/llm /usr/local/bin/llm-real

llm wrapper script /usr/local/bin/llm

#!/bin/bash
export LLM_USER_PATH=/etc/llm
REAL=/usr/local/bin/llm-real

# default system prompt, if the file exists
SYS=()
[ -r /etc/llm/system.txt ] && SYS=(-s "$(cat /etc/llm/system.txt)")

if [ $# -eq 0 ] && [ -t 0 ]; then
  exec $REAL --help
fi

case "${1:-}" in
  chat|prompt)
    cmd=$1; shift
    exec $REAL "$cmd" -d :memory: "${SYS[@]}" "$@" ;;
  -h|--help|--version|aliases|collections|embed|embed-models|embed-multi|fragments|install|keys|logs|models|openai|plugins|schemas|similar|templates|tools|uninstall)
    exec $REAL "$@" ;;
  *)
    exec $REAL prompt -d :memory: "${SYS[@]}" "$@" ;;
esac

Make it executable

chmod +x /usr/local/bin/llm

llm config file /etc/llm/extra-openai-models.yaml

- model_id: deepseek-ai/DeepSeek-V4.1-Flash-1017
  model_name: deepseek-ai/DeepSeek-V4.1-Flash-1017
  api_base: "http://chat:8081/v1"
  aliases: ["hpc"]
  vision: true

system prompt file with specific knowledge about HPC cluster

STYLE CONTRACT - applies to every reply, outranks every habit

The user reads your reply in a terminal. 
. . .
. . .
. . .
## 1. Role
Interactive command-line assistant for Slurm-based HPC cluster, used from a
terminal via Simon Willison's `llm` CLI. Help users write, debug and submit jobs,
understand partitions/accounts/quotas/storage, and read scheduler output.
. . .
. . .

Set default model in file /etc/llm/default_model.txt

hpc

Examples

$ llm "I cannot create new file"

$ llm "What is my dsik quota?"

$ echo "create a simple pyton script and a job file" | llm

$ llm "how to find my account?"

$ llm "My job failed these are my files" -f hello-25741694.out -f hello.py -f run_hello.sbatch

$ llm chat

$ llm chat -f hello-25741694.out -f hello.py -f run_hello.sbatch

$ llm "Extract text from this photo " -a photo_a1.jpg