LLM (Large Language Model) client, for communicating with an LLM backend. This can be used for generating texts for AI npcs, or for fine-tuning the LLM on a given prompt.

Note that running a LLM locally requires a lot of power, and ideally a powerful GPU. Testing this with CPU mode on a beefy laptop, still takes some 4s just on a very small model.

The server defaults to output suitable for a local server https://github.com/oobabooga/text-generation-webui, but could be used for other LLM servers too.

See the LLM instructions on that page for how to set up the server. You’ll also need a model file - there are thousands to try out on https://huggingface.co/models (you want Text Generation models specifically).

Optional Evennia settings (if not given, these defaults are used)

DEFAULT_LLM_HOST = “http://localhost:5000” DEFAULT_LLM_PATH = “/api/v1/generate” DEFAULT_LLM_HEADERS = {“Content-Type”: “application/json”} DEFAULT_LLM_PROMPT_KEYNAME = “prompt” DEFAULT_LLM_REQUEST_BODY = {…} # see below, this controls how to prompt the LLM server.

class evennia.contrib.rpg.llm.llm_client.StringProducer(body)[source]

Bases: object

Used for feeding a request body to the HTTP client.


Initialize self. See help(type(self)) for accurate signature.

class evennia.contrib.rpg.llm.llm_client.SimpleResponseReceiver(status_code, d)[source]

Bases: twisted.internet.protocol.Protocol

Used for pulling the response body out of an HTTP response.

__init__(status_code, d)[source]

Initialize self. See help(type(self)) for accurate signature.


Called whenever data is received.

Use this method to translate to a higher-level message. Usually, some callback will be made upon the receipt of each complete protocol message.

@param data: a string of indeterminate length. Please keep in mind

that you will probably need to buffer some data, as partial (or multiple) protocol messages may be received! I recommend that unit tests for protocols call through to this method with differing chunk sizes, down to one byte at a time.

connectionLost(reason=<twisted.python.failure.Failure twisted.internet.error.ConnectionDone: Connection was closed cleanly.>)[source]

Called when the connection is shut down.

Clear any circular references here, and any external references to this Protocol. The connection has been closed.

@type reason: L{twisted.python.failure.Failure}

class evennia.contrib.rpg.llm.llm_client.QuietHTTP11ClientFactory(quiescentCallback, metadata)[source]

Bases: twisted.web.client._HTTP11ClientFactory

Silences the obnoxious factory start/stop messages in the default client.

noisy = False
class evennia.contrib.rpg.llm.llm_client.LLMClient(on_bad_request=None)[source]

Bases: object

A client for communicating with an LLM server.


Initialize self. See help(type(self)) for accurate signature.


Get a response from the LLM server for the given npc.


prompt (str or list) – The prompt to send to the LLM server. If a list, this is assumed to be the chat history so far, and will be added to the prompt in a way suitable for the api.



The generated text response. Will return an empty string

if there is an issue with the server, in which case the the caller is expected to handle this gracefully.