Table of Contents

  1. Customizing Controllers
  2. Customizing Models
  3. Customizing Views
  4. Customizing System Prompts
  5. Authorizing Model Completions
  6. Inference Cost Events
  7. Prompt Caching
  8. Adding LLM Models

Customizing Controllers

You can override Raif’s controllers by creating your own that inherit from Raif’s base controllers:

class ConversationsController < Raif::ConversationsController
  # Your customizations here
end

class ConversationEntriesController < Raif::ConversationEntriesController
  # Your customizations here
end

Then update the configuration:

Raif.configure do |config|
  config.conversations_controller = "ConversationsController"
  config.conversation_entries_controller = "ConversationEntriesController"
end

Customizing Models

By default, Raif models inherit from ApplicationRecord. You can change this:

Raif.configure do |config|
  config.model_superclass = "CustomRecord"
end

Customizing Views

You can customize Raif’s views by copying them to your application and modifying them. To copy the conversation-related views, run:

rails generate raif:views

This will copy all conversation and conversation entry views to your application in:

  • app/views/raif/conversations/
  • app/views/raif/conversation_entries/

These views will automatically override Raif’s default views. You can customize them to match your application’s look and feel while maintaining the same functionality.

Customizing System Prompts

If you don’t want to override the system prompt entirely in your task/conversation subclasses, you can customize the intro portion of the system prompts for conversations and tasks:

Raif.configure do |config|
  config.conversation_system_prompt_intro = "You are a helpful assistant who specializes in customer support."
  config.task_system_prompt_intro = "You are a helpful assistant who specializes in data analysis."

  # or with a lambda
  config.task_system_prompt_intro = ->(task) { "You are a helpful assistant who specializes in #{task.name}." }
  config.conversation_system_prompt_intro = ->(conversation) { "You are a helpful assistant talking to #{conversation.creator.email}. Today's date is #{Date.today.strftime('%B %d, %Y')}." }
end

Authorizing Model Completions

You can register a lambda that is called at the start of Raif::Llm#chat, before the Raif::ModelCompletion record is created or any provider API call is made. It receives llm: (the Raif::Llm instance) and source: (the completion’s source - typically a conversation entry, task, or agent). To veto the request, raise an exception - it propagates to the caller. Return values are ignored.

This is useful for enforcing per-account usage limits (e.g. a monthly inference spend budget):

Raif.configure do |config|
  config.model_completion_authorizer = ->(llm:, source:) {
    account = source.account if source.respond_to?(:account)
    raise MyApp::UsageLimitExceededError if account && !account.within_llm_usage_limits?
  }
end

Notes:

  • The authorizer runs before the llm_api_requests_enabled guard, so it applies even when API requests are disabled.
  • Because agents call Raif::Llm#chat on every iteration, raising from the authorizer also stops in-progress agent runs.
  • The veto propagates to the caller even through Raif::Task.run, which otherwise rescues StandardError: the task is marked failed and then the original exception is re-raised (rather than being logged and reported as an ordinary model failure), so callers can distinguish an intentional veto from a model error.
  • Conversation responses run in a background job (Raif::ConversationEntryJob), so there is no host caller to receive a raise. A veto there surfaces as a failed Raif::ConversationEntry (marked failed and broadcast to the UI) rather than propagating.
  • It does not apply to embedding generation or batch API submissions, which do not go through Raif::Llm#chat.

Inference Cost Events

Each time a Raif::ModelCompletion reaches a terminal state (completed or failed), Raif creates a durable Raif::InferenceCostEvent - a slim record of the completion’s token counts and costs. Cost events survive deletion of the completion row (and, later, its source), so cost reporting keeps working even after old completions are culled. This is enabled by default and can be disabled:

Raif.configure do |config|
  config.inference_cost_events_enabled = false
end

You can attach host application context (e.g. account or workflow ids) to each event via a metadata resolver. It receives model_completion: and returns a hash that is merged into the event’s metadata column:

Raif.configure do |config|
  config.inference_cost_event_metadata = ->(model_completion:) {
    source = model_completion.source
    { account_id: source.account_id } if source.respond_to?(:account_id)
  }
end

Notes:

  • Events are created for terminal completions only. Failed completions get events too (their cost columns are usually NULL, so cost sums are unaffected).
  • A sync failure never fails the completion save. The error is reported via Rails.error and the idempotent Raif::RepairInferenceCostEventsJob is enqueued to self-heal.
  • After upgrading, run rails raif:backfill_inference_cost_events to create events for existing completions. Raif’s admin stats read from events, so they show partial history until the backfill completes.

Prompt Caching

Anthropic and AWS Bedrock support prompt caching on supported Claude models. Raif does not enable prompt caching by default. To enable it on a per-class basis, use the enable_anthropic_prompt_caching and/or enable_bedrock_prompt_caching class directives on your Raif::Task, Raif::Conversation, or Raif::Agent subclasses:

class Raif::Tasks::DocumentSummarization < Raif::Task
  enable_anthropic_prompt_caching
  enable_bedrock_prompt_caching

  # ...
end

When enabled, Raif will set cache_control: { type: "ephemeral" } on the Anthropic request and add a cache_point to the system prompt and last message on Bedrock requests. Cached vs. fresh input tokens are tracked on Raif::ModelCompletion (as cache_read_input_tokens and cache_creation_input_tokens) and cost estimates in the admin account for the provider’s cached-token pricing.

Adding LLM Models

You can easily add new LLM models to Raif. The first argument is the provider adapter class to use and the second argument is a hash defining the specifics of the model:

# Register the model in Raif's LLM registry
Raif.register_llm(Raif::Llms::OpenRouter, {
  key: :open_router_gemini_flash_1_5_8b, # a unique key for the model
  api_name: "google/gemini-flash-1.5-8b", # name of the model to be used in API calls - needs to match the provider's API name
  input_token_cost: 0.038 / 1_000_000, # the cost per input token
  output_token_cost: 0.15 / 1_000_000, # the cost per output token
})

# Then use the model
llm = Raif.llm(:open_router_gemini_flash_1_5_8b)
llm.chat(message: "Hello, world!")

# Or set it as the default LLM model in your initializer
Raif.configure do |config|
  config.default_llm_model_key = "open_router_gemini_flash_1_5_8b"
end

Read next: Testing