Class: Raif::ModelCompletion

Inherits:
ApplicationRecord
  • Object
show all
Includes:
Concerns::BooleanTimestamp, Concerns::HasAvailableModelTools, Concerns::HasRuntimeDuration, Concerns::LlmResponseParsing, Concerns::ProviderManagedToolCalls
Defined in:
app/models/raif/model_completion.rb

Overview

Schema Information

Table name: raif_model_completions

id :bigint not null, primary key available_model_tools :jsonb not null cache_creation_input_tokens :integer cache_read_input_tokens :integer citations :jsonb completed_at :datetime completion_tokens :integer failed_at :datetime failure_error :string failure_reason :text failure_response_body :text failure_response_status :integer llm_model_key :string not null max_completion_tokens :integer messages :jsonb not null model_api_name :string not null output_token_cost :decimal(10, 6) prompt_token_cost :decimal(10, 6) prompt_tokens :integer raw_response :text request_settings :jsonb response_array :jsonb response_finish_reason :string response_format :integer default("text"), not null response_format_parameter :string response_tool_calls :jsonb retry_count :integer default(0), not null source_type :string started_at :datetime stream_response :boolean default(FALSE), not null system_prompt :text temperature :decimal(5, 3) tool_choice :string total_cost :decimal(10, 6) total_tokens :integer created_at :datetime not null updated_at :datetime not null batch_custom_id :string raif_model_completion_batch_id :bigint response_id :string source_id :bigint

Indexes

index_raif_model_completions_on_batch_custom_id (batch_custom_id) index_raif_model_completions_on_batch_id_and_custom_id (raif_model_completion_batch_id,batch_custom_id) UNIQUE WHERE (raif_model_completion_batch_id IS NOT NULL) index_raif_model_completions_on_completed_at (completed_at) index_raif_model_completions_on_created_at (created_at) index_raif_model_completions_on_failed_at (failed_at) index_raif_model_completions_on_raif_model_completion_batch_id (raif_model_completion_batch_id) index_raif_model_completions_on_source (source_type,source_id) index_raif_model_completions_on_started_at (started_at)

Foreign Keys

fk_rails_... (raif_model_completion_batch_id => raif_model_completion_batches.id)

Constant Summary collapse

REQUEST_SETTING_KEYS =

Every key request_settings may carry, with the provider parameter each one controls. Validated, so the bag stays a declared set rather than a place anything can be stashed. Each has a Raif.config default of the same name, and an absent key defers to it.

open_ai_store_responses      Raif::Llms::OpenAiResponses `store`
open_router_data_collection  Raif::Llms::OpenRouter `provider.data_collection`
open_router_zdr              Raif::Llms::OpenRouter `provider.zdr`

The prompt caching and parallel tool call flags below stay request-scoped. Persisting Anthropic's would start sending cache_control on batched requests that omit it today, moving the bill in a direction that depends on prefix reuse against a 5-minute cache TTL - a cost change that belongs on its own. The other two have no batched request to change: Bedrock has no batch API, and every parallel-tool-call read sits behind a tool_choice that Raif::Task#prepare_for_batch! never sets.

%w[
  open_ai_store_responses
  open_router_data_collection
  open_router_zdr
].freeze
TRUNCATED_FINISH_REASONS =

Raw provider-reported finish/stop reasons that indicate the response was cut off before completing - either at the maximum output token limit, or (on Anthropic models) because the request exhausted the model's context window (model_context_window_exceeded). The response (including any tool calls in it) is incomplete and should not be trusted.

Deliberately excluded: content-filter stops (e.g. OpenAI's "content_filter" / incomplete_details.reason "content_filter"). Those responses are also cut short, but the truncation-recovery guidance ("be more concise and retry") would be wrong for them; their partial tool calls are still rejected by argument validation.

%w[max_output_tokens length max_tokens MAX_TOKENS incomplete model_context_window_exceeded].freeze
FAILURE_RESPONSE_BODY_MAX_CHARS =

Maximum number of characters of an upstream HTTP body we persist on failure. The body usually carries the provider's actual error reason (e.g. OpenAI/Anthropic structured error JSON), which failure_reason cannot fit in 255 chars. 4 KB is enough to capture realistic error payloads without bloating storage.

4_000
INFERENCE_COST_EVENT_SYNCED_COLUMNS =

Columns copied onto the inference cost event. A post-terminal change to any of them (e.g. batch results applying token counts after completed_at was already set) re-syncs the event so it stays a faithful mirror.

%w[
  source_type
  source_id
  llm_model_key
  model_api_name
  prompt_tokens
  completion_tokens
  total_tokens
  cache_read_input_tokens
  cache_creation_input_tokens
  prompt_token_cost
  output_token_cost
  total_cost
  retry_count
  raif_model_completion_batch_id
].freeze

Constants included from Concerns::LlmResponseParsing

Concerns::LlmResponseParsing::ASCII_CONTROL_CHARS

Instance Attribute Summary collapse

Instance Method Summary collapse

Methods included from Concerns::ProviderManagedToolCalls

#provider_managed_tool_calls, #sanitized_citations

Methods included from Concerns::HasRuntimeDuration

#runtime_duration, #runtime_duration_seconds, #runtime_ended_at

Methods included from Concerns::HasAvailableModelTools

#available_model_tools_map

Methods included from Concerns::LlmResponseParsing

#parse_html_response, #parse_json_response, #parsed_response

Instance Attribute Details

#allow_parallel_tool_calls ⇒ Object

Request-scoped (not persisted): when true, the provider request permits the model to return multiple tool calls. Adapters that can disable parallel tool use map this onto their provider parameter. Any value other than true (including nil, the default for an instance built outside Raif::Llm#chat) is treated as false (single call).



100
101
102
 
# File 'app/models/raif/model_completion.rb', line 100

def allow_parallel_tool_calls
  @allow_parallel_tool_calls
end

#anthropic_prompt_caching_enabled ⇒ Object

Returns the value of attribute anthropic_prompt_caching_enabled.



71
72
73
 
# File 'app/models/raif/model_completion.rb', line 71

def anthropic_prompt_caching_enabled
  @anthropic_prompt_caching_enabled
end

#bedrock_prompt_caching_enabled ⇒ Object

Returns the value of attribute bedrock_prompt_caching_enabled.



71
72
73
 
# File 'app/models/raif/model_completion.rb', line 71

def bedrock_prompt_caching_enabled
  @bedrock_prompt_caching_enabled
end

Instance Method Details

#calculate_costs ⇒ Object 



238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
 
# File 'app/models/raif/model_completion.rb', line 238

def calculate_costs
  # Each retry resends the same prompt, so the provider charges input tokens
  # for every attempt. Factor in retry_count to reflect actual billing.
  total_attempts = (retry_count || 0) + 1

  if prompt_tokens.present? && llm_config[:input_token_cost].present?
    self.prompt_token_cost = calculate_prompt_token_cost(total_attempts)
  end

  if completion_tokens.present? && llm_config[:output_token_cost].present?
    self.output_token_cost = llm_config[:output_token_cost] * completion_tokens
  end

  if prompt_token_cost.present? || output_token_cost.present?
    self.total_cost = (prompt_token_cost || 0) + (output_token_cost || 0)
  end

  apply_batch_inference_discount if raif_model_completion_batch_id.present?
end

#json_response_schema ⇒ Object 



230
231
232
 
# File 'app/models/raif/model_completion.rb', line 230

def json_response_schema
  source.json_response_schema if source&.respond_to?(:json_response_schema)
end

#open_ai_store_responses ⇒ Object

Provider data retention settings, resolved against Raif.config. An absent key means "use the config value", so read request_settings directly to tell an unset setting from an explicit false.



135
136
137
138
 
# File 'app/models/raif/model_completion.rb', line 135

def open_ai_store_responses
  value = request_settings["open_ai_store_responses"]
  value.nil? ? Raif.config.open_ai_store_responses : value
end

#open_ai_store_responses=(value) ⇒ Object 



140
141
142
 
# File 'app/models/raif/model_completion.rb', line 140

def open_ai_store_responses=(value)
  write_request_setting("open_ai_store_responses", value)
end

#open_router_data_collection ⇒ Object 



144
145
146
 
# File 'app/models/raif/model_completion.rb', line 144

def open_router_data_collection
  (request_settings["open_router_data_collection"].presence || Raif.config.open_router_data_collection).to_s
end

#open_router_data_collection=(value) ⇒ Object 



148
149
150
 
# File 'app/models/raif/model_completion.rb', line 148

def open_router_data_collection=(value)
  write_request_setting("open_router_data_collection", value&.to_s.presence)
end

#open_router_zdr ⇒ Object 



152
153
154
155
 
# File 'app/models/raif/model_completion.rb', line 152

def open_router_zdr
  value = request_settings["open_router_zdr"]
  value.nil? ? Raif.config.open_router_zdr : value
end

#open_router_zdr=(value) ⇒ Object 



157
158
159
 
# File 'app/models/raif/model_completion.rb', line 157

def open_router_zdr=(value)
  write_request_setting("open_router_zdr", value)
end

#pending? ⇒ Boolean

Returns:

  • (Boolean) 


128
129
130
 
# File 'app/models/raif/model_completion.rb', line 128

def pending?
  started_at.nil? && completed_at.nil? && failed_at.nil?
end

#record_failure!(exception) ⇒ Object 



265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
 
# File 'app/models/raif/model_completion.rb', line 265

def record_failure!(exception)
  self.failed_at = Time.current
  self.failure_error = exception.class.name
  self.failure_reason = exception.message.truncate(255)
  # Always clear before re-populating so a second call with a different
  # exception kind doesn't leave stale response metadata attached.
  self.failure_response_status = nil
  self.failure_response_body = nil

  # Faraday errors carry the provider's HTTP status and response body —
  # the latter is where the actual provider-side error reason lives. Both
  # are nil when the failure happened before a response was received
  # (DNS/connection refused/timeout).
  if exception.is_a?(Faraday::Error)
    self.failure_response_status = exception.response_status
    body = exception.response_body
    self.failure_response_body = body.to_s.first(FAILURE_RESPONSE_BODY_MAX_CHARS) if body.present?
  end

  save!
end

#set_total_tokens ⇒ Object 



234
235
236
 
# File 'app/models/raif/model_completion.rb', line 234

def set_total_tokens
  self.total_tokens ||= completion_tokens.present? && prompt_tokens.present? ? completion_tokens + prompt_tokens : nil
end

#tool_call_summary ⇒ Object

Admin-friendly summary of the tool calls this completion requested, e.g. "5: google_search_tool (4), web_search". Combines developer-managed calls (response_tool_calls) with provider-managed calls (provider_managed_tool_calls, e.g. OpenAI/Anthropic web search). nil when no tool calls were requested.



181
182
183
184
185
186
187
188
189
 
# File 'app/models/raif/model_completion.rb', line 181

def tool_call_summary
  names = Array(response_tool_calls).map { |call| call["name"] }
  names += provider_managed_tool_calls.map { |call| call["tool_name"] }
  names = names.compact
  return if names.empty?

  tally = names.tally.map { |name, count| count > 1 ? "#{name} (#{count})" : name }
  "#{names.length}: #{tally.join(", ")}"
end

#truncated? ⇒ Boolean

Returns:

  • (Boolean) 


173
174
175
 
# File 'app/models/raif/model_completion.rb', line 173

def truncated?
  TRUNCATED_FINISH_REASONS.include?(response_finish_reason)
end