Class: Raif::ModelCompletion

Inherits:
ApplicationRecord
  • Object
show all
Includes:
Concerns::BooleanTimestamp, Concerns::HasAvailableModelTools, Concerns::HasRuntimeDuration, Concerns::LlmResponseParsing, Concerns::ProviderManagedToolCalls
Defined in:
app/models/raif/model_completion.rb

Overview

Schema Information

Table name: raif_model_completions

id :bigint not null, primary key available_model_tools :jsonb not null cache_creation_input_tokens :integer cache_read_input_tokens :integer citations :jsonb completed_at :datetime completion_tokens :integer failed_at :datetime failure_error :string failure_reason :text failure_response_body :text failure_response_status :integer llm_model_key :string not null max_completion_tokens :integer messages :jsonb not null model_api_name :string not null output_token_cost :decimal(10, 6) prompt_token_cost :decimal(10, 6) prompt_tokens :integer raw_response :text request_settings :jsonb response_array :jsonb response_finish_reason :string response_format :integer default("text"), not null response_format_parameter :string response_tool_calls :jsonb retry_count :integer default(0), not null source_type :string started_at :datetime stream_response :boolean default(FALSE), not null system_prompt :text temperature :decimal(5, 3) tool_choice :string total_cost :decimal(10, 6) total_tokens :integer created_at :datetime not null updated_at :datetime not null batch_custom_id :string raif_model_completion_batch_id :bigint response_id :string source_id :bigint

Indexes

index_raif_model_completions_on_batch_custom_id (batch_custom_id) index_raif_model_completions_on_batch_id_and_custom_id (raif_model_completion_batch_id,batch_custom_id) UNIQUE WHERE (raif_model_completion_batch_id IS NOT NULL) index_raif_model_completions_on_completed_at (completed_at) index_raif_model_completions_on_created_at (created_at) index_raif_model_completions_on_failed_at (failed_at) index_raif_model_completions_on_raif_model_completion_batch_id (raif_model_completion_batch_id) index_raif_model_completions_on_source (source_type,source_id) index_raif_model_completions_on_started_at (started_at)

Foreign Keys

fk_rails_... (raif_model_completion_batch_id => raif_model_completion_batches.id)

Constant Summary collapse

REQUEST_SETTING_KEYS =

Every key request_settings may carry, with the provider parameter each one controls. Validated, so the bag stays a declared set rather than a place anything can be stashed. The first three each have a Raif.config default of the same name, and an absent key defers to it. An absent stop_sequences sends none. It is persisted because a batch request is built from this row at submit time, not from the task that created it.

open_ai_store_responses      Raif::Llms::OpenAiResponses `store`
open_router_data_collection  Raif::Llms::OpenRouter `provider.data_collection`
open_router_zdr              Raif::Llms::OpenRouter `provider.zdr`
stop_sequences               Raif::Llms::Anthropic `stop_sequences`, Raif::Llms::Google `stopSequences`

The prompt caching and parallel tool call flags below stay request-scoped. Persisting Anthropic's would start sending cache_control on batched requests that omit it today, moving the bill in a direction that depends on prefix reuse against a 5-minute cache TTL - a cost change that belongs on its own. The other two have no batched request to change: Bedrock has no batch API, and every parallel-tool-call read sits behind a tool_choice that Raif::Task#prepare_for_batch! never sets.

%w[
  open_ai_store_responses
  open_router_data_collection
  open_router_zdr
  stop_sequences
].freeze
MAX_STOP_SEQUENCES =

Google rejects a request with more than 5 stop sequences.

5
TRUNCATED_FINISH_REASONS =

Raw provider-reported finish/stop reasons that indicate the response was cut off before completing - either at the maximum output token limit, or (on Anthropic models) because the request exhausted the model's context window (model_context_window_exceeded), or (on Gemini 4 models) because a non-streaming request reached Google's per-request output cap (CONTINUATION). The response (including any tool calls in it) is incomplete and should not be trusted.

Deliberately excluded: content-filter stops (e.g. OpenAI's "content_filter" / incomplete_details.reason "content_filter"). Those responses are also cut short, but the truncation-recovery guidance ("be more concise and retry") would be wrong for them; their partial tool calls are still rejected by argument validation.

%w[
  max_output_tokens length max_tokens MAX_TOKENS CONTINUATION incomplete model_context_window_exceeded
].freeze
FAILURE_RESPONSE_BODY_MAX_CHARS =

Maximum number of characters of an upstream HTTP body we persist on failure. The body usually carries the provider's actual error reason (e.g. OpenAI/Anthropic structured error JSON), which failure_reason cannot fit in 255 chars. 4 KB is enough to capture realistic error payloads without bloating storage.

4_000
INFERENCE_COST_EVENT_SYNCED_COLUMNS =

Columns copied onto the inference cost event. A post-terminal change to any of them (e.g. batch results applying token counts after completed_at was already set) re-syncs the event so it stays a faithful mirror.

%w[
  source_type
  source_id
  llm_model_key
  model_api_name
  prompt_tokens
  completion_tokens
  total_tokens
  cache_read_input_tokens
  cache_creation_input_tokens
  prompt_token_cost
  output_token_cost
  total_cost
  retry_count
  raif_model_completion_batch_id
].freeze

Constants included from Concerns::LlmResponseParsing

Concerns::LlmResponseParsing::ASCII_CONTROL_CHARS

Instance Attribute Summary collapse

Class Method Summary collapse

Instance Method Summary collapse

Methods included from Concerns::ProviderManagedToolCalls

#provider_managed_image_output?, #provider_managed_tool_calls, #sanitized_citations, #web_search_source_urls

Methods included from Concerns::HasRuntimeDuration

#runtime_duration, #runtime_duration_seconds, #runtime_ended_at

Methods included from Concerns::HasAvailableModelTools

#available_model_tools_map

Methods included from Concerns::LlmResponseParsing

#parse_html_response, #parse_json_response, #parsed_response

Instance Attribute Details

#allow_parallel_tool_calls ⇒ Object

Request-scoped (not persisted): when true, the provider request permits the model to return multiple tool calls. Adapters that can disable parallel tool use map this onto their provider parameter. Any value other than true (including nil, the default for an instance built outside Raif::Llm#chat) is treated as false (single call).



104
105
106
# File 'app/models/raif/model_completion.rb', line 104

def allow_parallel_tool_calls
  @allow_parallel_tool_calls
end

#anthropic_prompt_caching_enabled ⇒ Object

Returns the value of attribute anthropic_prompt_caching_enabled.



71
72
73
# File 'app/models/raif/model_completion.rb', line 71

def anthropic_prompt_caching_enabled
  @anthropic_prompt_caching_enabled
end

#bedrock_prompt_caching_enabled ⇒ Object

Returns the value of attribute bedrock_prompt_caching_enabled.



71
72
73
# File 'app/models/raif/model_completion.rb', line 71

def bedrock_prompt_caching_enabled
  @bedrock_prompt_caching_enabled
end

Class Method Details

.valid_stop_sequences?(stop_sequences) ⇒ Boolean

Returns:

  • (Boolean)


168
169
170
171
172
# File 'app/models/raif/model_completion.rb', line 168

def self.valid_stop_sequences?(stop_sequences)
  stop_sequences.is_a?(Array) &&
    stop_sequences.size.between?(1, MAX_STOP_SEQUENCES) &&
    stop_sequences.all? { |sequence| sequence.is_a?(String) && sequence.present? }
end

Instance Method Details

#calculate_costs ⇒ Object



262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
# File 'app/models/raif/model_completion.rb', line 262

def calculate_costs
  # Each retry resends the same prompt, so the provider charges input tokens
  # for every attempt. Factor in retry_count to reflect actual billing.
  total_attempts = (retry_count || 0) + 1
  rates = token_rates

  if prompt_tokens.present? && rates[:input].present?
    self.prompt_token_cost = calculate_prompt_token_cost(total_attempts, rates)
  end

  if completion_tokens.present? && rates[:output].present?
    self.output_token_cost = rates[:output] * completion_tokens
  end

  if prompt_token_cost.present? || output_token_cost.present?
    self.total_cost = (prompt_token_cost || 0) + (output_token_cost || 0)
  end

  apply_batch_inference_discount if raif_model_completion_batch_id.present?
end

#json_response_schema ⇒ Object



254
255
256
# File 'app/models/raif/model_completion.rb', line 254

def json_response_schema
  source.json_response_schema if source&.respond_to?(:json_response_schema)
end

#open_ai_store_responses ⇒ Object

Provider data retention settings, resolved against Raif.config. An absent key means "use the config value", so read request_settings directly to tell an unset setting from an explicit false.



139
140
141
142
# File 'app/models/raif/model_completion.rb', line 139

def open_ai_store_responses
  value = request_settings["open_ai_store_responses"]
  value.nil? ? Raif.config.open_ai_store_responses : value
end

#open_ai_store_responses=(value) ⇒ Object



144
145
146
# File 'app/models/raif/model_completion.rb', line 144

def open_ai_store_responses=(value)
  write_request_setting("open_ai_store_responses", value)
end

#open_router_data_collection ⇒ Object



148
149
150
# File 'app/models/raif/model_completion.rb', line 148

def open_router_data_collection
  (request_settings["open_router_data_collection"].presence || Raif.config.open_router_data_collection).to_s
end

#open_router_data_collection=(value) ⇒ Object



152
153
154
# File 'app/models/raif/model_completion.rb', line 152

def open_router_data_collection=(value)
  write_request_setting("open_router_data_collection", value&.to_s.presence)
end

#open_router_zdr ⇒ Object



156
157
158
159
# File 'app/models/raif/model_completion.rb', line 156

def open_router_zdr
  value = request_settings["open_router_zdr"]
  value.nil? ? Raif.config.open_router_zdr : value
end

#open_router_zdr=(value) ⇒ Object



161
162
163
# File 'app/models/raif/model_completion.rb', line 161

def open_router_zdr=(value)
  write_request_setting("open_router_zdr", value)
end

#pending? ⇒ Boolean

Returns:

  • (Boolean)


132
133
134
# File 'app/models/raif/model_completion.rb', line 132

def pending?
  started_at.nil? && completed_at.nil? && failed_at.nil?
end

#record_failure!(exception) ⇒ Object



290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
# File 'app/models/raif/model_completion.rb', line 290

def record_failure!(exception)
  self.failed_at = Time.current
  self.failure_error = exception.class.name
  self.failure_reason = exception.message.truncate(255)
  # Always clear before re-populating so a second call with a different
  # exception kind doesn't leave stale response metadata attached.
  self.failure_response_status = nil
  self.failure_response_body = nil

  # Faraday errors carry the provider's HTTP status and response body —
  # the latter is where the actual provider-side error reason lives. Both
  # are nil when the failure happened before a response was received
  # (DNS/connection refused/timeout).
  if exception.is_a?(Faraday::Error)
    self.failure_response_status = exception.response_status
    body = exception.response_body
    self.failure_response_body = body.to_s.first(FAILURE_RESPONSE_BODY_MAX_CHARS) if body.present?
  end

  save!
end

#set_total_tokens ⇒ Object



258
259
260
# File 'app/models/raif/model_completion.rb', line 258

def set_total_tokens
  self.total_tokens ||= completion_tokens.present? && prompt_tokens.present? ? completion_tokens + prompt_tokens : nil
end

#stop_sequences ⇒ Object



174
175
176
# File 'app/models/raif/model_completion.rb', line 174

def stop_sequences
  request_settings["stop_sequences"]
end

#stop_sequences=(value) ⇒ Object



178
179
180
# File 'app/models/raif/model_completion.rb', line 178

def stop_sequences=(value)
  write_request_setting("stop_sequences", value.presence)
end

#tool_call_summary ⇒ Object

Admin-friendly summary of the tool calls this completion requested, e.g. "5: google_search_tool (4), web_search". Combines developer-managed calls (response_tool_calls) with provider-managed calls (provider_managed_tool_calls, e.g. OpenAI/Anthropic web search). nil when no tool calls were requested.



205
206
207
208
209
210
211
212
213
# File 'app/models/raif/model_completion.rb', line 205

def tool_call_summary
  names = Array(response_tool_calls).map { |call| call["name"] }
  names += provider_managed_tool_calls.map { |call| call["tool_name"] }
  names = names.compact
  return if names.empty?

  tally = names.tally.map { |name, count| count > 1 ? "#{name} (#{count})" : name }
  "#{names.length}: #{tally.join(", ")}"
end

#truncated? ⇒ Boolean

Returns:

  • (Boolean)


197
198
199
# File 'app/models/raif/model_completion.rb', line 197

def truncated?
  TRUNCATED_FINISH_REASONS.include?(response_finish_reason)
end