Class: Raif::ModelCompletion
- Inherits:
-
ApplicationRecord
- Object
- ApplicationRecord
- Raif::ModelCompletion
- Includes:
- Concerns::BooleanTimestamp, Concerns::HasAvailableModelTools, Concerns::HasRuntimeDuration, Concerns::LlmResponseParsing, Concerns::ProviderManagedToolCalls
- Defined in:
- app/models/raif/model_completion.rb
Overview
Schema Information
Table name: raif_model_completions
id :bigint not null, primary key available_model_tools :jsonb not null cache_creation_input_tokens :integer cache_read_input_tokens :integer citations :jsonb completed_at :datetime completion_tokens :integer failed_at :datetime failure_error :string failure_reason :text failure_response_body :text failure_response_status :integer llm_model_key :string not null max_completion_tokens :integer messages :jsonb not null model_api_name :string not null output_token_cost :decimal(10, 6) prompt_token_cost :decimal(10, 6) prompt_tokens :integer raw_response :text request_settings :jsonb response_array :jsonb response_finish_reason :string response_format :integer default("text"), not null response_format_parameter :string response_tool_calls :jsonb retry_count :integer default(0), not null source_type :string started_at :datetime stream_response :boolean default(FALSE), not null system_prompt :text temperature :decimal(5, 3) tool_choice :string total_cost :decimal(10, 6) total_tokens :integer created_at :datetime not null updated_at :datetime not null batch_custom_id :string raif_model_completion_batch_id :bigint response_id :string source_id :bigint
Indexes
index_raif_model_completions_on_batch_custom_id (batch_custom_id) index_raif_model_completions_on_batch_id_and_custom_id (raif_model_completion_batch_id,batch_custom_id) UNIQUE WHERE (raif_model_completion_batch_id IS NOT NULL) index_raif_model_completions_on_completed_at (completed_at) index_raif_model_completions_on_created_at (created_at) index_raif_model_completions_on_failed_at (failed_at) index_raif_model_completions_on_raif_model_completion_batch_id (raif_model_completion_batch_id) index_raif_model_completions_on_source (source_type,source_id) index_raif_model_completions_on_started_at (started_at)
Foreign Keys
fk_rails_... (raif_model_completion_batch_id => raif_model_completion_batches.id)
Constant Summary collapse
- REQUEST_SETTING_KEYS =
Every key request_settings may carry, with the provider parameter each one controls. Validated, so the bag stays a declared set rather than a place anything can be stashed. The first three each have a Raif.config default of the same name, and an absent key defers to it. An absent stop_sequences sends none. It is persisted because a batch request is built from this row at submit time, not from the task that created it.
open_ai_store_responses Raif::Llms::OpenAiResponses `store` open_router_data_collection Raif::Llms::OpenRouter `provider.data_collection` open_router_zdr Raif::Llms::OpenRouter `provider.zdr` stop_sequences Raif::Llms::Anthropic `stop_sequences`, Raif::Llms::Google `stopSequences`The prompt caching and parallel tool call flags below stay request-scoped. Persisting Anthropic's would start sending
cache_controlon batched requests that omit it today, moving the bill in a direction that depends on prefix reuse against a 5-minute cache TTL - a cost change that belongs on its own. The other two have no batched request to change: Bedrock has no batch API, and every parallel-tool-call read sits behind a tool_choice that Raif::Task#prepare_for_batch! never sets. %w[ open_ai_store_responses open_router_data_collection open_router_zdr stop_sequences ].freeze
- MAX_STOP_SEQUENCES =
Google rejects a request with more than 5 stop sequences.
5- TRUNCATED_FINISH_REASONS =
Raw provider-reported finish/stop reasons that indicate the response was cut off before completing - either at the maximum output token limit, or (on Anthropic models) because the request exhausted the model's context window (model_context_window_exceeded), or (on Gemini 4 models) because a non-streaming request reached Google's per-request output cap (CONTINUATION). The response (including any tool calls in it) is incomplete and should not be trusted.
Deliberately excluded: content-filter stops (e.g. OpenAI's "content_filter" / incomplete_details.reason "content_filter"). Those responses are also cut short, but the truncation-recovery guidance ("be more concise and retry") would be wrong for them; their partial tool calls are still rejected by argument validation.
%w[ max_output_tokens length max_tokens MAX_TOKENS CONTINUATION incomplete model_context_window_exceeded ].freeze
- FAILURE_RESPONSE_BODY_MAX_CHARS =
Maximum number of characters of an upstream HTTP body we persist on failure. The body usually carries the provider's actual error reason (e.g. OpenAI/Anthropic structured error JSON), which
failure_reasoncannot fit in 255 chars. 4 KB is enough to capture realistic error payloads without bloating storage. 4_000- INFERENCE_COST_EVENT_SYNCED_COLUMNS =
Columns copied onto the inference cost event. A post-terminal change to any of them (e.g. batch results applying token counts after completed_at was already set) re-syncs the event so it stays a faithful mirror.
%w[ source_type source_id llm_model_key model_api_name prompt_tokens completion_tokens total_tokens cache_read_input_tokens cache_creation_input_tokens prompt_token_cost output_token_cost total_cost retry_count raif_model_completion_batch_id ].freeze
Constants included from Concerns::LlmResponseParsing
Concerns::LlmResponseParsing::ASCII_CONTROL_CHARS
Instance Attribute Summary collapse
-
#allow_parallel_tool_calls ⇒ Object
Request-scoped (not persisted): when true, the provider request permits the model to return multiple tool calls.
-
#anthropic_prompt_caching_enabled ⇒ Object
Returns the value of attribute anthropic_prompt_caching_enabled.
-
#bedrock_prompt_caching_enabled ⇒ Object
Returns the value of attribute bedrock_prompt_caching_enabled.
Class Method Summary collapse
Instance Method Summary collapse
- #calculate_costs ⇒ Object
- #json_response_schema ⇒ Object
-
#open_ai_store_responses ⇒ Object
Provider data retention settings, resolved against Raif.config.
- #open_ai_store_responses=(value) ⇒ Object
- #open_router_data_collection ⇒ Object
- #open_router_data_collection=(value) ⇒ Object
- #open_router_zdr ⇒ Object
- #open_router_zdr=(value) ⇒ Object
- #pending? ⇒ Boolean
- #record_failure!(exception) ⇒ Object
- #set_total_tokens ⇒ Object
- #stop_sequences ⇒ Object
- #stop_sequences=(value) ⇒ Object
-
#tool_call_summary ⇒ Object
Admin-friendly summary of the tool calls this completion requested, e.g.
- #truncated? ⇒ Boolean
Methods included from Concerns::ProviderManagedToolCalls
#provider_managed_image_output?, #provider_managed_tool_calls, #sanitized_citations, #web_search_source_urls
Methods included from Concerns::HasRuntimeDuration
#runtime_duration, #runtime_duration_seconds, #runtime_ended_at
Methods included from Concerns::HasAvailableModelTools
Methods included from Concerns::LlmResponseParsing
#parse_html_response, #parse_json_response, #parsed_response
Instance Attribute Details
#allow_parallel_tool_calls ⇒ Object
Request-scoped (not persisted): when true, the provider request permits the model to return multiple tool calls. Adapters that can disable parallel tool use map this onto their provider parameter. Any value other than true (including nil, the default for an instance built outside Raif::Llm#chat) is treated as false (single call).
104 105 106 |
# File 'app/models/raif/model_completion.rb', line 104 def allow_parallel_tool_calls @allow_parallel_tool_calls end |
#anthropic_prompt_caching_enabled ⇒ Object
Returns the value of attribute anthropic_prompt_caching_enabled.
71 72 73 |
# File 'app/models/raif/model_completion.rb', line 71 def anthropic_prompt_caching_enabled @anthropic_prompt_caching_enabled end |
#bedrock_prompt_caching_enabled ⇒ Object
Returns the value of attribute bedrock_prompt_caching_enabled.
71 72 73 |
# File 'app/models/raif/model_completion.rb', line 71 def bedrock_prompt_caching_enabled @bedrock_prompt_caching_enabled end |
Class Method Details
.valid_stop_sequences?(stop_sequences) ⇒ Boolean
168 169 170 171 172 |
# File 'app/models/raif/model_completion.rb', line 168 def self.valid_stop_sequences?(stop_sequences) stop_sequences.is_a?(Array) && stop_sequences.size.between?(1, MAX_STOP_SEQUENCES) && stop_sequences.all? { |sequence| sequence.is_a?(String) && sequence.present? } end |
Instance Method Details
#calculate_costs ⇒ Object
262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 |
# File 'app/models/raif/model_completion.rb', line 262 def calculate_costs # Each retry resends the same prompt, so the provider charges input tokens # for every attempt. Factor in retry_count to reflect actual billing. total_attempts = (retry_count || 0) + 1 rates = token_rates if prompt_tokens.present? && rates[:input].present? self.prompt_token_cost = calculate_prompt_token_cost(total_attempts, rates) end if completion_tokens.present? && rates[:output].present? self.output_token_cost = rates[:output] * completion_tokens end if prompt_token_cost.present? || output_token_cost.present? self.total_cost = (prompt_token_cost || 0) + (output_token_cost || 0) end apply_batch_inference_discount if raif_model_completion_batch_id.present? end |
#json_response_schema ⇒ Object
254 255 256 |
# File 'app/models/raif/model_completion.rb', line 254 def json_response_schema source.json_response_schema if source&.respond_to?(:json_response_schema) end |
#open_ai_store_responses ⇒ Object
Provider data retention settings, resolved against Raif.config. An absent key means "use the config value", so read request_settings directly to tell an unset setting from an explicit false.
139 140 141 142 |
# File 'app/models/raif/model_completion.rb', line 139 def open_ai_store_responses value = request_settings["open_ai_store_responses"] value.nil? ? Raif.config.open_ai_store_responses : value end |
#open_ai_store_responses=(value) ⇒ Object
144 145 146 |
# File 'app/models/raif/model_completion.rb', line 144 def open_ai_store_responses=(value) write_request_setting("open_ai_store_responses", value) end |
#open_router_data_collection ⇒ Object
148 149 150 |
# File 'app/models/raif/model_completion.rb', line 148 def open_router_data_collection (request_settings["open_router_data_collection"].presence || Raif.config.open_router_data_collection).to_s end |
#open_router_data_collection=(value) ⇒ Object
152 153 154 |
# File 'app/models/raif/model_completion.rb', line 152 def open_router_data_collection=(value) write_request_setting("open_router_data_collection", value&.to_s.presence) end |
#open_router_zdr ⇒ Object
156 157 158 159 |
# File 'app/models/raif/model_completion.rb', line 156 def open_router_zdr value = request_settings["open_router_zdr"] value.nil? ? Raif.config.open_router_zdr : value end |
#open_router_zdr=(value) ⇒ Object
161 162 163 |
# File 'app/models/raif/model_completion.rb', line 161 def open_router_zdr=(value) write_request_setting("open_router_zdr", value) end |
#pending? ⇒ Boolean
132 133 134 |
# File 'app/models/raif/model_completion.rb', line 132 def pending? started_at.nil? && completed_at.nil? && failed_at.nil? end |
#record_failure!(exception) ⇒ Object
290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 |
# File 'app/models/raif/model_completion.rb', line 290 def record_failure!(exception) self.failed_at = Time.current self.failure_error = exception.class.name self.failure_reason = exception..truncate(255) # Always clear before re-populating so a second call with a different # exception kind doesn't leave stale response metadata attached. self.failure_response_status = nil self.failure_response_body = nil # Faraday errors carry the provider's HTTP status and response body — # the latter is where the actual provider-side error reason lives. Both # are nil when the failure happened before a response was received # (DNS/connection refused/timeout). if exception.is_a?(Faraday::Error) self.failure_response_status = exception.response_status body = exception.response_body self.failure_response_body = body.to_s.first(FAILURE_RESPONSE_BODY_MAX_CHARS) if body.present? end save! end |
#set_total_tokens ⇒ Object
258 259 260 |
# File 'app/models/raif/model_completion.rb', line 258 def set_total_tokens self.total_tokens ||= completion_tokens.present? && prompt_tokens.present? ? completion_tokens + prompt_tokens : nil end |
#stop_sequences ⇒ Object
174 175 176 |
# File 'app/models/raif/model_completion.rb', line 174 def stop_sequences request_settings["stop_sequences"] end |
#stop_sequences=(value) ⇒ Object
178 179 180 |
# File 'app/models/raif/model_completion.rb', line 178 def stop_sequences=(value) write_request_setting("stop_sequences", value.presence) end |
#tool_call_summary ⇒ Object
Admin-friendly summary of the tool calls this completion requested, e.g. "5: google_search_tool (4), web_search". Combines developer-managed calls (response_tool_calls) with provider-managed calls (provider_managed_tool_calls, e.g. OpenAI/Anthropic web search). nil when no tool calls were requested.
205 206 207 208 209 210 211 212 213 |
# File 'app/models/raif/model_completion.rb', line 205 def tool_call_summary names = Array(response_tool_calls).map { |call| call["name"] } names += provider_managed_tool_calls.map { |call| call["tool_name"] } names = names.compact return if names.empty? tally = names.tally.map { |name, count| count > 1 ? "#{name} (#{count})" : name } "#{names.length}: #{tally.join(", ")}" end |
#truncated? ⇒ Boolean
197 198 199 |
# File 'app/models/raif/model_completion.rb', line 197 def truncated? TRUNCATED_FINISH_REASONS.include?(response_finish_reason) end |