Module: Raif::Evals::ModelCompletionSink

Defined in:

lib/raif/evals/model_completion_sink.rb

Overview

Collects the Raif::ModelCompletion records created while one execution runs. EvalSet opens it around the whole execution and splits the result into the eval block's own calls and what setup and teardown spent around them.

Holds the same objects Raif::Llm mutates in place as a request progresses, so tokens and cost are read off the live records rather than re-queried out of a transaction that is about to be rolled back.

Scoped per thread (per fiber under isolation_level: :fiber), so concurrent evals stay correct. The trade is that a completion created on another thread mid-eval is not captured.

Constant Summary

STATE_KEY =

:raif_evals_model_completion_sink

Class Method Summary

• .close ⇒ Object
• .open ⇒ Object

  Starts collecting and returns the array completions will be pushed onto.

• .record(model_completion) ⇒ Object

  Called from the create.raif_model_completion subscriber in raif/evals.rb, so it sees every completion however it was created, including ones a host app's own code or a factory creates.

Class Method Details

.close ⇒ Object

# File 'lib/raif/evals/model_completion_sink.rb', line 25

def close
  ActiveSupport::IsolatedExecutionState.delete(STATE_KEY)
end

.open ⇒ Object

Starts collecting and returns the array completions will be pushed onto. The caller holds that array, so it can read what was collected after #close.

# File 'lib/raif/evals/model_completion_sink.rb', line 21

def open
  ActiveSupport::IsolatedExecutionState[STATE_KEY] = []
end

.record(model_completion) ⇒ Object

Called from the create.raif_model_completion subscriber in raif/evals.rb, so it sees every completion however it was created, including ones a host app's own code or a factory creates. A no-op unless an eval has opened a sink on this thread.

# File 'lib/raif/evals/model_completion_sink.rb', line 32

def record(model_completion)
  ActiveSupport::IsolatedExecutionState[STATE_KEY]&.push(model_completion)
end