Class: Raif::Evals::ScoreResult

Inherits:
Object
  • Object
show all
Defined in:
lib/raif/evals/score_result.rb

Overview

A number recorded by an eval, as opposed to the yes/no an ExpectationResult carries.

scale and higher_is_better travel with the value because evals:compare cannot otherwise tell an improvement from a regression - a latency that halved and a score that halved are the same arithmetic and opposite news.

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(name:, value:, scale: nil, higher_is_better: true, min: nil, max: nil) ⇒ ScoreResult

Returns a new instance of ScoreResult.



13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
 
# File 'lib/raif/evals/score_result.rb', line 13

def initialize(name:, value:, scale: nil, higher_is_better: true, min: nil, max: nil)
  # Checked rather than left to to_f, which turns nil and any non-numeric into a silent
  # 0.0 - dragging the mean toward zero and, against a max: ceiling, reporting a missing
  # measurement as the best possible result and passing the gate on it.
  unless value.is_a?(Numeric)
    raise ArgumentError, "score #{name.to_s.inspect} was given #{value.inspect}; a score value must be numeric"
  end

  @name = name.to_s
  @value = value.to_f
  @scale = scale
  @higher_is_better = higher_is_better
  @min = min
  @max = max
end

Instance Attribute Details

#max ⇒ Object (readonly)

Returns the value of attribute max.



11
12
13
 
# File 'lib/raif/evals/score_result.rb', line 11

def max
  @max
end

#min ⇒ Object (readonly)

Returns the value of attribute min.



11
12
13
 
# File 'lib/raif/evals/score_result.rb', line 11

def min
  @min
end

#name ⇒ Object (readonly)

Returns the value of attribute name.



11
12
13
 
# File 'lib/raif/evals/score_result.rb', line 11

def name
  @name
end

#scale ⇒ Object (readonly)

Returns the value of attribute scale.



11
12
13
 
# File 'lib/raif/evals/score_result.rb', line 11

def scale
  @scale
end

#value ⇒ Object (readonly)

Returns the value of attribute value.



11
12
13
 
# File 'lib/raif/evals/score_result.rb', line 11

def value
  @value
end

Instance Method Details

#formatted_value ⇒ Object

Integral values print without a trailing zero, so a 1-5 rubric score reads as "4" in a console line that already has several numbers in it.



53
54
55
 
# File 'lib/raif/evals/score_result.rb', line 53

def formatted_value
  (value % 1).zero? ? value.to_i.to_s : value.round(4).to_s
end

#gate_description ⇒ Object

Becomes the gating expectation's description, which evals:compare matches results on across runs.



43
44
45
46
47
48
49
 
# File 'lib/raif/evals/score_result.rb', line 43

def gate_description
  bounds = []
  bounds << ">= #{min}" unless min.nil?
  bounds << "<= #{max}" unless max.nil?

  "#{name} score #{bounds.join(" and ")}"
end

#gated? ⇒ Boolean

Returns:

  • (Boolean) 


33
34
35
 
# File 'lib/raif/evals/score_result.rb', line 33

def gated?
  !min.nil? || !max.nil?
end

#higher_is_better? ⇒ Boolean

Returns:

  • (Boolean) 


29
30
31
 
# File 'lib/raif/evals/score_result.rb', line 29

def higher_is_better?
  @higher_is_better
end

#passed? ⇒ Boolean

Returns:

  • (Boolean) 


37
38
39
 
# File 'lib/raif/evals/score_result.rb', line 37

def passed?
  (min.nil? || value >= min) && (max.nil? || value <= max)
end

#to_h ⇒ Object 



57
58
59
60
61
62
63
64
65
66
67
 
# File 'lib/raif/evals/score_result.rb', line 57

def to_h
  {
    name: name,
    value: value,
    scale: scale&.to_s,
    higher_is_better: higher_is_better?,
    min: min,
    max: max,
    passed: (passed? if gated?)
  }.compact
end