Skip to content

Conversation

@rvagg
Copy link
Member

@rvagg rvagg commented Jun 7, 2023

Getting this work up in a Draft for now so I can pause and think about something else for a little while.

This is an attempt to create a simulation of a set of retrievals in such a way that we can tweak the scoring mechanism (weights etc.) to see how/if they make a difference.

This works, (run main in pkg/session/model/cmd), but the averaging of provider behaviour across runs is causing any tweaking to get lost in the wash. I need to set up a static list of providers with fixed(ish?) characteristics up front so they can be sorted in more meaningful ways and their score comparisons have more impact.

@codecov-commenter
Copy link

Codecov Report

Merging #298 (1bcd218) into main (fe9e760) will decrease coverage by 3.29%.
The diff coverage is 3.28%.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #298      +/-   ##
==========================================
- Coverage   72.55%   69.27%   -3.29%     
==========================================
  Files          68       72       +4     
  Lines        6217     6519     +302     
==========================================
+ Hits         4511     4516       +5     
- Misses       1458     1753     +295     
- Partials      248      250       +2     
Impacted Files Coverage Δ
pkg/session/model/cmd/main.go 0.00% <0.00%> (ø)
pkg/session/model/pop.go 0.00% <0.00%> (ø)
pkg/session/model/prob.go 0.00% <0.00%> (ø)
pkg/session/model/sim.go 0.00% <0.00%> (ø)
pkg/session/state.go 92.11% <90.90%> (+0.30%) ⬆️

... and 2 files with indirect coverage changes

@rvagg rvagg force-pushed the rvagg/score-modelling branch from 1bcd218 to f7f497d Compare June 7, 2023 09:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants