Tweak score function

[nuwang]

supercedes: #134

[coveralls]

Pull Request Test Coverage Report for Build 22258746196

Details

• 13 of 13 (100.0%) changed or added relevant lines in 1 file are covered.
• No unchanged relevant lines lost coverage.
• Overall coverage increased (+0.02%) to 95.604%

---

Totals
Change from base Build 22251655819:	0.02%
Covered Lines:	1384
Relevant Lines:	1412

---

💛 - Coveralls

[cat-bro]

Hi Nuwan, does this mean that score is still additive with accept and require affecting the outcome? If that is so, then my example from #134 still holds. Could the numbers instead require: 0, prefer: 1, accept: 0? Require and accept should be neutral.

[nuwang]

Yes and no. The require counts the same as accept, and therefore, both destinations will score equally, as it should?

[cat-bro]

I disagree. There is no reason for accept and require to add to score.

[nuwang]

Take a look at the limbo tool in test_job_args_match_helper. I think that's a good example for why simply ignoring require tags won't work.

[cat-bro]

I can see that prefer needs to be multiplied by a non-zero to add to the score, however accept x require will add, and all absent tags subtract. I’m not convinced that this is going to lead to fair scores when only prefer should matter. If you look at AU’s destinations, each one has 4-10 accept tags that will have nothing to do with most of the tools, and with ‘accept’ now having weight 1, does this now come into play in the subtraction? I’d argue that the implementation of this function has always been a bit strange, even if it works for our less detailed examples in the test cases. I’ll try to make a test case next week to demonstrate this.

[nuwang]

Sure. Sounds good. Just to be clear - I'd be much happier with a simpler approach - however, it needs to be something that deals with a variety of conditions, including:

1. The presence of multiple tags should be a "vote" that can mutually reinforce the score.
2. The absence of a tag should count as an absence of a vote. (e.g. tool a accepts tag A and B, but a destination that accepts only one should count less than a destination that accepts both)
3. Two tags that mutually reinforce each other should probably count more than just two tags that tolerate each other. E.g. if a tool prefers a destination, and a destination prefers a tool, should it not count more than a destination that merely accepts the tool?
4. If a destination requires a tag, and a tool prefers that tag (limbo example), then that should not count the same as a destination that completely lacks the tag. Put differently, the presence of the require tag is a mutually reinforcing vote in this case.

I think our test cases contain most of these scenarios, so if we can get the tests to pass, chances are it should be a good replacement.

[cat-bro]

(3) makes sense if only ‘prefer’ combined with another positive tag (it doesn’t matter which one) adds to the score, and a number of prefer/positive-tag combos adding up does make sense.

(1), (2) and (4) are what I am trying to avoid. The bias that they introduce seems unhelpful, and has been unhelpful for real-world scheduling on AU. I’ll have to tap out of this debate until Monday :)