feat: try Suggestion of Blacksmith

[Vinay-Umrethe]

Trying this "drop in replacement" suggestion of using: blacksmith-4vcpu-ubuntu-2404 instead of ubuntu-latest for Github CI tests...

Not sure how reliable it would be or even worth it or not. Let's see.

Suggested by @red40maxxer on discord.

[gemini-code-assist]

Note

Gemini is unable to generate a review for this pull request due to the file types involved not being currently supported.

[Vinay-Umrethe]

I've cancelled the CI to run since it probably requires installing blacksmith to heretic / p-e-w org first so the request is actually sent.

/cc
@p-e-w
@red40maxxer

[p-e-w]

Thanks for kicking this off!

I'm looking at Blacksmith's offerings and I'm not sure I understand them correctly. What part is actually free? The listing is confusing because it gives a price of $0.004/minute for Ubuntu and $0.08/minute for macOS, but both pages claim 3000 free minutes per month. It's a bit difficult to believe that they would give 3000 free minutes on macOS, since that would equate to $240/month at their billing rate.

@red40maxxer I believe you mentioned that you have experience with Blacksmith. Could you provide some insight on this?

The macOS setup could be a game-changer because an M4 can run real models, so we could test whether abliteration actually suppresses refusals.

[Vinay-Umrethe]

Good to know they even provide M4 wow.

[red40maxxer]

It's a bit difficult to believe that they would give 3000 free minutes on macOS, since that would equate to $240/month at their billing rate.

As far as I'm aware, this is allowed as I've tried this out with both macOS and Windows runners and haven't been billed. I don't think this price point is something we should build infrastructure around as it's clearly a temporary VC subsidy, but we 100% could replace all of our existing Github runners with the Ubuntu ones at the current usage rate and not pay anything, it's TBD whether this helps with the reproducibility issue though

[p-e-w]

I finally tried to set up Blacksmith today, but I found to my surprise that they require permission to "Act on your behalf". I don't think I'm ready to give that kind of authorization to a third-party app at this time 😞