Thoughts on samurai?

[N-R-K]

Samurai is a reimplementation of ninja in C. It feels a lot closer in spirit of w64dk than the official ninja implementation. It's used as the default ninja provider in alpine (which I use for chroot/sandbox dev env) and I haven't run into any issue with it there.

Although going through it's README, it seems to expect quite a bit of POSIX-ism and windows support doesn't seem to be first class (?). Admittedly I did not try samurai on windows, so I can't tell how well it would or wouldn't work on windows/w64dk.

Also worth noting that muon - which does have first class windows support - ships with a samurai fork that patches samurai to use muon's "platform layer". Not sure if it's worth trying to pluck it out so that the fork can be compiled as a standalone binary.

(As an aside, muon is fairly well written with lots of "handmade" tech like platform layer, arena etc. I don't quite like meson, but if meson support is ever needed in w64dk, it could be worthwhile candidate to consider).

---

Not quite related to this issue, but I'm also thinking that llama.mak is probably useless now that w64dk ships with cmake+ninja? Or is there any reason to still maintain it?

[Peter0x44]

I have no comment or opinion on samurai or muon yet, just commenting on llama.mak

It "has superior build artifacts" to cmake, and it's not much effort to maintain (I test it periodically)
#364 (comment)

It serves at least as an important reference for anyone else trying to bypass cmake in such codebases.

[skeeto]

For me, over the past few months Ninja went from "that weird build tool CMake can use" to "impressive, efficient piece of systems software infrastructure." It's why I rigged w64dk's CMake to use Ninja by default, in contrast to CMake everywhere else (where I regret every time I forget -G Ninja). It's lived up to the robustness I've always wanted from GNU Make. I've already landed a patch upstream as a result of my w64dk Ninja usage. So I honestly have little reason to consider anything else!

My only complaints would be that its image is about 10x larger than an "ideal" Ninja due to C++ (though all my recent projects are looking like this now). That it doesn't work on XP (moot since CMake doesn't work there either). That it requires a UTF-8 manifest (moot since it doesn't work on XP). That's really about it.

I see samurai's os.h sort-of platform layer (you mentioned), though not yet exercised given the POSIX interfaces outside this file (poll, path assumptions, etc.). Plus, IMHO, platform layers must contain the entry point to be complete. I don't like that it builds up strings of POSIX shell commands then evaluates them with a shell. Where can I find muon's samurai patches (googling for "samurai patches" is hopeless)?

While writing this up, I came up with this:
https://github.com/skeeto/samurai/commits/windows/?author=skeeto

Took longer than I hoped. It requires sh.exe and a UTF-8 manifest (not included, I used LDLIBS=libwinsane.o). It's complete enough to build CMake and seems to work fine. A little more work could eliminate the UTF-8 manifest. A moderate amount of work could eliminate the shell dependency, though that's best tackled with some infrastructure improvements to not construct shell strings in the first place.

I'm aware of Muon, and I'm glad it exists (building systems software in Python is insanity), but most of the times I've tried it the thing I tried to build was too dependent on Meson to be built by Muon. It's been a couple years, though.

As previously discussed, I've done almost a 180 on CMake. So many sharp edges that trips up literally everyone from beginner to intermediate, but it's so, so powerful once you understand it. Being able to debug-step through builds with dcmake has already been more useful than I anticipated, and I miss it when facing Autoconf or Makefiles (and I'm sure I'd feel that again with Meson/Muon). I'm using dcmake in my professional work, and its development effort has already paid for itself.

Or is there any reason to still maintain it?

I'll keep it around as long as someone wants to maintain it! @Peter0x44's been staying on top of it. (Thanks!)

[Peter0x44]

I still use make under CMake occasionally, but only for very specific reasons.
As far as I can tell, the Ninja generator doesn't emit targets like this (though I could be wrong):
... src/dcmake.i
... src/dcmake.s
... src/dcmake.obj

It's quite convenient that I can use e.g. make src/dcmake.i to get preprocessed source or assembly directly out of a build tree when I need it. Helps when reporting compiler bugs.

[Peter0x44]

the llama.mak build for vulkan is currently broken, but this should resolve it:
ggml-org/llama.cpp#21918

[N-R-K]

Where can I find muon's samurai patches (googling for "samurai patches" is hopeless)?

I linked to muon's samurai fork here: https://github.com/muon-build/muon/tree/master/src/external/samurai

I don't think they maintain separate patches. So you'll probably just need to run some kind of directory diff to figure out what changed between this and upstream.

and I'm sure I'd feel that again with Meson/Muon

Muon actually has an LSP server. No DAP tho. But given that they have LSP already, they probably wouldn't object to adding DAP support similar to cmake --debugger if someone asks for it.

but most of the times I've tried it the thing I tried to build was too dependent on Meson to be built by Muon. It's been a couple years, though.

I think the situation probably has changed now. At least most of the meson stuff I come across seem to build fine with muon nowadays.

It "has superior build artifacts" to cmake

This kind of raises more questions than it answers :) Superior in terms of performance or not linking to dynamic libs or something else?

[skeeto]

the Ninja generator doesn't emit targets like this

How about?

$ cmake -B build -G Ninja -DCMAKE_CXX_FLAGS=-save-temps=obj

I linked to muon's samurai fork

Sorry, I didn't notice the link. I see the fork leverages Muon's platform layer. Seems like a decent port. I'll have to investigate all this more.

This kind of raises more questions than it answers :)

The official Mingw-w64 builds can't producellama.dll, while our llama.mak produces a DLL with a precise set of exports. (I keep thinking of maybe using it in a project someday.) Our llama-server.exe is a little smaller, though it looks like that gap has closed.

Also llama.mak broke last week on llama.cpp's 6690e2f1 and needs fixing. Had to re-check the above at an older commit.

[Peter0x44]

I pushed a fix before you even wrote the comment!

[Peter0x44]

Btw you should try qwen3.6 35moe it's really impressive.

[skeeto]

I pushed a fix before you even wrote the comment!

I've got to get used to the idea that new commits might appear in w64devkit without my doing!

[Peter0x44]

the Ninja generator doesn't emit targets like this

How about?

$ cmake -B build -G Ninja -DCMAKE_CXX_FLAGS=-save-temps=obj

This is going to require doing a reconfigure of the whole build. And the flags are naturally a bit fragile because stuff can choose to ignore/override them. But it probably would work for some cases.
Really, I haven't had any issues/bugs using the make generator. It's just slower than ninja.

[N-R-K]

Muon actually has an LSP server. No DAP tho. But given that they have LSP already, they probably wouldn't object to adding DAP support similar to cmake --debugger if someone asks for it.

Turns out they already have some WIP debugging support:
https://docs.muon.build/features.html?highlight=debugg#debugger-wip

No DAP (yet), I've opened an issue about it: muon-build/muon#249.

[skeeto]

Hmm, if it had DAP compatible enough with CMake then I could support it in dcmake, making this a pretty compelling addition to w64dk...

[Peter0x44]

Muon actually has an LSP server. No DAP tho. But given that they have LSP already, they probably wouldn't object to adding DAP support similar to cmake --debugger if someone asks for it.

Turns out they already have some WIP debugging support: https://docs.muon.build/features.html?highlight=debugg#debugger-wip

No DAP (yet), I've opened an issue about it: muon-build/muon#249.

Are you going to implement it if they say they want it?

Nice to see there is interest to do it upstream.