vNUMA support

[ohault]

Any experiences yet of NUMA support with Ski?

I would like to try vNUMA project.

Reference:

• https://trustworthy.systems/projects/OLD/virtualisation/vnuma

[johnny-mnemonic]

Any experiences yet of NUMA support with Ski?

Not yet, but I have recently extensively tested multiple Ski instances running as a distcc compile cluster (1 client, 6 volunteers, running EPIC Slack)*, which worked quite well after some modifications to distcc (mainly timeouts) to take into account the limited performance of the simulation (e.g. 106 D(hrystone)MIPS on an i7 4790K overclocked to 5 GHz on boost). I also used a modified memory map (3 GiB w/o holes) provided by the ski-bootloader as sort of firmware. No crashes ovserved so far.

*) Simulated on a Ryzen 7 3700X (4.4 GHz max) based sytem with 16 GiB of memory and disk images served from a btrfs volume with snapshots for client and volunteer images to save on-disk space and located on a SATA SSD.

But Single System Image clustering would be a far greater feat.

Are you gonna forward-port those VNUMA patches? You can find ia64 patches for Linux stable on https://github.com/linux-ia64/linux-stable-rc/, Ski patches for Linux stable w/ia64 support on https://github.com/linux-ia64/ski/tree/hp-sim-for-linux. Or if you prefer mainline, you can also use the respective per release (candidate) branches on https://github.com/johnny-mnemonic/linux-ia64/, the latest HP-Sim platform patch should work on the latest patch set, too.

Prior test results for Ski running Linux mainline can be accessed from http://epic-linux.org/#!/machines/hp-sim/ .

[trofi]

Any experiences yet of NUMA support with Ski
I would like to try vNUMA project.
Reference:
https://trustworthy.systems/projects/OLD/virtualisation/vnuma

I vaguely remember you asking the same question over email. All copy the answer as is:

I'm not sure if ski supports Itanium hypervisor
extension. But maybe the extension is not needed if it's just a matter
of handling remote page faults. Note that currently ski does not
support SMP and emulates just a single CPU. Networking (or most of
devices, even a serial port) do not emulate real hardware either and
instead rely on ski / guest-specific interface to pass data around.
That will probably make vnuma a challenge to run on ski as is.

[ohault]

The amazing goal would be to get it back working on IA64 meanwhile on modern X64 architecture such "distributed" hypervisor would not be yet on any open source projects as far as I know. Porting vNUMA to X64 would be another story.

[johnny-mnemonic]

Looks like the developers presented at USENIX 2009:
https://www.usenix.org/conference/usenix-09/vnuma-virtual-shared-memory-multiprocessor

They also produced a paper: http://usenix.org/event/usenix09/tech/full_papers/chapman/chapman.pdf

Maybe this paper is already detailed enough to determine if vNUMA can work w/o VT-i. I'll definitely have a read.