feat(miniooni): use a remote miniooni instance to measure

[bassosimone]

This commit extends miniooni to be able to use a remote instance of miniooni for performing OONI measurements.

The main use case for this functionality is to run measurements inside another network that has censorship for QA purposes.

We currently support two transports for implementing remoting:

1. "tcp", which uses a cleartext TCP connection

2. "ssh", which uses SSH

The tcp transport is useful when you trust the network that transports packets from and to the two miniooni instances.

The ssh transport covers the case where you want a secure channel to transport packets between the two miniooni instances.

To start the remotetcp server, run this command as root:

./miniooni remotetcp

To start the remotessh server, run this command as root:

./miniooni remotessh

On the local side, you need to create $HOME/.miniooni/remote/config.yaml, which must look like the following:

remotes:
  foobar_tcp:
    address: "1.2.3.4:5555"
    transport: "tcp"

  foobar_ssh:
    address: "1.2.3.4:2222"
    transport: "ssh"
    ssh:
      user: "root"

The ports in the above example are the default ones used by remotetcp and remotessh.

On the command line, you can enable remoting by adding --remote=NAME, where NAME is the remote name. For example:

./miniooni -n --remote foobar_ssh example

Note that this command will only work if you have an active instance of the ssh-agent (i.e., SSH_AGENT_SOCK is defined).

Having described the functionality, let us explain how we implemented all of this.

We have reintroduced the possibility of completely taking over the basic netxlite primitives:

1. dialing a TCP or UDP conn

2. looking up a domain name using getaddrinfo

3. creating a listening UDP socket

When you use --remote NAME, the code will do the following:

1. read the config file

2. figure out the right remote

3. establish a connection with the remote

4. create a TCP/IP stack in userspace

5. create a TUN device in userspace that gets all the packets generated by the TCP/IP stack in userspace

6. overwrite netxlite primitives so they use the TCP/IP stack in userspace instead of the Go standard library

7. setup routing between the connection with the remote and the TUN device in userspace so we route packets

8. run the desired experiments as usual

On the server side, we do something similar, except that we use a real TUN device as implemented by Linux. (We only support Linux servers at the moment.)

Once the real TUN device has been created we setup masquerading for it, and then we we route between the connection with the miniooni client and the real TUN device.

We don't currently implement censorship, but the real TUN device is clearly the right place where to do that.

See ooni/probe#2340.

[bassosimone]

I tested this code for ~30 minutes and it eventually crashed like this:

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x2 addr=0x18 pc=0x10571da50]

goroutine 26486 [running]:
github.com/ooni/probe-cli/v3/internal/measurexlite.(*connTrace).Read(0x1400330b590, {0x140029816f3, 0x794d, 0x794d})
	/Users/sbs/src/monorepo/repo/probe-cli/internal/measurexlite/conn.go:44 +0x40
github.com/ooni/oocrypto/tls.(*atLeastReader).Read(0x14003480e88, {0x140029816f3?, 0x0?, 0x104f48fe8?})
	/Users/sbs/go/pkg/mod/github.com/ooni/oocrypto@v0.1.2/tls/conn.go:786 +0x44
bytes.(*Buffer).ReadFrom(0x140001aa978, {0x10607cff8, 0x14003480e88})
	/Users/sbs/sdk/go1.18.6/src/bytes/buffer.go:204 +0x94
github.com/ooni/oocrypto/tls.(*Conn).readFromUntil(0x140001aa700, {0x12ec801d8?, 0x1400330b590}, 0x104f9149c?)
	/Users/sbs/go/pkg/mod/github.com/ooni/oocrypto@v0.1.2/tls/conn.go:808 +0xd8
github.com/ooni/oocrypto/tls.(*Conn).readRecordOrCCS(0x140001aa700, 0x0)
	/Users/sbs/go/pkg/mod/github.com/ooni/oocrypto@v0.1.2/tls/conn.go:660 +0x8ac
github.com/ooni/oocrypto/tls.(*Conn).readRecord(...)
	/Users/sbs/go/pkg/mod/github.com/ooni/oocrypto@v0.1.2/tls/conn.go:583
github.com/ooni/oocrypto/tls.(*Conn).Read(0x140001aa700, {0x14002715000, 0x1000, 0x1076ef820?})
	/Users/sbs/go/pkg/mod/github.com/ooni/oocrypto@v0.1.2/tls/conn.go:1286 +0x180
github.com/ooni/probe-cli/v3/internal/netxlite.(*httpTLSConnWithReadTimeout).Read(0x1400276a260, {0x14002715000, 0x1000, 0x1000})
	/Users/sbs/src/monorepo/repo/probe-cli/internal/netxlite/http.go:335 +0xe0
bufio.(*Reader).Read(0x140030771a0, {0x14002609620, 0x9, 0xd?})
	/Users/sbs/sdk/go1.18.6/src/bufio/bufio.go:236 +0x1e4
io.ReadAtLeast({0x10607c458, 0x140030771a0}, {0x14002609620, 0x9, 0x9}, 0x9)
	/Users/sbs/sdk/go1.18.6/src/io/io.go:331 +0xa4
io.ReadFull(...)
	/Users/sbs/sdk/go1.18.6/src/io/io.go:350
github.com/ooni/oohttp.http2readFrameHeader({0x14002609620?, 0x9?, 0x140024fce10?}, {0x10607c458?, 0x140030771a0?})
	/Users/sbs/go/pkg/mod/github.com/ooni/oohttp@v0.2.2/h2_bundle.go:1566 +0x5c
github.com/ooni/oohttp.(*http2Framer).ReadFrame(0x140026095e0)
	/Users/sbs/go/pkg/mod/github.com/ooni/oohttp@v0.2.2/h2_bundle.go:1830 +0x84
github.com/ooni/oohttp.(*http2clientConnReadLoop).run(0x1400184bf88)
	/Users/sbs/go/pkg/mod/github.com/ooni/oohttp@v0.2.2/h2_bundle.go:8820 +0x100
github.com/ooni/oohttp.(*http2ClientConn).readLoop(0x14000641080)
	/Users/sbs/go/pkg/mod/github.com/ooni/oohttp@v0.2.2/h2_bundle.go:8716 +0x60
created by github.com/ooni/oohttp.(*http2Transport).newClientConn
	/Users/sbs/go/pkg/mod/github.com/ooni/oohttp@v0.2.2/h2_bundle.go:7444 +0x924

I suspect using a SSH channel in the middle exposed issues of the implementation. The crash, in fact, does not seem related to new code I added today but rather to previously existing code.