# Testing with cases

> Test moat rules with case files: describe packets and expected verdicts, run them in the kernel with BPF_PROG_TEST_RUN or in moat's model, and see why with moat explain.

Source: https://moatfw.dev/docs/testing

`moat test rules.moat cases.toml` compiles the file, loads it into the
running kernel without attaching it to any interface, and runs each case
with `BPF_PROG_TEST_RUN`. The verifier that checks it is the one that
will check it on the interface.

```toml
[sets]
verified = ["192.0.2.10"] # added to the set before the cases run
banned = ["198.51.100.77 expired"] # inserted with an expiry in the past

[[case]]
name = "query from an unverified source"
packet = { proto = "udp", src = "192.0.2.50", dport = 27015, payload = "hex:ffffffff" }
expect = { verdict = "drop", rule = "query" }
```

`rule` is optional; `"default"` names the default action.

## When a case fails

`moat explain` shows which rule decided and why.

```sh
$ moat explain rules.moat cases.toml [CASE...]
```

For the last packet of each case, or only of the cases named, it prints
the packet as moat parsed it, then every rule up to the one that decided,
each with why it did not match or what it did:

```
query with the wrong header
  packet: UDP over IPv4 192.0.2.8:40000 > 198.51.100.1:27015, 4 bytes of payload
  strict_mode: no match, strict is false
  bad_flags: no match, tcp.flags == {fin, syn}, but the packet is UDP over IPv4
  probes: no match, tcp.dport in {23, 2323}: the packet has no TCP header
  query: no match, payload[0:4] == 0xffffffff, but the packet has 0x00ffffff there
  no rule decides: the default, pass
  FAIL query with the wrong header: got pass by default, want drop
```

Every case still runs, so limits and sets fill as they do in `moat test`,
and a rule over its rate says so: `query: matches, over the rate: adds
192.0.2.7 to banned, drop`. `moat explain` always runs in
[moat's model of the program](#without-a-kernel) and says so on its last
line. It exits with 1 when a case fails, and 2 for a usage error or a
case name that is not in the file.

## Without a kernel

```sh
$ moat test rules.moat cases.toml --model
$ moat test rules.moat --pcap capture.pcap [cases.toml] --model
```

`--model` runs the cases in moat's model of the program instead: a Go
version of what the generated program does, with the same packet
parsing, the same tests, the same rate limits with the same rounding,
set expiry, adds that never shorten an entry, and audit mode. It needs no
kernel and no root, so it works on macOS and in unprivileged CI. The
summary line ends with `(model)`, as in `9 of 9 cases passed (model)`.

Off Linux, `moat test` uses the model on its own and says so on stderr.
On Linux without the privileges to load a program, the error suggests
`--model`.

> moat's own tests check the model against the kernel packet by packet,
> but the model is not the program. On Linux, run `moat test` without
> `--model` before trusting a ruleset. The model also keeps every rate
> limit key and set entry, where the kernel's maps evict the least
> recently used ones when they are full.

## Packets

Keys: `proto` (`tcp`, `udp`, `icmp`, `icmp6`, `other`), `src`, `dst`,
`sport`, `dport`, `ttl`, `flags` (for example `"syn,ack"`), `window`,
`mss`, `icmp_type`, `icmp_code`, `vlan`, `frag` (a fragment other than
the first), `payload` (text, or `hex:...`, which may contain spaces) and
`udp_len`. `raw = "hex:..."` gives the whole frame instead, for malformed
packets. Unknown keys are errors. A key left out gets a default; a
written 0 for `sport`, `dport`, `ttl` or `window` means 0.

## Time

Time is simulated, so tests of limits and expiring entries are exact. A
run starts at time zero; `time = "1500ms"` sends a case's packet that
long after the start, and without it a case follows the previous one.

`repeat = 20` and `every = "10ms"` send the packet several times. The
expected verdict and rule are those of the last packet, and `drops = 3`
checks how many of them were dropped. `every` needs `repeat` above 1, and
`drops` cannot be more than the packets sent. A top-level `audit = true`
runs the cases in audit mode, and a `[vars]` table sets variables.

## Captures and timing

```sh
$ moat test rules.moat --pcap capture.pcap
$ moat bench rules.moat cases.toml
```

With `--pcap`, every packet of a capture runs through the rules with the
capture's timestamps as the clock, and moat prints how many packets each
rule decided. It reads pcap and pcapng files with Ethernet, raw IPv4 or
IPv6, loopback (as macOS writes `lo0`) or Linux cooked frames; a record
too short for its own header is skipped and counted. A case file can
follow the capture, `moat test FILE --pcap CAPTURE CASES`: its `[sets]`,
`[vars]` and `audit` apply, and it cannot hold `[[case]]` entries then
(exit 2).

`moat bench` runs each case through the production build and prints the
nanoseconds per run. `--repeat N` sets how many runs, 100000 by default,
and `--pcap CAPTURE` times the packets of a capture instead of the
cases.

`moat test` exits with 1 when the rules have errors or a case fails, 2
when the command line or the case file is wrong, and 3 when the kernel
rejects the program.