|
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [RFC PATCH v3 1/2] xen: add libafl-qemu fuzzer support
LibAFL, which is a part of AFL++ project is a instrument that allows us to perform fuzzing on beremetal code (Xen hypervisor in this case) using QEMU as an emulator. It employs QEMU's ability to create snapshots to run many tests relatively quickly: system state is saved right before executing a new test and restored after the test is finished. This patch adds all necessary plumbing to run aarch64 build of Xen inside that LibAFL-QEMU fuzzer. While, most of the code is in common section and can be used by any supported architecture, final calls to LibAFL-QEMU are arch-specific and were tested only on aarch64 for now. But LibAFL-QEMU itself supports many different architectures, including x86_64 and riscv. >From the Xen perspective we need to do following things: 1. Able to communicate with LibAFL-QEMU fuzzer. This is done by executing special opcodes, that only LibAFL-QEMU can handle. 2. Use interface from p.1 to tell the fuzzer about code Xen section, so fuzzer know which part of code to track and gather coverage data. 3. Report fuzzer about crash. This is done in panic() function. 4. Prevent test harness from shooting itself in knee. Right now test harness is an external component, because we want to test external Xen interfaces, but it is possible to fuzz internal code if we want to. Test harness is implemented XTF-based test-case(s). As test harness can issue hypercall that shuts itself down, KConfig option CONFIG_FUZZER_PASS_BLOCKING was added. It basically tells fuzzer that test was completed successfully if Dom0 tries to shut itself (or the whole machine) down. Signed-off-by: Volodymyr Babchuk <volodymyr_babchuk@xxxxxxxx> --- Changes in v3: - Added fuzzer.h - Kconfig entries were reworked to be more generic and support other fuzzers in the future - Moved all the code into common area, as there is nothing arch-specific in it - Created arch-specific header file form ARM - Removed not used definitions in libafl-qemu.h - Removed not used functions from libafl-qemu.c - Folded libafl-qemu-defs.h into libafl-qemu.h as we don't need two separate headers - Aligned code with xen coding style - Added SPDX identifiers with MIT license to libafl-* files --- docs/hypervisor-guide/fuzzing.rst | 91 ++++++++++++++++++++++++++ xen/arch/arm/Kconfig.debug | 37 +++++++++++ xen/arch/arm/include/asm/libafl-qemu.h | 48 ++++++++++++++ xen/arch/arm/psci.c | 5 ++ xen/common/Makefile | 1 + xen/common/domain.c | 3 + xen/common/libafl-qemu.c | 80 ++++++++++++++++++++++ xen/common/sched/core.c | 6 ++ xen/common/shutdown.c | 3 + xen/drivers/char/console.c | 3 + xen/include/xen/fuzzer.h | 52 +++++++++++++++ xen/include/xen/libafl-qemu.h | 63 ++++++++++++++++++ 12 files changed, 392 insertions(+) create mode 100644 docs/hypervisor-guide/fuzzing.rst create mode 100644 xen/arch/arm/include/asm/libafl-qemu.h create mode 100644 xen/common/libafl-qemu.c create mode 100644 xen/include/xen/fuzzer.h create mode 100644 xen/include/xen/libafl-qemu.h diff --git a/docs/hypervisor-guide/fuzzing.rst b/docs/hypervisor-guide/fuzzing.rst new file mode 100644 index 0000000000..895d858edc --- /dev/null +++ b/docs/hypervisor-guide/fuzzing.rst @@ -0,0 +1,91 @@ +.. SPDX-License-Identifier: CC-BY-4.0 + +Fuzzing +======= + +It is possible to use LibAFL-QEMU for fuzzing hypervisor. Right now +only aarch64 is supported and only hypercall fuzzing is enabled in the +test harness, but there are plans to add vGIC interface fuzzing, PSCI +fuzzing and vPL011 fuzzing as well. + + +Principle of operation +---------------------- + +LibAFL-QEMU is a part of American Fuzzy lop plus plus (AKA AFL++) +project. It uses special build of QEMU, that allows to fuzz baremetal +software like Xen hypervisor or Linux kernel. Basic idea is that we +have software under test (Xen hypervisor in our case) and a test +harness application. Test harness uses special protocol to communicate +with LibAFL outside of QEMU to get input data and report test +result. LibAFL monitors which branches are taken by Xen and mutates +input data in attempt to discover new code paths that eventually can +lead to a crash or other unintended behavior. + +LibAFL uses QEMU's `snapshot` feature to run multiple test without +restarting the whole system every time. This speeds up fuzzing process +greatly. + +So, to try Xen fuzzing we need three components: LibAFL-based fuzzer, +test harness and Xen itself. + +Building Xen for fuzzing with LibAFL-QEMU +----------------------------------------- + +Xen hypervisor should be built with these three options:: + + CONFIG_FUZZING=y + CONFIG_FUZZER_LIBAFL_QEMU=y + CONFIG_FUZZER_PASS_BLOCKING=y + +Building LibAFL-QEMU based fuzzer +--------------------------------- + +Fuzzer is written in Rust, so you need Rust toolchain and `cargo` tool +in your system. Please refer to your distro documentation on how to +obtain them. + +Once Rust is ready, fetch and build the fuzzer:: + + # git clone https://github.com/xen-troops/xen-fuzzer-rs + # cd xen-fuzzer-rs + # cargo build + +Building test harness +--------------------- + +We need to make low-level actions, like issuing random hypercalls, so +for test harness we use special build of XTF (Xen Testing Framework). +You can build XTF manually, or let fuzzer to do this:: + + # cargo make build_xtf + +This fill download and build XTF for ARM. + +Running the fuzzer +------------------ + +Please refer to README.md that comes with the fuzzer, but the most +versatile way is to run it like this:: + + # target/debug/xen_fuzzer -t 3600 /path/to/xen \ + target/xtf/tests/arm-vgic-fuzzer/test-mmu64le-arm-vgic-fuzzer + +(assuming that you built XTF with `cargo make build_xtf`) + +Any inputs that led to crashes will be found in `crashes` directory. + +You can replay a crash with `-r` option:: + + # target/debug/xen_fuzzer -r crashes/0195e4fc65828c17 run \ + /path/to/xen \ + /path/to/harness + + +Fuzzer will return non-zero error code if it encountered any crashes. + +TODOs +----- + + - Add x86 support. + - Implement fuzzing of other external hypervisor interfaces. diff --git a/xen/arch/arm/Kconfig.debug b/xen/arch/arm/Kconfig.debug index 5a03b220ac..1a51c5d221 100644 --- a/xen/arch/arm/Kconfig.debug +++ b/xen/arch/arm/Kconfig.debug @@ -190,3 +190,40 @@ config EARLY_PRINTK_INC default "debug-mvebu.inc" if EARLY_UART_MVEBU default "debug-pl011.inc" if EARLY_UART_PL011 default "debug-scif.inc" if EARLY_UART_SCIF + +config FUZZING + bool "Build Xen for fuzzing" + help + Enable this option only if you are going to run the hypervisor + inside a fuzzer. Do not try to run run Xen built with this option + on any real hardware, because it will likely crash during boot. + +choice FUZZER + depends on FUZZING + prompt "Fuzzer" + +config FUZZER_LIBAFL_QEMU + depends on ARM_64 + bool "LibAFL-QEMU" + help + This option enables support for LibAFL-QEMU fuzzer. Choose this + option only when you are going to run hypervisor inside LibAFL-QEMU. + Xen will report code section to LibAFL and will report about + crash when it panics. + +endchoice + +config FUZZER_PASS_BLOCKING + depends on FUZZING + bool "Fuzzing: Report any attempt to suspend/destroy a domain as a success" + help + When fuzzing hypercalls, a fuzzer might make Xen to do something + that prevents from returning to the caller: reboot or turn off the + machine, block calling vCPU, crash a domain, etc. Depending on + fuzzing goal this may be a valid behavior, but as control is not + returned to the fuzzing harness, it can't tell the fuzzer about + success. With this option enabled, Xen will do this by itself. + + Enable this option only if fuzzing attempt can lead to a + correct stop, like when fuzzing hypercalls or PSCI. + diff --git a/xen/arch/arm/include/asm/libafl-qemu.h b/xen/arch/arm/include/asm/libafl-qemu.h new file mode 100644 index 0000000000..9b87eafca9 --- /dev/null +++ b/xen/arch/arm/include/asm/libafl-qemu.h @@ -0,0 +1,48 @@ +/* SPDX-License-Identifier: MIT */ +/* + * Arch-specific portions of LibAFL-QEMU interface + */ +#ifndef __ASM_ARM_LIBAFL_QEMU_H +#define __ASM_ARM_LIBAFL_QEMU_H + +#define LIBAFL_DEFINE_FUNCTIONS(name, opcode) \ + libafl_word _libafl_##name##_call0( \ + libafl_word action) { \ + register unsigned long r0 ASM_REG(0) = action; \ + __asm__ volatile ( \ + ".word " XSTRINGIFY(opcode) "\n" \ + : "+r"(r0) \ + : \ + : "memory" \ + ); \ + return r0; \ + } \ + \ + libafl_word _libafl_##name##_call1( \ + libafl_word action, libafl_word arg1) { \ + register unsigned long r0 ASM_REG(0) = action; \ + register unsigned long r1 ASM_REG(1) = arg1; \ + __asm__ volatile ( \ + ".word " XSTRINGIFY(opcode) "\n" \ + : "+r"(r0) \ + : "r"(r1) \ + : "memory" \ + ); \ + return r0; \ + } \ + \ + libafl_word _libafl_##name##_call2( \ + libafl_word action, libafl_word arg1, libafl_word arg2) { \ + register unsigned long r0 ASM_REG(0) = action; \ + register unsigned long r1 ASM_REG(1) = arg1; \ + register unsigned long r2 ASM_REG(2) = arg2; \ + __asm__ volatile ( \ + ".word " XSTRINGIFY(opcode) "\n" \ + : "+r"(r0) \ + : "r"(r1), "r"(r2) \ + : "memory" \ + ); \ + return r0; \ + } + +#endif diff --git a/xen/arch/arm/psci.c b/xen/arch/arm/psci.c index b6860a7760..43253b3f71 100644 --- a/xen/arch/arm/psci.c +++ b/xen/arch/arm/psci.c @@ -10,6 +10,7 @@ #include <xen/acpi.h> +#include <xen/fuzzer.h> #include <xen/types.h> #include <xen/init.h> #include <xen/mm.h> @@ -62,12 +63,16 @@ void call_psci_cpu_off(void) void call_psci_system_off(void) { + fuzzer_on_block(); + if ( psci_ver > PSCI_VERSION(0, 1) ) arm_smccc_smc(PSCI_0_2_FN32_SYSTEM_OFF, NULL); } void call_psci_system_reset(void) { + fuzzer_on_block(); + if ( psci_ver > PSCI_VERSION(0, 1) ) arm_smccc_smc(PSCI_0_2_FN32_SYSTEM_RESET, NULL); } diff --git a/xen/common/Makefile b/xen/common/Makefile index 98f0873056..f2fbf54911 100644 --- a/xen/common/Makefile +++ b/xen/common/Makefile @@ -78,6 +78,7 @@ extra-y := symbols-dummy.o obj-$(CONFIG_COVERAGE) += coverage/ obj-y += sched/ obj-$(CONFIG_UBSAN) += ubsan/ +obj-$(CONFIG_FUZZER_LIBAFL_QEMU) += libafl-qemu.o obj-$(CONFIG_NEEDS_LIBELF) += libelf/ obj-$(CONFIG_LIBFDT) += libfdt/ diff --git a/xen/common/domain.c b/xen/common/domain.c index abf1969e60..e63a80c26e 100644 --- a/xen/common/domain.c +++ b/xen/common/domain.c @@ -5,6 +5,7 @@ */ #include <xen/compat.h> +#include <xen/fuzzer.h> #include <xen/init.h> #include <xen/lib.h> #include <xen/ctype.h> @@ -1317,6 +1318,8 @@ int domain_shutdown(struct domain *d, u8 reason) spin_unlock(&d->shutdown_lock); + fuzzer_on_block(); + return 0; } diff --git a/xen/common/libafl-qemu.c b/xen/common/libafl-qemu.c new file mode 100644 index 0000000000..a09a2931c6 --- /dev/null +++ b/xen/common/libafl-qemu.c @@ -0,0 +1,80 @@ +/* SPDX-License-Identifier: MIT */ +/* + This file is based on libafl_qemu_impl.h, libafl_qemu_qemu_arch.h + and libafl_qemu_defs.h from LibAFL project. +*/ +#include <xen/lib.h> +#include <xen/init.h> +#include <xen/kernel.h> +#include <xen/spinlock.h> +#include <xen/libafl-qemu.h> +#include <asm/libafl-qemu.h> + +/* Generates sync exit functions */ +LIBAFL_DEFINE_FUNCTIONS(sync_exit, LIBAFL_SYNC_EXIT_OPCODE) + + void libafl_qemu_end(enum LibaflQemuEndStatus status) +{ + _libafl_sync_exit_call1(LIBAFL_QEMU_COMMAND_END, status); +} + +void libafl_qemu_internal_error(void) +{ + _libafl_sync_exit_call0(LIBAFL_QEMU_COMMAND_INTERNAL_ERROR); +} + +void lqprintf(const char *fmt, ...) +{ + static DEFINE_SPINLOCK(lock); + static char buffer[LIBAFL_QEMU_PRINTF_MAX_SIZE] = {0}; + va_list args; + int res; + + spin_lock(&lock); + + va_start(args, fmt); + res = vsnprintf(buffer, LIBAFL_QEMU_PRINTF_MAX_SIZE, fmt, args); + va_end(args); + + if ( res >= LIBAFL_QEMU_PRINTF_MAX_SIZE ) + { + /* buffer is not big enough, either recompile the target with more */ + /* space or print less things */ + libafl_qemu_internal_error(); + } + + _libafl_sync_exit_call2(LIBAFL_QEMU_COMMAND_LQPRINTF, + (libafl_word)buffer, res); + spin_unlock(&lock); +} + +void libafl_qemu_trace_vaddr_range(libafl_word start, + libafl_word end) +{ + _libafl_sync_exit_call2(LIBAFL_QEMU_COMMAND_VADDR_FILTER_ALLOW, start, end); +} + +static int init_afl(void) +{ + vaddr_t xen_text_start = (vaddr_t)_stext; + vaddr_t xen_text_end = (vaddr_t)_etext; + + lqprintf("Telling AFL about code section: %lx - %lx\n", xen_text_start, + xen_text_end); + + libafl_qemu_trace_vaddr_range(xen_text_start, xen_text_end); + + return 0; +} + +__initcall(init_afl); + +/* + * Local variables: + * mode: C + * c-file-style: "BSD" + * c-basic-offset: 4 + * indent-tabs-mode: nil + * End: + */ + diff --git a/xen/common/sched/core.c b/xen/common/sched/core.c index 9043414290..b109a8de44 100644 --- a/xen/common/sched/core.c +++ b/xen/common/sched/core.c @@ -16,6 +16,7 @@ #ifndef COMPAT #include <xen/init.h> #include <xen/lib.h> +#include <xen/fuzzer.h> #include <xen/param.h> #include <xen/sched.h> #include <xen/sections.h> @@ -1429,6 +1430,8 @@ void vcpu_block(void) TRACE_TIME(TRC_SCHED_BLOCK, v->domain->domain_id, v->vcpu_id); raise_softirq(SCHEDULE_SOFTIRQ); } + + fuzzer_on_block(); } static void vcpu_block_enable_events(void) @@ -1502,6 +1505,8 @@ static long do_poll(const struct sched_poll *sched_poll) TRACE_TIME(TRC_SCHED_BLOCK, d->domain_id, v->vcpu_id); raise_softirq(SCHEDULE_SOFTIRQ); + fuzzer_on_block(); + return 0; out: @@ -1529,6 +1534,7 @@ long vcpu_yield(void) TRACE_TIME(TRC_SCHED_YIELD, current->domain->domain_id, current->vcpu_id); raise_softirq(SCHEDULE_SOFTIRQ); + return 0; } diff --git a/xen/common/shutdown.c b/xen/common/shutdown.c index c47341b977..8e82678626 100644 --- a/xen/common/shutdown.c +++ b/xen/common/shutdown.c @@ -1,5 +1,6 @@ #include <xen/init.h> #include <xen/lib.h> +#include <xen/fuzzer.h> #include <xen/param.h> #include <xen/sched.h> #include <xen/sections.h> @@ -32,6 +33,8 @@ static void noreturn reboot_or_halt(void) void hwdom_shutdown(unsigned char reason) { + fuzzer_on_block(); + switch ( reason ) { case SHUTDOWN_poweroff: diff --git a/xen/drivers/char/console.c b/xen/drivers/char/console.c index c3150fbdb7..45048351d5 100644 --- a/xen/drivers/char/console.c +++ b/xen/drivers/char/console.c @@ -16,6 +16,7 @@ #include <xen/event.h> #include <xen/console.h> #include <xen/param.h> +#include <xen/fuzzer.h> #include <xen/serial.h> #include <xen/softirq.h> #include <xen/keyhandler.h> @@ -1289,6 +1290,8 @@ void panic(const char *fmt, ...) kexec_crash(CRASHREASON_PANIC); + fuzzer_crash(); + if ( opt_noreboot ) machine_halt(); else diff --git a/xen/include/xen/fuzzer.h b/xen/include/xen/fuzzer.h new file mode 100644 index 0000000000..852917fe50 --- /dev/null +++ b/xen/include/xen/fuzzer.h @@ -0,0 +1,52 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +#ifndef XEN__FUZZER_H +#define XEN__FUZZER_H + +#include <xen/compiler.h> + +#ifdef CONFIG_FUZZER_LIBAFL_QEMU +#include <xen/libafl-qemu.h> +#endif + +/* Unconditional failure */ +static always_inline void fuzzer_crash(void) +{ +#ifdef CONFIG_FUZZER_LIBAFL_QEMU + libafl_qemu_end(LIBAFL_QEMU_END_CRASH); +#endif +} + +/* Unconditional success */ +static always_inline void fuzzer_success(void) +{ +#ifdef CONFIG_FUZZER_LIBAFL_QEMU + libafl_qemu_end(LIBAFL_QEMU_END_OK); +#endif +} + +/* + * Conditional success + * + * Sometimes a fuzzer might make Xen to do something that prevents + * from returning to the caller: reboot or turn off the machine, block + * calling vCPU, crash a domain, etc. Depending on fuzzing goal this + * may be a valid behavior, but as control is not returned to the + * fuzzing harness, it can't tell the fuzzer about success, so we need + * to do this ourselves. + */ +static always_inline void fuzzer_on_block(void) +{ +#ifdef CONFIG_FUZZER_PASS_BLOCKING + fuzzer_success(); +#endif +} + +#endif +/* + * Local variables: + * mode: C + * c-file-style: "BSD" + * c-basic-offset: 4 + * indent-tabs-mode: nil + * End: + */ diff --git a/xen/include/xen/libafl-qemu.h b/xen/include/xen/libafl-qemu.h new file mode 100644 index 0000000000..f3b32adeca --- /dev/null +++ b/xen/include/xen/libafl-qemu.h @@ -0,0 +1,63 @@ +/* SPDX-License-Identifier: MIT */ +#ifndef __XEN_LIBAFL_QEMU_H +#define __XEN_LIBAFL_QEMU_H + +#include <xen/stdint.h> +#define LIBAFL_QEMU_PRINTF_MAX_SIZE 4096 + +#define LIBAFL_STRINGIFY(s) #s +#define XSTRINGIFY(s) LIBAFL_STRINGIFY(s) + +#define LIBAFL_SYNC_EXIT_OPCODE 0x66f23a0f + +typedef enum LibaflQemuCommand +{ + LIBAFL_QEMU_COMMAND_START_VIRT = 0, + LIBAFL_QEMU_COMMAND_START_PHYS = 1, + LIBAFL_QEMU_COMMAND_INPUT_VIRT = 2, + LIBAFL_QEMU_COMMAND_INPUT_PHYS = 3, + LIBAFL_QEMU_COMMAND_END = 4, + LIBAFL_QEMU_COMMAND_SAVE = 5, + LIBAFL_QEMU_COMMAND_LOAD = 6, + LIBAFL_QEMU_COMMAND_VERSION = 7, + LIBAFL_QEMU_COMMAND_VADDR_FILTER_ALLOW = 8, + LIBAFL_QEMU_COMMAND_INTERNAL_ERROR = 9, + LIBAFL_QEMU_COMMAND_LQPRINTF = 10, + LIBAFL_QEMU_COMMAND_TEST = 11, +} LibaflExit; + +typedef uint64_t libafl_word; + +/** + * LibAFL QEMU header file. + * + * This file is a portable header file used to build target harnesses more + * conveniently. Its main purpose is to generate ready-to-use calls to + * communicate with the fuzzer. The list of commands is available at the bottom + * of this file. The rest mostly consists of macros generating the code used by + * the commands. + */ + +enum LibaflQemuEndStatus +{ + LIBAFL_QEMU_END_UNKNOWN = 0, + LIBAFL_QEMU_END_OK = 1, + LIBAFL_QEMU_END_CRASH = 2, +}; + +void libafl_qemu_end(enum LibaflQemuEndStatus status); + +void libafl_qemu_internal_error(void); + +void __attribute__((format(printf, 1, 2))) lqprintf(const char *fmt, ...); + +void libafl_qemu_trace_vaddr_range(libafl_word start, libafl_word end); + +static always_inline void libafl_qemu_success_on_block(void) +{ +#ifdef CONFIG_LIBAFL_QEMU_FUZZER_PASS_BLOCKING + libafl_qemu_end(LIBAFL_QEMU_END_OK); +#endif +} + +#endif -- 2.48.1
|
 |
Lists.xenproject.org is hosted with RackSpace, monitoring our |