Abstraction of Paging Support in Xen Xen recently introduced a new shadow paging implementation (shadow2), which substantially improves the performance/clearness over shadow1. On the other hand, hardware assisted paging is on the horizon. AMD’s new processors will support nested paging which provides hardware-level paging acceleration. Intel’s extended paging will have similar purpose. To fully utilize the performance promised by hardware-assisted paging, we are currently working on a common interface to support different paging supports in Xen. Specifically, we intend to support shadow paging, AMD’s nested paging, and Intel’s extended paging, under this common interface. We have been working with Tim Deegan from XenSource on paging interface abstraction. 1. Design We breakdown the original shadow data structure (in domain.h and shadow.h) into common paging components and shadow paging components. More specifically, the following structs are defined: * struct paging_domain and struct shadow_domain (defined in domain.h) * struct paging_vcpu and struct paging_vcpu (defined in domain.h) * struct paging_mode and struct shadow_paging_mode (defined in paging.h) For instance, paging_mode is defined as the following struct paging_mode { int (*page_fault )(struct vcpu *v, …); int (*invlpg )(struct vcpu *v, …); paddr_t (*gva_to_gpa )(struct vcpu *v, …); unsigned long (*gva_to_gfn )(struct vcpu *v, …); void (*update_cr3 )(struct vcpu *v); void (*update_paging_modes)(struct vcpu *v, …); /* paging support extension */ struct shadow_paging_mode shadow; } We provide wrapper functions in paging.h for each function. Basically, these wrapper functions delegates the job to function pointers. All existing shadow paging function calls only need to change to wrapper functions. * paging_page_fault * paging_invlpg * paging_gva_to_gpa * paging_gva_to_gfn * paging_update_cr3 *paging_update_paging_modes The following is one wrapper function example for shadow_page_fault: static inline int paging_page_fault(unsigned long va, struct cpu_user_regs *regs) { struct vcpu *v = current; return v->arch.paging.mode->page_fault(v, va, regs); } The paging initialization happens in paging.c (paging_domain_init() and paging_vcpu_init()). Shadow code or hardware assisted paging (hap) is responsible for installed all the function pointers defined in “struct paging_domain” and “struct paging_mode”. 2. Hardware Assisted Paging Hardware assisted paging is enabled through “hap” option in Xen boot menu (see paging.c code). This patch only supports shadow paging; but it provides interface for other hardware assisted paging support (such as nested paging and extended paging). 3. Files Two new files (paging.c and paging.h) are added for the paging interface code. The following files are modified by these patches b/xen/arch/x86/paging.c | 53 +++++ b/xen/include/asm-x86/paging.h | 258 ++++++++++++++++++++++++++ xen/arch/x86/Makefile | 1 xen/arch/x86/domain.c | 20 +- xen/arch/x86/domain_build.c | 4 xen/arch/x86/domctl.c | 2 xen/arch/x86/hvm/hvm.c | 4 xen/arch/x86/hvm/platform.c | 6 xen/arch/x86/hvm/svm/svm.c | 22 +- xen/arch/x86/hvm/svm/vmcb.c | 2 xen/arch/x86/hvm/vmx/vmcs.c | 2 xen/arch/x86/hvm/vmx/vmx.c | 16 - xen/arch/x86/mm.c | 6 xen/arch/x86/mm/shadow/common.c | 385 ++++++++++++++++++++------------------- xen/arch/x86/mm/shadow/multi.c | 116 ++++++----- xen/arch/x86/mm/shadow/multi.h | 2 xen/arch/x86/mm/shadow/private.h | 60 +++--- xen/arch/x86/mm/shadow/types.h | 2 xen/arch/x86/traps.c | 4 xen/arch/x86/x86_64/domctl.c | 6 xen/include/asm-x86/domain.h | 83 +++++--- xen/include/asm-x86/shadow.h | 176 +---------------- 22 files changed, 724 insertions(+), 506 deletions(-) 4. Coding Approaches Since the patches are a little bit invasive, we took a conservative approach by breaking it into different phases. Tim Deegan gave us a lot of suggestions on the design. The attached PhaseDoc.pdf tells each step of our progress. All changes happen inside x86 directory. Therefore, we don’t believe that they break other architectures (such as PowerPC and IA64). 5. Tested Enviroment * AMD SVM boxes - 64-bit Xen: 32-bit WinXP SP2, 32-bit SUSE10, 32-bit SUSE 10 PAE BigSMP, and 64-bit RHEL4 - 32-bit PAE Xen: 32-bit WinXP SP2, 32-bit SUSE10, 32-bit SUSE 10 PAE BigSMP, and SLES 9.3 * Intel VT boxes: - 32-bit PAE Xen: 32-bit SUSE10, 32-bit WinXP SP2, 32b SUSE 10 PAE BigSMP, and 32-bit Windows 2003 enterprise server - 64-bit Xen: 32-bit WinXP SP2, 32-bit Windows 2003 enterprise server 32-bit SUSE10, 32-bit SUSE10 PAE BigSMP, and 64Bit RHEL 4