|
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [Xen-devel] [PATCH 00/27] Running benchmarks via OSSTest
Hello everyone, This is a highly reworked and much more mature version of this old RFC series: http://lists.xen.org/archives/html/xen-devel/2014-06/msg03429.html It is about integrating benchmarking capabilities into OSSTest. The series is a lot bigger, because it is now capable of doing a lot more stuff. :-) All the comments I got during review of the RFC have been addressed, of course... More comments are of course welcome! The series is available here: git://xenbits.xen.org/people/dariof/osstest.git benchmarking-with-osstest http://xenbits.xen.org/gitweb/?p=people/dariof/osstest.git;a=shortlog;h=refs/heads/benchmarking-with-osstest So, about the patches. Patches 01 to 04, as well as patch 20, are preliminary changes and fixes, necessary for the rest of the series to work properly, but they are general enough to be considered separately from the series, if interesting. Patches 05 to 12 make it possible to run an instance of the Unixbench (XXX) benchmark via OSSTest. Management scripts, test scripts, jobs and recipes are provided to download the benchmark, prepare the guest (PV or HVM), build the benchmark in the guest, run it, and retrieve and process (include some super fancy plots! :-D) the results. To do so, do as follows (in standalone mode): $ ./standalone-reset -t bench $ ./mg-unixbench-download $ ./standalone run-job -h ultralisk bench-unixbench-amd64-credit-pv-4vcpus-4096ram One thing about patch 12. I tried, but I can't tell how I should invoke ./standalone in order to test the new parameter I'm adding... Can someone help with this? IanC? To have an idea of how the 'output' of one of this jobs looks like, have a look here: http://xenbits.xen.org/people/dariof/osstest-bench/bench-kernbench-amd64-credit-hvm-4vcpus-4096ram.log http://xenbits.xen.org/people/dariof/osstest-bench/bench-kernbench-amd64-credit-hvm-4vcpus-4096ram/ http://xenbits.xen.org/people/dariof/osstest-bench/bench-kernbench-amd64-credit-hvm-4vcpus-4096ram/debianhvm.guest.osstest--kernbench-n4-PLOT.png Or here: http://xenbits.xen.org/people/dariof/osstest-bench/bench-unixbench-amd64-credit-pv-4vcpus-4096ram.log http://xenbits.xen.org/people/dariof/osstest-bench/bench-unixbench-amd64-credit-pv-4vcpus-4096ram/ http://xenbits.xen.org/people/dariof/osstest-bench/bench-unixbench-amd64-credit-pv-4vcpus-4096ram/debian.guest.osstest--unixbench-i1-c4-PLOT.png Patches 13 to 19 does the very same, but for Kernbench (XXX): $ ./mg-kernbench-download $ ./standalone run-job -h ultralisk bench-kernbench-amd64-credit-hvm-4vcpus-4096ram Patches 21 to 27 aims at allowing running a benchmark both in a guest and on baremetal, to investigate the performances loss due to the virtualization overhead. To do that, for Unixbench and Kernbench, one just issue the following commands (in standalone mode): $ ./standalone run-job -h ultralisk bench-hostcmp-unixbench-amd64-credit $ ./standalone run-job -h ultralisk bench-hostcmp-kernbench-amd64-credit2 To have an idea of how the 'output' of one of these jobs looks like, have a look here: http://xenbits.xen.org/people/dariof/osstest-bench/bench-hostcmp-kernbench-amd64-credit.log http://xenbits.xen.org/people/dariof/osstest-bench/bench-hostcmp-kernbench-amd64-credit/ http://xenbits.xen.org/people/dariof/osstest-bench/bench-hostcmp-kernbench-amd64-credit/bench-hostcmp-kernbench-amd64-credit-PLOT.png TODOs: * in patch 02/27, I haven't actually checked whether also grub1 and/or uboot needs fixing * When comparing performance, I think it would be best to always use the same kernel, in host and guests. This happens by default for PV guest, while it does not for HVM guests. This of course can be solved pretty easily, and I plan to do it either in future posting of this series, or with a follow-up patch. Regards, Dario --- Dario Faggioli (27): ts-devbian-hvm-install: prune "cdrom:" from install sources Osstest/Debian.pm: fix identifying a Linux baremetal grub2 entry Guest setup: allow the amount of RAM to be a runvar Osstest/TestSupport.pm: Introduce target_getfile_[root_]stash() mg-unixbench-download: new script for downloading the unixbench archive ts-unixbench-build: prep the environment for running unixbench ts-unixbench-run: kick off the benchmark on the target ts-unixbench-reslts: for retrieving the results ts-unixbench-reslts: process and plot bench results sg-run-job: recipes for the unixbench jobs make-bench-flight: to create a benchmarking flight standalone-reset: introduce a new -t option mg-kernbench-download: new script for downloading kernbench ts-kernbench-build: prep the environment for running kernbench ts-kernbench-run: kick off the benchmark on the target ts-unixbench-reslts: retrieve and stash kernbench results ts-kernbench-reslts: process and plot bench results sg-run-job: recipes for the kernbench jobs make-bench-flight: create kernbench jobs Osstest/TestSupport.pm: read hosts' hardware characteristics ts-bench-hostcmp-guest-prep: new script ts-bench-hostcmp-host-prep: new script ts-bench-hostcmp-host-reset: new script Recipes and jobs for running unixbench both on host and guest ts-bench-hostcmp-post: add plotting facilities Kernbench perf comparison between host and guest ts-bench-hostcmp-post: add plotting facilities Osstest/Benchmarking.pm | 210 +++++++++++++++++++++++++++++++++++++++++++ Osstest/Debian.pm | 54 +++++++---- Osstest/TestSupport.pm | 166 +++++++++++++++++++++++++++++++++- README | 36 +++++++ cr-daily-branch | 3 - make-bench-flight | 153 +++++++++++++++++++++++++++++++ mg-host-hw-specs | 35 +++++++ mg-kernbench-download | 49 ++++++++++ mg-unixbench-download | 40 ++++++++ sg-run-job | 102 ++++++++++++++++++++- standalone | 7 + standalone-reset | 9 +- ts-bench-hostcmp-guest-prep | 87 ++++++++++++++++++ ts-bench-hostcmp-host-prep | 74 +++++++++++++++ ts-bench-hostcmp-post | 81 +++++++++++++++++ ts-debian-fixup | 3 + ts-debian-hvm-install | 3 - ts-kernbench-build | 86 ++++++++++++++++++ ts-kernbench-reslts | 75 +++++++++++++++ ts-kernbench-run | 63 +++++++++++++ ts-unixbench-build | 77 ++++++++++++++++ ts-unixbench-reslts | 84 +++++++++++++++++ ts-unixbench-run | 67 ++++++++++++++ ts-xen-install | 38 -------- 24 files changed, 1534 insertions(+), 68 deletions(-) create mode 100644 Osstest/Benchmarking.pm create mode 100755 make-bench-flight create mode 100755 mg-host-hw-specs create mode 100755 mg-kernbench-download create mode 100755 mg-unixbench-download create mode 100755 ts-bench-hostcmp-guest-prep create mode 100755 ts-bench-hostcmp-host-prep create mode 100755 ts-bench-hostcmp-post create mode 100755 ts-kernbench-build create mode 100755 ts-kernbench-reslts create mode 100755 ts-kernbench-run create mode 100755 ts-unixbench-build create mode 100755 ts-unixbench-reslts create mode 100755 ts-unixbench-run -- <<This happens because I choose it to happen!>> (Raistlin Majere) ------------------------------------------------------------------- Dario Faggioli, Ph.D, http://retis.sssup.it/people/faggioli Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK) _______________________________________________ Xen-devel mailing list Xen-devel@xxxxxxxxxxxxx http://lists.xen.org/xen-devel
|
 |
Lists.xenproject.org is hosted with RackSpace, monitoring our |
