|
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] Re: [Xen-devel] FAILED/MISSING cstate/cpufreq/cpupower support with Xen 4.13 + kernel 5.4.14; withOUT xen/hypervisor, WORKS. bug or config?
On 28.01.2020 18:41, PGNet Dev wrote:
> ( posted this already to xen-users, and @ distro list; advised to bring it
> here )
>
> I'm running linux kernel
>
> lsb_release -rd
> Description: openSUSE Leap 15.1
> Release: 15.1
>
> uname -rm
> 5.4.14-25.g170524c-default x86_64
>
> dmesg | grep DMI:
> [ 0.000000] DMI: Supermicro X10SAT/X10SAT, BIOS 3.0
> 05/26/2015
>
> cat /proc/cpuinfo | grep "model name" | head -n 1
> model name : Intel(R) Xeon(R) CPU E3-1220 v3 @ 3.10GHz
>
> BIOS *is* setup for max cstate support.
> Xeon E3-1220 does support intel_pstate driver.
>
> WITH Xen/hypervisor, I've *NO* cpufreq/scaling support.
>
> withOUT Xen/hypervisor, works as expected.
>
>
> Testing first,
>
> (1) boot, NO XEN
>
> pstate driver's init'd
>
> dmesg | egrep -i "intel_pstate"
> [ 6.132964] intel_pstate: Intel P-state driver initializing
>
> pstate/cstate info
>
> cat /sys/module/intel_idle/parameters/max_cstate
> 9
>
> cd /sys/devices/system/cpu/cpu0/cpuidle
> for state in state{0..9}
> do echo c-$state `cat $state/name` `cat $state/latency`
> done
> c-state0 POLL 0
> c-state1 C1 2
> c-state2 C1E 10
> c-state3 C3 33
> c-state4 C6 133
> c-state5 C7s 166
> cat: state6/name: No such file or directory
> cat: state6/latency: No such file or directory
> c-state6
> cat: state7/name: No such file or directory
> cat: state7/latency: No such file or directory
> c-state7
> cat: state8/name: No such file or directory
> cat: state8/latency: No such file or directory
> c-state8
> cat: state9/name: No such file or directory
> cat: state9/latency: No such file or directory
> c-state9
>
> cpufreq scaling info's available,
>
> cpupower frequency-info
> analyzing CPU 0:
> driver: intel_pstate
> CPUs which run at the same hardware frequency: 0
> CPUs which need to have their frequency coordinated by
> software: 0
> maximum transition latency: Cannot determine or is not
> supported.
> hardware limits: 800 MHz - 3.50 GHz
> available cpufreq governors: performance powersave
> current policy: frequency should be within 800 MHz and 3.50
> GHz.
> The governor "powersave" may decide which
> speed to use
> within this range.
> current CPU frequency: Unable to call hardware
> current CPU frequency: 799 MHz (asserted by call to kernel)
> boost state support:
> Supported: yes
> Active: yes
>
> & scaling IS in effect,
>
> cat /proc/cpuinfo | grep MHz
> cpu MHz : 798.106
> cpu MHz : 798.129
> cpu MHz : 798.964
> cpu MHz : 798.154
>
> (2) boot, WITH Xen 4.13
>
> rpm -qa | grep -i xen | sort
> grub2-x86_64-xen-2.04-lp151.6.5.noarch
> xen-4.13.0_04-lp151.688.2.x86_64
> xen-libs-4.13.0_04-lp151.688.2.x86_64
> xen-tools-4.13.0_04-lp151.688.2.x86_64
>
> Xen cmd line includes,
>
> grep options= /boot/grub2/xen-4.13.0_04-lp151.688.cfg
> [config.1]
> options=dom0=pvh dom0-iommu=map-reserved
> dom0_mem=4016M,max:4096M dom0_max_vcpus=4 cpufreq=xen cpuidle ucode=scan ...
>
> cstate info IS available
>
> xenpm start 5
> Timeout set to 5 seconds
> Start sampling, waiting for CTRL-C or SIGINT or SIGALARM signal
> ...
> Elapsed time (ms): 5000
>
> CPU0: Residency(ms) Avg Res(ms)
> C0 19 ( 0.40%) 0.03
> C1 0 ( 0.00%) 0.00
> C2 0 ( 0.00%) 0.00
> C3 1 ( 0.02%) 0.22
> C4 1 ( 0.02%) 0.59
> C5 4978 (99.56%) 8.07
>
> Avg freq -302378336 KHz
>
> CPU1: Residency(ms) Avg Res(ms)
> C0 3 ( 0.08%) 0.03
> C1 0 ( 0.00%) 0.00
> C2 0 ( 0.00%) 0.00
> C3 0 ( 0.00%) 0.00
> C4 0 ( 0.00%) 0.00
> C5 4996 (99.92%) 35.94
>
> Avg freq -302378336 KHz
>
> CPU2: Residency(ms) Avg Res(ms)
> C0 5000 (100.00%) 5000.27
> C1 0 ( 0.00%) 0.00
> C2 0 ( 0.00%) 0.00
> C3 0 ( 0.00%) 0.00
> C4 0 ( 0.00%) 0.00
> C5 0 ( 0.00%) 0.00
>
> Avg freq -302378336 KHz
>
> CPU3: Residency(ms) Avg Res(ms)
> C0 5 ( 0.11%) 0.03
> C1 0 ( 0.00%) 0.00
> C2 0 ( 0.00%) 0.00
> C3 0 ( 0.01%) 0.32
> C4 0 ( 0.00%) 0.00
> C5 4994 (99.88%) 25.48
>
> Avg freq -302378336 KHz
>
> Socket 0
> PC1 0 ms 0.00%
> PC2 0 ms 0.00%
> PC3 0 ms 0.00%
> Core 0 CPU 0
> CC1 0 ms 0.00%
> CC2 0 ms 0.00%
> CC3 1 ms 0.02%
> CC4 0 ms 0.00%
> CC5 0 ms 0.00%
> CC6 1 ms 0.02%
> CC7 4965 ms 99.30%
> Core 1 CPU 1
> CC1 0 ms 0.00%
> CC2 0 ms 0.00%
> CC3 0 ms 0.00%
> CC4 0 ms 0.00%
> CC5 0 ms 0.00%
> CC6 0 ms 0.00%
> CC7 4993 ms 99.86%
> Core 2 CPU 2
> CC1 0 ms 0.00%
> CC2 0 ms 0.00%
> CC3 0 ms 0.00%
> CC4 0 ms 0.00%
> CC5 0 ms 0.00%
> CC6 0 ms 0.00%
> CC7 0 ms 0.00%
> Core 3 CPU 3
> CC1 0 ms 0.00%
> CC2 0 ms 0.00%
> CC3 0 ms 0.01%
> CC4 0 ms 0.00%
> CC5 0 ms 0.00%
> CC6 0 ms 0.00%
> CC7 4989 ms 99.79%
>
> xenpm get-cpuidle-states 0
> All C-states allowed
>
> cpu id : 0
> total C-states : 6
> idle time(ms) : 46635271
> C0 : transition [ 3242686]
> residency [ 162089 ms]
> C1 : transition [ 9214]
> residency [ 1018 ms]
> C2 : transition [ 10254]
> residency [ 2871 ms]
> C3 : transition [ 23259]
> residency [ 17544 ms]
> C4 : transition [ 7252]
> residency [ 11896 ms]
> C5 : transition [ 3192706]
> residency [ 46522535 ms]
> pc2 : [ 1685 ms]
> pc3 : [ 30695 ms]
> cc3 : [ 17158 ms]
> cc6 : [ 11742 ms]
> cc7 : [ 46455767 ms]
>
> intel_pstate support is now DISABLED for this cpu
>
> xl dmesg | grep pstate
> [ 6.851121] intel_pstate: CPU model not supported
>
> NO cpufreq info is available
>
> xenpm get-cpufreq-states
> (empty)
>
> xenpm get-cpufreq-para
> [CPU0] failed to get cpufreq parameter
> [CPU1] failed to get cpufreq parameter
> [CPU2] failed to get cpufreq parameter
> [CPU3] failed to get cpufreq parameter
>
> xenpm enable-turbo-mode
> [CPU0] failed to enable turbo mode (13 - Permission denied)
> [CPU1] failed to enable turbo mode (13 - Permission denied)
> [CPU2] failed to enable turbo mode (13 - Permission denied)
> [CPU3] failed to enable turbo mode (13 - Permission denied)
>
> NO cpupower frequency-info is available
>
> cpupower frequency-info
> analyzing CPU 0:
> no or unknown cpufreq driver is active on this CPU
> CPUs which run at the same hardware frequency: Not Available
> CPUs which need to have their frequency coordinated by
> software: Not Available
> maximum transition latency: Cannot determine or is not
> supported.
> Not Available
> available cpufreq governors: Not Available
> Unable to determine current policy
> current CPU frequency: Unable to call hardware
> current CPU frequency: Unable to call to kernel
> boost state support:
> Supported: no
> Active: no
>
> and scaling is NOT in effect
>
> cat /proc/cpuinfo | grep MHz
> cpu MHz : 3092.828
> cpu MHz : 3092.828
> cpu MHz : 3092.828
> cpu MHz : 3092.828
>
> attempts to add acpi-cpufreq or xen-acpi-processor modules fail
>
> lsmod | egrep "xen|acpi"
> xen_pciback 81920 0
> xen_netback 73728 0
> xen_blkback 53248 0
> xen_gntalloc 20480 0
> xen_gntdev 45056 1
> xen_evtchn 16384 0
> xenfs 16384 1
> xen_privcmd 28672 17 xenfs
>
> lsmod | egrep "acpi-cpufreq|xen-acpi-processor"
> (empty)
>
> find /lib/modules/ | egrep "acpi-cpufreq|xen-acpi-processor"
>
> /lib/modules/5.4.14-25.g170524c-default/kernel/drivers/cpufreq/acpi-cpufreq.ko
>
> /lib/modules/5.4.14-25.g170524c-default/kernel/drivers/xen/xen-acpi-processor.ko
>
> modprobe acpi-cpufreq
> modprobe: ERROR: could not insert 'acpi_cpufreq': No such device
> insmod
> /lib/modules/5.4.14-25.g170524c-default/kernel/drivers/cpufreq/acpi-cpufreq.ko
> insmod: ERROR: could not insert module
> /lib/modules/5.4.14-25.g170524c-default/kernel/drivers/cpufreq/acpi-cpufreq.ko:
> No such device
> modprobe xen-acpi-processor
> modprobe: ERROR: could not insert 'xen_acpi_processor': No such
> device
> insmod
> /lib/modules/5.4.14-25.g170524c-default/kernel/drivers/xen/xen-acpi-processor.ko
> insmod: ERROR: could not insert module
> /lib/modules/5.4.14-25.g170524c-default/kernel/drivers/xen/xen-acpi-processor.ko:
> No such device
>
> fwiw, i did find this _old_ (2013) post,
>
> Linux 3.4 dom0 kernel error loading xen-acpi-processor: Input/output
> error
> https://lists.gt.net/xen/devel/274864
>
> which similarly manifests no cpufreq/turbomode in Dom0, and a fail to load
> 'xen_acpi_processor' mod unless/until
>
> > I found this:
> http://en.community.dell.com/techcenter/power-cooling/w/wiki/best-practices-in-power-management.aspx
> >
> > So I had to enable "OS Control" for "Power Management" in the Dell
> server BIOS,
> > and after that the CPU P-states are available in the ACPI tables,
> > and xen-acpi-processor driver loads and works OK in the dom0 kernel!
>
> checking the mobo/bios man
>
> https://www.supermicro.com/manuals/motherboard/C226/MNL-1544.pdf
>
> SM's BIOS doesn't have an identical "OS Control" option ...
>
> atm, boot config's,
>
> options=dom0=pvh dom0-iommu=map-reserved dom0_max_vcpus=4
> dom0_mem=4016M,max:4096M bootscrub=false spec-ctrl=ssbd,l1d-flush=true
> pv-l1tf=dom0=true,domu=true smt=true iommu=required,intremap,verbose,debug
> reboot=a ucode=scan sched=credit2 cpufreq=xen:ondemand,verbose cpuidle=on
> cpuinfo=on acpi=force vga=gfx-1920x1080x16 com1=115200,8n1,pci
> console=com1,vga console_timestamps console_to_ring conring_size=64
> log_buf_len=16M loglvl=all guest_loglvl=all noreboot=false sync_console=true
> sched_debug iommu=verbose apic_verbosity=verbose
Because of "console_to_ring" here, ...
> kernel=vmlinuz-5.4.14-25.g170524c-default root=/dev/mapper/VG0-ROOT
> rd.shell intel_idle.max_cstate=0 softlevel=xen rd.shell=1 rd.debug=0
> rd.auto=1 rd.systemd.show_status=auto rd.udev.log_priority=3 mds=full
> l1tf=flush smt=1 dolvm lvmwait=/dev/mapper/VG0-ROOT root=/dev/mapper/VG0-ROOT
> rootfstype=ext4 rootflags=journal_checksum noresume video=vesa:off
> video=efifb:1024x768 video=HDMI-A-1:1920x1080@60 modeset=1 nouveau.modeset=1
> xencons=xvc console=tty0 console=hvc0 pcie_asmp=off mce=bootlog
> fsck.mode=auto fsck.repair=preen reboot=acpi nmi_watchdog=0 clocksource=tsc
> tsc=reliable clocksource_failover=tsc force_tsc_stable=1 consistent_tscs
> iommu=1 apparmor=0 enforcing=0 selinux=0 rd.plymouth=0 plymouth.enable=0
> scsi_mod.use_blk_mq=1 elevator=mq-deadline enable_mtrr_cleanup
> mtrr_spare_reg_nr=1 mtrr_gran_size=32M mtrr_chunk_size=128M net.ifnames=1
> biosdevname=0 showopts noquiet log_buf_len=10M print_fatal_signals=1
> systemd.log_level=debug systemd.log_target=kmsg earlyprintk=xen,keep debug
> loglevel=8 lapic=debug apic_verbosity=debug apic=debug audit=0
> ramdisk=initrd-5.4.14-25.g170524c-default
>
> yes, i've been tweaking it a lot, trying to troubleshoot. feel free to
> suggest any other config to boot&log.
>
> here's the current verbose Xen boot log (if serial console output's req'd,
> can do that ... l8r, after cable replacement :-/). not sure if it's
> sufficient; pls advise what add'l info is useful.
>
> xl dmesg
... there is (afaics) exactly one hypervisor message in all of the
provided output. You either need to drastically increase the ring
size, or drop that option. I would anyway suggest to reduce both
command lines to the set of options you truly need. And of course
all of this goes along with Jürgen's request to see whether things
work with Dom0 in PV mode.
As an aside I think it would also have helped to see the actual
problem points if you had omitted some of the irrelevant info on
e.g. non-Xen kernel drivers not working, or user space tools not
being Xen-aware, which was already pointed out in reply to your
@distro report.
Jan
_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel
|
 |
Lists.xenproject.org is hosted with RackSpace, monitoring our |