Re: [Xen-devel] [PATCH v2] x86/apicv: fix RTC periodic timer and apicv issue

["Xuquan (Quan Xu)" <xuquan8@xxxxxxxxxx>]

On October 11, 2016 3:49 PM, Tian, Kevin <Kevin.tian@xxxxxxxxx>
>> From: Xuquan (Quan Xu) [mailto:xuquan8@xxxxxxxxxx]
>> Sent: Monday, October 10, 2016 6:49 PM
>>
>> On October 10, 2016 5:40 PM, Jan Beulich < JBeulich@xxxxxxxx > wrote:
>> >>>>>> >>> On 20.09.16 at 15:30, <xuquan8@xxxxxxxxxx> wrote:
>> >>>>>> > --- a/xen/arch/x86/hvm/vlapic.c
>> >>>>>> > +++ b/xen/arch/x86/hvm/vlapic.c
>> >>>>>> > @@ -433,6 +433,12 @@ void vlapic_EOI_set(struct vlapic
>> >>>>>> > *vlapic) void vlapic_handle_EOI(struct vlapic *vlapic, u8 vector)  {
>> >>>>>> >      struct domain *d = vlapic_domain(vlapic);
>> >>>>>> > +    struct vcpu *v = vlapic_vcpu(vlapic);
>> >>>>>> > +    struct hvm_intack pt_intack;
>> >>>>>> > +
>> >>>>>> > +    pt_intack.vector = vector;
>> >>>>>> > +    pt_intack.source = hvm_intsrc_lapic;
>> >>>>>> > +    pt_intr_post(v, pt_intack);
>> >>>>>>
>> >>>>>> This also sits on the EOI LAPIC register write path, i.e. the
>> >>>>>> change then also affects non-apicv environments.
>> >>>>>
>> >>>>>The new logic should be entered only when EOI-induced exit happens.
>> >>>>>
>> >>>>
>> >>>> Yes, more that the EOI-induced exit is conditional, specifically,
>> >>>> the bitmap is set by vmx_set_eoi_exit_bitmap().
>> >>>> Jan, what do you think? While I recall from v1 discussion, you
>> >>>> have the same comment. We can dig it deep..
>> >>>
>> >>>See my reply to Kevin sent a minute ago. As I'm not sure what Kevin
>> >>>means to state with several of his responses, I can't properly
>> >>>respond for now. And then what you say doesn't really address my
>> >>>concern - things being conditional elsewhere doesn't mean we won't
>> >>>get here too in the non-apicv case, at least not in a way that I can 
>> >>>follow
>right away.
>> >>
>> >> Jan, any idea now?
>> >
>> >I don't think there was anything left open on the other sub-thread;
>> >if there is, please point out specific aspects which are still unclear.
>> >
>>
>> Sorry, I overlooked the other sub-thread after holiday(10.1-10.7)..
>> I will read it again..
>>
>> Quan
>
>Is there any discussion after 10.1? I didn't see it.
>
>Back to the main open before holiday - multiple EOIs may come to clear
>irq_issued before guest actually handles the very vpt injection (possible if 
>vpt
>vector is shared with other sources). I don't see a good solution on that 
>open... :/
>
>We've discussed various options which all fail in one or another place - either
>miss an injection, or incur undesired injections.
>Possibly we should consider another direction - fall back to non-apicv path 
>when
>we see vpt vector pending but it's not the highest one.
>
>Original condition to enter virtual intr delivery:
>           else if ( cpu_has_vmx_virtual_intr_delivery &&
>              intack.source != hvm_intsrc_pic &&
>              intack.source != hvm_intsrc_vector )
>
>now new condition:
>           else if ( cpu_has_vmx_virtual_intr_delivery &&
>              intack.source != hvm_intsrc_pic &&
>              intack.source != hvm_intsrc_vector &&
>                       (pt_vector == -1 || intack.vector == pt_vector) )
>
>Thoughts?
>
Kevin, 
When I try to fix it as your suggestion, I cannot boot the guest, with below 
message(from xl dmesg):


(d1) HVM Loader
(d1) Detected Xen v4.8-unstable
(d1) Xenbus rings @0xfeffc000, event channel 1
(d1) System requested SeaBIOS
(d1) CPU speed is 2394 MHz
(d1) Relocating guest memory for lowmem MMIO space disabled
(XEN) irq.c:275: Dom1 PCI link 0 changed 0 -> 5
(d1) PCI-ISA link 0 routed to IRQ5
(XEN) irq.c:275: Dom1 PCI link 1 changed 0 -> 10
(d1) PCI-ISA link 1 routed to IRQ10
(XEN) irq.c:275: Dom1 PCI link 2 changed 0 -> 11
(d1) PCI-ISA link 2 routed to IRQ11
(XEN) irq.c:275: Dom1 PCI link 3 changed 0 -> 5
(d1) PCI-ISA link 3 routed to IRQ5
(d1) pci dev 01:3 INTA->IRQ10
(d1) pci dev 02:0 INTA->IRQ11
(d1) RAM in high memory; setting high_mem resource base to 20f800000
(d1) pci dev 03:0 bar 10 size 002000000: 0f0000008
(d1) pci dev 02:0 bar 14 size 001000000: 0f2000008
(d1) pci dev 03:0 bar 30 size 000010000: 0f3000000
(d1) pci dev 03:0 bar 14 size 000001000: 0f3010000
(d1) pci dev 02:0 bar 10 size 000000100: 00000c001
(d1) pci dev 01:1 bar 20 size 000000010: 00000c101
(d1) Multiprocessor initialisation:
(d1)  - CPU0 ... 46-bit phys ... fixed MTRRs ... var MTRRs [1/8] ... done.
(d1)  - CPU1 ... 46-bit phys ... fixed MTRRs ... var MTRRs [1/8] ... done.
(d1) Testing HVM environment:
(d1)  - REP INSB across page boundaries ... passed
(d1)  - GS base MSRs and SWAPGS ... passed
(d1) Passed 2 of 2 tests
(d1) Writing SMBIOS tables ...
(d1) Loading SeaBIOS ...
(d1) Creating MP tables ...
(d1) Loading ACPI ...
(d1) vm86 TSS at fc00a300
(d1) BIOS map:
(d1)  10000-100e3: Scratch space
(d1)  c0000-fffff: Main BIOS
(d1) E820 table:
(d1)  [00]: 00000000:00000000 - 00000000:000a0000: RAM
(d1)  HOLE: 00000000:000a0000 - 00000000:000c0000
(d1)  [01]: 00000000:000c0000 - 00000000:00100000: RESERVED
(d1)  [02]: 00000000:00100000 - 00000000:f0000000: RAM
(d1)  HOLE: 00000000:f0000000 - 00000000:fc000000
(d1)  [03]: 00000000:fc000000 - 00000001:00000000: RESERVED
(d1)  [04]: 00000001:00000000 - 00000002:0f800000: RAM
(d1) Invoking SeaBIOS ...
(d1) SeaBIOS (version rel-1.9.3-0-ge2fc41e)
(d1) BUILD: gcc: (SUSE Linux) 4.3.4 [gcc-4_3-branch revision 152973] binutils: 
(GNU
(d1) Binutils; SUSE Linux Enterprise 11) 2.23.1
(d1)
(d1) Found Xen hypervisor signature at 40000000
(d1) Running on QEMU (i440fx)
(d1) xen: copy e820...
(d1) Relocating init from 0x000d8fa0 to 0xeffabc40 (size 82736)
(d1) Found 6 PCI devices (max PCI bus is 00)
(d1) Allocated Xen hypercall page at effff000
(d1) Detected Xen v4.8-unstable
(d1) xen: copy BIOS tables...
(d1) Copying SMBIOS entry point from 0x00010020 to 0x000f5b60
(d1) Copying MPTABLE from 0xfc001170/fc001180 to 0x000f5a60
(d1) Copying PIR from 0x00010040 to 0x000f59e0
(d1) Copying ACPI RSDP from 0x000100c0 to 0x000f59b0
(d1) Using pmtimer, ioport 0xb008
(d1) Scan for VGA option rom
(d1) Running option rom at c000:0003
(XEN) stdvga.c:174:d1v0 entering stdvga mode
(d1) pmm call arg1=0
(d1) Turning on vga text mode console
(d1) SeaBIOS (version rel-1.9.3-0-ge2fc41e)
(d1) Machine UUID 59e20ef4-565a-49cb-9559-cde6d391cdf4
(d1) All threads complete.
(d1) Found 0 lpt ports
(d1) Found 0 serial ports
(d1) ATA controller 1 at 1f0/3f4/0 (irq 14 dev 9)
(d1) ATA controller 2 at 170/374/0 (irq 15 dev 9)
(d1) PS2 keyboard initialized
(d1) ata0-0: QEMU HARDDISK ATA-7 Hard-Disk (30720 MiBytes)
(d1) Searching bootorder for: /pci@i0cf8/*@1,1/drive@0/disk@0
(d1) All threads complete.
(d1) Scan for option roms
(d1)
(d1) Press ESC for boot menu.
(d1)
(d1) Searching bootorder for: HALT
(d1) drive 0x000f5940: PCHS=16383/16/63 translation=lba LCHS=1024/255/63 
s=62914561
(d1) Space available for UMB: c9800-ec800, f5380-f5940
(d1) Returned 258048 bytes of ZoneHigh
(d1) e820 map has 7 items:
(d1)   0: 0000000000000000 - 000000000009fc00 = 1 RAM
(d1)   1: 000000000009fc00 - 00000000000a0000 = 2 RESERVED
(d1)   2: 00000000000f0000 - 0000000000100000 = 2 RESERVED
(d1)   3: 0000000000100000 - 00000000effff000 = 1 RAM
(d1)   4: 00000000effff000 - 00000000f0000000 = 2 RESERVED
(d1)   5: 00000000fc000000 - 0000000100000000 = 2 RESERVED
(d1)   6: 0000000100000000 - 000000020f800000 = 1 RAM
(d1) enter handle_19:
(d1)   NULL
(d1) Booting from Hard Disk...
(d1) Booting from 0000:7c00
(XEN) stdvga.c:179:d1v0 leaving stdvga mode
(XEN) Failed vm entry (exit reason 0x80000021) caused by invalid guest state 
(0).
(XEN) ************* VMCS Area **************
(XEN) *** Guest State ***
(XEN) CR0: actual=0x0000000080010031, shadow=0x0000000080010031, 
gh_mask=ffffffffffffffff
(XEN) CR4: actual=0x00000000000426f8, shadow=0x00000000000406b8, 
gh_mask=ffffffffffffffff
(XEN) CR3 = 0x0000000000185000
(XEN) PDPTE0 = 0x0000000000186001  PDPTE1 = 0x0000000000187001
(XEN) PDPTE2 = 0x0000000000188001  PDPTE3 = 0x0000000000189001
(XEN) RSP = 0x000000008ce53a6c (0x000000008ce53a6c)  RIP = 0x000000008161dd21 
(0x000000008161dd21)
(XEN) RFLAGS=0x00200046 (0x00200046)  DR7 = 0x0000000000000400
(XEN) Sysenter RSP=000000008078b000 CS:RIP=0008:000000008168c0c0
(XEN)        sel  attr  limit   base
(XEN)   CS: 0008 0c09b ffffffff 0000000000000000
(XEN)   DS: 0023 0c0f3 ffffffff 0000000000000000
(XEN)   SS: 0010 0c093 ffffffff 0000000000000000
(XEN)   ES: 0023 0c0f3 ffffffff 0000000000000000
(XEN)   FS: 0030 04093 00003748 0000000081779c00
(XEN)   GS: 0000 1c000 ffffffff 0000000000000000
(XEN) GDTR:            000003ff 0000000081553000
(XEN) LDTR: 0000 1c000 ffffffff 0000000000000000
(XEN) IDTR:            000007ff 0000000081553400
(XEN)   TR: 0028 0008b 000020ab 00000000801ad000
(XEN) EFER = 0x0000000000000000  PAT = 0x0007010600070106
(XEN) PreemptionTimer = 0x00000000  SM Base = 0x00000000
(XEN) DebugCtl = 0x0000000000000000  DebugExceptions = 0x0000000000000000
(XEN) Interruptibility = 00000000  ActivityState = 00000000
(XEN) InterruptStatus = d100
(XEN) *** Host State ***
(XEN) RIP = 0xffff82d0801ff560 (vmx_asm_vmexit_handler)  RSP = 
0xffff83187e20ff90
(XEN) CS=e008 SS=0000 DS=0000 ES=0000 FS=0000 GS=0000 TR=e040
(XEN) FSBase=0000000000000000 GSBase=0000000000000000 TRBase=ffff830839dfec00
(XEN) GDTBase=ffff83187e36f000 IDTBase=ffff83187e37b000
(XEN) CR0=0000000080050033 CR3=0000000821f6e000 CR4=00000000001526e0
(XEN) Sysenter RSP=ffff83187e20ffc0 CS:RIP=e008:ffff82d080244a00
(XEN) EFER = 0x0000000000000000  PAT = 0x0000050100070406
(XEN) *** Control State ***
(XEN) PinBased=000000bf CPUBased=b6a065fa SecondaryExec=0000576b
(XEN) EntryControls=000051ff ExitControls=000fefff
(XEN) ExceptionBitmap=00060002 PFECmask=00000000 PFECmatch=00000000
(XEN) VMEntry: intr_info=800000e1 errcode=00000000 ilen=00000000
(XEN) VMExit: intr_info=00000000 errcode=00000000 ilen=00000006
(XEN)         reason=80000021 qualification=0000000000000000
(XEN) IDTVectoring: info=00000000 errcode=00000000
(XEN) TSC Offset = 0xfffd7fc7c2ca5a1c  TSC Multiplier = 0x0000000000000000
(XEN) TPR Threshold = 0x00  PostedIntrVec = 0xf4
(XEN) EPT pointer = 0x0000000821f8501e  EPTP index = 0x0000
(XEN) PLE Gap=00000080 Window=00001000
(XEN) Virtual processor ID = 0x5f7a VMfunc controls = 0000000000000000
(XEN) **************************************
(XEN) domain_crash called from vmx.c:3111
(XEN) Domain 1 (vcpu#0) crashed on cpu#12:
(XEN) ----[ Xen-4.8-unstable  x86_64  debug=y   Not tainted ]----
(XEN) CPU:    12
(XEN) RIP:    0008:[<000000008161dd21>]
(XEN) RFLAGS: 0000000000200046   CONTEXT: hvm guest (d1v0)
(XEN) rax: 00000000000c0000   rbx: 000000000000000a   rcx: 00000000000c00d1
(XEN) rdx: 0000000000000000   rsi: 0000000000000000   rdi: 000000008ce53acc
(XEN) rbp: 000000008ce53a6c   rsp: 000000008ce53a6c   r8:  0000000000000000
(XEN) r9:  0000000000000000   r10: 0000000000000000   r11: 0000000000000000
(XEN) r12: 0000000000000000   r13: 0000000000000000   r14: 0000000000000000
(XEN) r15: 0000000000000000   cr0: 0000000080010031   cr4: 00000000000406b8
(XEN) cr3: 0000000000185000   cr2: 000000008cc09000
(XEN) ds: 0023   es: 0023   fs: 0030   gs: 0000   ss: 0010   cs: 0008
(XEN) HVM2 save: CPU
(XEN) HVM2 save: PIC
(XEN) HVM2 save: IOAPIC
(XEN) HVM2 save: LAPIC
(XEN) HVM2 save: LAPIC_REGS
(XEN) HVM2 save: PCI_IRQ
(XEN) HVM2 save: ISA_IRQ
(XEN) HVM2 save: PCI_LINK
(XEN) HVM2 save: PIT
(XEN) HVM2 save: RTC
(XEN) HVM2 save: HPET
(XEN) HVM2 save: PMTIMER
(XEN) HVM2 save: MTRR
(XEN) HVM2 save: VIRIDIAN_DOMAIN
(XEN) HVM2 save: CPU_XSAVE
(XEN) HVM2 save: VIRIDIAN_VCPU
(XEN) HVM2 save: VMCE_VCPU
(XEN) HVM2 save: TSC_ADJUST
(XEN) HVM2 restore: CPU 0


Quan

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
https://lists.xen.org/xen-devel