Re: [Xen-devel] [BUG] xen_gntdev - gntdev_vma_find_special_page - unable to handle kernel paging request

[Arthur Borsboom <arthurborsboom@xxxxxxxxx>]

Hi Juergen,

I have noticed that a patch has been developed an awaits the upcoming Linux kernel 4.14, maybe 4.15.

In the mean time the server keeps crashing about every 2 days.

Do you know a workaround I can apply?

Best regards,

Arthur Borsboom.

On 25 October 2017 at 16:13, Juergen Gross <jgross@xxxxxxxx> wrote:

On 25/10/17 13:17, Arthur Borsboom wrote:
> Since about a month, possibly due to software updates, after a couple of
> days running several VMs, one of the VM guests crashes and the VM host
> is not stable anymore. I need to shutdown all the remaining VM guests
> (if possible) and reboot the server by hardware (sudo reboot hangs).
>
> Does anybody have a suggestion how to analyze/resolve this?

Hmm, it seems as if gntdev_mmap() is mapping only some pages and then
exits with an error. This will leave map->pages_vm_start as 0 leading
to a problem when the already mapped pages are being unmapped again.

Patch will come soon...

Can I add you as "Reported-by:" ?


Juergen

> All help is appreciated!
>
> Xen: 4.9.0
> OS (Dom0): Arch Linux 4.13.7
> Dmesg:
>
> [131395.101610] BUG: unable to handle kernel paging request at
> ffff88401920c018
> [131395.101715] IP: gntdev_vma_find_special_page+0x1d/0x30 [xen_gntdev]
> [131395.101796] PGD 1a0a067 
> [131395.101797] P4D 1a0a067 
> [131395.101832] PUD 0 
> [131395.101922] Oops: 0000 [#1] PREEMPT SMP
> [131395.101975] Modules linked in: xt_nat xt_physdev br_netfilter
> xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4
> iptable_nat nf_nat_ipv4 nf_nat tun bridge stp llc ebtable_filter
> ebtables devlink ipt_REJECT nf_reject_ipv4 xt_tcpudp nf_conntrack_ipv4
> nf_defrag_ipv4 xt_conntrack nf_conntrack libcrc32c crc32c_generic
> ip6table_filter iptable_filter ip6_tables snd_hda_codec_realtek amdkfd
> snd_hda_codec_generic snd_hda_codec_hdmi amd_iommu_v2 snd_hda_intel
> radeon joydev snd_hda_codec mousedev evdev input_leds led_class mac_hid
> ppdev wmi_bmof snd_hda_core i2c_algo_bit ttm snd_hwdep snd_pcm
> edac_mce_amd drm_kms_helper crct10dif_pclmul crc32_pclmul crc32c_intel
> ghash_clmulni_intel snd_timer pcbc r8169 aesni_intel psmouse aes_x86_64
> crypto_simd glue_helper tpm_infineon drm cryptd tpm_tis snd pcspkr
> [131395.102862]  agpgart sp5100_tco tpm_tis_core mii syscopyarea
> sysfillrect tpm i2c_piix4 sysimgblt soundcore parport_pc parport
> fb_sys_fops fam15h_power k10temp wmi shpchp button sch_fq_codel
> xen_acpi_processor xen_pciback xen_netback xen_blkback xenfs
> xen_gntalloc xen_gntdev xen_evtchn xen_privcmd ip_tables x_tables ext4
> crc16 mbcache jbd2 fscrypto hid_generic usbhid hid sd_mod ata_generic
> pata_acpi ohci_pci pata_atiixp serio_raw atkbd libps2 ahci ehci_pci
> libahci ehci_hcd ohci_hcd libata usbcore scsi_mod usb_common i8042 serio
> [131395.103469] CPU: 0 PID: 10887 Comm: qemu-dm Not tainted 4.13.7-1-ARCH #1
> [131395.103554] Hardware name: MICRO-STAR INTERNATIONAL CO.,LTD
> MS-7596/760GM-E51(MS-7596), BIOS V3.6 10/26/2012
> [131395.103677] task: ffff8800483b1e00 task.stack: ffffc90046598000
> [131395.103759] RIP: e030:gntdev_vma_find_special_page+0x1d/0x30
> [xen_gntdev]
> [131395.103852] RSP: e02b:ffffc9004659bb60 EFLAGS: 00010212
> [131395.103927] RAX: ffff88001ef0a360 RBX: ffff8800119a0cb8 RCX:
> 0000000000624684
> [131395.104018] RDX: 8000000624684367 RSI: 00000007ff460397 RDI:
> ffff88003dd7b240
> [131395.104108] RBP: ffffc9004659bb70 R08: ffff88003dd7b240 R09:
> 00007ff4603a2000
> [131395.104198] R10: 0000000000000001 R11: 00003ffffffff000 R12:
> 00007ff460397000
> [131395.104288] R13: 8000000624684367 R14: 00007ff460398000 R15:
> ffffc9004659bce0
> [131395.104390] FS:  00007ff4605667c0(0000) GS:ffff880055000000(0000)
> knlGS:0000000000000000
> [131395.104490] CS:  e033 DS: 0000 ES: 0000 CR0: 0000000080050033
> [131395.104564] CR2: ffff88401920c018 CR3: 000000001cb0f000 CR4:
> 0000000000040660
> [131395.104655] Call Trace:
> [131395.104695]  ? vm_normal_page+0x5d/0xa0
> [131395.104748]  unmap_page_range+0x4e3/0x930
> [131395.104804]  unmap_single_vma+0x7d/0xf0
> [131395.104857]  unmap_vmas+0x51/0xb0
> [131395.104904]  unmap_region+0xbd/0x130
> [131395.104953]  ? gnttab_map_refs+0xc4/0x160
> [131395.105009]  ? gntdev_mmap+0x3a4/0x610 [xen_gntdev]
> [131395.105074]  mmap_region+0x461/0x5f0
> [131395.105122]  do_mmap+0x2b3/0x400
> [131395.105167]  vm_mmap_pgoff+0xcc/0x120
> [131395.105217]  SyS_mmap_pgoff+0x1bc/0x230
> [131395.105271]  SyS_mmap+0x1b/0x30
> [131395.108716]  entry_SYSCALL_64_fastpath+0x1a/0xa5
> [131395.112161] RIP: 0033:0x7ff45dcc3e63
> [131395.115644] RSP: 002b:00007ffdf0c57648 EFLAGS: 00000246 ORIG_RAX:
> 0000000000000009
> [131395.119149] RAX: ffffffffffffffda RBX: 000000000000a000 RCX:
> 00007ff45dcc3e63
> [131395.122587] RDX: 0000000000000002 RSI: 000000000000b000 RDI:
> 0000000000000000
> [131395.126005] RBP: 0000000000001000 R08: 000000000000002a R09:
> 000000000000c000
> [131395.129509] R10: 0000000000000001 R11: 0000000000000246 R12:
> 0000000000000001
> [131395.132989] R13: 0000560471ae8e00 R14: 0000560471a66290 R15:
> 0000000000000000
> [131395.136568] Code: 5b 5d c3 90 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44
> 00 00 48 8b 87 a8 00 00 00 55 48 89 e5 48 2b 70 68 48 8b 40 60 5d 48 c1
> ee 0c <48> 8b 04 f0 c3 0f 1f 40 00 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 
> [131395.143798] RIP: gntdev_vma_find_special_page+0x1d/0x30 [xen_gntdev]
> RSP: ffffc9004659bb60
> [131395.147367] CR2: ffff88401920c018
> [131395.150844] ---[ end trace bf61e71da2f22d1c ]---
>
> xl info:
>
> host                   : orion1695
> release                : 4.13.7-1-ARCH
> version                : #1 SMP PREEMPT Sat Oct 14 20:13:26 CEST 2017
> machine                : x86_64
> nr_cpus                : 8
> max_cpu_id             : 23
> nr_nodes               : 1
> cores_per_socket       : 4
> threads_per_core       : 2
> cpu_mhz                : 2300
> hw_caps                :
> 178bf3ff:b698320b:2e500800:0069bfff:00000000:00000008:00000000:00000500
> virt_caps              : hvm
> total_memory           : 32510
> free_memory            : 30751
> sharing_freed_memory   : 0
> sharing_used_memory    : 0
> outstanding_claims     : 0
> free_cpus              : 0
> xen_major              : 4
> xen_minor              : 9
> xen_extra              : .0
> xen_version            : 4.9.0
> xen_caps               : xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32
> hvm-3.0-x86_32p hvm-3.0-x86_64 
> xen_scheduler          : credit
> xen_pagesize           : 4096
> platform_params        : virt_start=0xffff800000000000
> xen_changeset          : 
> xen_commandline        : /boot/xen-4.9.0.gz dom0_mem=1400M,max:1400M
> dom0pvh=1 ucode=-1
> cc_compiler            : gcc (GCC) 7.1.1 20170630
> cc_compile_by          : arthur

> cc_compile_domain      : startdedicated.com <http://startdedicated.com>
> cc_compile_date        : Mon Aug 28 11:12:45 CEST 2017
> build_id               : 3fb8c28a1eec9d8b7f15f6ba17bf8feb9c20c99c
> xend_config_format     : 4
>
> Best Regards,
> Arthur Borsboom
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@xxxxxxxxxxxxx
> https://lists.xen.org/xen-devel
>

--

Arthur Borsboom

Mob: +31629089953

Email: arthurborsboom@xxxxxxxxx

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
https://lists.xen.org/xen-devel