Re: [Xen-users] AMD-Vi/Intel VT-d: Passthrough or virtualization?

[Andrew Bobulsky <rulerof@xxxxxxxxx>]

On Tue, Jun 4, 2013 at 2:52 PM, Zir Blazer <zir_blazer@xxxxxxxxxxx> wrote:
>>> My understanding was that this relied on the number of Virtual
>>> Functions a PCI device was equipped with in the firmware. This at
>>> least is the case for network cards...
>
> According to this:
>
> http://wiki.xen.org/wiki/Xen_VGA_Passthrough#Why_is_VGA_passthrough_different_from_normal_PCI_passthrough.3F
>
> Xen uses a different type of passthrough with Video Cards that it does with
> Network Cards. That should be why you can't do that.
>
>
>
>
>>> Not to chase tails here however, can we step back and figure out which
>>> of the chipset manufactures (AMD vs. Intel) provides a stable platform
>>> that can be used in production. We are not necessarily interested in
>>> GPUs but we are interested in passing through network cards QLogic,
>>> Intel etc... I would imagine this would still be important to the
>>> gamers, and Justin.tv broadcasters as well....
>
> I have made another Thread with that intention, here:
>
> http://lists.xen.org/archives/html/xen-users/2013-06/msg00010.html
>
>
>
>
>>> We can understand why the chipset, cpu, and even pci hardware
>>> manufactures would play this cat and mouse game with virtualiztion
>>> since to them it equates to less sales......
>
> Indeed. Manufacturers doesn't want to let consumers get all the useful
> Server features that they sell at a premium on your typical Desktop
> computer. Its not that they will lose sales, but their sales will have a
> much lower profit margin. There are many enthusiasts users that would
> happily purchase a cheap Processor and overclock it instead of paying the
> full price of what a Processor with that nominal Frequency got (Some Server
> guys may argue with the reliability issue of running out-of-spec, but we're
> assuming that the guy knows what he is doing and can stress test it to
> guarantee rock solid stability), and on the professional lines like Intel
> Xeon, AMD Opteron, nVidia Quadro and AMD FirePro, for many parts they're
> selling you the same silicon that for the consumer market but with a much
> higher price.
>
> Just check around for the price on nVidia GRID K2 that is supposed to be
> used for cloud gaming. And from a Hardware perspective, it just looks like a
> GeForce 690.
>
>
>
>
>>> >> Casey DeLorme
>>> >> From my experience if VT-d or IOMMU are not explicitly mentioned in
>>> >> the user
>>> >> manuals >> (available for download off the net before you spend a dime
>>> >> on
>>> >> the board) then it likely
>>> >> does not have support for it.
>>>
>>> Interesting... We do something similar when purchasing IBMs. We look
>>> to see if there are BIOS firmware updates that involve virtualization
>>> such as this:
>>>
>>>
>>> http://www-947.ibm.com/support/entry/portal/docdisplay?lndocid=MIGR-5086623
>
> I don't agree with just checking the manual for a single reason: That
> doesn't guarantees that it will work. Some people says that they have an
> option on the BIOS to enable VT-d/AMD-Vi, but the support is buggy or badly
> implemented. A BIOS upgrade can break support or fix it, so sometimes you
> have to downgrade, or expect your Motherboard manufacturer to be interesed
> in fixing it. Not all of them do. This means that your safest bet it to get
> someone with the Motherboard you want and ask him if he got it working, and
> in what BIOS version.
>
>
>
>
>>> I came in a little late in the game for this conversation however, can
>>> we please iron out some issues here. At an abstract level (i.e.,
>>> chipsets, cpus, gpu, network interfaces), without mentioning any
>>> motherboard manufactures such as ASRock, Asus, Saphire etc.. can we
>>> determine which combination will work. Both on the AMD and Intel
>>> platform. The reason for this is because not too many people deploy
>>> white boxes for production. it's strictly SuperMicro, IBM, Dell etc...
>
> I already did a recollection of possible supporting Hardware, that needs to
> be confirmed or discarded:
>
>
> AMD Socket AM3: Chipsets AMD 890FX and 990FX has official IOMMU support
> build in in the Chipset itself. Of interesing note, is that albeit there
> seems to be other people that got the other 9xx series Chipsets working with
> AMD-Vi (On Xen wiki 970 and 990X are included, but not the 980G), AMD says
> on a Tech Doc that only the 990FX Chipset got support for it (Page 9, 1.1):
>
> http://support.amd.com/us/ChipsetMotherboard_TechDocs/48691.pdf
>
> Additionally, I heared that Bulldozer based Processors (Including AMD FX
> series Zambezi and Vishera, APUs Trinity and Richland) have another IOMMU
> built in. Considering this, you could potentially have two IOMMUs on Socket
> AM3+ if you have a Bulldozer based Processor with one of the previous two
> Chipsets. I have not confirmed this through. So the following combinations
> are possible:
>
> K10 based Processor on 890FX or 990FX Chipset *MUST WORK*
> K10 based Processor on 970, 980G or 990X *SHOULD NOT WORK*
> Bulldozer based Processor on ANY Chipset *SHOULD WORK*
> Bulldozer based Processor on 890FX or 990FX Chipset *MUST WORK*... just what
> IOMMU it uses?
>
>
> AMD Socket FM1: There should be NO support on this platform. Llano, being
> K10 based, doesn't have a build in IOMMU, and the Chipsets doesn't have it,
> either.
>
> AMD Socket FM2: As every Processor here is Bulldozer derived, you should
> have IOMMU support in all them. Besides the newer A85X, the other Chipsets
> are the same that on FM1 platform. ASRock released two beta BIOSes that
> claims to include IOMMU support on at least two Motherboard models that
> includes A55 (FM2A55 Pro) and A75 (FM2A75M-DGS) Chipsets:
>
> http://www.asrock.com/mb/overview.asp?cat=Download&os=Beta&Model=FM2A75M-DGS
> http://www.asrock.com/mb/AMD/FM2A55%20Pro/?cat=Beta
>
> So I should suppose that information was correct.
>
>
> Intel platforms are a bit more complicated. Intel usually likes to sell you
> features in a Processor/Chipset combo, so you usually need support from both
> things or get that feature artifficially crippled. If you have a K series
> Processor and want to overclock the CPU component, you need a P or Z series
> Chipset, you can't do it on a B, H or Q. I don't know if VT-d recibes
> similar treatment, but at least for vPro you DO need a Q series Chipset.
> With just one LGA 2011 exception, Intel disabled VT-d, TXT, vPro, and on
> Haswell, the newly introduced TSX on ALL K series Processors. Seems that
> they don't want overclockers virtualizing.
> I didn't hear anyone claiming that you need a specific Chipset for VT-d
> support, until I asked yesterday a question related to this to a guy that
> work on ASUS, that claims that most VT-d features are getting moved to Q
> series Chipsets only:
>
> http://www.xtremesystems.org/forums/showthread.php?286345-ASUS-Z87-Motherboards-Overview-Guides-and-Official-Support&p=5191428&viewfull=1#post5191428
>(snippity-snip!)

I apologize that I don't have links for you, but VT-d and the K-series
chips was brought up specifically in a Reddit AMA with an Intel
architect a few months ago.  He wrote something along the lines of:

>VT-d was planned for support in the K-series chips, but late testing of the 
>features on >the pre-production runs showed that they were failing Intel's 
>feature tests, and VT-d >support was disabled on them as an unfortunate 
>last-minute move.

I got a little less upset about not being able to buy a 2600K, and
decided to skip the generation of chips altogether.  But reading that
VT-d support will disappear entirely is hopefully too disappointing to
be true....  :(

Regards,
Andrew Bobulsky

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxx
http://lists.xen.org/xen-users