[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Xen-users] AMD-Vi/Intel VT-d: Passthrough or virtualization?



>> My understanding was that this relied on the number of Virtual
>> Functions a PCI device was equipped with in the firmware. This at
>> least is the case for network cards...

According to this:

http://wiki.xen.org/wiki/Xen_VGA_Passthrough#Why_is_VGA_passthrough_different_from_normal_PCI_passthrough.3F

Xen uses a different type of passthrough with Video Cards that it does with Network Cards. That should be why you can't do that.



>> Not to chase tails here however, can we step back and figure out which
>> of the chipset manufactures (AMD vs. Intel) provides a stable platform
>> that can be used in production. We are not necessarily interested in
>> GPUs but we are interested in passing through network cards QLogic,
>> Intel etc... I would imagine this would still be important to the
>> gamers, and Justin.tv broadcasters as well....

I have made another Thread with that intention, here:

http://lists.xen.org/archives/html/xen-users/2013-06/msg00010.html



>> We can understand why the chipset, cpu, and even pci hardware
>> manufactures would play this cat and mouse game with virtualiztion
>> since to them it equates to less sales......

Indeed. Manufacturers doesn't want to let consumers get all the useful Server features that they sell at a premium on your typical Desktop computer. Its not that they will lose sales, but their sales will have a much lower profit margin. There are many enthusiasts users that would happily purchase a cheap Processor and overclock it instead of paying the full price of what a Processor with that nominal Frequency got (Some Server guys may argue with the reliability issue of running out-of-spec, but we're assuming that the guy knows what he is doing and can stress test it to guarantee rock solid stability), and on the professional lines like Intel Xeon, AMD Opteron, nVidia Quadro and AMD FirePro, for many parts they're selling you the same silicon that for the consumer market but with a much higher price.

Just check around for the price on nVidia GRID K2 that is supposed to be used for cloud gaming. And from a Hardware perspective, it just looks like a GeForce 690.



>> >> Casey DeLorme
>> >> From my experience if VT-d or IOMMU are not explicitly mentioned in the user
>> >> manuals >> (available for download off the net before you spend a dime on
>> >> the board) then it likely
>> >> does not have support for it.
>>
>> Interesting... We do something similar when purchasing IBMs. We look
>> to see if there are BIOS firmware updates that involve virtualization
>> such as this:
>>
>> http://www-947.ibm.com/support/entry/portal/docdisplay?lndocid=MIGR-5086623

I don't agree with just checking the manual for a single reason: That doesn't guarantees that it will work. Some people says that they have an option on the BIOS to enable VT-d/AMD-Vi, but the support is buggy or badly implemented. A BIOS upgrade can break support or fix it, so sometimes you have to downgrade, or expect your Motherboard manufacturer to be interesed in fixing it. Not all of them do. This means that your safest bet it to get someone with the Motherboard you want and ask him if he got it working, and in what BIOS version.



>> I came in a little late in the game for this conversation however, can
>> we please iron out some issues here. At an abstract level (i.e.,
>> chipsets, cpus, gpu, network interfaces), without mentioning any
>> motherboard manufactures such as ASRock, Asus, Saphire etc.. can we
>> determine which combination will work. Both on the AMD and Intel
>> platform. The reason for this is because not too many people deploy
>> white boxes for production. it's strictly SuperMicro, IBM, Dell etc...

I already did a recollection of possible supporting Hardware, that needs to be confirmed or discarded:


AMD Socket AM3: Chipsets AMD 890FX and 990FX has official IOMMU support build in in the Chipset itself. Of interesing note, is that albeit there seems to be other people that got the other 9xx series Chipsets working with AMD-Vi (On Xen wiki 970 and 990X are included, but not the 980G), AMD says on a Tech Doc that only the 990FX Chipset got support for it (Page 9, 1.1):

http://support.amd.com/us/ChipsetMotherboard_TechDocs/48691.pdf

Additionally, I heared that Bulldozer based Processors (Including AMD FX series Zambezi and Vishera, APUs Trinity and Richland) have another IOMMU built in. Considering this, you could potentially have two IOMMUs on Socket AM3+ if you have a Bulldozer based Processor with one of the previous two Chipsets. I have not confirmed this through. So the following combinations are possible:

K10 based Processor on 890FX or 990FX Chipset *MUST WORK*
K10 based Processor on 970, 980G or 990X *SHOULD NOT WORK*
Bulldozer based Processor on ANY Chipset *SHOULD WORK*
Bulldozer based Processor on 890FX or 990FX Chipset *MUST WORK*... just what IOMMU it uses?


AMD Socket FM1: There should be NO support on this platform. Llano, being K10 based, doesn't have a build in IOMMU, and the Chipsets doesn't have it, either.

AMD Socket FM2: As every Processor here is Bulldozer derived, you should have IOMMU support in all them. Besides the newer A85X, the other Chipsets are the same that on FM1 platform. ASRock released two beta BIOSes that claims to include IOMMU support on at least two Motherboard models that includes A55 (FM2A55 Pro) and A75 (FM2A75M-DGS) Chipsets:

http://www.asrock.com/mb/overview.asp?cat=Download&os=Beta&Model=FM2A75M-DGS
http://www.asrock.com/mb/AMD/FM2A55%20Pro/?cat=Beta

So I should suppose that information was correct.


Intel platforms are a bit more complicated. Intel usually likes to sell you features in a Processor/Chipset combo, so you usually need support from both things or get that feature artifficially crippled. If you have a K series Processor and want to overclock the CPU component, you need a P or Z series Chipset, you can't do it on a B, H or Q. I don't know if VT-d recibes similar treatment, but at least for vPro you DO need a Q series Chipset. With just one LGA 2011 exception, Intel disabled VT-d, TXT, vPro, and on Haswell, the newly introduced TSX on ALL K series Processors. Seems that they don't want overclockers virtualizing.
I didn't hear anyone claiming that you need a specific Chipset for VT-d support, until I asked yesterday a question related to this to a guy that work on ASUS, that claims that most VT-d features are getting moved to Q series Chipsets only:

http://www.xtremesystems.org/forums/showthread.php?286345-ASUS-Z87-Motherboards-Overview-Guides-and-Official-Support&p=5191428&viewfull=1#post5191428

I don't know if that info is true or not, as there are at least some guys with Z series Chipsets that can use VT-d. However, ASUS Motherboards do have a overally bad reputation regarding VT-d/AMD-Vi support, so I can't say if it just a mere excuse to justify that. I need to gather a few links and facts to reply to him about that. ASRock has been steadily adding IOMMU support to many Motherboards, and I heared that Gigabyte also sends custom BIOS to those people that requested support for it.



>> Once determining this, it would be nice to discuss which of the
>> manufactures are able to support stable platforms without rendering 3
>> out of the 4 PCIe/x slots useless. This would hurt!

This is one of the main issues. On a low level, you usually don't know the exact layout of a Motherboard and what is connected where. For what I readed, you can get all the stuff from the CPU and Chipset working, the usual things that are not compatible are bridges (The nVidia chip for adding PCIe lanes that was already mentioned), external controllers like the extra SATA Controllers that high end Motherboards have to provide you with more SATA connectors, and the like. Considering this, a good quality mainstream Motherboard should be a safer bet that a high end Motherboard full of additional components and chips that you may not have success trying to make them to work.



>> Buying hardware just got a lot more trickier folks, let's hope this
>> thread can shed some light on our lost souls, and make us complete
>> again....

With both Intel Haswell and AMD Richland have just been released, it may be worth to do this research on the new platforms for those that want to be early adopters.
_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxx
http://lists.xen.org/xen-users

 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.