|
> I suspect the reason you never got a reply is to do with the
> lack of conciseness of your post - many people probably gave
> up on reading before you actually got to the point.
Actually, I consider very important that people understand why I'm going through all this ordeal even though, as you said later, I'm mostly a Windows user. These paragraph were so you can get an idea about how I see Xen as a gateway to merge the best of both Windows and Linux worlds, along with solving a few of my personal issues that this type of virtualization could help me with.
> If you want something more recent that that, GTX680 can be
> modified into a Quadro K5000 or half of a Grid K2, but this
> requires a bit of soldering.
>
> The upshot of going for a cheap Quadro or a Quadrified
> Nvidia card is that rebooting VMs doesn't cause problems
> which ATI cards are widely reported to suffer from.
I know about the possibility of hard modding GeForces into Quadros, but purchasing a new card for that purpose is totally out of my budget. So I'm limited to whatever I can do with my current ones.
My understanding is that ATI Video Cards like the ones I already own were easier to passthrough than non-modded GeForces. No idea about the VM rebooting issues or what generations it applies to.
> You should be aware that Intel have changed VID control on Haswell
> and later CPUs, so undervolt tuning based on clock multiplier
> (e.g. using something like RMClock on Windows or PHC on Linux)
> no longer works. If you want to use this functionality, you would
> be better off picking a pre-Haswell CPU. I have this problem with
> my Chromebook Pixel, which runs uncomfortably (if you keep it on
> your lap) hot, even when mostly idle.
For mobile computers, pre-Haswell Processors VID can't be touched by Software, either. I purchased a year ago a Notebook with a Sandy Bridge and after googling around looking for info on that, there are no tools to undervolt neither mobile Sandy Bridge, Ivy Bridge or Haswell. It has to be done via BIOS only, which no Notebook BIOS will let you do. On Desktop platforms I think there were a few tools that did worked to undervolt via Software, but I'm not sure on Haswell generation. People that appreciates undervolt are just a small niche.
> I have more or less given up on buying any non-Asus motherboards.
> Switching to an EVGA SR-2 after completely trouble-free 5 years
> with my Asus Maximus Extreme has really shown me just how good
> Asus are compared to other manufacturers.
>
> All things being equal, if I was doing my rig for similar purposes
> as you (two virtual gaming VMs with VGA passthrough, one for me,
> one for the wife), I would probably get something like an Asus
> Sabertooth or Crosshair motherboard with an 8-Core AMD CPU. They
> are reasonably priced, support ECC, and seem to have worked
> quite well for VGA passthrough for may people on the list.
I find hard to believe that someone here would recommend an ASUS Motherboard considering that they were know to be the less reliable when it came to IOMMU support, which is top priority for my purpose. They're fine from the Hardware side, but these days where most of the important components are part of either the Processor or Chipset you can't really go wrong unless its a very, very low end model. Most expensive Motherboards are full of worthless features I don't need (And even makes virtualization harder, like that PCIe bridge). Besides that, what annoys me the most is the ASUS official "we don't support Linux, won't going to spend dev resources on fixing the BIOS so it follows standard specifications instead of expecting Windows to override settings and bad ACPI tables to make it work anyways". That's the very reason why I'm avoiding them at all cost currently, and suggesting other people to do the same. Oh, and every now and then ASUS has several low end models that are to be avoided, too. Maximus Extreme, Sabertooth or Crosshair aren't budget Motherboards, most manufacturers don't screw up badly if you're spending around 200 U$D or more on a Motherboard - yet ASUS does with IOMMU support.
Supermicro is quite expensive, but also a very reputable Server based brand. They're know for being 24/7 rock solid workhouses, reason why I'm spending soo much on one.
> Avoid anything featuring Nvidia NF200 PCIe bridges at all cost.
> That way lies pain and suffering. I'm in the process of working
> on two patches for Xen just to make things workable on my
> EVGA SR-2 (which has ALL if it's PCIe slots behind NF200
> bridges).
Supermicro X10SAT got the three PCIe 1x slots and 4 USB3 ports behind a PLX 8606 chip. Not sure if that one is supported or not, but I don't plan at the moment to fill those slots, neither.
> I flat out refuse to run anything without ECC memory these days.
> This is a major reason why I don't consider Core i chips an
> option.
There are several Xeons models that do support ECC Memory and are around the same price than a Core iX of similar specs. I think you also need a Server Motherboard based on the C222, C224 or C226 Chipsets too, another one of Intel artificial limits to force you to spend on a more expensive platform. But depending on your budget and what you were going to buy, adding ECC support on an Intel platform could be relatively cheap if you pick proper parts.
> Did you stability test them? GPUs come pre-overclocked to within
> 1% of death from the factory.
The default for my pair of Video Cards was 850 MHz @ 1.1V for GPU, which is the same than reference Radeons 5770, so these are not factory overclocked. They were capable of being 100% stable for BitCoin mining at 850 MHz @ 1V which was what I had them at, along with the VRAM @ 300 MHz. I think that other people with my model (Sapphire Radeon 5770 FLEX) were capable of around 1 GHz with the default cooling, but I didn't did it because I didn't wanted to push GPUs that were going to be on for 24/7 nor stressing out the Power Supply. I barely had them running games for more than 10 minutes, they were not the purpose that I purchased them for. But considering that currently I don't mine any longer, I could repurpose them to other VMs.
> I also wouldn't consider putting any non-expendable data on
> anything but ZFS - silent corruption happens far more often
> than most people imagine, especially on consumer grade desktop
> disks.
I would need to purchase a new, 2 TB+ HD to allow for ZFS redundancy on single disk systems. I have around 800 GB of data on this one. At the moment, can't.
> I didn't think firing up two D3D games at the same time was even
> possible - or are you talking about just minimizing one?
You can have two D3D games open at the same time even on WXP. However, as there is no proper GPU priority (Like CPU), you literally have no control of where you want the video performance at - League of Legends FPS are usually on the 15-20 FPS mark, and if I have Eve Online on the screen like on the screenshot its around 7-10. Also, remember that both games are running simultaneously powered by my integrated Radeon 4200 that is 5 years or so old, even the Haswell IGP should be 3-5 times more powerful than this.
http://imageshack.us/a/img24/4909/3uck.png
However, while it IS possible currently for me to run two games simultaneously, I wouldn't recommend it. Besides performance issues because my old IGP is less than adequate even for a single game, let alone two games at the same time, it is overally buggy. If I were playing Eve Online for some time and tried to open League of Legends, or if I was already logged on, tried to start a game, chances are it gives a D3D error, crashes, or glitches as described before. This is why I believe that having different games simultaneously on separated VMs could increase reliability for use cases that D3D doesn't handle good enough.
Additionally, sometimes you MAY want to use an older Driver version for whatever reason. For example, for my Radeon 4200 I don't want to use any recent Catalyst Drivers versions because at some point they introduced a 2D acceleration bug that locks my machine. Check this Thread:
http://forums.anandtech.com/showthread.php?t=2332798
I can live with these older Drivers with no issue. However, if I want to use the Radeons 5770 for BitCoin/LiteCoin mining or whatever other GPGPU purpose, I need a later version of the Catalyst Drivers because a newer one is required to install some AMD APP SDK versions that I need to use OpenCL. This means that I have to choose between a more stable system, or OpenCL with the newer Drivers but risk some additional freeze scenarios. This could apply to a game-by-game basis if I know that a game doesn't play nice with an specific Driver, so I could have a VHD with a WXP/W7 install and another set of Drivers specific for that game.
For me as a Desktop power user, the power of Xen is all about workarounding against OS and Software limitations that exist because at the time no one considered you could try doing stuff in some specific fashion, usually by running assymetric setups like mine (IGP + 2 discrete GPUs), multimonitor, etc. It is to make Frankenstein builds works as intended when the OS doesn't know how to do it. If my current machine had IOMMU support, I believe Xen could have helped me by letting me to have a WXP VM with the Radeon 4200 with the stable Drivers, then having a totally separated VM with the pair of Radeons 5770 with newer Drivers to mine, everything working conflictless assuming a perfect scenario.
> My understanding was that cost effectiveness of electricity + ammortized
> hardware cost of hardware was nowdays such that it makes mining
> cost-ineffective. But whatever floats your boat.
Where I live at, is STILL profitable, but barely so. Not enough to bother to be honest. I'm not doing it from some months ago due to the Driver issue described above. But originally it was one of the possible uses.
> I think your list of requirements is already spiraling out of
> control, and if you really are mostly a Windows user, you may
> find the amount of effort required to achieve a system this
> complex is not worth the benefits. As I said, I'm finding myself
> having to write patches to get my system working passably well,
> and the complexity level on that is considerably lower than
> what you are proposing.
I have been sitting on WXP for one decade. Whatever I do now with Xen will probabily last for years, reason why I consider that the time will be worth it considering WXP will fade sooner or later and I will be left in the cold if that happens. With a Hypervisor added to the mix I have much more flexibility to follow almost any path, and simultaneously.
The bad part of this is that while I do have the free time to waste making something out of Xen work, I'm NOT a programmer. I barely can do anything above a Hello World. This means that whatever I want to do, will be limited to what Xen can do either by itself or with patches, along with reading tons of guides, etc. If there are things that Xen currently can't do, they will be dropped out of my list of requeriments until the next Xen version, for as long as it is possible to do via Software modifications.
I may be dreaming a bit. My proposed requeriments are for my ultimate production setup, what can be archivable will obviously be less than that. However, for what you're telling me what can be currently done isn't really that far from that.
> This was only announced as a preview feature a few days ago.
> I wouldn't count on it being as production-ready as you might hope.
It is not new. I recall hearing about nVidia Grid and the GPU Hypervisor around 6 months ago. I think I readed somewhere that they already suffered more than a year of delays. Considering the market that it is aimed to and how much the nVidia GRIDs cost, I would be surprise if they are still far from production ready.
> In your case I would do as follows (if you accept my suggestion of dumping
> the slim hypervisor):
The problem is that if I run Xen over a full fledged OS and use it as my everyday Desktop, I'l risk that if I somehow screw it up with its configuration, have to restart it, or whatever else, I will also lose my current work/game session in the Windows VMs. If my main Desktop was inside a VM, I could restart it without disturbing the others VMs, or even restore a full backup VHD if needed.
Now, considering that Linux is mostly renowned for being secure and stable (Lets not talk about human errors, specially if they're beginners...), chances are that doesn't happen often enough to actually not being a workable setup. But my goal would be the slim Hypervisor for the previous reason, albeit you can argue that its overkill redundancy.
> Not sure about your screens but mine and many I have seen allow switching
> between multiple inputs (if they have).
The Samsung SyncMaster P2370H does have an extra HDMI input, the SyncMaster 932N+ has only one input. I never though about entering the Monitor menu to manually switch input, and how it could serve this purpose.
> You might be interested in a recent announcement I saw on the xen-devel mailing list, around work on a graphics virtualization solution : http://lists.xenproject.org/archives/html/xen-devel/2013-09/msg00681.html
Excellent to hear. Wasn't aware of that one, would give me yet another possible choice.
Overally I'm confident I will get it working close to what I intend. I have to purchase the new parts first before being actually able to even test it.
|
