|
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] Re: [Xen-users] Linux Fiber or iSCSI SAN
Hello everyone, At my current workplace, we've been evaluating solutions from DDN vs. NetApp vs. in-house. The requirement was to have a low entry price for at least 1/3 PT storage, as a starting point, high IO/bandwidth, low latency and Hadoop compatibility, and target capacity of 1PT with further The DDN and NetApp solutions were all $300k+ with limited flexibility, overpriced replacement drives and limited expandability options. After evaluating our own solution on old hardware we had lying around, we've decided to give it a shot. There was obviously some risks, convincing management to sign the PO for $25k and explaining the risks and benefits, with a worst case scenario - using it as more traditional storage nodes. We've purchased 4 x 3U SuperMicro chassis with 36 x 3.5 HDDs and additional internal slots for OS drives. Along with few used $150 Infiniband 40Gig cards and IB switch (most expensive single piece of equipment here ~ $5-7k). The resulted 4 node GlusterFS cluster running over RDMA transport, ZFS bricks (10 HDD in raidz + 1 SSD cache + 1 spare), with 200 nano-second fabric latency, highly configurable replication (we use 3x) and flexible expandability. In out tests so far with this system, we've seen 18GB/sec fabric bandwidth, reading from all 3 replicas (which is what Gluster does when you replicate - it spreads IO) at 6GB/sec per replica. 6GB per second is pretty much the most you can squeeze out of 40GB Infiniband (aka QDR), but that was a sequential read test. However, by increasing number of Gluster nodes and bricks, you can achieve greater throughput for I suppose, you could do DRBD over RDMA (SDP or SuperSockets as per DRBD Docs: http://www.drbd.org/users-guide/s-replication-transports.html) if your environment requires it, over Gluster.. Infiniband is now part of the linux kernel, compare to few years ago.. and used hardware is not that expensive.. not much different from Fiber Channel. 56Gig (aka FDR) is also available, albeit more expensive.. Imho, Infiniband is going to become more relevant and universal in the upcoming years.. Cheers, Anastas S sysadmin++ On Wed, Jun 12, 2013 at 12:00 PM, <xen-users-request@xxxxxxxxxxxxx> wrote: > Send Xen-users mailing list submissions to > xen-users@xxxxxxxxxxxxx > > To subscribe or unsubscribe via the World Wide Web, visit > http://lists.xen.org/cgi-bin/mailman/listinfo/xen-users > or, via email, send a message with subject or body 'help' to > xen-users-request@xxxxxxxxxxxxx > > You can reach the person managing the list at > xen-users-owner@xxxxxxxxxxxxx > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of Xen-users digest..." > > > Today's Topics: > > 1. Re: Linux Fiber or iSCSI SAN (Gordan Bobic) > 2. Re: Linux Fiber or iSCSI SAN (Nick Khamis) > 3. Re: Linux Fiber or iSCSI SAN (Gordan Bobic) > 4. Re: Linux Fiber or iSCSI SAN (Errol Neal) > 5. Re: Linux Fiber or iSCSI SAN (Nick Khamis) > 6. Re: Linux Fiber or iSCSI SAN (Nick Khamis) > 7. Re: Linux Fiber or iSCSI SAN (Errol Neal) > 8. Re: Linux Fiber or iSCSI SAN (Nick Khamis) > 9. Re: Linux Fiber or iSCSI SAN (Errol Neal) > 10. Re: Linux Fiber or iSCSI SAN (Nick Khamis) > 11. Re: Linux Fiber or iSCSI SAN (Gordan Bobic) > 12. Re: Linux Fiber or iSCSI SAN (Gordan Bobic) > 13. Xen 4.1 compile from source and install on Fedora 17 > (ranjith krishnan) > 14. Re: Xen 4.1 compile from source and install on Fedora 17 (Wei Liu) > 15. pv assign pci device (jacek burghardt) > 16. Re: pv assign pci device (Gordan Bobic) > 17. Re: Blog: Installing the Xen hypervisor on Fedora 19 > (Dario Faggioli) > 18. Xen Test Day is today! (Dario Faggioli) > 19. Re: [Xen-devel] Xen Test Day is today! (Fabio Fantoni) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Tue, 11 Jun 2013 17:52:02 +0100 > From: Gordan Bobic <gordan@xxxxxxxxxx> > To: Nick Khamis <symack@xxxxxxxxx> > Cc: xen-users <xen-users@xxxxxxxxxxxxxxxxxxx> > Subject: Re: [Xen-users] Linux Fiber or iSCSI SAN > Message-ID: > <1bdcfbd8f2994ee32483e1646fcbe5ec@xxxxxxxxxxxxxxxxxxxxxxxxx> > Content-Type: text/plain; charset=UTF-8; format=flowed > > On Tue, 11 Jun 2013 12:29:22 -0400, Nick Khamis <symack@xxxxxxxxx> > wrote: >> On Tue, Jun 11, 2013 at 11:30 AM, Nick Khamis wrote: >> >> Hello Everyone, >> >> I am speaking for everyone when saying that we are really interested >> in knowing what people are >> using in deployment. This would be active/active replicated, block >> level storage solutions at the: >> >> NAS Level: FreeNAS, OpenFiler (I know it's not linux), IET >> FS Level: ZFS, OCFS/2, GFS/2, GlusterFS >> Replication Level: DRBD vs GlusterFS >> Cluster Level: OpenAIS with Pacemaker etc... >> >> Our hope is for an educated breakdown (i.e., comparisons, benefits, >> limitation) of different setups, as opposed to >> a war of words on which NAS solution is better than the other. >> Comparing black boxes would also be interesting >> at a performance level. Talk about pricing, not so much since we >> already know that they cost and arm and a leg. >> >> Kind Regards, >> >> Nick. >> >> There was actually one more level I left out >> >> Hardware Level: PCIe bus (8x 16x V2 etc..), Interface cards (FC and >> RJ), SAS (Seagate vs WD) >> >> I hope this thread takes off, and individuals interested in the same >> topic can get some really valuable info. >> >> On a side note, and interesting comment I received was on the risks >> that are associated with such a custom build, as >> well as the lack of flexibility in some sense. > > The risk issue I might entertain to some extent (although > personally I think the risk is LOWER if you built the system > yourself and you have it adequately mirrored and backed up - if > something goes wrong you actually understand how it all hangs > together and can fix it yourself quickly, as opposed to hours > of downtime while an engineer on the other end of the phone > tries to guess what is actually wrong). > > But the flexibility argument is completely bogus. If you are > building the solution yourself you have the flexibility to do > whatever you want. When you buy and off the shelf > all-in-one-black-box appliance you are straitjacketed by > whatever somebody else decided might be useful without any > specific insight into your particular use case. > > Gordan > > > > ------------------------------ > > Message: 2 > Date: Tue, 11 Jun 2013 13:03:12 -0400 > From: Nick Khamis <symack@xxxxxxxxx> > To: Gordan Bobic <gordan@xxxxxxxxxx> > Cc: xen-users <xen-users@xxxxxxxxxxxxxxxxxxx> > Subject: Re: [Xen-users] Linux Fiber or iSCSI SAN > Message-ID: > <CAGWRaZZga+SuBc4iV0FO=D=HLthY=DNNJ-fuDEa1re8DQygZZA@xxxxxxxxxxxxxx> > Content-Type: text/plain; charset="iso-8859-1" > > On Tue, Jun 11, 2013 at 12:52 PM, Gordan Bobic <gordan@xxxxxxxxxx> wrote: > >> >> The risk issue I might entertain to some extent (although >> personally I think the risk is LOWER if you built the system >> yourself and you have it adequately mirrored and backed up - if >> something goes wrong you actually understand how it all hangs >> together and can fix it yourself quickly, as opposed to hours >> of downtime while an engineer on the other end of the phone >> tries to guess what is actually wrong). >> > > Very True!! > > But apples vs apples. It comes down to the warranty on your > iscsi raid controller, cpu etc.. vs. whatever guts are in the > powervault. And I agree with both trains of thoughts... > Warranty through adaptec or Dell, in either case there > will be downtime. > > >> >> But the flexibility argument is completely bogus. If you are >> building the solution yourself you have the flexibility to do >> whatever you want. When you buy and off the shelf >> all-in-one-black-box appliance you are straitjacketed by >> whatever somebody else decided might be useful without any >> specific insight into your particular use case. >> >> Gordan >> > > For sure... The inflexibility I was referring to are instance where > one starts out an endeavour to build a replicated NAS, and finds > out the hard way regarding size limitations of DRBD, lack of > clustering capabilities of FreeNAS, or instability issues of OpenFiler > with large instances. > > There is also SCSI-3 persistent reservations issues which is needed > by some of the virtualization systems that may of may not be supported > by FreeNAS (last I checked)... > > N. > > N. > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: > <http://lists.xen.org/archives/html/xen-users/attachments/20130611/3e06eae9/attachment.html> > > ------------------------------ > > Message: 3 > Date: Tue, 11 Jun 2013 18:13:58 +0100 > From: Gordan Bobic <gordan@xxxxxxxxxx> > To: Nick Khamis <symack@xxxxxxxxx> > Cc: xen-users <xen-users@xxxxxxxxxxxxxxxxxxx> > Subject: Re: [Xen-users] Linux Fiber or iSCSI SAN > Message-ID: > <7d0db81b985d5c4e76781d94626b0cd9@xxxxxxxxxxxxxxxxxxxxxxxxx> > Content-Type: text/plain; charset=UTF-8; format=flowed > > On Tue, 11 Jun 2013 13:03:12 -0400, Nick Khamis <symack@xxxxxxxxx> > wrote: >> On Tue, Jun 11, 2013 at 12:52 PM, Gordan Bobic wrote: >> >> The risk issue I might entertain to some extent (although >> personally I think the risk is LOWER if you built the system >> yourself and you have it adequately mirrored and backed up - if >> something goes wrong you actually understand how it all hangs >> together and can fix it yourself quickly, as opposed to hours >> of downtime while an engineer on the other end of the phone >> tries to guess what is actually wrong). >> >> Very True!! >> >> But apples vs apples. It comes down to the warranty on your >> iscsi raid controller, cpu etc.. vs. whatever guts are in the >> powervault. And I agree with both trains of thoughts... >> Warranty through adaptec or Dell, in either case there >> will be downtime. > > If you build it yourself you will save enough money that you can > have 5 of everything sitting on the shelf for spares. And it'll > all still be covered by a warranty. > >> But the flexibility argument is completely bogus. If you are >> building the solution yourself you have the flexibility to do >> whatever you want. When you buy and off the shelf >> all-in-one-black-box ?appliance you are straitjacketed by >> whatever somebody else decided might be useful without any >> specific insight into your particular use case. >> >> For sure... The inflexibility I was referring to are instance where >> one starts out an endeavour to build a replicated NAS, and finds >> out the hard way regarding size limitations of DRBD, lack of >> clustering capabilities of FreeNAS, or instability issues of >> OpenFiler with large instances. > > Heavens forbid we should do some research, prototyping and > testing before building the whole solution... > > It ultimately comes down to what your time is worth and > how much you are saving. If you are looking to deploy 10 > storage boxes at $10K each vs. $50K each, you can spend > a year prototyping and testing and still save a fortune. > If you only need one, it may or may not be worthwhile > depending on your hourly rate. > > Gordan > > > > ------------------------------ > > Message: 4 > Date: Tue, 11 Jun 2013 13:17:10 -0400 > From: Errol Neal <eneal@xxxxxxxxxxxxxxxxx> > To: Nick Khamis <symack@xxxxxxxxx> > Cc: "xen-users@xxxxxxxxxxxxxxxxxxx" <xen-users@xxxxxxxxxxxxxxxxxxx> > Subject: Re: [Xen-users] Linux Fiber or iSCSI SAN > Message-ID: <1370971030353245500@xxxxxxxxxxxxxxxxx> > Content-Type: text/plain > > >> > I've built a number of white box SANs using everything from OpenSolaris >> > and COMSTAR, Open-E, OpenFiler, SCST, IET... etc.using iSCSI and FC. >> > I've settled Ubuntu boxes booted via DRBD running SCST OR ESOS. >> > From a performance perspective, I have pretty large customer that two XCP >> > pools running off a Dell MD3200F using 4GB FC. To compare, I took a Dell >> > 2970 or something like that, stuck 8 Seatgate 2.5" Constellation Drives in >> > it, a 4GB HBA and installed ESOS on it. >> > I never got around to finishing my testing, but the ESOS box can >> > definitely keep up and things like LSI cachecade would really help to bring >> > it to a more enterprise-level performance with respect to random reads and >> > writes. >> > Lastly, there is such an abundance of DIRT CHEAP, lightly used 4GB FC >> > equipment on the market today that I find it interesting that people still >> > prefer iSCSI. iSCSI is good if you have 10GBE which is still far to >> > expensive per port IMO. However, you can get 2 - 4 port, 4GB FC Hbas on >> > ebay for under 100 bucks and I generally am able to purchase fully loaded >> > switches (brocade 200e) for somewhere in the neighborhood of 300 bucks >> > each! >> > MPIO with 2 FC ports from an initiator to a decent target can easily >> > saturate the link on basic sequential r/w write tests. Not to mention, >> > improved latency, access times, etc for random i/o. >> > >> >> Hello Eneal, >> >> Thank you so much for your response. Did you experience any problems with >> ESOS and your FS SAN in terms of stability. >> We already have our myrinet FC cards and switches, and I agree, it was dirt >> cheap. >> > > ESOS by all means is not perfect. I'm running an older release because it's > impossible to upgrade a production system without downtime using ESOS > (currently) but I was impressed with it non the less and i can see where it's > going. > I think what has worked better for me is using SCST on Ubuntu. As long as > your hardware is stable, you should have no issues. > At another site, I have two boxes in production (running iSCSI at this site) > and I've had zero non-hardware-related issues and I've been running them in > prod for 1 - 2 years. > > > > > ------------------------------ > > Message: 5 > Date: Tue, 11 Jun 2013 13:23:05 -0400 > From: Nick Khamis <symack@xxxxxxxxx> > To: eneal@xxxxxxxxxxxxxxxxx > Cc: "xen-users@xxxxxxxxxxxxxxxxxxx" <xen-users@xxxxxxxxxxxxxxxxxxx> > Subject: Re: [Xen-users] Linux Fiber or iSCSI SAN > Message-ID: > <CAGWRaZbBjH_bMS-Zgd-qN8f5b8zey2ng-ZZaGZ8QUkoaiKZ+XQ@xxxxxxxxxxxxxx> > Content-Type: text/plain; charset=ISO-8859-1 > >>> ESOS by all means is not perfect. I'm running an older release because it's >>> impossible to >>> upgrade a production system without downtime using ESOS (currently) but I >>> was >>> impressed with it non the less and i can see where it's going. > > Thanks again Errol. Just our of curiosity was any of this replicated? > > N. > > > > ------------------------------ > > Message: 6 > Date: Tue, 11 Jun 2013 13:27:24 -0400 > From: Nick Khamis <symack@xxxxxxxxx> > To: Gordan Bobic <gordan@xxxxxxxxxx> > Cc: xen-users <xen-users@xxxxxxxxxxxxxxxxxxx> > Subject: Re: [Xen-users] Linux Fiber or iSCSI SAN > Message-ID: > <CAGWRaZbY4uqZaq5b-CWam27vG_3K=qQnZBOcM5F_7UV3jya_qw@xxxxxxxxxxxxxx> > Content-Type: text/plain; charset=ISO-8859-1 > > On 6/11/13, Gordan Bobic <gordan@xxxxxxxxxx> wrote: >> Heavens forbid we should do some research, prototyping and >> testing before building the whole solution... >> >> It ultimately comes down to what your time is worth and >> how much you are saving. If you are looking to deploy 10 >> storage boxes at $10K each vs. $50K each, you can spend >> a year prototyping and testing and still save a fortune. >> If you only need one, it may or may not be worthwhile >> depending on your hourly rate. >> >> Gordan >> > > And hence the purpose of this thread :). Gordon, you mentioned that > you did use DRBD > for separate instances outside of the NAS. I am curious to know of > your experience with NAS level replication. What you feel would be a > more stable and scalable fit. > > N. > > > > ------------------------------ > > Message: 7 > Date: Tue, 11 Jun 2013 13:26:00 -0400 > From: Errol Neal <eneal@xxxxxxxxxxxxxxxxx> > To: Gordan Bobic <gordan@xxxxxxxxxx> > Cc: "xen-users@xxxxxxxxxxxxxxxxxxx" <xen-users@xxxxxxxxxxxxxxxxxxx>, > Nick Khamis <symack@xxxxxxxxx> > Subject: Re: [Xen-users] Linux Fiber or iSCSI SAN > Message-ID: <1370971560963286500@xxxxxxxxxxxxxxxxx> > Content-Type: text/plain > > On Tue, 06/11/2013 01:13 PM, Gordan Bobic <gordan@xxxxxxxxxx> wrote: >> >> Heavens forbid we should do some research, prototyping and >> testing before building the whole solution... >> >> It ultimately comes down to what your time is worth and >> how much you are saving. If you are looking to deploy 10 >> storage boxes at $10K each vs. $50K each, you can spend >> a year prototyping and testing and still save a fortune. >> If you only need one, it may or may not be worthwhile >> depending on your hourly rate. > > This is a really key point. I don't like to toot my own horn, but I've done > EXTENSIVE and EXHAUSTIVE research into this. I built my first Open-E iSCSI > box in like 2006. The right combination of hard disk, hdd firmware, raid > controller, controller firmware, motherboard, memory, cpu, nics, hbas.. > everything is critical and by the time you narrow all of this down and test > sufficiently and are ready to go into production, you've spent a significant > amount of time and money. > Now that said, if you able to piggy back off the knowledge of others, then > you get a nice short cut and to be fair, the open source software has > advanced and matured so much that it's really production ready for certain > workloads and environments. > > > > ------------------------------ > > Message: 8 > Date: Tue, 11 Jun 2013 13:27:55 -0400 > From: Nick Khamis <symack@xxxxxxxxx> > To: Gordan Bobic <gordan@xxxxxxxxxx> > Cc: xen-users <xen-users@xxxxxxxxxxxxxxxxxxx> > Subject: Re: [Xen-users] Linux Fiber or iSCSI SAN > Message-ID: > <CAGWRaZYxn6y5D-q3HnTo-H92NyDaORWh7fSKR7Q6HWnF48xsqw@xxxxxxxxxxxxxx> > Content-Type: text/plain; charset=ISO-8859-1 > > Gordan, sorry for the typo! > > N. > > > > ------------------------------ > > Message: 9 > Date: Tue, 11 Jun 2013 13:28:50 -0400 > From: Errol Neal <eneal@xxxxxxxxxxxxxxxxx> > To: Nick Khamis <symack@xxxxxxxxx> > Cc: "xen-users@xxxxxxxxxxxxxxxxxxx" <xen-users@xxxxxxxxxxxxxxxxxxx> > Subject: Re: [Xen-users] Linux Fiber or iSCSI SAN > Message-ID: <1370971730454167500@xxxxxxxxxxxxxxxxx> > Content-Type: text/plain > > On Tue, 06/11/2013 01:23 PM, Nick Khamis <symack@xxxxxxxxx> wrote: >> >> ESOS by all means is not perfect. I'm running an older release because >> >> it's impossible to >> >> upgrade a production system without downtime using ESOS (currently) but I >> >> was >> >> impressed with it non the less and i can see where it's going. >> >> Thanks again Errol. Just our of curiosity was any of this replicated? >> > > That is my next step. I had been planning of using Ininiband, SDP and DRBD, > but there are some funky issues there. I just never got around to it. > I think what's necessary over replication is a dual head configuration. > A combination of RAID1, CLVM, Pacemaker, SCST and shared storage between two > nodes should suffice. > > > > ------------------------------ > > Message: 10 > Date: Tue, 11 Jun 2013 13:32:16 -0400 > From: Nick Khamis <symack@xxxxxxxxx> > To: eneal@xxxxxxxxxxxxxxxxx > Cc: Gordan Bobic <gordan@xxxxxxxxxx>, "xen-users@xxxxxxxxxxxxxxxxxxx" > <xen-users@xxxxxxxxxxxxxxxxxxx> > Subject: Re: [Xen-users] Linux Fiber or iSCSI SAN > Message-ID: > <CAGWRaZZzsxXSRuH+XgfULrVcX7AGiSueA9f9WLzarMgseByNpA@xxxxxxxxxxxxxx> > Content-Type: text/plain; charset=ISO-8859-1 > >> Now that said, if you able to piggy back off the knowledge of others, then >> you get a nice short cut and to be fair, the open source software has >> advanced and matured so much that it's really production ready for certain >> workloads and environments. >> > > We run our BGP links on Quagga linux boxes on IBM machines and > transmitting an average of 700Mbps with packet sizes upwards of > 900-1000 bytes. I don't loose sleep over them.... > > N. > > > > ------------------------------ > > Message: 11 > Date: Tue, 11 Jun 2013 19:17:04 +0100 > From: Gordan Bobic <gordan@xxxxxxxxxx> > To: Nick Khamis <symack@xxxxxxxxx> > Cc: xen-users <xen-users@xxxxxxxxxxxxxxxxxxx> > Subject: Re: [Xen-users] Linux Fiber or iSCSI SAN > Message-ID: <51B769A0.8040301@xxxxxxxxxx> > Content-Type: text/plain; charset=ISO-8859-1; format=flowed > > On 06/11/2013 06:27 PM, Nick Khamis wrote: >> On 6/11/13, Gordan Bobic <gordan@xxxxxxxxxx> wrote: >>> Heavens forbid we should do some research, prototyping and >>> testing before building the whole solution... >>> >>> It ultimately comes down to what your time is worth and >>> how much you are saving. If you are looking to deploy 10 >>> storage boxes at $10K each vs. $50K each, you can spend >>> a year prototyping and testing and still save a fortune. >>> If you only need one, it may or may not be worthwhile >>> depending on your hourly rate. >>> >>> Gordan >>> >> >> And hence the purpose of this thread :). Gordon, you mentioned that >> you did use DRBD >> for separate instances outside of the NAS. I am curious to know of >> your experience with NAS level replication. What you feel would be a >> more stable and scalable fit. > > It largely depends on what exactly do you want to do with it. For a NAS, > I use ZFS + lsyncd for near-synchronous replication (rsync-on-write). > > For a SAN I tend to use ZFS with zvols exported over iSCSI, with period > ZFS send to the backup NAS. If you need real-time replication for > fail-over purposes, I would probably run DRBD on top of ZFS zvols. > > Gordan > > > > ------------------------------ > > Message: 12 > Date: Tue, 11 Jun 2013 19:28:40 +0100 > From: Gordan Bobic <gordan@xxxxxxxxxx> > To: eneal@xxxxxxxxxxxxxxxxx > Cc: "xen-users@xxxxxxxxxxxxxxxxxxx" <xen-users@xxxxxxxxxxxxxxxxxxx>, > Nick Khamis <symack@xxxxxxxxx> > Subject: Re: [Xen-users] Linux Fiber or iSCSI SAN > Message-ID: <51B76C58.6090303@xxxxxxxxxx> > Content-Type: text/plain; charset=ISO-8859-1; format=flowed > > On 06/11/2013 06:28 PM, Errol Neal wrote: >> On Tue, 06/11/2013 01:23 PM, Nick Khamis <symack@xxxxxxxxx> wrote: >>>>> ESOS by all means is not perfect. I'm running an older release because >>>>> it's impossible to >>>>> upgrade a production system without downtime using ESOS (currently) but I >>>>> was >>>>> impressed with it non the less and i can see where it's going. >>> >>> Thanks again Errol. Just our of curiosity was any of this replicated? >>> >> >> That is my next step. I had been planning of using Ininiband, >> SDP and DRBD, but there are some funky issues there. I just >> never got around to it. > > The first thing that jumps out at me here is infiniband. Do you have the > infrastructure and cabling in place to actually do that? This can be > very relevant depending on your environment. If you are planning to get > some cheap kit on eBay to do this, that's all well and good, but will > you be able to get a replacement if something breaks in a year or three? > One nice thing about ethernet is that it will always be around, it will > always be cheap, and it will always be compatible. > > For most uses multiple gigabit links bonded together are ample. Remember > that you will get, on a good day, about 120 IOPS per disk. Assuming a > typical 4K operation size that's 480KB/s/disk. At 16KB/op that is still > 1920KB/s/disk. At that rate you'd need 50 disks to saturate a single > gigabit channel. And you can bond a bunch of them together for next to > nothing in switch/NIC costs. > >> I think what's necessary over replication is a dual head >> configuration. > > Elaborate? > >> A combination of RAID1, CLVM, Pacemaker, SCST and shared storage >> between two nodes should suffice. > > In what configuration? > > Gordan > > > > ------------------------------ > > Message: 13 > Date: Tue, 11 Jun 2013 16:39:39 -0500 > From: ranjith krishnan <ranjithkrishnan1@xxxxxxxxx> > To: xen-users@xxxxxxxxxxxxx > Subject: [Xen-users] Xen 4.1 compile from source and install on Fedora > 17 > Message-ID: > <CAEybL6wFUpGJJa_BHumwR_TgVnN63qJ4ZHGF+EmdPF9mcaD7mQ@xxxxxxxxxxxxxx> > Content-Type: text/plain; charset="iso-8859-1" > > Hello, > > I am relatively new to Xen and need help compiling and installing Xen from > source. > > Using some tutorials online, I have got Xen working with the 'yum install > xen' method. > I used virt-manager and was able to get 2 domUs working ( CentOS 5, and > Fedora 16). > My domUs reside on Logical Volumes in an LVM, on a second hard disk sda2, > while my dom0 is installed on sda1. Everything is working fine in this > configuration. > I want to use Xen 4.1 since I want to continue using > virt-install/virt-manager for domU provisioning. > > For my work now, I want to install Xen from source and try to modify some > source code files and test things out. > I have seen some tutorials online, and I am not sure they give the complete > picture. > For ex, > http://wiki.xen.org/wiki/Xen_4.2_Build_From_Source_On_RHEL_CentOS_Fedora > Fedora 17 uses grub 2. When we do a yum install, the grub entries are taken > care of and things just work. > When I install from source, this is not the case. Are there any tutorials > which give a complete picture? > Or if someone has got Xen working from source on Fedora 16, 17 or 18, can > you give me tips on how to edit grub configuration so that xen boots ok. > I have tried and failed once compiling and installing Xen on Fedora 16, > which is when I used yum. > > > -- > Ranjith krishnan > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: > <http://lists.xen.org/archives/html/xen-users/attachments/20130611/34655873/attachment.html> > > ------------------------------ > > Message: 14 > Date: Tue, 11 Jun 2013 23:40:04 +0100 > From: Wei Liu <wei.liu2@xxxxxxxxxx> > To: ranjith krishnan <ranjithkrishnan1@xxxxxxxxx> > Cc: xen-users@xxxxxxxxxxxxx, wei.liu2@xxxxxxxxxx > Subject: Re: [Xen-users] Xen 4.1 compile from source and install on > Fedora 17 > Message-ID: <20130611224004.GA25483@xxxxxxxxxxxxxxxxxxxxx> > Content-Type: text/plain; charset="us-ascii" > > Hello, > > I've seen your mail to xen-devel as well. Given that you're still in > configuration phase, my gut feeling is that this is the proper list to > post. When you have questions about Xen code / development workflow you > can ask them on xen-devel. > > On Tue, Jun 11, 2013 at 04:39:39PM -0500, ranjith krishnan wrote: >> Hello, >> >> I am relatively new to Xen and need help compiling and installing Xen from >> source. >> >> Using some tutorials online, I have got Xen working with the 'yum install >> xen' method. >> I used virt-manager and was able to get 2 domUs working ( CentOS 5, and >> Fedora 16). >> My domUs reside on Logical Volumes in an LVM, on a second hard disk sda2, >> while my dom0 is installed on sda1. Everything is working fine in this >> configuration. >> I want to use Xen 4.1 since I want to continue using >> virt-install/virt-manager for domU provisioning. >> >> For my work now, I want to install Xen from source and try to modify some >> source code files and test things out. >> I have seen some tutorials online, and I am not sure they give the complete >> picture. >> For ex, >> http://wiki.xen.org/wiki/Xen_4.2_Build_From_Source_On_RHEL_CentOS_Fedora >> Fedora 17 uses grub 2. When we do a yum install, the grub entries are taken >> care of and things just work. >> When I install from source, this is not the case. Are there any tutorials >> which give a complete picture? >> Or if someone has got Xen working from source on Fedora 16, 17 or 18, can >> you give me tips on how to edit grub configuration so that xen boots ok. >> I have tried and failed once compiling and installing Xen on Fedora 16, >> which is when I used yum. > > For the grub entry, the simplest method is to place your binary under > /boot and invoke update-grub2 (which is also invoked when you do 'yum > install' if I'm not mistaken). In theory it should do the right thing. > > Another method to solve your problem is to modify grub.conf yourself. > Just copy the entry that 'yum install' adds in grub.conf, replace the > binary file name with the one you compile and you're all set. > > You might also find this page useful if you're to develop Xen. > http://wiki.xen.org/wiki/Xen_Serial_Console > (it also contains sample entries for legacy grub and grub2, nice ;-) ) > > > Wei. > >> >> >> -- >> Ranjith krishnan > >> _______________________________________________ >> Xen-users mailing list >> Xen-users@xxxxxxxxxxxxx >> http://lists.xen.org/xen-users > > > > > ------------------------------ > > Message: 15 > Date: Tue, 11 Jun 2013 19:01:33 -0600 > From: jacek burghardt <jaceksburghardt@xxxxxxxxx> > To: xen-users <xen-users@xxxxxxxxxxxxx> > Subject: [Xen-users] pv assign pci device > Message-ID: > <CAHyyzzQ53ZHYExKQ15TQSMdaXuN6t7_+wuJnFMFywvwJDYrBGA@xxxxxxxxxxxxxx> > Content-Type: text/plain; charset="iso-8859-1" > > I have xeon quad core server I wonder if is possible to assign pci usb > device to pv if the server does not suport iommu vd-t > I had blacklisted usb modules and hid devices and devices are listed as > assignable > but when I add them to pv I get this error libxl: error: libxl: error: > libxl_pci.c:989:libxl__device_pci_reset: The kernel doesn't support reset > from sysfs for PCI device 0000:00:1d.0 > libxl: error: libxl_pci.c:989:libxl__device_pci_reset: The kernel doesn't > support reset from sysfs for PCI device 0000:00:1d.1 > Daemon running with PID 897 > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: > <http://lists.xen.org/archives/html/xen-users/attachments/20130611/6e5ccfba/attachment.html> > > ------------------------------ > > Message: 16 > Date: Wed, 12 Jun 2013 07:13:15 +0100 > From: Gordan Bobic <gordan@xxxxxxxxxx> > To: jacek burghardt <jaceksburghardt@xxxxxxxxx> > Cc: xen-users <xen-users@xxxxxxxxxxxxx> > Subject: Re: [Xen-users] pv assign pci device > Message-ID: <51B8117B.3020404@xxxxxxxxxx> > Content-Type: text/plain; charset=ISO-8859-1; format=flowed > > On 06/12/2013 02:01 AM, jacek burghardt wrote: >> I have xeon quad core server I wonder if is possible to assign pci usb >> device to pv if the server does not suport iommu vd-t >> I had blacklisted usb modules and hid devices and devices are listed as >> assignable >> but when I add them to pv I get this error libxl: error: libxl: error: >> libxl_pci.c:989:libxl__device_pci_reset: The kernel doesn't support >> reset from sysfs for PCI device 0000:00:1d.0 >> libxl: error: libxl_pci.c:989:libxl__device_pci_reset: The kernel >> doesn't support reset from sysfs for PCI device 0000:00:1d.1 >> Daemon running with PID 897 > > I don't think that is a fatal error. I get that on, for example, the VGA > card passed through to the VM, but it still works inside the domU. It > just means the device doesn't support FLR. > > Gordan > > > > > ------------------------------ > > Message: 17 > Date: Wed, 12 Jun 2013 00:30:06 +0200 > From: Dario Faggioli <dario.faggioli@xxxxxxxxxx> > To: Ian Campbell <Ian.Campbell@xxxxxxxxxx> > Cc: xen-users@xxxxxxxxxxxxx, Russ Pavlicek > <russell.pavlicek@xxxxxxxxxxxxxx> > Subject: Re: [Xen-users] Blog: Installing the Xen hypervisor on Fedora > 19 > Message-ID: <1370989806.20028.51.camel@Solace> > Content-Type: text/plain; charset="utf-8" > > On gio, 2013-06-06 at 09:52 +0100, Ian Campbell wrote: >> On Wed, 2013-06-05 at 22:11 -0400, Russ Pavlicek wrote: >> > Saw this post from Major Hayden of Rackspace: >> > >> > http://major.io/2013/06/02/installing-the-xen-hypervisor-on-fedora-19/ >> >> It'd be good to get this linked from >> http://wiki.xen.org/wiki/Category:Fedora >> > Well, although I'm very happy about blog posts like these starting to > come up spontaneously all around the place, allow me to say tat we have > the Fedora host install page on the Wiki > (http://wiki.xen.org/wiki/Fedora_Host_Installation) that contains > exactly the same information (it actually has much more info, and it is > of course part of the Fedora wiki category!) > > That being said, I guess I can add a section there (in the Fedora > Category page) about 'external' pages, posts, etc... Let me think how > and where to put it... > > Thanks and Regards, > Dario > > -- > <<This happens because I choose it to happen!>> (Raistlin Majere) > ----------------------------------------------------------------- > Dario Faggioli, Ph.D, http://about.me/dario.faggioli > Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK) > > -------------- next part -------------- > A non-text attachment was scrubbed... > Name: signature.asc > Type: application/pgp-signature > Size: 198 bytes > Desc: This is a digitally signed message part > URL: > <http://lists.xen.org/archives/html/xen-users/attachments/20130612/67c37d4d/attachment.pgp> > > ------------------------------ > > Message: 18 > Date: Wed, 12 Jun 2013 09:01:56 +0200 > From: Dario Faggioli <dario.faggioli@xxxxxxxxxx> > To: xen-devel@xxxxxxxxxxxxx > Cc: xen-users@xxxxxxxxxxxxx, xen-api@xxxxxxxxxxxxx > Subject: [Xen-users] Xen Test Day is today! > Message-ID: <1371020516.9946.5.camel@Abyss> > Content-Type: text/plain; charset="utf-8" > > Hi everybody, > > Allow me to remind you that the 4th Xen Test Day is happening today, so > come and join us on #xentest on freenode! > > We will be testing Xen 4.3 RC4, released yesterday and, probably, *the* > *last* release candidate! For more info, see: > > - on Xen Test Days: > http://wiki.xen.org/wiki/Xen_Test_Days > > - on getting and testing RC4: > http://wiki.xen.org/wiki/Xen_4.3_RC4_test_instructions > > - for generic testing information: > http://wiki.xen.org/wiki/Testing_Xen > > See you all on freenode, channel #xentest. > > Regards > Dario > > -- > <<This happens because I choose it to happen!>> (Raistlin Majere) > ----------------------------------------------------------------- > Dario Faggioli, Ph.D, http://about.me/dario.faggioli > Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK) > > -------------- next part -------------- > A non-text attachment was scrubbed... > Name: signature.asc > Type: application/pgp-signature > Size: 198 bytes > Desc: This is a digitally signed message part > URL: > <http://lists.xen.org/archives/html/xen-users/attachments/20130612/2fcb2e25/attachment.pgp> > > ------------------------------ > > Message: 19 > Date: Wed, 12 Jun 2013 09:44:02 +0200 > From: Fabio Fantoni <fabio.fantoni@xxxxxxx> > To: Dario Faggioli <dario.faggioli@xxxxxxxxxx> > Cc: xen-users@xxxxxxxxxxxxx, xen-api@xxxxxxxxxxxxx, > xen-devel@xxxxxxxxxxxxx > Subject: Re: [Xen-users] [Xen-devel] Xen Test Day is today! > Message-ID: <51B826C2.3030706@xxxxxxx> > Content-Type: text/plain; charset="iso-8859-1"; Format="flowed" > > Il 12/06/2013 09:01, Dario Faggioli ha scritto: >> Hi everybody, >> >> Allow me to remind you that the 4th Xen Test Day is happening today, so >> come and join us on #xentest on freenode! >> >> We will be testing Xen 4.3 RC4, released yesterday and, probably, *the* >> *last* release candidate! For more info, see: >> >> - on Xen Test Days: >> http://wiki.xen.org/wiki/Xen_Test_Days >> >> - on getting and testing RC4: >> http://wiki.xen.org/wiki/Xen_4.3_RC4_test_instructions >> >> - for generic testing information: >> http://wiki.xen.org/wiki/Testing_Xen >> >> See you all on freenode, channel #xentest. >> >> Regards >> Dario >> >> >> >> _______________________________________________ >> Xen-devel mailing list >> Xen-devel@xxxxxxxxxxxxx >> http://lists.xen.org/xen-devel > I saw that qemu upstrem tag is not updated (on Config.mk > QEMU_UPSTREAM_REVISION ?= qemu-xen-4.3.0-rc1) but on git there are new > patches, why? > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: > <http://lists.xen.org/archives/html/xen-users/attachments/20130612/d02fbfa4/attachment.html> > > ------------------------------ > > _______________________________________________ > Xen-users mailing list > Xen-users@xxxxxxxxxxxxx > http://lists.xen.org/xen-users > > > End of Xen-users Digest, Vol 100, Issue 17 > ****************************************** _______________________________________________ Xen-users mailing list Xen-users@xxxxxxxxxxxxx http://lists.xen.org/xen-users
|
 |
Lists.xenproject.org is hosted with RackSpace, monitoring our |