|
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [Xen-devel] [PATCH 11 of 17] docs: remove some fatally out of date documentation
# HG changeset patch # User Ian Campbell <ian.campbell@xxxxxxxxxx> # Date 1321541678 0 # Node ID 55ce23cb7b2af3c497b3a10658e09d5fc11eb45f # Parent 617f5d6e9e69b4f6362df91f078b7dc2abdbd80a docs: remove some fatally out of date documentation I think these are better off deleted than remaining to confuse people. docs/misc/blkif-drivers-explained.txt: - Talks about Xen 2.0 beta, talks about the old pre-xenstored IPC mechanism. docs/misc/cpuid-config-for-guest.txt: - Doesn't really say anything, in particular doesn't actually describe how to configure CPUID. docs/misc/hg-cheatsheet.txt: - Not out of date per-se wrt mercural but talk about Xen 2.0 and gives URLs under www.cl.cam.ac.uk. Talks a lot about bitkeeper. Given that mercurial is hardly unusual anymore I think there must be better guides out there so this one is not worth resurecting. docs/misc/network_setup.txt: - This is more comprehensively documented on the wiki these days. docs/misc/VMX_changes.txt: - Is basically a changelog from the initial implementation of VMX in 2004. I'm not sure about some of the other docs, but these ones seemed fairly obvious. Signed-off-by: Ian Campbell <ian.campbell@xxxxxxxxxx> diff -r 617f5d6e9e69 -r 55ce23cb7b2a docs/misc/VMX_changes.txt --- a/docs/misc/VMX_changes.txt Thu Nov 17 14:54:12 2011 +0000 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,90 +0,0 @@ -Changes to Xen in support of Intel(R) Vanderpool Technology -------------------------------------------------------------- - -Our VT extensions to the Xen hypervisor provide full platform -virtualization, including CPU(s), memory, and I/O infrastructure. The -generic code in Xen handles and schedules those virtual machines as it -does for the existing para-virtualized domains. - -Full virtualization required by the OS guests requires full device -virtualization as well. The device models in BOCHS -(http://bochs.sourceforge.net/) were decoupled from the CPU -virtualization, and are used to virtualize the legacy devices (such as -keyboard, mouse, VGA, IDE) in the PC platform. At this point, the -device models run in user mode on domain 0, not in the Xen hypervisor. - -We would like to thank Ian Pratt and Keir Fraser for reviewing our -design and code intensively, and for providing numerous useful -suggestions to improve the architecture and code. - -We have a list of Intel team members who take credit for making this -release happen: Yunhong Jiang, Nitin Kamble, Chengyuan Li, Xin Li, -Xiaofeng Ling, Benjamin Liu, Asit Mallick, Jun Nakajima, Sunil Saxena, -Arun Sharma, Edwin Zhai, Jeff Zheng, and Louis Zhuang. We'll continue -to add more features to complete full virtualization in Xen using VT. - -The notes document the changes to the Xen hypervisor in order to add -VT support. The changes to other areas, such as Control Panel will be -added as we deliver the code. - -Summary of changes for the first release ----------------------------------------- -December 15, 2004 - - * VT specific event handling and domain management were added. - - * Shadow mode was extended to support full 32-bit guests - - * Domain switching code was extended to support VT domain - - * I/O request handling was added to communicate with the device model - - * Domain builder was extended to provide the environment when the - guest enters the protected mode, including E820 memory and VGA - info, typically obtained by BIOS calls. - -New code: ---------- - VT (Vanderpool Technology) is based on the new VMX (Virtual - Machine Extensions) architecture. The current release of the - software supports 32-bit only. - - * arch/x86/vmx.[ch] and arch/x86/vmx_*.[ch]: created to handle - VMX-specific events in order to provide virtual machine. - - * arch/x86/x86_32/entry.S: new code path was added to have the - first-level handler from VM exits. The first-level handler calls - the second-level handler in arch/x86/vmx.c. - - * arch/x86/setup.c: new function start_vmx() to init_intel() to - enable VMX mode. - - * include/asm-x86/config.h: #ifdef CONFIG_VMX was added. - - * arch/x86/domain.c: new code patch was added to create a VMX - domain given the flag from the control panel. - - * include/public/io/ioreq.h: A new data structure was added to - define the I/O requests between the Xen hypervisor and the - device models. - -Changes to the existing code: ------------------------------ - - * arch/x86/shadow.[ch]: new mode SHM_full_32 was added to support - full virtualization. The current Xen code assumes that the guest - page directory and tables have _machine_ (or host) physical page - frame numbers, and the new code allows to support _guest_ - physical page frame numbers - - * include/asm-x86/processor.h: struct arch_vmx_struct arch_vmx has - been added to the thread_struct data structure. The arch_vmx has - the additional VMX-related CPU context. - - * arch/x86/io_apic.c: reverse mapping between vector and irq has - been added. We will revisit this code when considering MSI - support. - ---- Jun - - diff -r 617f5d6e9e69 -r 55ce23cb7b2a docs/misc/blkif-drivers-explained.txt --- a/docs/misc/blkif-drivers-explained.txt Thu Nov 17 14:54:12 2011 +0000 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,485 +0,0 @@ -=== How the Blkif Drivers Work === -Andrew Warfield -andrew.warfield@xxxxxxxxxxxx - -The intent of this is to explain at a fairly detailed level how the -split device drivers work in Xen 1.3 (aka 2.0beta). The intended -audience for this, I suppose, is anyone who intends to work with the -existing blkif interfaces and wants something to help them get up to -speed with the code in a hurry. Secondly though, I hope to break out -the general mechanisms that are used in the drivers that are likely to -be necessary to implement other drivers interfaces. - -As a point of warning before starting, it is worth mentioning that I -anticipate much of the specifics described here changing in the near -future. There has been talk about making the blkif protocol -a bit more efficient than it currently is. Keir's addition of grant -tables will change the current remapping code that is used when shared -pages are initially set up. - -Also, writing other control interface types will likely need support -from Xend, which at the moment has a steep learning curve... this -should be addressed in the future. - -For more information on the driver model as a whole, read the -"Reconstructing I/O" technical report -(http://www.cl.cam.ac.uk/Research/SRG/netos/papers/2004-xenngio.pdf). - -==== High-level structure of a split-driver interface ==== - -Why would you want to write a split driver in the first place? As Xen -is a virtual machine manager and focuses on isolation as an initial -design principle, it is generally considered unwise to share physical -access to devices across domains. The reasons for this are obvious: -when device resources are shared, misbehaving code or hardware can -result in the failure of all of the client applications. Moreover, as -virtual machines in Xen are entire OSs, standard device drives that -they might use cannot have multiple instantiations for a single piece -of hardware. In light of all this, the general approach in Xen is to -give a single virtual machine hardware access to a device, and where -other VMs want to share the device, export a higher-level interface to -facilitate that sharing. If you don't want to share, that's fine. -There are currently Xen users actively exploring running two -completely isolated X-Servers on a Xen host, each with it's own video -card, keyboard, and mouse. In these situations, the guests need only -be given physical access to the necessary devices and left to go on -their own. However, for devices such as disks and network interfaces, -where sharing is required, the split driver approach is a good -solution. - -The structure is like this: - - +--------------------------+ +--------------------------+ - | Domain 0 (privileged) | | Domain 1 (unprivileged) | - | | | | - | Xend ( Application ) | | | - | Blkif Backend Driver | | Blkif Frontend Driver | - | Physical Device Driver | | | - +--------------------------+ +--------------------------+ - +--------------------------------------------------------+ - | X E N | - +--------------------------------------------------------+ - - -The Blkif driver is in two parts, which we refer to as frontend (FE) -and a backend (BE). Together, they serve to proxy device requests -between the guest operating system in an unprivileged domain, and the -physical device driver in the physical domain. An additional benefit -to this approach is that the FE driver can provide a single interface -for a whole class of physical devices. The blkif interface mounts -IDE, SCSI, and our own VBD-structured disks, independent of the -physical driver underneath. Moreover, supporting additional OSs only -requires that a new FE driver be written to connect to the existing -backend. - -==== Inter-Domain Communication Mechanisms ==== - -===== Event Channels ===== - -Before getting into the specifics of the block interface driver, it is -worth discussing the mechanisms that are used to communicate between -domains. Two mechanisms are used to allow the construction of -high-performance drivers: event channels and shared-memory rings. - -Event channels are an asynchronous interdomain notification -mechanism. Xen allows channels to be instantiated between two -domains, and domains can request that a virtual irq be attached to -notifications on a given channel. The result of this is that the -frontend domain can send a notification on an event channel, resulting -in an interrupt entry into the backend at a later time. - -The event channel between two domains is instantiated in the Xend code -during driver startup (described later). Xend's channel.py -(tools/python/xen/xend/server/channel.py) defines the function - - -def eventChannel(dom1, dom2): - return xc.evtchn_bind_interdomain(dom1=dom1, dom2=dom2) - - -which maps to xc_evtchn_bind_interdomain() in tools/libxc/xc_evtchn.c, -which in turn generates a hypercall to Xen to patch the event channel -between the domains. Only a privileged domain can request the -creation of an event channel. - -Once the event channel is created in Xend, its ends are passed to both the -front and backend domains over the control channel. The end that is -passed to a domain is just an integer "port" uniquely identifying the -event channel's local connection to that domain. An example of this -setup code is in linux-2.6.x/drivers/xen/blkfront/blkfront.c in -blkif_connect(), which receives several status change events as -the driver starts up. It is passed an event channel end in a -BLKIF_INTERFACE_STATUS_CONNECTED message, and patches it in like this: - - - blkif_evtchn = status->evtchn; - blkif_irq = bind_evtchn_to_irq(blkif_evtchn); - if ( (rc = request_irq(blkif_irq, blkif_int, - SA_SAMPLE_RANDOM, "blkif", NULL)) ) - printk(KERN_ALERT"blkfront request_irq failed (%ld)\n",rc); - - -This code associates a virtual irq with the event channel, and -attaches the function blkif_int() as an interrupt handler for that -irq. blkif_int() simply handles the notification and returns, it does -not need to interact with the channel at all. - -An example of generating a notification can also be seen in blkfront.c: - - -static inline void flush_requests(void) -{ - DISABLE_SCATTERGATHER(); - wmb(); /* Ensure that the frontend can see the requests. */ - blk_ring->req_prod = req_prod; - notify_via_evtchn(blkif_evtchn); -} -}}} - -notify_via_evtchn() issues a hypercall to set the event waiting flag on -the other domain's end of the channel. - -===== Communication Rings ===== - -Event channels are strictly a notification mechanism between domains. -To move large chunks of data back and forth, Xen allows domains to -share pages of memory. We use communication rings as a means of -managing access to a shared memory page for message passing between -domains. These rings are not explicitly a mechanism of Xen, which is -only concerned with the actual sharing of the page and not how it is -used, they are however worth discussing as they are used in many -places in the current code and are a useful model for communicating -across a shared page. - -A shared page is set up by a front end guest first allocating and passing -the address of a page in its own address space to the backend driver. - -Consider the following code, also from blkfront.c. Note: this code -is in blkif_disconnect(). The driver transitions from STATE_CLOSED -to STATE_DISCONNECTED before becoming CONNECTED. The state automata -is in blkif_status(). - - blk_ring = (blkif_ring_t *)__get_free_page(GFP_KERNEL); - blk_ring->req_prod = blk_ring->resp_prod = resp_cons = req_prod = 0; - ... - /* Construct an interface-CONNECT message for the domain controller. */ - cmsg.type = CMSG_BLKIF_FE; - cmsg.subtype = CMSG_BLKIF_FE_INTERFACE_CONNECT; - cmsg.length = sizeof(blkif_fe_interface_connect_t); - up.handle = 0; - up.shmem_frame = virt_to_machine(blk_ring) >> PAGE_SHIFT; - memcpy(cmsg.msg, &up, sizeof(up)); - - -blk_ring will be the shared page. The producer and consumer pointers -are then initialised (these will be discussed soon), and then the -machine address of the page is send to the backend via a control -channel to Xend. This control channel itself uses the notification -and shared memory mechanisms described here, but is set up for each -domain automatically at startup. - -The backend, which is a privileged domain then takes the page address -and maps it into its own address space (in -linux26/drivers/xen/blkback/interface.c:blkif_connect()): - - -void blkif_connect(blkif_be_connect_t *connect) - - ... - unsigned long shmem_frame = connect->shmem_frame; - ... - - if ( (vma = get_vm_area(PAGE_SIZE, VM_IOREMAP)) == NULL ) - { - connect->status = BLKIF_BE_STATUS_OUT_OF_MEMORY; - return; - } - - prot = __pgprot(_PAGE_PRESENT | _PAGE_RW | _PAGE_DIRTY | _PAGE_ACCESSED); - error = direct_remap_area_pages(&init_mm, VMALLOC_VMADDR(vma->addr), - shmem_frame<<PAGE_SHIFT, PAGE_SIZE, - prot, domid); - - ... - - blkif->blk_ring_base = (blkif_ring_t *)vma->addr -}}} - -The machine address of the page is passed in the shmem_frame field of -the connect message. This is then mapped into the virtual address -space of the backend domain, and saved in the blkif structure -representing this particular backend connection. - -NOTE: New mechanisms will be added very shortly to allow domains to -explicitly grant access to their pages to other domains. This "grant -table" support is in the process of being added to the tree, and will -change the way a shared page is set up. In particular, it will remove -the need of the remapping domain to be privileged. - -Sending data across shared rings: - -Shared rings avoid the potential for write interference between -domains in a very cunning way. A ring is partitioned into a request -and a response region, and domains only work within their own space. -This can be thought of as a double producer-consumer ring -- the ring -is described by four pointers into a circular buffer of fixed-size -records. Pointers may only advance, and may not pass one another. - - - resp_cons----+ - V - +----+----+----+----+----+----+----+ - | | | free(A) |RSP1|RSP2| - +----+----+----+----+----+----+----+ - req_prod->| | --------> |RSP3| - +----+ +----+ - |REQ8| | |<-resp_prod - +----+ +----+ - |REQ7| | | - +----+ +----+ - |REQ6| <-------- | | - +----+----+----+----+----+----+----+ - |REQ5|REQ4| free(B) | | | - +----+----+----+----+----+----+----+ - req_cons---------^ - - - -By adopting the convention that every request will receive a response, -not all four pointers need be shared and flow control on the ring -becomes very easy to manage. Each domain manages its own -consumer pointer, and the two producer pointers are visible to both -(xen/include/public/io/blkif.h): - - -/* NB. Ring size must be small enough for sizeof(blkif_ring_t) <=PAGE_SIZE.*/ - #define BLKIF_RING_SIZE 64 - - ... - -/* - * We use a special capitalised type name because it is _essential_ that all - * arithmetic on indexes is done on an integer type of the correct size. - */ -typedef u32 BLKIF_RING_IDX; - -/* - * Ring indexes are 'free running'. That is, they are not stored modulo the - * size of the ring buffer. The following macro converts a free-running counter - * into a value that can directly index a ring-buffer array. - */ -#define MASK_BLKIF_IDX(_i) ((_i)&(BLKIF_RING_SIZE-1)) - -typedef struct { - BLKIF_RING_IDX req_prod; /* 0: Request producer. Updated by front-end. */ - BLKIF_RING_IDX resp_prod; /* 4: Response producer. Updated by back-end. */ - union { /* 8 */ - blkif_request_t req; - blkif_response_t resp; - } PACKED ring[BLKIF_RING_SIZE]; -} PACKED blkif_ring_t; - - - -As shown in the diagram above, the rules for using a shared memory -ring are simple. - - 1. A ring is full when a domain's producer and consumer pointers are - equal (e.g. req_prod == resp_cons). In this situation, the - consumer pointer must be advanced. Furthermore, if the consumer - pointer is equal to the other domain's producer pointer, - (e.g. resp_cons = resp_prod), then the other domain has all the - buffers. - -2. Producer pointers point to the next buffer that will be written to. - (So blk_ring[MASK_BLKIF_IDX(req_prod)] should not be consumed.) - -3. Consumer pointers point to a valid message, so long as they are not - equal to the associated producer pointer. - -4. A domain should only ever write to the message pointed - to by its producer index, and read from the message at it's - consumer. More generally, the domain may be thought of to have - exclusive access to the messages between its consumer and producer, - and should absolutely not read or write outside this region. - - Thus the front end has exclusive access to the free(A) region - in the figure above, and the back end driver has exclusive - access to the free(B) region. - -In general, drivers keep a private copy of their producer pointer and -then set the shared version when they are ready for the other end to -process a set of messages. Additionally, it is worth paying attention -to the use of memory barriers (rmb/wmb) in the code, to ensure that -rings that are shared across processors behave as expected. - -==== Structure of the Blkif Drivers ==== - -Now that the communications primitives have been discussed, I'll -quickly cover the general structure of the blkif driver. This is -intended to give a high-level idea of what is going on, in an effort -to make reading the code a more approachable task. - -There are three key software components that are involved in the blkif -drivers (not counting Xen itself). The frontend and backend driver, -and Xend, which coordinates their initial connection. Xend may also -be involved in control-channel signalling in some cases after startup, -for instance to manage reconnection if the backend is restarted. - -===== Frontend Driver Structure ===== - -The frontend domain uses a single event channel and a shared memory -ring to trade control messages with the backend. These are both setup -during domain startup, which will be discussed shortly. The shared -memory ring is called blkif_ring, and the private ring indexes are -resp_cons, and req_prod. The ring is protected by blkif_io_lock. -Additionally, the frontend keeps a list of outstanding requests in -rec_ring[]. These are uniquely identified by a guest-local id number, -which is associated with each request sent to the backend, and -returned with the matching responses. Information about the actual -disks are stored in major_info[], of which only the first nr_vbds -entries are valid. Finally, the global 'recovery' indicates that the -connection between the backend and frontend drivers has been broken -(possibly due to a backend driver crash) and that the frontend is in -recovery mode, in which case it will attempt to reconnect and reissue -outstanding requests. - -The frontend driver is single-threaded and after setup is entered only -through three points: (1) read/write requests from the XenLinux guest -that it is a part of, (2) interrupts from the backend driver on its -event channel (blkif_int()), and (3) control messages from Xend -(blkif_ctrlif_rx). - -===== Backend Driver Structure ===== - -The backend driver is slightly more complex as it must manage any -number of concurrent frontend connections. For each domain it -manages, the backend driver maintains a blkif structure, which -describes all the connection and disk information associated with that -particular domain. This structure is associated with the interrupt -registration, and allows the backend driver to have immediate context -when it takes a notification from some domain. - -All of the blkif structures are stored in a hash table (blkif_hash), -which is indexed by a hash of the domain id, and a "handle", really a -per-domain blkif identifier, in case it wants to have multiple connections. - -The per-connection blkif structure is of type blkif_t. It contains -all of the communication details (event channel, irq, shared memory -ring and indexes), and blk_ring_lock, which is the backend mutex on -the shared ring. The structure also contains vbd_rb, which is a -red-black tree, containing an entry for each device/partition that is -assigned to that domain. This structure is filled by xend passing -disk information to the backend at startup, and is protected by -vbd_lock. Finally, the blkif struct contains a status field, which -describes the state of the connection. - -The backend driver spawns a kernel thread at startup -(blkio_schedule()), which handles requests to and from the actual disk -device drivers. This scheduler thread maintains a list of blkif -structures that have pending requests, and services them round-robin -with a maximum per-round request limit. blkifs are added to the list -in the interrupt handler (blkif_be_int()) using -add_to_blkdev_list_tail(), and removed in the scheduler loop after -calling do_block_io_op(), which processes a batch of requests. The -scheduler thread is explicitly activated at several points in the code -using maybe_trigger_blkio_schedule(). - -Pending requests between the backend driver and the physical device -drivers use another ring, pending_ring. Requests are placed in this -ring in the scheduler thread and issued to the device. A completion -callback, end_block_io_op, indicates that requests have been serviced -and generates a response on the appropriate blkif ring. pending -reqs[] stores a list of outstanding requests with the physical drivers. - -So, control entries to the backend are (1) the blkio scheduler thread, -which sends requests to the real device drivers, (2) end_block_io_op, -which is called as serviced requests complete, (3) blkif_be_int() -handles notifications from the frontend drivers in other domains, and -(4) blkif_ctrlif_rx() handles control messages from xend. - -==== Driver Startup ==== - -Prior to starting a new guest using the frontend driver, the backend -will have been started in a privileged domain. The backend -initialisation code initialises all of its data structures, such as -the blkif hash table, and starts the scheduler thread as a kernel -thread. It then sends a driver status up message to let xend know it -is ready to take frontend connections. - -When a new domain that uses the blkif frontend driver is started, -there are a series of interactions between it, xend, and the specified -backend driver. These interactions are as follows: - -The domain configuration given to xend will specify the backend domain -and disks that the new guest is to use. Prior to actually running the -domain, xend and the backend driver interact to setup the initial -blkif record in the backend. - -(1) Xend sends a BLKIF_BE_CREATE message to backend. - - Backend does blkif_create(), having been passed FE domid and handle. - It creates and initialises a new blkif struct, and puts it in the - hash table. - It then returns a STATUS_OK response to xend. - -(2) Xend sends a BLKIF_BE_VBD_CREATE message to the backend. - - Backend adds a vbd entry in the red-black tree for the - specified (dom, handle) blkif entry. - Sends a STATUS_OK response. - -(3) Xend sends a BLKIF_BE_VBD_GROW message to the backend. - - Backend takes the physical device information passed in the - message and assigns them to the newly created vbd struct. - -(2) and (3) repeat as any additional devices are added to the domain. - -At this point, the backend has enough state to allow the frontend -domain to start. The domain is run, and eventually gets to the -frontend driver initialisation code. After setting up the frontend -data structures, this code continues the communications with xend and -the backend to negotiate a connection: - -(4) Frontend sends Xend a BLKIF_FE_DRIVER_STATUS_CHANGED message. - - This message tells xend that the driver is up. The init function - now spin-waits until driver setup is complete in order to prevent - Linux from attempting to boot before the disks are connected. - -(5) Xend sends the frontend an INTERFACE_STATUS_CHANGED message - - This message specifies that the interface is now disconnected - (instead of closed). - The domain updates it's state, and allocates the shared blk_ring - page. Next, - -(6) Frontend sends Xend a BLKIF_INTERFACE_CONNECT message - - This message specifies the domain and handle, and includes the - address of the newly created page. - -(7) Xend sends the backend a BLKIF_BE_CONNECT message - - The backend fills in the blkif connection information, maps the - shared page, and binds an irq to the event channel. - -(8) Xend sends the frontend an INTERFACE_STATUS_CHANGED message - - This message takes the frontend driver to a CONNECTED state, at - which point it binds an irq to the event channel and calls - xlvbd_init to initialise the individual block devices. - -The frontend Linux is stall spin waiting at this point, until all of -the disks have been probed. Messaging now is directly between the -front and backend domain using the new shared ring and event channel. - -(9) The frontend sends a BLKIF_OP_PROBE directly to the backend. - - This message includes a reference to an additional page, that the - backend can use for it's reply. The backend responds with an array - of the domains disks (as vdisk_t structs) on the provided page. - -The frontend now initialises each disk, calling xlvbd_init_device() -for each one. diff -r 617f5d6e9e69 -r 55ce23cb7b2a docs/misc/cpuid-config-for-guest.txt --- a/docs/misc/cpuid-config-for-guest.txt Thu Nov 17 14:54:12 2011 +0000 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,23 +0,0 @@ -CPUID emulation for guest -------------------------- - -When HVM guest tries to execute CPUID, or PV guest tries to execute XEN_CPUID, -the xen hypervior traps and emultes them. - -For HVM guest and PV DomU guest, xen's CPUID emulation can be adjusted using -the guest configation file if necessary (e.g., to supply better support for -guest live migration). The CPUID syntax in guest configration file is -described in detail in the examples like /etc/xen/xmexample.hvm, -/etc/xen/xmexample.hvm-stubdom. - -However, a user (or an administrator) must be aware that the CPUID in guest -configuration file can NOT be configured casually. The default CPUID -configuration should be safe, but illegal configuration can cause unexpected -behaviors of guest -- even can crash guest. - -For example, we should not expose the MONITOR CPUID feature flag (ECX bit 3; -CPUID executed EAX = 1) to HVM guest, otherwise, on guest's attempt of -executing MWAIT, the VMExit handler in Xen would inject #UD (Invalid Opcode -Exception) into the HVM guest, and guest kernel would panic. - -/* We can add more unsafe CPUID configuration here in future. */ diff -r 617f5d6e9e69 -r 55ce23cb7b2a docs/misc/hg-cheatsheet.txt --- a/docs/misc/hg-cheatsheet.txt Thu Nov 17 14:54:12 2011 +0000 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,438 +0,0 @@ - -Mercurial(hg) Cheatsheet for Xen -================================ - -Written by Andrew Warfield, extended by Michael Fetterman and Ian Pratt -June 29, 2005, extended by Grzegorz Milos 04 July 2005. - -Overview --------- -The Xen project has moved from BitKeeper to Mercurial for source -control. This note aims to provide a quick guide to getting up and -running with the new tools as quickly as possible, and is written from -the perspective of someone who has been using BK. - -For a more detailed exposition, see the mercurial tutorial: - http://www.serpentine.com/mercurial/index.cgi?Tutorial - -The Hg manpage is available at: - http://www.selenic.com/mercurial/hg.1.html - -There's also a very useful FAQ that explains the terminology: - http://www.selenic.com/mercurial/FAQ.html - -There's also a good README: - http://www.selenic.com/mercurial/README - -Necessary software ------------------- -Mercurial is available at: - http://www.selenic.com/mercurial/ - -You will also need a Python version >= 2.3 - -How Mercurial is different from BK ----------------------------------- -There are several pertinent differences between hg and bk. This -section aims to give an overview of the conceptual differences between -the two SCMs -- if you just want examples to get going, skip ahead to -"Getting Xen". The key differences are: - - - No explicit per-file locking. You do not need to explicitly - check a file out before editing it. - - No notion (currently) of file renames. - - A repository can have multiple active heads. - - Automatic merge support is currently inferior to BK's. - - No graphical tools. - - No per-file revision history, only per-changeset (we never really used this anyhow) - - Hg repositories tend to be rather bigger than Bk ones, but Hg does seem faster. - -Mercurial is based on the notion of changesets as complete, immutable, -versions of the repository. You make changes to a working version of -the repository that is based on a particular changeset. When you -commit, you will generate a new child changeset with whatever changes -you choose to apply. - -A major difference between Hg and BK is that you aren't forced to -resolve conflicts immediately: BK forced you to resolve conflicts -immediately on any merge, and it then immediately created a changeset -with those conflicts' resolutions. Frequently, you then had to add -yet another changeset to fixup the things for which the automatic -merge yielded bad results. Hg puts the results of the merge into your -work directory, and remembers what you merged with (so that it can -later record both of the merge parents, if you decide to make a -changeset), but it doesn't immediately create a changeset. - -A further feature of Hg is that it allows a repository to have -multiple heads. This means that you can have changesets with no common -descendent in one repository -- something BK won't allow. This is -actually pretty neat. For example, it would in principle enable you to -have both the 2.0-testing and unstable trees in a single -repository. We shyed away from doing this as we thought the risk of -committing to the wrong head was too great. - -One slightly confusing aspect of Hg is that many of the commands have -aliases, and hence when looking things up in the man page its not -always obvious what the underlying command is. For example 'co' is -actually an alias for the 'update' command, but 'co' seems to make -more sense, at least to RCS refugees like me. - - -Getting Xen ------------ - -The URL for the mainline Xen mercurial repository is: - - http://xenbits.xensource.com/xen-unstable.hg - (similarly for xen-2.0 and xen-2.0-testing) - -You can point a browser and this and use Hg's web interface to view -revision history, or use it as the nominated source when issuing -"hg init" or "hg pull" commands. - -However, to avoid taxing the Mercurial server with a complete pull of -the Xen repository, it is best to download a tarball of a seed -repository from: - - http://www.cl.cam.ac.uk/Research/SRG/netos/xen/downloads/xen-unstable.hg.tar.gz - - (or copy from /usr/groups/netos/html/xen/downloads/xen-unstable.hg.tar.gz) - -Untar the repository on your disk, cd into it, and then pull the most -recent changes: - - hg pull -u - -By default hg does not automatically checkout ('update') files from -the repository as used to happen with bk. The above is equivalent to -"hg pull; hg co" - -The repository parent is stored in a repository configuration file, -.hg/hgrc, from the repository root. If you look at this file, you -will see: - - | [paths] - | default = http://xenbits.xensource.com/xen-unstable.hg - -"default" specifies the appropriate parent repository for hg to pull -from. Hg allows you to pull additional repositories, for instance if -you want to work between unstable and testing concurrently. - -The command "hg pull" simply adds changesets to your repository, -without any merging of any kind. "hg pull -u" implies merging with -the current state of your working directory. If you weren't already -"updated" to your local repository's tip, you might be surprised to -find yourself merging the results of the pull with a non-tip node in -your local repository. - - -Revision History ----------------- - -You can view the repository revision history with: - - hg history - -In practice, you'll probably want to use pipe the output through -'head' or 'more' as it prints the entire history. - -Looking at the first few lines of output, you can see the changeset at -the head of the current branch, known as the 'tip' (the tip is -automatically given a special tag to make it easy to refer to): - - | changeset: 5599:6cbf9ec05cd9e05c0c46a85df7fc00262633cd3d - | tag: tip - | user: kaf24@xxxxxxxxxxxxxxxxxxxx - | date: Tue Jun 28 18:47:14 2005 - | summary: bitkeeper revision 1.1768 (42c18d2259NPELcGV7ohyZNh72ufSw) - -By default, Hg just shows the first line of the changset comments. You -can find further information with "hg -v history". - -The changeset identifier has two parts, a _local_ monotonically -increasing changeset id, 5599 above, and a _global_ hash, which -follows the colon on the changeset line. The hash uniquely identifies -the changeset and its lineage back to the root of the changeset tree --- it is useful for distributed management and so on. However, as it -is a bit unruly, the local id will allow you to work easily with the -local repo. Hg commands will take either identifier. Additionally, a -tags mechanism lets you give common names to specific changesets. - -You should always use the global hash when referring to versions of -the mainline Xen respoitory. With Bk you could often get away with -using the shortform version, but with Hg the local ids are pretty much -guaranteed to be different. - - -Creating a child repository from an existing repository -------------------------------------------------------- -If you wanted to create additional local child repositories, - - hg init [path or url] - -is effectively equivalent to bk clone. The major difference is that -it should be run from the root of your new repository. So: - - bk clone /foo/bar - -would be replaced with: - - mkdir bar - cd bar - hg init /foo/bar - -NB: newer version of Hg support a 'clone' command that works in the -same manner as bk. - -Editing files -------------- - -Normal edits may be made in place. File creation needs explicit -marking, though deletes should be picked up automatically - -creation: - - touch a.txt (or otherwise created a file) - hg add a.txt - -You can see what has changed using: - - hg status - - | C foo/foo.c - | R foo/bar.c - | ? a.txt - -This shows that in the current repo, foo.c has been changed, bar.c has -been deleted, and a.txt is new, but has not been added. '?' changes -to 'A' after "hg add a.txt". There is a .hgignore file which contains -regexps of files that should be ignored when scanning for new -files. We try to ensure that all the generated files in a build are -covered by the regexps. - -You can add all the new files in a repository with "hg addremove". If -you discover that you've added a file you didn't want, you can remove -it from the list of files to be included in the next commit using -"hg forget". - -Committing changes ------------------ - -After you've checked that hg knows about any new files you've created, -you probably want to see a diff of what you're about to commit. You -can do this with: - - hg diff - -Once you're happy with what you have, invoke: - - hg commit - -This will pop up an editor with a list of files to be committed to the -repository. It will look vaguely like this: - - | - | HG: manifest hash 6397b04bd5c2a992482d973b685a7e5e498788e7 - | HG: changed doc/thesis/new.tex - | HG: removed doc/2005-hotdep-protection/paper.tex - -Your comments can go anywhere in this file. The first line is the -most important, as it will show as the changeset description in -non-verbose-mode history listings. - -You can do commits without the editor and of partial sets of files -using command-line switches. See: - - hg help commit - -You can use the -A (--addremove) flag to commit e.g. "hg -A commit" to -ask mercurial to scan the tree looking for newly created files to add -in to the changeset. This avoids having to explicitly use "hg add", -but you probably want to be careful of adding any new generated files -too. - - -Generating a patch ------------------- -Generating a patch is easy, - - hg export [changeset] - -will generate a patch describing the diff between that changeset and -its parent. - -To generate a patch between two specified revisions use: - hg diff -r A -r B [files] -NB: BK syntax -rA..B isn't supported by Hg. - - -Pushing changesets to a parent repository ------------------------------------------ - - hg push - -Pushes changes up to a parent. You can't push if you pulled the -repository off the web interface. In fact, you can currently only push -to an ssh target -- filesystem directory targets don't work, but this -will be fixed soon. -For now it is possible to set up asymmetric pull/push paths. Pulls can -be done via web interface while pushes via ssh. Example of .hg/hgrc config -file: - | [paths] - | default = http://your.server/repository_name - | default-push = ssh://[username@]your.server//repository_location - - -Repository history ------------------- - -Here are a collection of common commands to get you started: - - hg history | less - -shows the history of changesets, starting from the most recent. You -want to pipe it to some sort of pager. For more complete details, - - hg -v history | less - -will include files modified and full (not just first-line) comments. - -Additionally, you can see just the tip (head of the current -branch) of the repository by typing: - - hg [-v] tip - - -Moving to a specific changeset ------------------------------- - -The co command lets you change the working version of the repository -to a different changeset. - - hg co [changeset] - -NB: 'co' is an alias for 'update' - -This command enables you to rewind the working repository to previous -changesets, for example to isolate the changeset in which a bug is -introduced. - -If you try and do a 'co' but have modified files in your repository Hg -won't let you unless you ask it explicitly to merge the checked out -version into the current tree using the "-m" option. The "-C" -(--clean) option will force overwrite any locally modified files. - -Any commits that are made to non-head changesets will obviously fork -the tree, creating a new head. You can see all the heads in a tree with -"hg heads". - -In general, "hg co" does the right thing, although it doesn't -currently seem to clean up unused directories that have been created -by other checked-out versions. This can confuse the Xen build -system. Hg will probably get fixed soon, but in the meantime you can -cleanup with "find -depth -type d -print | xargs -r rmdir". - -You can return to the tip by omitting an explicit changeset id. - -The manifest command lets you see the contents of the repository for -the current changeset. - - hg manifest - -This will print a bunch of records of the form: - - | 98856c45c35a504bc6da06a62b7787ddfdfd1c8b 644 COPYING - | f28971eedc5b54e7a9b26dd18d52992955354981 644 Config.mk - | a3575cc4db59e50bbac8a039a0c74f081a8dfc4f 644 Makefile - | 7fc869aae2945a9f4626fad96552db3103e61cb9 644 README - | ... - -This lists the hash of each file, its 1-bit 'executable' attribute -(either file permission mode 644 or 755), and the file name. So, to -determine the files that change across two changesets, you would dump -the respective manifests to files, and use diff. - - -Managing changeset tags ------------------------ -To create a tag to the current changeset: - - hg tag tagname - -This will _immediately_ generate a changeset with a change to the file -.hgtags in the repository root. The new tag in this file will look -something like: - - | 35159ed4b30538e7a52c60ad0a63f7e9af156e4c tagname - -and may be used to identify that changeset throughout the repo. -Storing tags in this file and generating changesets immediately -forces people to merge and keep tags up to date across the repository. - -Note that tags are resolved by searching .hgtags in each of the -repository heads, sequentially, and using the first match. "hg heads" -lists the current heads. - -The "hg tags" command, will lists all the currently valid tags. - - -Hg server and source browser ----------------------------- - - hg serve -p port - -Launches a web server on the specified port, serving a source browser -for the repository. This browser may be used to examine the -changeset history, view annotated source files, generate diffs. -Additionally "hg pull" may be run against it. - -Additional useful commands -(that probably only need one-line descriptions) ------------------------------------------------ - -(Slightly) more detail on all of these is available with - - hg help [command] - -Shows the differences between whatever changeset you most recently -checked out, and your current working directory: - - hg diff - -View an annotated version of a source file: - - hg annotate - -Get a historical version of a file: - - hg cat - - NB: Most commands accepting a version number want the changeset's - version number. "hg cat" is different in that it wants the - *file*'s version number. - -Unadd a file to the current commit: - - hg forget - -List all heads in the current repository: - - hg heads - -Undo exactly one (and ONLY one) changeset: - - hg undo - -Show the parents of a changeset: - - hg parents - - NB: Changesets have either one or two parent changesets. If your - working directory contains the uncommitted results of a merge, then - you have two parents. Otherwise, the single parent is the changeset - which you most recently checked out. - -Show the revision history for a single file - - hg [-v] log <filename> - diff -r 617f5d6e9e69 -r 55ce23cb7b2a docs/misc/network_setup.txt --- a/docs/misc/network_setup.txt Thu Nov 17 14:54:12 2011 +0000 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,196 +0,0 @@ -Native OS bridge configuration -============================== - -The traditional "network-bridge" script attempts to modify existing active -network interfaces to enable bridging. For non-trivial network configurations -though this can be error prone, and the temporary disruption to network -connectivity can upset some applications. This document outlines how to -configure bridging using an OS' native network configuration files. - -Disabling Xen's network scripts -------------------------------- - -The first step is to check XenD's network bridge is disabled by -editing /etc/xen/xend-config.sxp and changing the line - - (network-script network-bridge) - -To be - - (network-script /bin/true) - - -Fedora/RHEL Bridging -==================== - -This outlines how to setup bridging using standard network initscripts -present in Fedora or RHEL distros and their derivatives - - -Disabling NetworkManager ------------------------- - -As of time of writing (Fedora 14) NetworkManager does not support bridging, -so it is neccessary to disable NetworkManager, and revert to "classic" -network initscripts - - # chkconfig NetworkManager off - # chkconfig network on - # service NetworkManager stop - # service network start - -NB, as an alternative to turning off NetworkManager, you can also add a line -"NM_CONTROLLED=no" to the ifcfg-XXX scripts below - -Creating network initscripts ----------------------------- - -In the /etc/sysconfig/network-scripts directory it is necccessary to create -2 config files. The first (ifcfg-eth0) defines your physical network interface, -and says that it will be part of a bridge: - -# cat > ifcfg-eth0 <<EOF -DEVICE=eth0 -HWADDR=00:16:76:D6:C9:45 -ONBOOT=yes -BRIDGE=br0 -EOF - -Obviously change the HWADDR to match your actual NIC's address. You may also -wish to configure the device's MTU here using e.g. MTU=9000. - -The second config file (ifcfg-br0) defines the bridge device: - -# cat > ifcfg-br0 <<EOF -DEVICE=br0 -TYPE=Bridge -BOOTPROTO=dhcp -ONBOOT=yes -DELAY=0 -EOF - -WARNING: The line TYPE=Bridge is case-sensitive - it must have uppercase -'B' and lower case 'ridge' - -After changing this restart networking (or better still reboot) - - # service network restart - - -The final step is to configure iptables to allow all traffic to be -forwarded across the bridge - -# echo "-I FORWARD -m physdev --physdev-is-bridged -j ACCEPT" > /etc/sysconfig/iptables-forward-bridged -# lokkit --custom-rules=ipv4:filter:/etc/sysconfig/iptables-forward-bridged -# service libvirtd reload - -Alternatively, you can prevent bridged traffic getting pushed through -the host's iptables rules completely. In /etc/sysctl.conf add - - # cat >> /etc/sysctl.conf <<EOF - net.bridge.bridge-nf-call-ip6tables = 0 - net.bridge.bridge-nf-call-iptables = 0 - net.bridge.bridge-nf-call-arptables = 0 - EOF - # sysctl -p /etc/sysctl.conf - -You should now have a "shared physical device", to which guests can be -attached and have full LAN access - - # brctl show - bridge name bridge id STP enabled interfaces - br0 8000.000e0cb30550 no eth0 - - - -Debian/Ubuntu Bridging -======================= - -This outlines how to setup bridging using standard network interface config files -on Debian / Ubuntu distributions and their derivatives - -Disabling NetworkManager ------------------------- - -Stop network manager - - sudo /etc/dbus-1/event.d/26NetworkManagerDispatcher stop - sudo /etc/dbus-1/event.d/25NetworkManager stop - -Create two files with only the word 'exit' in them. These files are: - - /etc/default/NetworkManager - /etc/default/NetworkManagerDispatcher - - -Altering the interface config ------------------------------ - -First take down the interface you wish to bridge - - ifdown eth0 - -Edit /etc/network/interfaces and find the config for the physical -interface, which looks something like - - allow-hotplug eth0 - iface eth0 inet static - address 192.168.2.4 - netmask 255.255.255.0 - network 192.168.2.0 - broadcast 192.168.2.255 - gateway 192.168.2.2 - -Remove the 'allow-hotplug eth0' line, replacing it with 'auto br0', -and change the next line with iface name to 'br0', so it now starts -with - - auto br0 - iface br0 inet static - -And then define the interface as being a bridge and specify its ports - - bridge_ports eth0 - bridge_stp off - bridge_maxwait 5 - -The complete config should now look like - - auto br0 - iface br0 inet static - address 192.168.2.4 - netmask 255.255.255.0 - network 192.168.2.0 - broadcast 192.168.2.255 - gateway 192.168.2.2 - bridge_ports eth0 - bridge_stp off - bridge_maxwait 5 - -The interface can now be started with - - ifup br0 - -Finally add the '/etc/sysctl.conf' settings - -net.bridge.bridge-nf-call-ip6tables = 0 -net.bridge.bridge-nf-call-iptables = 0 -net.bridge.bridge-nf-call-arptables = 0 - -And then load the settings with - - sysctl -p /etc/sysctl.conf - - -You should now have a "shared physical device", to which guests -can be attached and have full LAN access - - # brctl show - bridge name bridge id STP enabled interfaces - br0 8000.000e0cb30550 no eth0 - - -Other operating systems / distributions -======================================= - -[...send patches to this file with instructions....] _______________________________________________ Xen-devel mailing list Xen-devel@xxxxxxxxxxxxxxxxxxx http://lists.xensource.com/xen-devel
|
 |
Lists.xenproject.org is hosted with RackSpace, monitoring our |