Xen project Mailing List

On Fri, Jun 24, 2011 at 4:54 AM, Ian Campbell <Ian.Campbell@xxxxxxxxxxxxx> wrote:

On Thu, 2011-06-23 at 16:16 +0100, Shriram Rajagopalan wrote:

> Actually, after the last iteration, the primary sends a
> XC_SAVE_ID_ENABLE_COMPRESSION and then for further rounds, the
> receiver stops expecting page data following the pfn array. Instead it
> waits for XC_SAVE_ID_COMPRESSED_DATA. Thanks for pointing it out. I ll
> document it.

[...]

> Answered above. Its a separate trigger. That said, if you meant "can I
> expect a +ve chunk 'after' a XC_SAVE_ID_COMPRESSED_DATA", yes that is
> possible. It happens when there are too many dirty pages to fit in the
> sender's compression buffer. Sender basically blocks, sends out the
> compressed chunk and moves on to the next batch of pages. This is a
> corner case.

I think I'm misunderstanding. When there are too many dirty pages you
send out a standard +ve chunk, including the page data? (presumably in
order to catch up). If so then how does this mesh with the statement
that once you've seen an XC_SAVE_ID_COMPRESSED_DATA you don't expect
page data in a +ve chunk anymore?

That was a good catch. I thought I ll keep the corner cases out of xg_save_restore.h
but from the way you pointed out, I think I ll document everything. Answers below.

Back in the original patch:

> + * compressed page data : variable length data of size indicated above.
> + * This chunk consists of compressed page data. The
> + * number of pages in one chunk varies with respect
> + * to amount of space available in the sender's
> + * output buffer.

What's the format of this compressed page data?

compressed_data = (BEGIN_PAGE,<deltas>*) | (FULL_PAGE,4096 bytes)
delta = <+ve offset in page (2byte), value (4byte)>
BEGIN_PAGE = is a dummy delta with offset = -100 and value = 0
FULL_PAGE = is a dummy delta with offset = -101 and value = 0

So is the sequence:
+16 (e.g.) +ve chunk
unsigned long[16] PFN array
NOT page-data (because we've seen XC_SAVE_ID_COMPRESSED_DATA)

XC_SAVE_ID_COMPRESSED_DATA TAG
N Length of compressed data batch#1

N bytes of DATA, batch #1 Decompresses to e.g. 7 pages
XC SAVE_ID_COMPRESSED_DATA TAG
M Length of compressed data batch#2
M bytes of DATA, batch #2 Decompresses to e.g. 9 pages

So now we have the originally specified 16 pages.

The sequence goes like this.
      +16       +ve chunk
unsigned long[16] PFN array
NO page-data (because we've seen XC_SAVE_ID_ENABLE_COMPRESSION)
       +100    +ve chunk
unsigned long[100] PFN array
       +50 (e.g.)       +ve chunk
unsigned long[50] PFN array

XC_SAVE_ID_COMPRESSED_DATA TAG
   N Length of compressed data (no batches. explained below)
   N bytes of DATA Decompresses to e.g. 166 pages
       XC_SAVE_ID_*            other xc save tags..
       tailbuf

Do we guarantee that
we will always see enough instances of XC_SAVE_ID_COMPRESSED_DATA to
total to the +ve chunk specified number of pages?

Yes. If the number of dirty pages < the sender's page buffer size in xc_remus.c
(not to be confused with pagebuf structure in xc_domain_restore),
then there will be only one instance of XC_SAVE_ID_COMPRESSED_DATA,
holding all the compressed pages.

The page buffer can hold upto 8196 pages. So, most of the time, there will be only one
XC_SAVE_ID_COMPRESSED_DATA, followed by the compressed chunk

Are they always
contiguous?

The order of compressed pages arriving, is same as the order in which they would have
been sent if compression logic was not present.

How does the sequence of events differ in the corner case of too many
dirty pages? Do we abort e.g. before the second
XC_SAVE_ID_COMPRESSED_DATA and go back to the +ve chunk stage?

In case there are too many dirty pages, there would be a XC_SAVE_ID_COMPRESSED_DATA in
the midst of an otherwise contiguous series of +ve chunks. For e.g. say if there are 9000 dirty pages,
all of which are valid.

   +1024 +ve chunk (batch #1)
unsigned long[1024]     PFN array

   +1024    +ve chunk (batch #2)
unsigned long[1024] PFN array

   +1024    +ve chunk (batch #3)
unsigned long[1024] PFN array

   +1024 +ve chunk (batch #4)
unsigned long[1024]     PFN array

   +1024    +ve chunk (batch #5)
unsigned long[1024] PFN array

   +1024    +ve chunk (batch #6)
unsigned long[1024] PFN array

   +1024    +ve chunk (batch #7)
unsigned long[1024] PFN array

   +1024    +ve chunk (batch #8)
unsigned long[1024] PFN array

       XC_SAVE_ID_COMPRESSED_DATA TAG
        N #length of compressed data
        N bytes of DATA (decompresses to 8196 pages)

     +4               +ve chunk
unsigned long[5] PFN array

       XC_SAVE_ID_COMPRESSED_DATA TAG
        M #length of compressed data
        M bytes of DATA (decompresses to 4 pages)

       XC_SAVE_ID_*            other xc save tags..
       tailbuf

shriram

Ian.

Re: [Xen-devel] [PATCH] tools/libxc: Document checkpoint compression in xg_save_restore.h