From minios-devel-bounces@lists.xenproject.org Fri Sep 10 06:40:57 2021 Return-path: Envelope-to: archives@lists.xenproject.org Delivery-date: Fri, 10 Sep 2021 06:40:57 +0000 Received: from list by lists.xenproject.org with outflank-mailman.184081.332627 (Exim 4.92) (envelope-from ) id 1mOaDX-0004RJ-Eh; Fri, 10 Sep 2021 06:40:55 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 184081.332627; Fri, 10 Sep 2021 06:40:55 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mOaDX-0004RA-B0; Fri, 10 Sep 2021 06:40:55 +0000 Received: by outflank-mailman (input) for mailman id 184081; Fri, 10 Sep 2021 06:40:54 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mOaDW-0004AU-CO for minios-devel@lists.xenproject.org; Fri, 10 Sep 2021 06:40:54 +0000 Received: from smtp-out2.suse.de (unknown [195.135.220.29]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 07450c1e-1202-11ec-b211-12813bfff9fa; Fri, 10 Sep 2021 06:40:47 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id EB2921FD52; Fri, 10 Sep 2021 06:40:45 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id BF20813D12; Fri, 10 Sep 2021 06:40:45 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id IUb8LO39OmGIIAAAMHmgww (envelope-from ); Fri, 10 Sep 2021 06:40:45 +0000 X-BeenThere: minios-devel@lists.xenproject.org List-Id: Mini-os development list List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: minios-devel-bounces@lists.xenproject.org Precedence: list Sender: "Minios-devel" X-Inumbo-ID: 07450c1e-1202-11ec-b211-12813bfff9fa DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1631256045; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=i6QZbTx5XiHB9TsuBBNdDhq37UbkhMYg1y+2T87ru54=; b=VGjhwRu2GxmPQY5YcDcKf4+pHq9zLYlyFZwFyAZFpkwKDJUQnfybEtJ5zCcXM0IDBbp650 ICipyAR1+kxWKu6eyuYnjbaKlZ3k/6kl8a2OiMhfhqWubfIHnXt0TunipdTqo9ZOiRcaVG v/LeKssGWFa/GNhDfPBejN5dbWGmLII= Subject: Re: [PATCH] mini-os: xenbus: support large messages To: minios-devel@lists.xenproject.org, xen-devel@lists.xenproject.org Cc: samuel.thibault@ens-lyon.org, wl@xen.org References: <20210818152610.6114-1-jgross@suse.com> From: Juergen Gross Message-ID: <722703e5-efa5-9868-5926-79a449e90285@suse.com> Date: Fri, 10 Sep 2021 08:40:45 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.12.0 MIME-Version: 1.0 In-Reply-To: <20210818152610.6114-1-jgross@suse.com> Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="X9mDkyBHbWuCbILhXetosgGLPk2Phjqj2" This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --X9mDkyBHbWuCbILhXetosgGLPk2Phjqj2 Content-Type: multipart/mixed; boundary="W7fjh8u4KjmjIgkNXOz4RSFwVIIAMGvrI"; protected-headers="v1" From: Juergen Gross To: minios-devel@lists.xenproject.org, xen-devel@lists.xenproject.org Cc: samuel.thibault@ens-lyon.org, wl@xen.org Message-ID: <722703e5-efa5-9868-5926-79a449e90285@suse.com> Subject: Re: [PATCH] mini-os: xenbus: support large messages References: <20210818152610.6114-1-jgross@suse.com> In-Reply-To: <20210818152610.6114-1-jgross@suse.com> --W7fjh8u4KjmjIgkNXOz4RSFwVIIAMGvrI Content-Type: multipart/mixed; boundary="------------B3256F1D978D2E4AA868E07A" Content-Language: en-US This is a multi-part message in MIME format. --------------B3256F1D978D2E4AA868E07A Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: quoted-printable On 18.08.21 17:26, Juergen Gross wrote: > Today the implementation of the xenbus protocol in Mini-OS will only > allow to transfer the complete message to or from the ring page buffer.= > This is limiting the maximum message size to lower values as the xenbus= > protocol normally would allow. >=20 > Change that by allowing to transfer the xenbus message in chunks as > soon as they are available. >=20 > Avoid crashing Mini-OS in case of illegal data read from the ring > buffer. >=20 > Signed-off-by: Juergen Gross Ping? Juergen > --- > xenbus/xenbus.c | 212 ++++++++++++++++++++++++++++-------------------= - > 1 file changed, 124 insertions(+), 88 deletions(-) >=20 > diff --git a/xenbus/xenbus.c b/xenbus/xenbus.c > index 23de61e..3fbb122 100644 > --- a/xenbus/xenbus.c > +++ b/xenbus/xenbus.c > @@ -29,6 +29,7 @@ > #include > #include > #include > +#include > =20 > #define min(x,y) ({ \ > typeof(x) tmpx =3D (x); \ > @@ -46,6 +47,7 @@ > static struct xenstore_domain_interface *xenstore_buf; > static DECLARE_WAIT_QUEUE_HEAD(xb_waitq); > DECLARE_WAIT_QUEUE_HEAD(xenbus_watch_queue); > +static __DECLARE_SEMAPHORE_GENERIC(xb_write_sem, 1); > =20 > xenbus_event_queue xenbus_events; > static struct watch { > @@ -231,75 +233,105 @@ char *xenbus_wait_for_state_change(const char* p= ath, XenbusState *state, xenbus_ > } > =20 > =20 > +static void xenbus_read_data(char *buf, unsigned int len) > +{ > + unsigned int off =3D 0; > + unsigned int prod; > + unsigned int size; > + int notify; > + > + while (off !=3D len) > + { > + if (xenstore_buf->rsp_prod =3D=3D xenstore_buf->rsp_cons) > + wait_event(xb_waitq, > + xenstore_buf->rsp_prod !=3D xenstore_buf->rsp_c= ons); > + > + prod =3D xenstore_buf->rsp_prod; > + DEBUG("Rsp_cons %d, rsp_prod %d.\n", xenstore_buf->rsp_cons, p= rod); > + size =3D min(len - off, prod - xenstore_buf->rsp_cons); > + memcpy_from_ring(xenstore_buf->rsp, buf + off, > + MASK_XENSTORE_IDX(xenstore_buf->rsp_cons), si= ze); > + off +=3D size; > + notify =3D (xenstore_buf->rsp_cons + XENSTORE_RING_SIZE =3D=3D= > + xenstore_buf->rsp_prod); > + rmb(); > + xenstore_buf->rsp_cons +=3D size; > + wmb(); > + if (notify) > + notify_remote_via_evtchn(xenbus_evtchn); > + } > +} > + > static void xenbus_thread_func(void *ign) > { > struct xsd_sockmsg msg; > - unsigned prod =3D xenstore_buf->rsp_prod; > + char *data; > =20 > for (;;) { > - wait_event(xb_waitq, prod !=3D xenstore_buf->rsp_prod); > - while (1) { > - prod =3D xenstore_buf->rsp_prod; > - DEBUG("Rsp_cons %d, rsp_prod %d.\n", xenstore_buf->rsp_con= s, > - xenstore_buf->rsp_prod); > - if (xenstore_buf->rsp_prod - xenstore_buf->rsp_cons < size= of(msg)) > - break; > - rmb(); > - memcpy_from_ring(xenstore_buf->rsp, &msg, > - MASK_XENSTORE_IDX(xenstore_buf->rsp_cons)= , > - sizeof(msg)); > - DEBUG("Msg len %d, %d avail, id %d.\n", msg.len + sizeof(m= sg), > - xenstore_buf->rsp_prod - xenstore_buf->rsp_cons, msg= =2Ereq_id); > - > - if (xenstore_buf->rsp_prod - xenstore_buf->rsp_cons < > - sizeof(msg) + msg.len) > - break; > - > - DEBUG("Message is good.\n"); > - > - if (msg.type =3D=3D XS_WATCH_EVENT) { > - struct xenbus_event *event =3D malloc(sizeof(*event) += msg.len); > - xenbus_event_queue *events =3D NULL; > - char *data =3D (char*)event + sizeof(*event); > - struct watch *watch; > - > - memcpy_from_ring(xenstore_buf->rsp, data, > - MASK_XENSTORE_IDX(xenstore_buf->rsp_cons + sizeof(= msg)), > - msg.len); > - > - event->path =3D data; > - event->token =3D event->path + strlen(event->path) + 1= ; > - > - mb(); > - xenstore_buf->rsp_cons +=3D msg.len + sizeof(msg); > - > - for (watch =3D watches; watch; watch =3D watch->next) > - if (!strcmp(watch->token, event->token)) { > - events =3D watch->events; > - break; > - } > - > - if (events) { > - event->next =3D *events; > - *events =3D event; > - wake_up(&xenbus_watch_queue); > - } else { > - printk("unexpected watch token %s\n", event->token= ); > - free(event); > + xenbus_read_data((char *)&msg, sizeof(msg)); > + DEBUG("Msg len %d, %d avail, id %d.\n", msg.len + sizeof(msg),= > + xenstore_buf->rsp_prod - xenstore_buf->rsp_cons, msg.req= _id); > + > + if (msg.len > XENSTORE_PAYLOAD_MAX) { > + printk("Xenstore violates protocol, message longer than al= lowed.\n"); > + return; > + } > + > + if (msg.type =3D=3D XS_WATCH_EVENT) { > + struct xenbus_event *event =3D malloc(sizeof(*event) + msg= =2Elen); > + xenbus_event_queue *events =3D NULL; > + struct watch *watch; > + char *c; > + int zeroes =3D 0; > + > + data =3D (char*)event + sizeof(*event); > + xenbus_read_data(data, msg.len); > + > + for (c =3D data; c < data + msg.len; c++) > + if (!*c) > + zeroes++; > + if (zeroes !=3D 2) { > + printk("Xenstore: illegal watch event data\n"); > + free(event); > + continue; > + } > + > + event->path =3D data; > + event->token =3D event->path + strlen(event->path) + 1; > + > + for (watch =3D watches; watch; watch =3D watch->next) > + if (!strcmp(watch->token, event->token)) { > + events =3D watch->events; > + break; > } > + > + if (events) { > + event->next =3D *events; > + *events =3D event; > + wake_up(&xenbus_watch_queue); > } else { > - req_info[msg.req_id].reply =3D malloc(sizeof(msg) + ms= g.len); > - memcpy_from_ring(xenstore_buf->rsp, req_info[msg.req_i= d].reply, > - MASK_XENSTORE_IDX(xenstore_buf->rsp_c= ons), > - msg.len + sizeof(msg)); > - mb(); > - xenstore_buf->rsp_cons +=3D msg.len + sizeof(msg); > - wake_up(&req_info[msg.req_id].waitq); > + printk("Xenstore: unexpected watch token %s\n", event-= >token); > + free(event); > } > =20 > - wmb(); > - notify_remote_via_evtchn(xenbus_evtchn); > + continue; > } > + > + data =3D malloc(sizeof(msg) + msg.len); > + memcpy(data, &msg, sizeof(msg)); > + xenbus_read_data(data + sizeof(msg), msg.len); > + > + if (msg.req_id >=3D NR_REQS || !req_info[msg.req_id].in_use) {= > + printk("Xenstore: illegal request id %d\n", msg.req_id); > + free(data); > + continue; > + } > + > + DEBUG("Message is good.\n"); > + > + req_info[msg.req_id].reply =3D data; > + > + wake_up(&req_info[msg.req_id].waitq); > } > } > =20 > @@ -451,36 +483,40 @@ static void xb_write(int type, int req_id, xenbus= _transaction_t trans_id, > =20 > cur_req =3D &header_req; > =20 > - BUG_ON(len > XENSTORE_RING_SIZE); > - /* Wait for the ring to drain to the point where we can send the > - message. */ > - prod =3D xenstore_buf->req_prod; > - if (prod + len - xenstore_buf->req_cons > XENSTORE_RING_SIZE) > - { > - /* Wait for there to be space on the ring */ > - DEBUG("prod %d, len %d, cons %d, size %d; waiting.\n", > - prod, len, xenstore_buf->req_cons, XENSTORE_RING_SIZE)= ; > - wait_event(xb_waitq, > - xenstore_buf->req_prod + len - xenstore_buf->req_cons = <=3D > - XENSTORE_RING_SIZE); > - DEBUG("Back from wait.\n"); > - prod =3D xenstore_buf->req_prod; > - } > + BUG_ON(len > XENSTORE_PAYLOAD_MAX); > =20 > - /* We're now guaranteed to be able to send the message without > - overflowing the ring. Do so. */ > + /* Make sure we are the only thread trying to write. */ > + down(&xb_write_sem); > + > + /* Send the message in chunks using free ring space when available= =2E */ > total_off =3D 0; > req_off =3D 0; > - while (total_off < len) > + while (total_off < len) > { > + prod =3D xenstore_buf->req_prod; > + if (prod - xenstore_buf->req_cons >=3D XENSTORE_RING_SIZE) > + { > + /* Send evtchn to notify remote */ > + notify_remote_via_evtchn(xenbus_evtchn); > + > + /* Wait for there to be space on the ring */ > + DEBUG("prod %d, len %d, cons %d, size %d; waiting.\n", pro= d, > + len - total_off, xenstore_buf->req_cons, XENSTORE_RI= NG_SIZE); > + wait_event(xb_waitq, > + prod - xenstore_buf->req_cons < XENSTORE_RING_S= IZE); > + DEBUG("Back from wait.\n"); > + } > + > this_chunk =3D min(cur_req->len - req_off, > - XENSTORE_RING_SIZE - MASK_XENSTORE_IDX(prod)); > + XENSTORE_RING_SIZE - MASK_XENSTORE_IDX(prod))= ; > + this_chunk =3D min(this_chunk, > + xenstore_buf->req_cons + XENSTORE_RING_SIZE -= prod); > memcpy((char *)xenstore_buf->req + MASK_XENSTORE_IDX(prod), > - (char *)cur_req->data + req_off, this_chunk); > + (char *)cur_req->data + req_off, this_chunk); > prod +=3D this_chunk; > req_off +=3D this_chunk; > total_off +=3D this_chunk; > - if (req_off =3D=3D cur_req->len) > + if (req_off =3D=3D cur_req->len) > { > req_off =3D 0; > if (cur_req =3D=3D &header_req) > @@ -488,20 +524,20 @@ static void xb_write(int type, int req_id, xenbus= _transaction_t trans_id, > else > cur_req++; > } > + > + /* Remote must see entire message before updating indexes */ > + wmb(); > + xenstore_buf->req_prod =3D prod; > } > =20 > + /* Send evtchn to notify remote */ > + notify_remote_via_evtchn(xenbus_evtchn); > + > DEBUG("Complete main loop of xb_write.\n"); > BUG_ON(req_off !=3D 0); > BUG_ON(total_off !=3D len); > - BUG_ON(prod > xenstore_buf->req_cons + XENSTORE_RING_SIZE); > - > - /* Remote must see entire message before updating indexes */ > - wmb(); > - > - xenstore_buf->req_prod +=3D len; > =20 > - /* Send evtchn to notify remote */ > - notify_remote_via_evtchn(xenbus_evtchn); > + up(&xb_write_sem); > } > =20 > /* Send a mesasge to xenbus, in the same fashion as xb_write, and >=20 --------------B3256F1D978D2E4AA868E07A Content-Type: application/pgp-keys; name="OpenPGP_0xB0DE9DD628BF132F.asc" Content-Transfer-Encoding: quoted-printable Content-Description: OpenPGP public key Content-Disposition: attachment; filename="OpenPGP_0xB0DE9DD628BF132F.asc" -----BEGIN PGP PUBLIC KEY BLOCK----- xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy= cWx w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O= f8Z d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y= 9bf IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq= G7/ 377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR= 3Jv c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA= QIe AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT= hpw dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0= MbD 1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv= oPH Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v= 5QL +qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ= 2Vu IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC= QoL BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr= Wf0 teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC= /nu AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh= ITT d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm= XBK 7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz= 80h SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC= AcD AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn= FOX gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1= jnD kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0= N51 N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l= otu fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1= tqS EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA= hsD BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3= g3O ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5= dM7 wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5= D+j LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm= V2x AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N= Eaw QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW= nHI s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR= wgn BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV= bVF LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk= pEd IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA= QAB wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw= Tbe 8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1= vJz Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8= VGi wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW= svi uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX= zXs ZDn8R38=3D =3D2wuH -----END PGP PUBLIC KEY BLOCK----- --------------B3256F1D978D2E4AA868E07A-- --W7fjh8u4KjmjIgkNXOz4RSFwVIIAMGvrI-- --X9mDkyBHbWuCbILhXetosgGLPk2Phjqj2 Content-Type: application/pgp-signature; name="OpenPGP_signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="OpenPGP_signature" -----BEGIN PGP SIGNATURE----- wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmE6/e0FAwAAAAAACgkQsN6d1ii/Ey8d DAf/QEDma3LEitNzsAM/u77x/bLeUxlNBaSDL76FNzjcOZvBMHtQ3ekqoYdjfjfojPJIQ3UVbhH6 rU5yGZZ3sF7oUzeFmK9YRBRAt59XMp+mxdwpMzR1kqObkPeFUwKQSiCtcJsQZTJqlBMy/84Yy+Z5 lZbd1mENpTplK6PGUz/xsf+XBse6QlKxBBRcQBkzq30N92cQI27obgTfx3SkoSZmDBRw5ji2Dsww hvAiG3YAQenbbJMjxLfVXyL7R2F1AAwPSoHgysaH54XRlnr+U7lIy68CUxcVKj/nPiW/JQeaNqLf mlRMNLuVceM/bgxN3A1VrZJ6yGKqbqi4dFHA/+NIVQ== =alK0 -----END PGP SIGNATURE----- --X9mDkyBHbWuCbILhXetosgGLPk2Phjqj2-- From minios-devel-bounces@lists.xenproject.org Mon Sep 13 19:05:24 2021 Return-path: Envelope-to: archives@lists.xenproject.org Delivery-date: Mon, 13 Sep 2021 19:05:24 +0000 Received: from list by lists.xenproject.org with outflank-mailman.162918 (Exim 4.92) (envelope-from ) id 1mPrGc-0004yA-NC; Mon, 13 Sep 2021 19:05:22 +0000 Received: by outflank-mailman (output) from mailman id 162918 (unknown outflank id, message too old?); Mon, 13 Sep 2021 19:05:22 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mPrGc-0004y3-Jn; Mon, 13 Sep 2021 19:05:22 +0000 Received: by outflank-mailman (input) for mailman id 162918; Mon, 02 Aug 2021 09:08:01 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mATvV-0004lS-Dz for minios-devel@lists.xenproject.org; Mon, 02 Aug 2021 09:08:01 +0000 X-Inumbo-ID: ef79e0f1-5193-4371-95a8-eacc33753c56 Received: from mr.tuwien.ac.at (unknown [128.130.2.110]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id ef79e0f1-5193-4371-95a8-eacc33753c56; Mon, 02 Aug 2021 09:07:57 +0000 (UTC) X-Virus-Scanned: amavisd-new at vc6.kom.tuwien.ac.at Received: from mail.infosys.tuwien.ac.at (mail.infosys.tuwien.ac.at [128.131.172.240]) by mr.tuwien.ac.at (8.14.4/8.14.4/Debian-2ubuntu2.1) with ESMTP id 17297uHv020915 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Mon, 2 Aug 2021 11:07:56 +0200 Received: from [128.131.172.36] (ogi-dock.infosys.tuwien.ac.at [128.131.172.36]) by mail.infosys.tuwien.ac.at (Postfix) with ESMTPSA id 10A80202C3A72 for ; Mon, 2 Aug 2021 11:07:55 +0200 (CEST) To: minios-devel@lists.xenproject.org From: Philipp Raith Subject: Mattermost add Message-ID: Date: Mon, 2 Aug 2021 11:07:55 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.11.0 MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="------------E8C5B6C46EFA5BA1452D7AF7" Content-Language: en-US X-Mailman-Approved-At: Mon, 13 Sep 2021 19:05:21 +0000 X-BeenThere: minios-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Mini-os development list List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: minios-devel-bounces@lists.xenproject.org Sender: "Minios-devel" This is a multi-part message in MIME format. --------------E8C5B6C46EFA5BA1452D7AF7 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Hi, my name is Philipp Raith and I'm currently working as a PhD Student at TU Wien. My research is focused on (Serverless) Edge Computing and I would like to explore Unikernel. I have seen an issue on Github [1] where the user has questions in combination with Jetson Nano (a device much common in Edge Computing) and wanted to ask if I could also get access to your Mattermost instance to possible view the discussion or actively discuss Unikraft. Thanks and best regards, Philipp Raith [1] https://github.com/unikraft/kraft/issues/25 -- Dipl.-Ing. Philipp Raith, BSc University Assistant TU Wien Institute of Information Systems Distributed Systems Group Argentinierstraße 8/194-02 A-1040 Vienna T: +43-1-58801-184831 E:p.raith@dsg.tuwien.ac.at W:https://dsg.tuwien.ac.at/staff/praith/ --------------E8C5B6C46EFA5BA1452D7AF7 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: 8bit

Hi,

my name is Philipp Raith and I'm currently working as a PhD Student at TU Wien.

My research is focused on (Serverless) Edge Computing and I would like to explore Unikernel.

I have seen an issue on Github [1] where the user has questions in combination with Jetson Nano (a device much common in Edge Computing)
and wanted to ask if I could also get access to your Mattermost instance to possible view the discussion or actively discuss Unikraft.

Thanks and best regards,
Philipp Raith

[1] https://github.com/unikraft/kraft/issues/25

-- 
Dipl.-Ing. Philipp Raith, BSc
University Assistant

TU Wien
Institute of Information Systems
Distributed Systems Group
Argentinierstraße 8/194-02
A-1040 Vienna

T: +43-1-58801-184831
E: p.raith@dsg.tuwien.ac.at
W: https://dsg.tuwien.ac.at/staff/praith/
--------------E8C5B6C46EFA5BA1452D7AF7-- From minios-devel-bounces@lists.xenproject.org Mon Sep 13 20:15:58 2021 Return-path: Envelope-to: archives@lists.xenproject.org Delivery-date: Mon, 13 Sep 2021 20:15:58 +0000 Received: from list by lists.xenproject.org with outflank-mailman.185903.334651 (Exim 4.92) (envelope-from ) id 1mPsMu-00046O-5G; Mon, 13 Sep 2021 20:15:56 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 185903.334651; Mon, 13 Sep 2021 20:15:56 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mPsMu-00046H-2W; Mon, 13 Sep 2021 20:15:56 +0000 Received: by outflank-mailman (input) for mailman id 185903; Mon, 13 Sep 2021 20:15:55 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mPsMt-00046B-4i for minios-devel@lists.xenproject.org; Mon, 13 Sep 2021 20:15:55 +0000 Received: from mail-wm1-x32b.google.com (unknown [2a00:1450:4864:20::32b]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 56e96a3d-db56-41cc-bb40-1115e8e8cc89; Mon, 13 Sep 2021 20:15:53 +0000 (UTC) Received: by mail-wm1-x32b.google.com with SMTP id d207-20020a1c1dd8000000b00307e2d1ec1aso826809wmd.5 for ; Mon, 13 Sep 2021 13:15:53 -0700 (PDT) Received: from localhost ([141.85.225.204]) by smtp.gmail.com with ESMTPSA id g5sm8353612wrq.80.2021.09.13.13.15.51 (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Mon, 13 Sep 2021 13:15:52 -0700 (PDT) X-BeenThere: minios-devel@lists.xenproject.org List-Id: Mini-os development list List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: minios-devel-bounces@lists.xenproject.org Precedence: list X-Inumbo-ID: 56e96a3d-db56-41cc-bb40-1115e8e8cc89 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=sender:from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version; bh=qw4wR4Z4YPGFAtirijQ+qZ+1V54hzoLM0+woVFFtmSw=; b=m1bRlXxUxaXxgLkC/aeNqMF0F+DvtWWtu2gn8q7BjJ+Kmh8TTokEXCprmbzkMZEe4K aAebt7iENTAFVW+uA07nh5gtOGV0zUzRrn9z9NvJosSGJt3R4yjpFKM4/g5dAXansAf9 NfRVzgUAKsGAjVQi1sG50belCzrQJv42eoxjR2oqkS6z5kJZSSlfCQr+z6oyocaB63cu lW9N1kFpEblK9X2Sdeaq4Iv5VafimWohRGVO1wN6TcIekjWI6i2kwBAKKkgO7DuwyOUz 2OmvCil6Iol+1VIWoxwICymyVx98suUG7tuT5uCCFFNcLm1L8V7q7GC4lcMSLpqlhfdn szFw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:sender:from:to:cc:subject:references:date :in-reply-to:message-id:user-agent:mime-version; bh=qw4wR4Z4YPGFAtirijQ+qZ+1V54hzoLM0+woVFFtmSw=; b=L6VY4MxRnOXDWN9YiTdqB8Wm0WgiNeevKr8HPFfQA+hmwbEuRUnFBjmv7VZlxKBRLY unrFsEOt1K65qINDlBTEQY9jHE1oj0Uu4AjoWM4fWba1GJeYi89qJhDMnKOYkeNcHLxh KugTZw/kGLMBThX3WOs0iTK+rwoSJNjA4b4X/I3zMdG2LzXqgLYTB7hFzI9kRBZxF7v4 mNtjIqGRYxYNRFTCyr8Ow0eRDLyKZLcsh6esggX9tf8k5sYywZPgsCyJ7+eVSY5Vnx64 T+f3vYnJ6KFVIHmpqnkJgdYHAryTEz8uCAPx93nL22GjP+bEuKf+fXN722KL7On9N0hr gI1A== X-Gm-Message-State: AOAM5312eHpqqym6ypG71o8apUz9ZY517e7KP/AP4j+LqJM1TorMemsu SKpDIzX4clK584z2h96ypEp5z6WUyMc= X-Google-Smtp-Source: ABdhPJw94XvHvJtEqmOIJXyPR32LBpdNPcnZjdpijaVqj5qICt/VFjWneAODXFLwkFQsi0ig4H57hQ== X-Received: by 2002:a1c:3182:: with SMTP id x124mr12743866wmx.35.1631564152401; Mon, 13 Sep 2021 13:15:52 -0700 (PDT) Sender: Razvan Deaconescu From: Razvan Deaconescu To: Philipp Raith Cc: minios-devel@lists.xenproject.org Subject: Re: Mattermost add References: Date: Mon, 13 Sep 2021 23:15:51 +0300 In-Reply-To: (Philipp Raith's message of "Mon, 2 Aug 2021 11:07:55 +0200") Message-ID: <86sfy8xmdk.fsf@drone.cs.pub.ro> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain Philipp Raith writes: > Hi, > > my name is Philipp Raith and I'm currently working as a PhD Student at > TU Wien. > > My research is focused on (Serverless) Edge Computing and I would like > to explore Unikernel. > > I have seen an issue on Github [1] where the user has questions in > combination with Jetson Nano (a device much common in Edge Computing) > and wanted to ask if I could also get access to your Mattermost > instance to possible view the discussion or actively discuss Unikraft. Hi, Philipp. The best way is to join the Discord server[2] and interact with the community there. See you soon on Discord[2]. > [1] https://secure-web.cisco.com/1TZXew7THy7Ta2DhQl5bxJWrlXK5S5LA5TkJ-JkuFNzuMeztvChvYjUF1T7cGnyL5GwI-aU6TYJoAbWnJc0lg1EZp10SMO5Fg9CyIdau-H-B8Rmj40pu_w7dzs2XLNOucCeFtV75nTrfRyuOSt2mjfwkR5XYnhfg987EGhmbtq6x0d0zfywypi1pXZj1ZWkVBgkeUrEgXHVuflWsRbPIZkSo9KIvXzHCagEfA1lIvTftRWU68bHIsFs0XlejhtjL12vAuNpt7BImF62EyE8gBJyreEuYtwku0EZmOTD2Z5XIFS7a_SgluKa59EUUJ3d2MlAsr96qNBdESvet0e4dWVQ/https%3A%2F%2Fgithub.com%2Funikraft%2Fkraft%2Fissues%2F25 [2] https://bit.ly/UnikraftDiscord Razvan From minios-devel-bounces@lists.xenproject.org Tue Sep 14 23:17:55 2021 Return-path: Envelope-to: archives@lists.xenproject.org Delivery-date: Tue, 14 Sep 2021 23:17:55 +0000 Received: from list by lists.xenproject.org with outflank-mailman.187067.335807 (Exim 4.92) (envelope-from ) id 1mQHgY-0006tz-45; Tue, 14 Sep 2021 23:17:54 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 187067.335807; Tue, 14 Sep 2021 23:17:54 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mQHgY-0006ts-19; Tue, 14 Sep 2021 23:17:54 +0000 Received: by outflank-mailman (input) for mailman id 187067; Tue, 14 Sep 2021 23:17:53 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mQHgX-0006cD-4E for minios-devel@lists.xenproject.org; Tue, 14 Sep 2021 23:17:53 +0000 Received: from hera.aquilenet.fr (unknown [2a0c:e300::1]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id c39529d0-a703-408e-911a-c82fd71502a0; Tue, 14 Sep 2021 23:17:45 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by hera.aquilenet.fr (Postfix) with ESMTP id ED04127B; Wed, 15 Sep 2021 01:17:43 +0200 (CEST) Received: from hera.aquilenet.fr ([127.0.0.1]) by localhost (hera.aquilenet.fr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 3XfStTJK8Dom; Wed, 15 Sep 2021 01:17:43 +0200 (CEST) Received: from begin (unknown [IPv6:2a01:cb19:956:1b00:de41:a9ff:fe47:ec49]) by hera.aquilenet.fr (Postfix) with ESMTPSA id 537014A; Wed, 15 Sep 2021 01:17:43 +0200 (CEST) Received: from samy by begin with local (Exim 4.95-RC2) (envelope-from ) id 1mQHgM-00DvMY-DX; Wed, 15 Sep 2021 01:17:42 +0200 X-BeenThere: minios-devel@lists.xenproject.org List-Id: Mini-os development list List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: minios-devel-bounces@lists.xenproject.org Precedence: list Sender: "Minios-devel" X-Inumbo-ID: c39529d0-a703-408e-911a-c82fd71502a0 X-Virus-Scanned: Debian amavisd-new at aquilenet.fr Date: Wed, 15 Sep 2021 01:17:42 +0200 From: Samuel Thibault To: Juergen Gross Cc: minios-devel@lists.xenproject.org, xen-devel@lists.xenproject.org, wl@xen.org Subject: Re: [PATCH] mini-os: xenbus: support large messages Message-ID: <20210914231742.pxpac2lhp2kk5ook@begin> Mail-Followup-To: Samuel Thibault , Juergen Gross , minios-devel@lists.xenproject.org, xen-devel@lists.xenproject.org, wl@xen.org References: <20210818152610.6114-1-jgross@suse.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20210818152610.6114-1-jgross@suse.com> Organization: I am not organized User-Agent: NeoMutt/20170609 (1.8.3) X-Spamd-Bar: -- Authentication-Results: hera.aquilenet.fr X-Rspamd-Server: hera X-Rspamd-Queue-Id: ED04127B X-Spamd-Result: default: False [-2.50 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; HAS_ORG_HEADER(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; RCVD_NO_TLS_LAST(0.10)[]; FROM_EQ_ENVFROM(0.00)[]; MID_RHS_NOT_FQDN(0.50)[]; BAYES_HAM(-3.00)[100.00%] Hello, Thanks for having worked on this! Juergen Gross, le mer. 18 août 2021 17:26:10 +0200, a ecrit: > +static void xenbus_read_data(char *buf, unsigned int len) > +{ > + unsigned int off = 0; > + unsigned int prod; > + unsigned int size; > + int notify; > + > + while (off != len) > + { > + if (xenstore_buf->rsp_prod == xenstore_buf->rsp_cons) > + wait_event(xb_waitq, > + xenstore_buf->rsp_prod != xenstore_buf->rsp_cons); The if is redundant since wait_event already tests it. > + prod = xenstore_buf->rsp_prod; > + DEBUG("Rsp_cons %d, rsp_prod %d.\n", xenstore_buf->rsp_cons, prod); > + size = min(len - off, prod - xenstore_buf->rsp_cons); > + memcpy_from_ring(xenstore_buf->rsp, buf + off, > + MASK_XENSTORE_IDX(xenstore_buf->rsp_cons), size); > + off += size; > + notify = (xenstore_buf->rsp_cons + XENSTORE_RING_SIZE == > + xenstore_buf->rsp_prod); This looks odd to me? We want to notify as soon as the ring is empty, which can happen at any place in the ring right? Linux' code uses (intf->rsp_prod - cons >= XENSTORE_RING_SIZE), *after* the rsp_cons increase. > + rmb(); rmb() must be placed before memcpy_from_ring, to make sure that the data we read from the buffer is up-to-date according to the read we made from rsp_prod. The rest seems ok to me. Samuel From minios-devel-bounces@lists.xenproject.org Wed Sep 15 10:48:50 2021 Return-path: Envelope-to: archives@lists.xenproject.org Delivery-date: Wed, 15 Sep 2021 10:48:50 +0000 Received: from list by lists.xenproject.org with outflank-mailman.187475.336346 (Exim 4.92) (envelope-from ) id 1mQSTA-0007ZG-N1; Wed, 15 Sep 2021 10:48:48 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 187475.336346; Wed, 15 Sep 2021 10:48:48 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mQSTA-0007ZC-KG; Wed, 15 Sep 2021 10:48:48 +0000 Received: by outflank-mailman (input) for mailman id 187475; Wed, 15 Sep 2021 10:48:47 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mQST9-0007Z2-Su for minios-devel@lists.xenproject.org; Wed, 15 Sep 2021 10:48:47 +0000 Received: from smtp-out1.suse.de (unknown [195.135.220.28]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 804632c8-1612-11ec-b4f9-12813bfff9fa; Wed, 15 Sep 2021 10:48:46 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 9A0EF2220E; Wed, 15 Sep 2021 10:48:45 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 7661113C2F; Wed, 15 Sep 2021 10:48:45 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id 7bKnG43PQWEmBAAAMHmgww (envelope-from ); Wed, 15 Sep 2021 10:48:45 +0000 X-BeenThere: minios-devel@lists.xenproject.org List-Id: Mini-os development list List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: minios-devel-bounces@lists.xenproject.org Precedence: list Sender: "Minios-devel" X-Inumbo-ID: 804632c8-1612-11ec-b4f9-12813bfff9fa DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1631702925; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc: mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=iTKSaRtAvb4MPwNlHHAs/1gnbRIWxnDB5cbkIAO1GYk=; b=t/rojrg2wrxx4pRgBWYGWnJ2Ze410a7OmVESnyHsEw5REFzBJObacrRkBjT+2UP7rqPRbY zBEkWKtm4HWZeH7du5fRizxnFtJWmqoDX5BO5eqNKcmF74RzIIM+UB0jtY9BK9kP9rOM7b kOu98XwmIq2F8Baq8wo98+MgEI4Jfa8= To: Samuel Thibault , minios-devel@lists.xenproject.org, xen-devel@lists.xenproject.org, wl@xen.org References: <20210818152610.6114-1-jgross@suse.com> <20210914231742.pxpac2lhp2kk5ook@begin> From: Juergen Gross Subject: Re: [PATCH] mini-os: xenbus: support large messages Message-ID: <2304b885-434c-8a19-78e0-14637a4adee3@suse.com> Date: Wed, 15 Sep 2021 12:48:44 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.12.0 MIME-Version: 1.0 In-Reply-To: <20210914231742.pxpac2lhp2kk5ook@begin> Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="a6FiRo5rvrV8rM1QpL0b6BtntmpTnTYqB" This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --a6FiRo5rvrV8rM1QpL0b6BtntmpTnTYqB Content-Type: multipart/mixed; boundary="dCjWhK2Ru03rTrBBaj7QlVmsqxnePxHnB"; protected-headers="v1" From: Juergen Gross To: Samuel Thibault , minios-devel@lists.xenproject.org, xen-devel@lists.xenproject.org, wl@xen.org Message-ID: <2304b885-434c-8a19-78e0-14637a4adee3@suse.com> Subject: Re: [PATCH] mini-os: xenbus: support large messages References: <20210818152610.6114-1-jgross@suse.com> <20210914231742.pxpac2lhp2kk5ook@begin> In-Reply-To: <20210914231742.pxpac2lhp2kk5ook@begin> --dCjWhK2Ru03rTrBBaj7QlVmsqxnePxHnB Content-Type: multipart/mixed; boundary="------------4AF975DB5AFDAF92EAD2B51A" Content-Language: en-US This is a multi-part message in MIME format. --------------4AF975DB5AFDAF92EAD2B51A Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: quoted-printable On 15.09.21 01:17, Samuel Thibault wrote: > Hello, >=20 > Thanks for having worked on this! >=20 > Juergen Gross, le mer. 18 ao=C3=BBt 2021 17:26:10 +0200, a ecrit: >> +static void xenbus_read_data(char *buf, unsigned int len) >> +{ >> + unsigned int off =3D 0; >> + unsigned int prod; >> + unsigned int size; >> + int notify; >> + >> + while (off !=3D len) >> + { >> + if (xenstore_buf->rsp_prod =3D=3D xenstore_buf->rsp_cons) >> + wait_event(xb_waitq, >> + xenstore_buf->rsp_prod !=3D xenstore_buf->rsp_= cons); >=20 > The if is redundant since wait_event already tests it. Ah, right. >=20 >> + prod =3D xenstore_buf->rsp_prod; >> + DEBUG("Rsp_cons %d, rsp_prod %d.\n", xenstore_buf->rsp_cons, = prod); >> + size =3D min(len - off, prod - xenstore_buf->rsp_cons); >> + memcpy_from_ring(xenstore_buf->rsp, buf + off, >> + MASK_XENSTORE_IDX(xenstore_buf->rsp_cons), s= ize); >> + off +=3D size; >> + notify =3D (xenstore_buf->rsp_cons + XENSTORE_RING_SIZE =3D=3D= >> + xenstore_buf->rsp_prod); >=20 > This looks odd to me? We want to notify as soon as the ring is empty, > which can happen at any place in the ring right? No, we want to notify if the ring was full and is about to gain some space again, as the other side was probably not able to put all data in and is now waiting for more space to become available. >=20 > Linux' code uses (intf->rsp_prod - cons >=3D XENSTORE_RING_SIZE), *afte= r* > the rsp_cons increase. >=20 >> + rmb(); >=20 > rmb() must be placed before memcpy_from_ring, to make sure that the dat= a > we read from the buffer is up-to-date according to the read we made fro= m > rsp_prod. Ah, yes. Thanks for spotting that one! >=20 > The rest seems ok to me. Thanks, Juergen --------------4AF975DB5AFDAF92EAD2B51A Content-Type: application/pgp-keys; name="OpenPGP_0xB0DE9DD628BF132F.asc" Content-Transfer-Encoding: quoted-printable Content-Description: OpenPGP public key Content-Disposition: attachment; filename="OpenPGP_0xB0DE9DD628BF132F.asc" -----BEGIN PGP PUBLIC KEY BLOCK----- xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy= cWx w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O= f8Z d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y= 9bf IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq= G7/ 377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR= 3Jv c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA= QIe AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT= hpw dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0= MbD 1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv= oPH Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v= 5QL +qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ= 2Vu IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC= QoL BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr= Wf0 teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC= /nu AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh= ITT d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm= XBK 7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz= 80h SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC= AcD AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn= FOX gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1= jnD kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0= N51 N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l= otu fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1= tqS EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA= hsD BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3= g3O ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5= dM7 wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5= D+j LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm= V2x AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N= Eaw QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW= nHI s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR= wgn BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV= bVF LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk= pEd IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA= QAB wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw= Tbe 8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1= vJz Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8= VGi wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW= svi uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX= zXs ZDn8R38=3D =3D2wuH -----END PGP PUBLIC KEY BLOCK----- --------------4AF975DB5AFDAF92EAD2B51A-- --dCjWhK2Ru03rTrBBaj7QlVmsqxnePxHnB-- --a6FiRo5rvrV8rM1QpL0b6BtntmpTnTYqB Content-Type: application/pgp-signature; name="OpenPGP_signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="OpenPGP_signature" -----BEGIN PGP SIGNATURE----- wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmFBz4wFAwAAAAAACgkQsN6d1ii/Ey+l dQf6ApRTwp9ksBqlRW19px1aRTw+8f8odlRkxEUfzEj0AFav0/wyCygTBCSdWev93acVbAF2Uo+T czuUxY6dWAiPdxrYS0XzfoG8C+Pq8TMDmxt+nl582VSMwDxLJzYIkZmwh0897je9jr86m1x5a2lS EnpBiZn0UJvBiYTn5cP6ocFWA6at5vW88fT9UmWX6NBwguH9u9y4Y9ZV7Wv8unJKyNjtsQ7mKGgA PL5+2yxM8Mwxvv1WCCT7fee7AYF//gpX/CYyZVe2nw5Chjf64NvoLJBLnVIMMGrn6SH46xx+ba3H cE81xv80CGJsiJWZJf+8hHt6mD0lQ1Aj1uWsLbKbCg== =Isq2 -----END PGP SIGNATURE----- --a6FiRo5rvrV8rM1QpL0b6BtntmpTnTYqB-- From minios-devel-bounces@lists.xenproject.org Wed Sep 15 11:16:34 2021 Return-path: Envelope-to: archives@lists.xenproject.org Delivery-date: Wed, 15 Sep 2021 11:16:34 +0000 Received: from list by lists.xenproject.org with outflank-mailman.187505.336383 (Exim 4.92) (envelope-from ) id 1mQSu1-0003td-KQ; Wed, 15 Sep 2021 11:16:33 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 187505.336383; Wed, 15 Sep 2021 11:16:33 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mQSu1-0003tW-HV; Wed, 15 Sep 2021 11:16:33 +0000 Received: by outflank-mailman (input) for mailman id 187505; Wed, 15 Sep 2021 11:16:32 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mQSu0-0003tQ-Ef for minios-devel@lists.xenproject.org; Wed, 15 Sep 2021 11:16:32 +0000 Received: from smtp-out1.suse.de (unknown [195.135.220.28]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id aef0d088-d611-48ba-b794-2221fe00e3e2; Wed, 15 Sep 2021 11:16:30 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 25B0A2221F; Wed, 15 Sep 2021 11:16:29 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id E6BCB13C37; Wed, 15 Sep 2021 11:16:28 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id 7evhNgzWQWEHFAAAMHmgww (envelope-from ); Wed, 15 Sep 2021 11:16:28 +0000 X-BeenThere: minios-devel@lists.xenproject.org List-Id: Mini-os development list List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: minios-devel-bounces@lists.xenproject.org Precedence: list Sender: "Minios-devel" X-Inumbo-ID: aef0d088-d611-48ba-b794-2221fe00e3e2 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1631704589; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding; bh=AjV0yLQDNp7wCmyLlXRUm2R34qvQRC0Ch6O/l5NYfDc=; b=OxfedwDzTzFCJ0byFYdt0blxi/vXV3CzSgJ3/wNV3jzZZvtjKTDJbJEOtaQX91Z1d4EXX8 YFNF0GOcjAzRpvPpPhUQiXIka4YPB/4R4Vfyauk1k1nlqn5lzQde12eNifiqJGDBH+RM0C Gke2pA1mEBRgVzmnU7T0Pzx1Rrxxy3Q= From: Juergen Gross To: minios-devel@lists.xenproject.org, xen-devel@lists.xenproject.org Cc: samuel.thibault@ens-lyon.org, wl@xen.org, Juergen Gross Subject: [PATCH v2] xenbus: support large messages Date: Wed, 15 Sep 2021 13:16:27 +0200 Message-Id: <20210915111627.19486-1-jgross@suse.com> X-Mailer: git-send-email 2.26.2 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Today the implementation of the xenbus protocol in Mini-OS will only allow to transfer the complete message to or from the ring page buffer. This is limiting the maximum message size to lower values as the xenbus protocol normally would allow. Change that by allowing to transfer the xenbus message in chunks as soon as they are available. Avoid crashing Mini-OS in case of illegal data read from the ring buffer. Signed-off-by: Juergen Gross --- V2: - drop redundant if (Samuel Thibault) - move rmb() (Samuel Thibault) --- xenbus/xenbus.c | 211 ++++++++++++++++++++++++++++-------------------- 1 file changed, 123 insertions(+), 88 deletions(-) diff --git a/xenbus/xenbus.c b/xenbus/xenbus.c index 23de61e..ac016e9 100644 --- a/xenbus/xenbus.c +++ b/xenbus/xenbus.c @@ -29,6 +29,7 @@ #include #include #include +#include #define min(x,y) ({ \ typeof(x) tmpx = (x); \ @@ -46,6 +47,7 @@ static struct xenstore_domain_interface *xenstore_buf; static DECLARE_WAIT_QUEUE_HEAD(xb_waitq); DECLARE_WAIT_QUEUE_HEAD(xenbus_watch_queue); +static __DECLARE_SEMAPHORE_GENERIC(xb_write_sem, 1); xenbus_event_queue xenbus_events; static struct watch { @@ -231,75 +233,104 @@ char *xenbus_wait_for_state_change(const char* path, XenbusState *state, xenbus_ } +static void xenbus_read_data(char *buf, unsigned int len) +{ + unsigned int off = 0; + unsigned int prod; + unsigned int size; + int notify; + + while (off != len) + { + wait_event(xb_waitq, xenstore_buf->rsp_prod != xenstore_buf->rsp_cons); + + prod = xenstore_buf->rsp_prod; + DEBUG("Rsp_cons %d, rsp_prod %d.\n", xenstore_buf->rsp_cons, prod); + size = min(len - off, prod - xenstore_buf->rsp_cons); + + rmb(); /* Make sure data read from ring is ordered with rsp_prod. */ + memcpy_from_ring(xenstore_buf->rsp, buf + off, + MASK_XENSTORE_IDX(xenstore_buf->rsp_cons), size); + off += size; + notify = (xenstore_buf->rsp_cons + XENSTORE_RING_SIZE == + xenstore_buf->rsp_prod); + xenstore_buf->rsp_cons += size; + wmb(); + if (notify) + notify_remote_via_evtchn(xenbus_evtchn); + } +} + static void xenbus_thread_func(void *ign) { struct xsd_sockmsg msg; - unsigned prod = xenstore_buf->rsp_prod; + char *data; for (;;) { - wait_event(xb_waitq, prod != xenstore_buf->rsp_prod); - while (1) { - prod = xenstore_buf->rsp_prod; - DEBUG("Rsp_cons %d, rsp_prod %d.\n", xenstore_buf->rsp_cons, - xenstore_buf->rsp_prod); - if (xenstore_buf->rsp_prod - xenstore_buf->rsp_cons < sizeof(msg)) - break; - rmb(); - memcpy_from_ring(xenstore_buf->rsp, &msg, - MASK_XENSTORE_IDX(xenstore_buf->rsp_cons), - sizeof(msg)); - DEBUG("Msg len %d, %d avail, id %d.\n", msg.len + sizeof(msg), - xenstore_buf->rsp_prod - xenstore_buf->rsp_cons, msg.req_id); - - if (xenstore_buf->rsp_prod - xenstore_buf->rsp_cons < - sizeof(msg) + msg.len) - break; - - DEBUG("Message is good.\n"); - - if (msg.type == XS_WATCH_EVENT) { - struct xenbus_event *event = malloc(sizeof(*event) + msg.len); - xenbus_event_queue *events = NULL; - char *data = (char*)event + sizeof(*event); - struct watch *watch; - - memcpy_from_ring(xenstore_buf->rsp, data, - MASK_XENSTORE_IDX(xenstore_buf->rsp_cons + sizeof(msg)), - msg.len); - - event->path = data; - event->token = event->path + strlen(event->path) + 1; - - mb(); - xenstore_buf->rsp_cons += msg.len + sizeof(msg); - - for (watch = watches; watch; watch = watch->next) - if (!strcmp(watch->token, event->token)) { - events = watch->events; - break; - } - - if (events) { - event->next = *events; - *events = event; - wake_up(&xenbus_watch_queue); - } else { - printk("unexpected watch token %s\n", event->token); - free(event); + xenbus_read_data((char *)&msg, sizeof(msg)); + DEBUG("Msg len %d, %d avail, id %d.\n", msg.len + sizeof(msg), + xenstore_buf->rsp_prod - xenstore_buf->rsp_cons, msg.req_id); + + if (msg.len > XENSTORE_PAYLOAD_MAX) { + printk("Xenstore violates protocol, message longer than allowed.\n"); + return; + } + + if (msg.type == XS_WATCH_EVENT) { + struct xenbus_event *event = malloc(sizeof(*event) + msg.len); + xenbus_event_queue *events = NULL; + struct watch *watch; + char *c; + int zeroes = 0; + + data = (char*)event + sizeof(*event); + xenbus_read_data(data, msg.len); + + for (c = data; c < data + msg.len; c++) + if (!*c) + zeroes++; + if (zeroes != 2) { + printk("Xenstore: illegal watch event data\n"); + free(event); + continue; + } + + event->path = data; + event->token = event->path + strlen(event->path) + 1; + + for (watch = watches; watch; watch = watch->next) + if (!strcmp(watch->token, event->token)) { + events = watch->events; + break; } + + if (events) { + event->next = *events; + *events = event; + wake_up(&xenbus_watch_queue); } else { - req_info[msg.req_id].reply = malloc(sizeof(msg) + msg.len); - memcpy_from_ring(xenstore_buf->rsp, req_info[msg.req_id].reply, - MASK_XENSTORE_IDX(xenstore_buf->rsp_cons), - msg.len + sizeof(msg)); - mb(); - xenstore_buf->rsp_cons += msg.len + sizeof(msg); - wake_up(&req_info[msg.req_id].waitq); + printk("Xenstore: unexpected watch token %s\n", event->token); + free(event); } - wmb(); - notify_remote_via_evtchn(xenbus_evtchn); + continue; + } + + data = malloc(sizeof(msg) + msg.len); + memcpy(data, &msg, sizeof(msg)); + xenbus_read_data(data + sizeof(msg), msg.len); + + if (msg.req_id >= NR_REQS || !req_info[msg.req_id].in_use) { + printk("Xenstore: illegal request id %d\n", msg.req_id); + free(data); + continue; } + + DEBUG("Message is good.\n"); + + req_info[msg.req_id].reply = data; + + wake_up(&req_info[msg.req_id].waitq); } } @@ -451,36 +482,40 @@ static void xb_write(int type, int req_id, xenbus_transaction_t trans_id, cur_req = &header_req; - BUG_ON(len > XENSTORE_RING_SIZE); - /* Wait for the ring to drain to the point where we can send the - message. */ - prod = xenstore_buf->req_prod; - if (prod + len - xenstore_buf->req_cons > XENSTORE_RING_SIZE) - { - /* Wait for there to be space on the ring */ - DEBUG("prod %d, len %d, cons %d, size %d; waiting.\n", - prod, len, xenstore_buf->req_cons, XENSTORE_RING_SIZE); - wait_event(xb_waitq, - xenstore_buf->req_prod + len - xenstore_buf->req_cons <= - XENSTORE_RING_SIZE); - DEBUG("Back from wait.\n"); - prod = xenstore_buf->req_prod; - } + BUG_ON(len > XENSTORE_PAYLOAD_MAX); - /* We're now guaranteed to be able to send the message without - overflowing the ring. Do so. */ + /* Make sure we are the only thread trying to write. */ + down(&xb_write_sem); + + /* Send the message in chunks using free ring space when available. */ total_off = 0; req_off = 0; - while (total_off < len) + while (total_off < len) { + prod = xenstore_buf->req_prod; + if (prod - xenstore_buf->req_cons >= XENSTORE_RING_SIZE) + { + /* Send evtchn to notify remote */ + notify_remote_via_evtchn(xenbus_evtchn); + + /* Wait for there to be space on the ring */ + DEBUG("prod %d, len %d, cons %d, size %d; waiting.\n", prod, + len - total_off, xenstore_buf->req_cons, XENSTORE_RING_SIZE); + wait_event(xb_waitq, + prod - xenstore_buf->req_cons < XENSTORE_RING_SIZE); + DEBUG("Back from wait.\n"); + } + this_chunk = min(cur_req->len - req_off, - XENSTORE_RING_SIZE - MASK_XENSTORE_IDX(prod)); + XENSTORE_RING_SIZE - MASK_XENSTORE_IDX(prod)); + this_chunk = min(this_chunk, + xenstore_buf->req_cons + XENSTORE_RING_SIZE - prod); memcpy((char *)xenstore_buf->req + MASK_XENSTORE_IDX(prod), - (char *)cur_req->data + req_off, this_chunk); + (char *)cur_req->data + req_off, this_chunk); prod += this_chunk; req_off += this_chunk; total_off += this_chunk; - if (req_off == cur_req->len) + if (req_off == cur_req->len) { req_off = 0; if (cur_req == &header_req) @@ -488,20 +523,20 @@ static void xb_write(int type, int req_id, xenbus_transaction_t trans_id, else cur_req++; } + + /* Remote must see entire message before updating indexes */ + wmb(); + xenstore_buf->req_prod = prod; } + /* Send evtchn to notify remote */ + notify_remote_via_evtchn(xenbus_evtchn); + DEBUG("Complete main loop of xb_write.\n"); BUG_ON(req_off != 0); BUG_ON(total_off != len); - BUG_ON(prod > xenstore_buf->req_cons + XENSTORE_RING_SIZE); - - /* Remote must see entire message before updating indexes */ - wmb(); - - xenstore_buf->req_prod += len; - /* Send evtchn to notify remote */ - notify_remote_via_evtchn(xenbus_evtchn); + up(&xb_write_sem); } /* Send a mesasge to xenbus, in the same fashion as xb_write, and -- 2.26.2 From minios-devel-bounces@lists.xenproject.org Wed Sep 15 11:20:31 2021 Return-path: Envelope-to: archives@lists.xenproject.org Delivery-date: Wed, 15 Sep 2021 11:20:31 +0000 Received: from list by lists.xenproject.org with outflank-mailman.187517.336409 (Exim 4.92) (envelope-from ) id 1mQSxq-0005pa-JW; Wed, 15 Sep 2021 11:20:30 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 187517.336409; Wed, 15 Sep 2021 11:20:30 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mQSxq-0005pR-GM; Wed, 15 Sep 2021 11:20:30 +0000 Received: by outflank-mailman (input) for mailman id 187517; Wed, 15 Sep 2021 11:20:29 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mQSxp-0005Wz-Gc for minios-devel@lists.xenproject.org; Wed, 15 Sep 2021 11:20:29 +0000 Received: from mail3-relais-sop.national.inria.fr (unknown [192.134.164.104]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id ac53fdfe-4486-4ce5-94bc-4f2a534ee4ad; Wed, 15 Sep 2021 11:20:22 +0000 (UTC) Received: from nat-eduroam-36-gw-01-bso.bordeaux.inria.fr (HELO begin) ([194.199.1.36]) by mail3-relais-sop.national.inria.fr with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 15 Sep 2021 13:20:20 +0200 Received: from samy by begin with local (Exim 4.95-RC2) (envelope-from ) id 1mQSxg-00F8lH-Bk; Wed, 15 Sep 2021 13:20:20 +0200 X-BeenThere: minios-devel@lists.xenproject.org List-Id: Mini-os development list List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: minios-devel-bounces@lists.xenproject.org Precedence: list Sender: "Minios-devel" X-Inumbo-ID: ac53fdfe-4486-4ce5-94bc-4f2a534ee4ad IronPort-HdrOrdr: =?us-ascii?q?A9a23=3AO0s7kKurlqaXSTNPnfL9oYNI7skDttV00zEX?= =?us-ascii?q?/kB9WHVpm5Oj5ruTdaUgpGbJYWgqKQkdcIa7WZVoJkmyyXcV2/h0AV7GZmPbUQ?= =?us-ascii?q?STQr2KgbGSoQEIeBeSygcH78ddmsFFYbWaMbEdt7eY3ODSKbodKbe8gcWVrNab?= =?us-ascii?q?5HMoaQFudq16qyZwDQufHkAefnggObMJULGZovddrz27YHINc4CHBn0IRfXKo8?= =?us-ascii?q?SjruOBXSI7?= X-IronPort-AV: E=Sophos;i="5.84,326,1620684000"; d="scan'208";a="393037543" Date: Wed, 15 Sep 2021 13:20:20 +0200 From: Samuel Thibault To: Juergen Gross Cc: minios-devel@lists.xenproject.org, xen-devel@lists.xenproject.org, wl@xen.org Subject: Re: [PATCH] mini-os: xenbus: support large messages Message-ID: <20210915112020.iy6vqn4qcvtbohyi@begin> Mail-Followup-To: Samuel Thibault , Juergen Gross , minios-devel@lists.xenproject.org, xen-devel@lists.xenproject.org, wl@xen.org References: <20210818152610.6114-1-jgross@suse.com> <20210914231742.pxpac2lhp2kk5ook@begin> <2304b885-434c-8a19-78e0-14637a4adee3@suse.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <2304b885-434c-8a19-78e0-14637a4adee3@suse.com> Organization: I am not organized User-Agent: NeoMutt/20170609 (1.8.3) Juergen Gross, le mer. 15 sept. 2021 12:48:44 +0200, a ecrit: > On 15.09.21 01:17, Samuel Thibault wrote: > > > + prod = xenstore_buf->rsp_prod; > > > + DEBUG("Rsp_cons %d, rsp_prod %d.\n", xenstore_buf->rsp_cons, prod); > > > + size = min(len - off, prod - xenstore_buf->rsp_cons); > > > + memcpy_from_ring(xenstore_buf->rsp, buf + off, > > > + MASK_XENSTORE_IDX(xenstore_buf->rsp_cons), size); > > > + off += size; > > > + notify = (xenstore_buf->rsp_cons + XENSTORE_RING_SIZE == > > > + xenstore_buf->rsp_prod); > > > > This looks odd to me? We want to notify as soon as the ring is empty, > > which can happen at any place in the ring right? > > No, we want to notify if the ring was full and is about to gain some > space again, as the other side was probably not able to put all data > in and is now waiting for more space to become available. Ok, that said, the producer may fill the ring between this test and the rsp_cons update, and thus the producer will go sleep and here the consumer will not notice it and thus never notify it. So we really need to make the test after the rsp_cons update, like Linux does: > > Linux' code uses (intf->rsp_prod - cons >= XENSTORE_RING_SIZE), *after* > > the rsp_cons increase. Samuel From minios-devel-bounces@lists.xenproject.org Wed Sep 15 12:03:43 2021 Return-path: Envelope-to: archives@lists.xenproject.org Delivery-date: Wed, 15 Sep 2021 12:03:43 +0000 Received: from list by lists.xenproject.org with outflank-mailman.187543.336439 (Exim 4.92) (envelope-from ) id 1mQTdb-0002yi-FG; Wed, 15 Sep 2021 12:03:39 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 187543.336439; Wed, 15 Sep 2021 12:03:39 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mQTdb-0002yb-C8; Wed, 15 Sep 2021 12:03:39 +0000 Received: by outflank-mailman (input) for mailman id 187543; Wed, 15 Sep 2021 12:03:38 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mQTda-0002yV-F1 for minios-devel@lists.xenproject.org; Wed, 15 Sep 2021 12:03:38 +0000 Received: from smtp-out2.suse.de (unknown [195.135.220.29]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id e30529ed-5b9f-496f-bba0-b9d1997f84a0; Wed, 15 Sep 2021 12:03:36 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 0F69A20187; Wed, 15 Sep 2021 12:03:36 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id DDABD13C29; Wed, 15 Sep 2021 12:03:35 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id sIa1NBfhQWFYLgAAMHmgww (envelope-from ); Wed, 15 Sep 2021 12:03:35 +0000 X-BeenThere: minios-devel@lists.xenproject.org List-Id: Mini-os development list List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: minios-devel-bounces@lists.xenproject.org Precedence: list Sender: "Minios-devel" X-Inumbo-ID: e30529ed-5b9f-496f-bba0-b9d1997f84a0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1631707416; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc: mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=ENhmOeCtsm5hfqnNK954lymn6nLMXDrcKR3qancrAyk=; b=JbRqSckOJdoT1BrjCsXvigpfRJUYL2Wj9ROLqtKkvTLGYrlfvmyIS/tHSvvKx/dfX0Ma1S UF9ZFAjxmMcrNt6QbgMQ0lAAuZADFJO78nYCNBE8Dx3wN4z4lQgxmvIeclDz2W2Kn3/dMF lQBYvEqjc7HmdSy7wtLybB51/WxVjv8= Subject: Re: [PATCH] mini-os: xenbus: support large messages To: Samuel Thibault , minios-devel@lists.xenproject.org, xen-devel@lists.xenproject.org, wl@xen.org References: <20210818152610.6114-1-jgross@suse.com> <20210914231742.pxpac2lhp2kk5ook@begin> <2304b885-434c-8a19-78e0-14637a4adee3@suse.com> <20210915112020.iy6vqn4qcvtbohyi@begin> From: Juergen Gross Message-ID: <20e4fa88-74a8-35a0-dffa-fd5e5765fe0d@suse.com> Date: Wed, 15 Sep 2021 14:03:35 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.12.0 MIME-Version: 1.0 In-Reply-To: <20210915112020.iy6vqn4qcvtbohyi@begin> Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="ScL0Va8R8fzjJJz3Nv9BCmmjeBa35z0nA" This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --ScL0Va8R8fzjJJz3Nv9BCmmjeBa35z0nA Content-Type: multipart/mixed; boundary="CMMeocb5gxwlK7fdJkh0AgWtsvXXjTXOp"; protected-headers="v1" From: Juergen Gross To: Samuel Thibault , minios-devel@lists.xenproject.org, xen-devel@lists.xenproject.org, wl@xen.org Message-ID: <20e4fa88-74a8-35a0-dffa-fd5e5765fe0d@suse.com> Subject: Re: [PATCH] mini-os: xenbus: support large messages References: <20210818152610.6114-1-jgross@suse.com> <20210914231742.pxpac2lhp2kk5ook@begin> <2304b885-434c-8a19-78e0-14637a4adee3@suse.com> <20210915112020.iy6vqn4qcvtbohyi@begin> In-Reply-To: <20210915112020.iy6vqn4qcvtbohyi@begin> --CMMeocb5gxwlK7fdJkh0AgWtsvXXjTXOp Content-Type: multipart/mixed; boundary="------------A0BD0CE41CC0BB783A29960C" Content-Language: en-US This is a multi-part message in MIME format. --------------A0BD0CE41CC0BB783A29960C Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: quoted-printable On 15.09.21 13:20, Samuel Thibault wrote: > Juergen Gross, le mer. 15 sept. 2021 12:48:44 +0200, a ecrit: >> On 15.09.21 01:17, Samuel Thibault wrote: >>>> + prod =3D xenstore_buf->rsp_prod; >>>> + DEBUG("Rsp_cons %d, rsp_prod %d.\n", xenstore_buf->rsp_cons= , prod); >>>> + size =3D min(len - off, prod - xenstore_buf->rsp_cons); >>>> + memcpy_from_ring(xenstore_buf->rsp, buf + off, >>>> + MASK_XENSTORE_IDX(xenstore_buf->rsp_cons),= size); >>>> + off +=3D size; >>>> + notify =3D (xenstore_buf->rsp_cons + XENSTORE_RING_SIZE =3D= =3D >>>> + xenstore_buf->rsp_prod); >>> >>> This looks odd to me? We want to notify as soon as the ring is empty= , >>> which can happen at any place in the ring right? >> >> No, we want to notify if the ring was full and is about to gain some >> space again, as the other side was probably not able to put all data >> in and is now waiting for more space to become available. >=20 > Ok, that said, the producer may fill the ring between this test and > the rsp_cons update, and thus the producer will go sleep and here the > consumer will not notice it and thus never notify it. >=20 > So we really need to make the test after the rsp_cons update, like Linu= x > does: >=20 >>> Linux' code uses (intf->rsp_prod - cons >=3D XENSTORE_RING_SIZE), *af= ter* >>> the rsp_cons increase. Oh, you are right, of course. How could I overlook this? Thanks, Juergen --------------A0BD0CE41CC0BB783A29960C Content-Type: application/pgp-keys; name="OpenPGP_0xB0DE9DD628BF132F.asc" Content-Transfer-Encoding: quoted-printable Content-Description: OpenPGP public key Content-Disposition: attachment; filename="OpenPGP_0xB0DE9DD628BF132F.asc" -----BEGIN PGP PUBLIC KEY BLOCK----- xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy= cWx w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O= f8Z d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y= 9bf IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq= G7/ 377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR= 3Jv c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA= QIe AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT= hpw dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0= MbD 1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv= oPH Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v= 5QL +qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ= 2Vu IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC= QoL BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr= Wf0 teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC= /nu AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh= ITT d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm= XBK 7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz= 80h SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC= AcD AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn= FOX gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1= jnD kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0= N51 N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l= otu fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1= tqS EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA= hsD BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3= g3O ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5= dM7 wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5= D+j LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm= V2x AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N= Eaw QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW= nHI s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR= wgn BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV= bVF LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk= pEd IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA= QAB wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw= Tbe 8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1= vJz Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8= VGi wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW= svi uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX= zXs ZDn8R38=3D =3D2wuH -----END PGP PUBLIC KEY BLOCK----- --------------A0BD0CE41CC0BB783A29960C-- --CMMeocb5gxwlK7fdJkh0AgWtsvXXjTXOp-- --ScL0Va8R8fzjJJz3Nv9BCmmjeBa35z0nA Content-Type: application/pgp-signature; name="OpenPGP_signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="OpenPGP_signature" -----BEGIN PGP SIGNATURE----- wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmFB4RcFAwAAAAAACgkQsN6d1ii/Ey8A ywf/RgULYRjT5cSCcoBw6DISFWGq+0kG0JV/52VpYOvEuHQ4QB2jfdSW2FnhiI+zi7AhhmurF2fI W9VacbvdcDjQU6YLWXVRGQkirLAIg9LPZxtE8XTU9kZ0GS+PKsfAuKvm0YmmNVzT9tltsPMRiivL aqzHi4sdioQhpBxMi2uUpTqZhoX1k2T3uokEA/WkGOQpaWFxsVUuy+9uwu8hndtJY07leHazIpun X5oH5XuBoJOOhH9XvRfXOZuZ6Deq+WbzKnyZLY2eCeOM9L5uwK6On/awQe+zuRgvVZ1LTKZ0JxXS wgIfraKDHxZ77q1+acbEVg706lemRhj2Xw3PHzPcCA== =0vWU -----END PGP SIGNATURE----- --ScL0Va8R8fzjJJz3Nv9BCmmjeBa35z0nA-- From minios-devel-bounces@lists.xenproject.org Wed Sep 15 12:20:25 2021 Return-path: Envelope-to: archives@lists.xenproject.org Delivery-date: Wed, 15 Sep 2021 12:20:25 +0000 Received: from list by lists.xenproject.org with outflank-mailman.187554.336464 (Exim 4.92) (envelope-from ) id 1mQTto-0005o6-76; Wed, 15 Sep 2021 12:20:24 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 187554.336464; Wed, 15 Sep 2021 12:20:24 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mQTto-0005nz-3p; Wed, 15 Sep 2021 12:20:24 +0000 Received: by outflank-mailman (input) for mailman id 187554; Wed, 15 Sep 2021 12:20:23 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mQTtn-0005Wn-4U for minios-devel@lists.xenproject.org; Wed, 15 Sep 2021 12:20:23 +0000 Received: from mail2-relais-roc.national.inria.fr (unknown [192.134.164.83]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 4819fa30-161f-11ec-b508-12813bfff9fa; Wed, 15 Sep 2021 12:20:16 +0000 (UTC) Received: from nat-eduroam-36-gw-01-bso.bordeaux.inria.fr (HELO begin) ([194.199.1.36]) by mail2-relais-roc.national.inria.fr with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 15 Sep 2021 14:20:13 +0200 Received: from samy by begin with local (Exim 4.95-RC2) (envelope-from ) id 1mQTtd-00FA9l-7I; Wed, 15 Sep 2021 14:20:13 +0200 X-BeenThere: minios-devel@lists.xenproject.org List-Id: Mini-os development list List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: minios-devel-bounces@lists.xenproject.org Precedence: list Sender: "Minios-devel" X-Inumbo-ID: 4819fa30-161f-11ec-b508-12813bfff9fa IronPort-HdrOrdr: =?us-ascii?q?A9a23=3A0bEInaoqM1KGkjgH0DbTKQkaV5oweYIsimQD?= =?us-ascii?q?101hICG9Ffb5qynOppUmPHDP4gr5NEtQ++xoW5PvfZqjz+8X3WB5B97LNzUO01?= =?us-ascii?q?HFEGgN1+bfKnHbdBHDyg=3D=3D?= X-IronPort-AV: E=Sophos;i="5.84,326,1620684000"; d="scan'208";a="528588671" Date: Wed, 15 Sep 2021 14:20:13 +0200 From: Samuel Thibault To: Juergen Gross Cc: minios-devel@lists.xenproject.org, xen-devel@lists.xenproject.org, wl@xen.org Subject: Re: [PATCH] mini-os: xenbus: support large messages Message-ID: <20210915122013.5fswso7poab5iyzq@begin> Mail-Followup-To: Samuel Thibault , Juergen Gross , minios-devel@lists.xenproject.org, xen-devel@lists.xenproject.org, wl@xen.org References: <20210818152610.6114-1-jgross@suse.com> <20210914231742.pxpac2lhp2kk5ook@begin> <2304b885-434c-8a19-78e0-14637a4adee3@suse.com> <20210915112020.iy6vqn4qcvtbohyi@begin> <20e4fa88-74a8-35a0-dffa-fd5e5765fe0d@suse.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20e4fa88-74a8-35a0-dffa-fd5e5765fe0d@suse.com> Organization: I am not organized User-Agent: NeoMutt/20170609 (1.8.3) Juergen Gross, le mer. 15 sept. 2021 14:03:35 +0200, a ecrit: > On 15.09.21 13:20, Samuel Thibault wrote: > > So we really need to make the test after the rsp_cons update, like Linux > > does: > > > > > > Linux' code uses (intf->rsp_prod - cons >= XENSTORE_RING_SIZE), *after* > > > > the rsp_cons increase. > > Oh, you are right, of course. How could I overlook this? One person alone will always overlook something, that's why we make code review, to have several pairs of eyes check whether we may have forgotten something :) Samuel From minios-devel-bounces@lists.xenproject.org Wed Sep 15 14:08:25 2021 Return-path: Envelope-to: archives@lists.xenproject.org Delivery-date: Wed, 15 Sep 2021 14:08:25 +0000 Received: from list by lists.xenproject.org with outflank-mailman.187693.336643 (Exim 4.92) (envelope-from ) id 1mQVaI-0000Ie-BT; Wed, 15 Sep 2021 14:08:22 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 187693.336643; Wed, 15 Sep 2021 14:08:22 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mQVaI-0000IX-8f; Wed, 15 Sep 2021 14:08:22 +0000 Received: by outflank-mailman (input) for mailman id 187693; Wed, 15 Sep 2021 14:08:20 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mQVaG-0000IR-SZ for minios-devel@lists.xenproject.org; Wed, 15 Sep 2021 14:08:20 +0000 Received: from smtp-out1.suse.de (unknown [195.135.220.28]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 99d7e694-6503-4ba8-85ca-645479a1cc2d; Wed, 15 Sep 2021 14:08:18 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 062BC2217F; Wed, 15 Sep 2021 14:08:17 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id CC95213C37; Wed, 15 Sep 2021 14:08:16 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id 3i++MFD+QWHsdgAAMHmgww (envelope-from ); Wed, 15 Sep 2021 14:08:16 +0000 X-BeenThere: minios-devel@lists.xenproject.org List-Id: Mini-os development list List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: minios-devel-bounces@lists.xenproject.org Precedence: list Sender: "Minios-devel" X-Inumbo-ID: 99d7e694-6503-4ba8-85ca-645479a1cc2d DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1631714897; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding; bh=Xozn4dABy+P/hAZ8HNwNB0Im8TWq1cxoQG3rmcU2AYY=; b=d6rTs/CCIY0Ek/83+mMi5ojBDEp7c1YgI/Qh6fHn1xc6nJ8n7tJEfOozvFsgD0hNqV9RN5 KSaAhzHzJHH/pq0I+0GtQ5av0qDqHf0ztMyym2xDDtBylUpyI92sx+7JFvvPWHZhRDF7Lj lUEMTf4esXwv3rYl3QR8yygg9P3FVJ8= From: Juergen Gross To: minios-devel@lists.xenproject.org, xen-devel@lists.xenproject.org Cc: samuel.thibault@ens-lyon.org, wl@xen.org, Juergen Gross Subject: [PATCH v3] xenbus: support large messages Date: Wed, 15 Sep 2021 16:08:15 +0200 Message-Id: <20210915140815.26736-1-jgross@suse.com> X-Mailer: git-send-email 2.26.2 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Today the implementation of the xenbus protocol in Mini-OS will only allow to transfer the complete message to or from the ring page buffer. This is limiting the maximum message size to lower values as the xenbus protocol normally would allow. Change that by allowing to transfer the xenbus message in chunks as soon as they are available. Avoid crashing Mini-OS in case of illegal data read from the ring buffer. Signed-off-by: Juergen Gross --- V2: - drop redundant if (Samuel Thibault) - move rmb() (Samuel Thibault) V3: - correct notification test (Samuel Thibault) --- xenbus/xenbus.c | 209 ++++++++++++++++++++++++++++-------------------- 1 file changed, 121 insertions(+), 88 deletions(-) diff --git a/xenbus/xenbus.c b/xenbus/xenbus.c index 23de61e..c8d358d 100644 --- a/xenbus/xenbus.c +++ b/xenbus/xenbus.c @@ -29,6 +29,7 @@ #include #include #include +#include #define min(x,y) ({ \ typeof(x) tmpx = (x); \ @@ -46,6 +47,7 @@ static struct xenstore_domain_interface *xenstore_buf; static DECLARE_WAIT_QUEUE_HEAD(xb_waitq); DECLARE_WAIT_QUEUE_HEAD(xenbus_watch_queue); +static __DECLARE_SEMAPHORE_GENERIC(xb_write_sem, 1); xenbus_event_queue xenbus_events; static struct watch { @@ -231,75 +233,102 @@ char *xenbus_wait_for_state_change(const char* path, XenbusState *state, xenbus_ } +static void xenbus_read_data(char *buf, unsigned int len) +{ + unsigned int off = 0; + unsigned int prod, cons; + unsigned int size; + + while (off != len) + { + wait_event(xb_waitq, xenstore_buf->rsp_prod != xenstore_buf->rsp_cons); + + prod = xenstore_buf->rsp_prod; + cons = xenstore_buf->rsp_cons; + DEBUG("Rsp_cons %d, rsp_prod %d.\n", cons, prod); + size = min(len - off, prod - cons); + + rmb(); /* Make sure data read from ring is ordered with rsp_prod. */ + memcpy_from_ring(xenstore_buf->rsp, buf + off, + MASK_XENSTORE_IDX(cons), size); + off += size; + xenstore_buf->rsp_cons += size; + wmb(); + if (xenstore_buf->rsp_prod - cons >= XENSTORE_RING_SIZE) + notify_remote_via_evtchn(xenbus_evtchn); + } +} + static void xenbus_thread_func(void *ign) { struct xsd_sockmsg msg; - unsigned prod = xenstore_buf->rsp_prod; + char *data; for (;;) { - wait_event(xb_waitq, prod != xenstore_buf->rsp_prod); - while (1) { - prod = xenstore_buf->rsp_prod; - DEBUG("Rsp_cons %d, rsp_prod %d.\n", xenstore_buf->rsp_cons, - xenstore_buf->rsp_prod); - if (xenstore_buf->rsp_prod - xenstore_buf->rsp_cons < sizeof(msg)) - break; - rmb(); - memcpy_from_ring(xenstore_buf->rsp, &msg, - MASK_XENSTORE_IDX(xenstore_buf->rsp_cons), - sizeof(msg)); - DEBUG("Msg len %d, %d avail, id %d.\n", msg.len + sizeof(msg), - xenstore_buf->rsp_prod - xenstore_buf->rsp_cons, msg.req_id); - - if (xenstore_buf->rsp_prod - xenstore_buf->rsp_cons < - sizeof(msg) + msg.len) - break; - - DEBUG("Message is good.\n"); - - if (msg.type == XS_WATCH_EVENT) { - struct xenbus_event *event = malloc(sizeof(*event) + msg.len); - xenbus_event_queue *events = NULL; - char *data = (char*)event + sizeof(*event); - struct watch *watch; - - memcpy_from_ring(xenstore_buf->rsp, data, - MASK_XENSTORE_IDX(xenstore_buf->rsp_cons + sizeof(msg)), - msg.len); - - event->path = data; - event->token = event->path + strlen(event->path) + 1; - - mb(); - xenstore_buf->rsp_cons += msg.len + sizeof(msg); - - for (watch = watches; watch; watch = watch->next) - if (!strcmp(watch->token, event->token)) { - events = watch->events; - break; - } - - if (events) { - event->next = *events; - *events = event; - wake_up(&xenbus_watch_queue); - } else { - printk("unexpected watch token %s\n", event->token); - free(event); + xenbus_read_data((char *)&msg, sizeof(msg)); + DEBUG("Msg len %d, %d avail, id %d.\n", msg.len + sizeof(msg), + xenstore_buf->rsp_prod - xenstore_buf->rsp_cons, msg.req_id); + + if (msg.len > XENSTORE_PAYLOAD_MAX) { + printk("Xenstore violates protocol, message longer than allowed.\n"); + return; + } + + if (msg.type == XS_WATCH_EVENT) { + struct xenbus_event *event = malloc(sizeof(*event) + msg.len); + xenbus_event_queue *events = NULL; + struct watch *watch; + char *c; + int zeroes = 0; + + data = (char*)event + sizeof(*event); + xenbus_read_data(data, msg.len); + + for (c = data; c < data + msg.len; c++) + if (!*c) + zeroes++; + if (zeroes != 2) { + printk("Xenstore: illegal watch event data\n"); + free(event); + continue; + } + + event->path = data; + event->token = event->path + strlen(event->path) + 1; + + for (watch = watches; watch; watch = watch->next) + if (!strcmp(watch->token, event->token)) { + events = watch->events; + break; } + + if (events) { + event->next = *events; + *events = event; + wake_up(&xenbus_watch_queue); } else { - req_info[msg.req_id].reply = malloc(sizeof(msg) + msg.len); - memcpy_from_ring(xenstore_buf->rsp, req_info[msg.req_id].reply, - MASK_XENSTORE_IDX(xenstore_buf->rsp_cons), - msg.len + sizeof(msg)); - mb(); - xenstore_buf->rsp_cons += msg.len + sizeof(msg); - wake_up(&req_info[msg.req_id].waitq); + printk("Xenstore: unexpected watch token %s\n", event->token); + free(event); } - wmb(); - notify_remote_via_evtchn(xenbus_evtchn); + continue; } + + data = malloc(sizeof(msg) + msg.len); + memcpy(data, &msg, sizeof(msg)); + xenbus_read_data(data + sizeof(msg), msg.len); + + if (msg.req_id >= NR_REQS || !req_info[msg.req_id].in_use) { + printk("Xenstore: illegal request id %d\n", msg.req_id); + free(data); + continue; + } + + DEBUG("Message is good.\n"); + + req_info[msg.req_id].reply = data; + + wake_up(&req_info[msg.req_id].waitq); } } @@ -451,36 +480,40 @@ static void xb_write(int type, int req_id, xenbus_transaction_t trans_id, cur_req = &header_req; - BUG_ON(len > XENSTORE_RING_SIZE); - /* Wait for the ring to drain to the point where we can send the - message. */ - prod = xenstore_buf->req_prod; - if (prod + len - xenstore_buf->req_cons > XENSTORE_RING_SIZE) - { - /* Wait for there to be space on the ring */ - DEBUG("prod %d, len %d, cons %d, size %d; waiting.\n", - prod, len, xenstore_buf->req_cons, XENSTORE_RING_SIZE); - wait_event(xb_waitq, - xenstore_buf->req_prod + len - xenstore_buf->req_cons <= - XENSTORE_RING_SIZE); - DEBUG("Back from wait.\n"); - prod = xenstore_buf->req_prod; - } + BUG_ON(len > XENSTORE_PAYLOAD_MAX); + + /* Make sure we are the only thread trying to write. */ + down(&xb_write_sem); - /* We're now guaranteed to be able to send the message without - overflowing the ring. Do so. */ + /* Send the message in chunks using free ring space when available. */ total_off = 0; req_off = 0; - while (total_off < len) + while (total_off < len) { + prod = xenstore_buf->req_prod; + if (prod - xenstore_buf->req_cons >= XENSTORE_RING_SIZE) + { + /* Send evtchn to notify remote */ + notify_remote_via_evtchn(xenbus_evtchn); + + /* Wait for there to be space on the ring */ + DEBUG("prod %d, len %d, cons %d, size %d; waiting.\n", prod, + len - total_off, xenstore_buf->req_cons, XENSTORE_RING_SIZE); + wait_event(xb_waitq, + prod - xenstore_buf->req_cons < XENSTORE_RING_SIZE); + DEBUG("Back from wait.\n"); + } + this_chunk = min(cur_req->len - req_off, - XENSTORE_RING_SIZE - MASK_XENSTORE_IDX(prod)); + XENSTORE_RING_SIZE - MASK_XENSTORE_IDX(prod)); + this_chunk = min(this_chunk, + xenstore_buf->req_cons + XENSTORE_RING_SIZE - prod); memcpy((char *)xenstore_buf->req + MASK_XENSTORE_IDX(prod), - (char *)cur_req->data + req_off, this_chunk); + (char *)cur_req->data + req_off, this_chunk); prod += this_chunk; req_off += this_chunk; total_off += this_chunk; - if (req_off == cur_req->len) + if (req_off == cur_req->len) { req_off = 0; if (cur_req == &header_req) @@ -488,20 +521,20 @@ static void xb_write(int type, int req_id, xenbus_transaction_t trans_id, else cur_req++; } + + /* Remote must see entire message before updating indexes */ + wmb(); + xenstore_buf->req_prod = prod; } + /* Send evtchn to notify remote */ + notify_remote_via_evtchn(xenbus_evtchn); + DEBUG("Complete main loop of xb_write.\n"); BUG_ON(req_off != 0); BUG_ON(total_off != len); - BUG_ON(prod > xenstore_buf->req_cons + XENSTORE_RING_SIZE); - /* Remote must see entire message before updating indexes */ - wmb(); - - xenstore_buf->req_prod += len; - - /* Send evtchn to notify remote */ - notify_remote_via_evtchn(xenbus_evtchn); + up(&xb_write_sem); } /* Send a mesasge to xenbus, in the same fashion as xb_write, and -- 2.26.2 From minios-devel-bounces@lists.xenproject.org Wed Sep 15 14:30:56 2021 Return-path: Envelope-to: archives@lists.xenproject.org Delivery-date: Wed, 15 Sep 2021 14:30:56 +0000 Received: from list by lists.xenproject.org with outflank-mailman.187727.336703 (Exim 4.92) (envelope-from ) id 1mQVw7-0005pC-HL; Wed, 15 Sep 2021 14:30:55 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 187727.336703; Wed, 15 Sep 2021 14:30:55 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mQVw7-0005p5-Df; Wed, 15 Sep 2021 14:30:55 +0000 Received: by outflank-mailman (input) for mailman id 187727; Wed, 15 Sep 2021 14:30:53 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mQVw5-0005gO-Jr for minios-devel@lists.xenproject.org; Wed, 15 Sep 2021 14:30:53 +0000 Received: from mail2-relais-roc.national.inria.fr (unknown [192.134.164.83]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 86930eac-1631-11ec-b536-12813bfff9fa; Wed, 15 Sep 2021 14:30:51 +0000 (UTC) Received: from nat-eduroam-36-gw-01-bso.bordeaux.inria.fr (HELO begin) ([194.199.1.36]) by mail2-relais-roc.national.inria.fr with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 15 Sep 2021 16:30:50 +0200 Received: from samy by begin with local (Exim 4.95-RC2) (envelope-from ) id 1mQVw1-00FD1a-0C; Wed, 15 Sep 2021 16:30:49 +0200 X-BeenThere: minios-devel@lists.xenproject.org List-Id: Mini-os development list List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: minios-devel-bounces@lists.xenproject.org Precedence: list Sender: "Minios-devel" X-Inumbo-ID: 86930eac-1631-11ec-b536-12813bfff9fa IronPort-HdrOrdr: =?us-ascii?q?A9a23=3A10s0O6PZyLiCZ8BcTqajsMiBIKoaSvp037BN?= =?us-ascii?q?7SFMoH1uHPBw+Pre/8jzuSWE6gr5O0tOpTn/Asm9qBrnnPYfi7X5Vo3PYOCJgg?= =?us-ascii?q?aVEL0=3D?= X-IronPort-AV: E=Sophos;i="5.84,326,1620684000"; d="scan'208";a="528623449" Date: Wed, 15 Sep 2021 16:30:48 +0200 From: Samuel Thibault To: Juergen Gross Cc: minios-devel@lists.xenproject.org, xen-devel@lists.xenproject.org, wl@xen.org Subject: Re: [PATCH v3] xenbus: support large messages Message-ID: <20210915143048.lluaekufonze6pic@begin> Mail-Followup-To: Samuel Thibault , Juergen Gross , minios-devel@lists.xenproject.org, xen-devel@lists.xenproject.org, wl@xen.org References: <20210915140815.26736-1-jgross@suse.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20210915140815.26736-1-jgross@suse.com> Organization: I am not organized User-Agent: NeoMutt/20170609 (1.8.3) Re, Juergen Gross, le mer. 15 sept. 2021 16:08:15 +0200, a ecrit: > + while (off != len) > + { > + wait_event(xb_waitq, xenstore_buf->rsp_prod != xenstore_buf->rsp_cons); > + > + prod = xenstore_buf->rsp_prod; > + cons = xenstore_buf->rsp_cons; > + DEBUG("Rsp_cons %d, rsp_prod %d.\n", cons, prod); > + size = min(len - off, prod - cons); > + > + rmb(); /* Make sure data read from ring is ordered with rsp_prod. */ > + memcpy_from_ring(xenstore_buf->rsp, buf + off, > + MASK_XENSTORE_IDX(cons), size); > + off += size; > + xenstore_buf->rsp_cons += size; > + wmb(); > + if (xenstore_buf->rsp_prod - cons >= XENSTORE_RING_SIZE) > + notify_remote_via_evtchn(xenbus_evtchn); > + } Reading it again, I'm still not convinced we covered all cases. There is at least one thing: Linux does virt_rmb(); memcpy(data, src, avail); /* Other side must not see free space until we've copied out */ virt_mb(); intf->rsp_cons += avail; which makes sense to me: we don't want the compiler or anything to reorder the write to rsp_cons respectively to the memcpy. So I believe we also need a full barrier before xenstore_buf->rsp_cons += size; in mini-os. Then there is xenstore_buf->rsp_cons += size; wmb(); if (xenstore_buf->rsp_prod - cons >= XENSTORE_RING_SIZE) notify_remote_via_evtchn(xenbus_evtchn); The Linux code does intf->rsp_cons += avail; /* Implies mb(): other side will see the updated consumer. */ if (intf->rsp_prod - cons >= XENSTORE_RING_SIZE) notify_remote_via_evtchn(xen_store_evtchn); (Note that the "Implies mb()" comment is about the notify_remote_via_evtchn call) I believe the Linux code itself is missing a full barrier here: before reading intf->rsp_prod, we want to make sure that our update of rsp_cons is seen by the producer. Otherwise it may happen that the producer doesn't see the rsp_cons and updates its rsp_prod (filling the ring and going to sleep), and the consumer does not see the rsp_prod update either, and thus we miss a notification and the producer stays stuck. Extremely rare scenario etc. but I believe we do want a full barrier between rsp_cons += and the if, both in mini-os and Linux. Samuel