[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Xen-devel] Should PV frontend drivers trust the backends?



On 04/26/2018 11:16 AM, Paul Durrant wrote:
-----Original Message-----
From: Oleksandr Andrushchenko [mailto:andr2000@xxxxxxxxx]
Sent: 26 April 2018 07:00
To: Paul Durrant <Paul.Durrant@xxxxxxxxxx>; 'Juergen Gross'
<jgross@xxxxxxxx>; xen-devel <xen-devel@xxxxxxxxxxxxxxxxxxxx>
Subject: Re: [Xen-devel] Should PV frontend drivers trust the backends?

On 04/25/2018 04:47 PM, Paul Durrant wrote:
-----Original Message-----
From: Xen-devel [mailto:xen-devel-bounces@xxxxxxxxxxxxxxxxxxxx] On
Behalf
Of Juergen Gross
Sent: 25 April 2018 13:43
To: xen-devel <xen-devel@xxxxxxxxxxxxxxxxxxxx>
Subject: [Xen-devel] Should PV frontend drivers trust the backends?

This is a followup of a discussion on IRC:

The main question of the discussion was: "Should frontend drivers
trust their backends not doing malicious actions?"

This IMO includes:

1. The data put by the backend on the ring page(s) is sane and
     consistent, meaning that e.g. the response producer index is always
     ahead of the consumer index.

2. Response data won't be modified by the backend after the producer
     index has been incremented signaling the response is valid.

3. Response data is sane, e.g. an I/O data length is not larger than
     the buffer originally was.

4. When a response has been sent all grants belonging to the request
     have been unmapped again by the backend, meaning that the frontend
     can assume the grants can be removed without conflict.

Today most frontend drivers (at least in the Linux kernel) seem to
assume all of the above is true (there are some exceptions, but never
for all items):

- they don't check sanity of ring index values
- they don't copy response data into local memory before looking at it
- they don't verify returned data length (or do so via BUG_ON())
- they BUG() in case of a conflict when trying to remove a grant

So the basic question is: should all Linux frontend drivers be modified
in order to be able to tolerate buggy or malicious backends? Or is the
list of trust above fine?

IMO even in case the frontends do trust the backends to behave sane this
doesn't mean driver domains don't make sense. Driver domains still make
a Xen host more robust as they e.g. protect the host against driver
failures normally leading to a crash of dom0.

I see the general question as being analogous to 'should a Linux device
driver trust its hardware' and I think the answer for a general purpose OS like
linux is 'yes'.
Now, having worked on fault tolerant systems in a past life, there are
definitely cases where you want your OS not to implicitly trust its peripheral
hardware and hence special device drivers are used.
So what do you do if counters provided by the untrusted HW are ok
and the payload is not?
Well, that depends on whether there is actually any way to verify the payload 
in a driver. Whatever layer in the system is responsible for the data needs to 
verify its integrity in a fault tolerant system. Generally the driver can only 
attempt to verify that it's hardware is working as expect and quiesce it if 
not. For that reason, in the systems I worked on, the driver had the ability to 
control FETs that disconnected peripheral h/w from the PCI bus.

I think the same would apply for virtual machines in situations where a
driver domain is not wholly controlled by a host administrator or is not
trusted to the same extent as dom0 for other reasons; i.e. they should have
specialist frontends.
I believe we might be able to express some common strategy for the
frontends.
I do understand though that it all needs to be decided on case by case
basis,
but common things could still be there, e.g. if prod/cons counters are
not in sync
what a frontend needs to do:
   - should it keep trying to get in sync - might be a bad idea as the
req/resp data
     may already become inconsistent (net can probably survive, but not
block)
   - should it tear down the connection with the backend - this may
render in the whole
     system instability, e.g. imagine you tear down a "/" block device
   - should it BUG_ON and die
To me the second option (tear down the connection) seems to be
more reasonable, although it can still render the guest unusable, but at
least it
gives a chance for the guest to recover in a proper way

Absolutely that can be done and it's certainly a good idea to be somewhat 
defensive but, as you say, it's quite likely that the PV pair is part of a 
critical subsystem for the guest and so a BUG() may well be the best option to 
make sure that the inevitable guest crash actually contains pertinent 
information.

And, if my assumption is correct, we still do trust the contents of the
requests
and responses, e.g. the payload is still trusted.
Why should the payload be any more trusted than the content of the shared ring? 
They are both shared with the backend and therefore can be corrupted to the 
same extent.
This is exactly my point: if we only try to protect from inconsistent prod/cons then this protection is still incomplete as the payload may be the source of failure.
This also questions
the approach,
e.g. if we don't trust backend's counters, then why do we trust the
payload it sends?
And there is no obvious way to check the payload integrity.
Quite, as I said above.

So, either we trust the backend and accept the risks or we need to
develop some
complex approach to address the above.

Indeed, hence my position that in the general case it's not a security issue 
for a frontend to trust its backend.
+1
I tend to trust the backends in general, but some critical PV pairs
*may* implement some logic to detect the corruption
Cheers,

   Paul

Thank you,
Oleksandr
    Paul

Juergen

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel
_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel

 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.