[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Xen-devel] [RFD] OP-TEE (and probably other TEEs) support



Hello,

On 28 November 2016 at 18:14, Julien Grall <julien.grall@xxxxxxx> wrote:
> On 24/11/16 21:10, Volodymyr Babchuk wrote:
>>
>> Hello all,
>
>
> Hello,
>
>> My name is Volodymyr Babchuk, I'm working on EPAM Systems with bunch
>> of other guys like Artem Mygaiev or Andrii Anisov. My responsibility
>> there is security in embedded systems.
>>
>> I would like to discuss approaches to OP-TEE support in XEN.
>
>
> Thank you for sharing this, I am CC-ing some people who showed interest on
> accessing trusted firmware from the guest.
>
> In the future, please try to CC relevant people (in this case ARM
> maintainers) to avoid any delay on the answer.
Thanks. I never worked with XEN community earlier, so I don't know who is who :)

>
>
>> But first small introduction for those who is not familiar with topic:
>>
>> There are such thing as Security Extensions for ARM cores. It is part
>> of bigger thing - ARM TrustZone. Security extensions allows CPU to
>> work in two states: Normal (Non-Secure) and Secure.
>> Other parts of TrustZone provide hardware protection for peripherals,
>> memory regions, etc. So, when CPU is running in secure state, it have
>> access to all peripherals and memories, also it can forbid access to
>> some of those resources from Normal state. Secure state have own
>> Supervisor and User modes, so we can run OS there.
>> Just to be clear, CPU can run in following modes; user, supervisor,
>> hypervisor, secure monitor, secure user and secure supervisor. Last
>> two basically are the same as normal USR and SVC, but in secure state.
>> And Secure Monitor is special mode, that can switch Secure state to
>> Non-secure state and back. SMC instruction takes into this mode.
>> So, we can run one OS in secure state and one in non-secure state.
>> They will be mostly independent from each other. In normal
>> (non-secure) mode we run, say, Linux. And in Secure mode we run some
>> Trusted OS (or Secure Os). Secure OS provides services like
>> cryptography, secure storage, DRM, virtual SIM, etc. to Normal world
>> OS (it also called "Rich OS". It can be Linux for example.).
>> There are standard created by GlobalPlatform that is called "Trusted
>> Execution Environment" (TEE). It defines requirements and APIs for the
>> Secure OS. There are number number of Secure OSes that tries to be
>> conform with TEE specification and OP-TEE is one of them.
>>
>> CPU is normally running in Non-Secure state. When normal world OS
>> needs some services from Secure OS, it issues SMC instruction with
>> command data. This instruction puts processor into Secure Monitor
>> Mode, where secure monitor code examines request. If it is request to
>> Secure OS, it switches processor to Secure Supervisor mode and passes
>> request to Secure OS. When Secure OS finishes work, it also issues SMC
>> instruction and Secure Monitor switches processor back to non-secure
>> state, returning control to Rich OS.
>> SMC instruction can be used not only to communicate with SecureOS, but
>> also for other services like power control, control of processor
>> cores, etc. ARM wrote document "SMC Calling Convention" which in
>> detail describes how this instruction can be used.  By the way, SMCCC
>> requires to pass Virtual Machine ID in one of the registers during SMC
>> call.
>>
>> Now let's get back to XEN. We want XEN to provide TEE services to Dom0
>> and guests. I can see different approaches to this:
>>
>>  - One of them I call "Emulated TEE". Guests does not have access to
>> real TEE OS. Instead somewhere (in Dom0) we run instance of TEE for
>> each of the Guests. This provides perfect isolation, as TEE instances
>> does not anything about each one. But we will miss all hardware
>> benefits like cryptographic acceleration.
>>  - Another way is to allow guests to work with real TEE running Secure
>> state. In this case TEE should be aware about guests to track shared
>> memories, opened sessions, etc. It requires some careful programming
>> to ensure that guest belongings are isolated from each other. But as
>> reward we are much closer to original TrustZone desing.
>>
>> Personally I prefer second approach. I, even, did small PoC that
>> allows different guests to work with OP-TEE (but not simultaneously!).
>
>
> I agree that the first approach is not good because you can't get advantage
> of the hardware. However, I have some concerns about the second (see below).
>
> Bear in mind that I don't know much about TEE :).
>
>> You can find patches at [1] if you are interested.
>> During working on this PoC I have identified main questions that
>> should be answered:
>>
>> On XEN side:
>> 1. SMC handling in XEN. There are many different SMCs and only portion
>> of them belong to TEE. We need some SMC dispatcher that will route
>> calls to different subsystems. Like PSCI calls to PSCI subsystem, TEE
>> calls to TEE subsystem.
>
>
> So from my understanding of this paragraph, all SMC TEE calls should have a
> guest ID in the command. We don't expect command affecting all TEE. Correct?
Yes. Idea is to trap SMC, alter it, add guest ID (into r7, as SMCCC
says) and then
do real SMC to pass it to TEE.

But I'm not get this: "We don't expect command affecting all TEE".
What did you mean?

>
>>
>> 2. Support for different TEEs. There are OP-TEE, Google Trusty, TI
>> M-Shield... They all work thru SMC, but have different protocols.
>> Currently, we are aimed only to OP-TEE. But we need some generic API
>> in XEN, so support for new TEE can be easily added.
>
> For instance you
Hm?
> Is there any  generic way to detect which TEE is been in used and the
> version?
Yes, according to SMCCC, there call number 0xBF00FF01 that should
return Trusted OS UID.
OP-TEE supports this call. I hope, other TEEs also support it. In this
way we can which TrustedOS is running on host.

>>
>> 3. TEE services. Hypervisor should inform TEE when new guest is
>> created or destroyed, it should tag SMCs to TEE with GuestID, so TEE
>> can isolate guest data on its side.
>>
>> 4. SMC mangling. RichOS communicates with TEE using shared buffers, by
>> providing physical memory addresses. Hypervisor should convert IPAs to
>> PAs.
>
>
> I am actually concerned about this bit. From my understanding, the
> hypervisor would need some knowledge of the SMC.
Yes, it was my first idea - separate subsystem in the hypervisor that
handles SMC calls for different TEEs. This subsystems has a number of
backends. One for each TEE.

> So are the OP-TEE SMC calls fully standardized? By that I mean they will not
> change across version?
No, they are not standardized and they can change in the future.
OP-TEE tries to be backward-compatible, though. So hypervisor can drop
unknown capability flags in SMC call GET_CAPABILITIES. In this way it
can ensure that guest will use only  APIs that are known by
hypervisor.

> How about other TEE?
I can't say for sure. But I think, situation is the same as with OP-TEE

> If not, then it might be worth to consider a 3rd solution where the TEE SMC
> calls are forwarded to a specific domain handling the SMC on behalf of the
> guests. This would allow to upgrade the TEE layer without having to upgrade
> the hypervisor.
Yes, this is good idea. How this can look? I imagine following flow:
Hypervisor traps SMC, uses event channel to pass request to Dom0. Some
userspace daemon receives it, maps pages with request data, alters is
(e.g. by replacing IPAs with PAs), sends request to hypervisor to
issue real SMC, then alters response and only then returns data back
to guest.

Is this even possible with current APIs available to dom0?

I can see only one benefit there - this code will be not in
hypervisor. And there are number of drawbacks:

Stability: if this userspace demon will crash or get killed by, say,
OOM, we will lose information about all opened sessions, mapped shared
buffers, etc.That would be complete disaster.
Performance: how big will be latency introduced by switching between
hypervisor, dom0 SVC and USR modes? I have seen use case where TEE is
part of video playback pipe (it decodes DRM media).
There also can be questions about security, but Dom0 in any case can
access any memory from any guest.

But I really like the idea, because I don't want to mess with
hypervisor when I don't need to. So, how do you think, how it will
affect performance?

>
>> Currently I'm rewriting parts of OP-TEE to make it support arbitrary
>> buffers originated from RichOS.
>>
>> 5. Events from TEE. This is hard topic. Sometimes OP-TEE needs some
>> services from RichOS. For example it wants Linux to service pending
>> IRQ request, or allocate portion of shared memory, or lock calling
>> thread, etc. This is called "RPC request". To do RPC request OP-TEE
>> initiates return to Normal World, but it sets special return code to
>> indicate that Linux should do some job for OP-TEE. When Linux finishes
>> work, it initiates another SMC with code like "I have finished RPC
>> request" and OP-TEE resumes its execution.
>> OP-TEE mutexes create problem there. We don't  want to sleep in secure
>> state, so when OP-TEE thread gets blocked on a mutex, it issues RPC
>> request that asks calling thread to wait on wait queue. When mutex
>> owner unlocks it, that another thread also issues RPC to wake up first
>> thread.
>> This works perfectly when there are one OS (or one guest). But when
>> there are many, it is possible that request from one guest blocks
>> another guest. That another guest will wait on wait queue, but there
>> will be no one, who can wake it up. So we need another mechanism to
>> wake up sleeping threads. Obvious candidate is IPI. There are 8
>> non-secure IPIs that all are used by linux kernel. By there are also 8
>> secure IPIs. I think OP-TEE can use one of those to deliver events to
>> Normal World. But this will require changes to OP-TEE, XEN and Linux
>> kernel.
>
>
> Before giving suggestion here. I would like to confirm that by IPI you mean
> SGI, right?
Yes. Kernel calls them IPI, so did I.
If I remember ARM GIC TRM well, it recommends to give 8 SGIs to Normal
World and 8 to Secure World.
Hypervisor can use one of secure SGIs to deliver events to guests.
But, actually, this feature will be needed only virtualized
environment, so I think we can use XEN events there.
Jens, can you please comment on this? I can't imagine use case when
OP-TEE need to send SGI to normal world, when there are no
virtualization. But maybe I'm missing something?

>
>>
>> 6. Some mechanism to control which guests can work with TEE. At this
>> time I have no idea how this should work.
>
>
> Probably a toolstack option "TEE enabled".
So there will be some additional flag in guest descriptor structure?

>
>>
>> On OP-TEE side:
>> 1. Shared memory redesign which is almost complete.
>> 2. Implement new SMCs  from hypervsiror to TEE to track VM lifecycle.
>> 3. Track VM IDs to isolated VM data.
>> 4. RPCs to sleeping guests.
>>
>> That is basically all what I currently have on my mind. It will be
>> great if all interested sides will share their opinions, requirements,
>> thoughts on this.
>
>
> Cheers,
>
>> [1] https://github.com/lorc/xen/commits/staging-4.7 - Xen + OP-TEE PoC
>>
>
> [2]
> http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.den0028a/index.html
Yep, thank you for link to ARM SMCCC.
>
> --
> Julien Grall



-- 
WBR Volodymyr Babchuk aka lorc [+380976646013]
mailto: vlad.babchuk@xxxxxxxxx

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
https://lists.xen.org/xen-devel

 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.