Re: [Xen-devel] [PATCH 1/1] cameraif: add ABI for para-virtual camera

[Oleksandr Andrushchenko <andr2000@xxxxxxxxx>]

On 09/03/2018 06:25 PM, Hans Verkuil wrote:
> Hi Oleksandr,
>
> On 09/03/2018 12:16 PM, Oleksandr Andrushchenko wrote:
> > On 08/21/2018 08:54 AM, Oleksandr Andrushchenko wrote:
> > > On 08/14/2018 11:30 AM, Juergen Gross wrote:
> > > > On 31/07/18 11:31, Oleksandr Andrushchenko wrote:
> > > > > From: Oleksandr Andrushchenko <oleksandr_andrushchenko@xxxxxxxx>
> > > > >
> > > > > This is the ABI for the two halves of a para-virtualized
> > > > > camera driver which extends Xen's reach multimedia capabilities even
> > > > > farther enabling it for video conferencing, In-Vehicle Infotainment,
> > > > > high definition maps etc.
> > > > >
> > > > > The initial goal is to support most needed functionality with the
> > > > > final idea to make it possible to extend the protocol if need be:
> > > > >
> > > > > 1. Provide means for base virtual device configuration:
> > > > >    - pixel formats
> > > > >    - resolutions
> > > > >    - frame rates
> > > > > 2. Support basic camera controls:
> > > > >    - contrast
> > > > >    - brightness
> > > > >    - hue
> > > > >    - saturation
> > > > > 3. Support streaming control
> > > > > 4. Support zero-copying use-cases
> > > > >
> > > > > Signed-off-by: Oleksandr Andrushchenko
> > > > > <oleksandr_andrushchenko@xxxxxxxx>
> > > > Some style issues below...
> > > Will fix all the below, thank you!
> > >
> > > I would like to draw some attention of the Linux/V4L community to this
> > > protocol as the plan is that once it is accepted for Xen we plan to
> > > upstream a Linux camera front-end kernel driver which will be based
> > > on this work and will be a V4L2 device driver (this is why I have sent
> > > this patch not only to Xen, but to the corresponding Linux mailing list
> > > as well)
> > ping
> Sorry, this got buried in my mailbox, I only came across it today. I'll try
> to review this this week, if not, just ping me again.
Thank you for your time
> I had one high-level question, though:
>
> What types of hardware do you intend to target? This initial version targets
> (very) simple webcams, but what about HDMI or SDTV receivers? Or hardware
> codecs? Or complex embedded video pipelines?
>
> In other words, where are you planning to draw the line?
>
> Even with just simple cameras there is a difference between regular UVC
> webcams and cameras used with embedded systems: for the latter you often
> need to provide more control w.r.t. white-balancing etc., things that a
> UVC webcam will generally do for you in the webcam's firmware.
The use-cases we want to implement are mostly in automotive/embedded domain,
so there are many performance restrictions apply.
We are not targeting virtualizing very complex hardware and have no 
intention
to make a 1:1 mapping of the real hardware: for that one can pass-through
a real HW device to a virtual machine (VM). The goal is to share a single
camera device to multiple virtual machines, no codecs, receivers etc.

Controlling the same HW device from different VMs doesn't look feasible:
what if the same control is set to different values from different VMs?
Of course, this can be achieved if the corresponding backend can 
post-process
original camera image with GPU, for example, thus applying different filters
for different VMs effectively emulating camera controls.
But this requires additional CPU/GPU power which we try to avoid.

System partitioning (camera and controls assignment) is done at 
configuration
time (remember we are in automotive/embedded world, so most of the time 
the set
of VMs requiring cameras is known at this stage and the configuration 
remains
static at run-time). So, when para-virtualized (PV) approach is used then we
only implement very basic controls (those found in the protocol), so one can
assign set of controls (all or some) to one of the VMs (main or mission 
critical
VM or whatever) allowing that VM to adjusts those for all VMs at once. 
For other
VMs think of it as firmware implemented adjustment. And the backend still
controls the rest of the controls of the real HW camera you mention.

Just an example of automotive use-case (we can imagine many more):
1. Driver Domain - owns real camera HW and runs the camera backend.
   Uses camera output for mission critical tasks, e.g. parking assistance.
2. In-Vehicle Infotainment domain - uses PV camera for infotainment 
purposes,
   e.g. taking pictures while in motion.
3. Navigation domain - uses PV camera for high definition maps

Hope, this helps understanding the possible uses of the proposed 
protocol, its
intention and restrictions.

> Regards,
>
>         Hans
Thank you,
Oleksandr

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel