FW: Xen FuSa meeting tomorrow Tue 17 November

[Artem Mygaiev <Artem_Mygaiev@xxxxxxxx>]

All,
Re-sending this to bump the thread on cppcheck usage

 -- Artem

-----Original Message-----
From: Artem Mygaiev 
Sent: вторник, 1 декабря 2020 г. 00:28
To: Norbert Manthey <nmanthey@xxxxxxxxx>; Bertrand Marquis 
<Bertrand.Marquis@xxxxxxx>; Stefano Stabellini <stefano.stabellini@xxxxxxxxxx>
Cc: Julien Grall <julien@xxxxxxx>; David Ward <david.ward@xxxxxxxxxxxxxxx>; 
Francesco Brancati <francesco.brancati@xxxxxxxxxxxxx>; pserwa@xxxxxxxxx; 
mszczepankiewicz@xxxxxxxxx; fusa-sig@xxxxxxxxxxxxxxxxxxxx
Subject: RE: Xen FuSa meeting tomorrow Tue 17 November

Hello all

I have played with cppcheck MISRA-2012 implementation a little, please see some 
preliminary results below

Tools:
 * https://github.com/danmar/cppcheck/releases/tag/2.2 built with default 
settings
 * Linaro's gcc-arm-10.2-2020.11-x86_64-aarch64-none-linux-gnu

Experimented with current Xen staging tree & Stefano's tiny64_defconfigxen

Commands to check only xen folder excluding userspace tools and kconfig:
$ export 
PATH=$PATH:~/downloads/gcc-arm-10.2-2020.11-x86_64-aarch64-none-linux-gnu/bin/
$ XEN_TARGET_ARCH=arm64 CROSS_COMPILE=aarch64-none-linux-gnu- make 
tiny64_defconfigxen
$ bear --exclude xen/tools make xen -j$NUMCPUS XEN_TARGET_ARCH=arm64 
CROSS_COMPILE=aarch64-none-linux-gnu-
$ ../cppcheck/cppcheck --include=xen/include/generated/autoconf.h 
--project=compile_commands.json --dump
$ find -name "*.dump" -print0 | xargs -0 python ../cppcheck/addons/misra.py

Results summary:
MISRA rules violations found:
        Undefined: 6134

MISRA rules violated:
        misra-c2012-2.7 (-): 10
        misra-c2012-3.1 (-): 42
        misra-c2012-5.5 (-): 4
        misra-c2012-7.3 (-): 52
        misra-c2012-8.11 (-): 20
        misra-c2012-9.5 (-): 4
        misra-c2012-10.1 (-): 47
        misra-c2012-10.4 (-): 195
        misra-c2012-10.6 (-): 1
        misra-c2012-11.3 (-): 10
        misra-c2012-11.4 (-): 26
        misra-c2012-11.5 (-): 131
        misra-c2012-11.6 (-): 8
        misra-c2012-11.8 (-): 7
        misra-c2012-12.1 (-): 36
        misra-c2012-12.3 (-): 83
        misra-c2012-13.3 (-): 11
        misra-c2012-13.4 (-): 56
        misra-c2012-13.5 (-): 13
        misra-c2012-14.2 (-): 2
        misra-c2012-14.4 (-): 96
        misra-c2012-15.1 (-): 7
        misra-c2012-15.5 (-): 215
        misra-c2012-15.6 (-): 4405
        misra-c2012-15.7 (-): 6
        misra-c2012-16.3 (-): 132
        misra-c2012-16.4 (-): 2
        misra-c2012-17.2 (-): 3
        misra-c2012-17.7 (-): 28
        misra-c2012-17.8 (-): 81
        misra-c2012-18.4 (-): 85
        misra-c2012-18.7 (-): 1
        misra-c2012-19.2 (-): 46
        misra-c2012-20.1 (-): 4
        misra-c2012-20.5 (-): 5
        misra-c2012-20.7 (-): 56
        misra-c2012-20.10 (-): 22
        misra-c2012-21.1 (-): 180
        misra-c2012-21.9 (-): 2

Would be interesting to compare with other tools' results.

Best regards,
 -- Artem

-----Original Message-----
From: Norbert Manthey <nmanthey@xxxxxxxxx> 
Sent: Friday, 20 November, 2020 10:46
To: Bertrand Marquis <Bertrand.Marquis@xxxxxxx>; Stefano Stabellini 
<stefano.stabellini@xxxxxxxxxx>
Cc: Artem Mygaiev <Artem_Mygaiev@xxxxxxxx>; Julien Grall <julien@xxxxxxx>; 
David Ward <david.ward@xxxxxxxxxxxxxxx>; Francesco Brancati 
<francesco.brancati@xxxxxxxxxxxxx>; pserwa@xxxxxxxxx; 
mszczepankiewicz@xxxxxxxxx; fusa-sig@xxxxxxxxxxxxxxxxxxxx
Subject: Re: Xen FuSa meeting tomorrow Tue 17 November

Hi all,

On 11/20/20 9:33 AM, Bertrand Marquis wrote:
> CAUTION: This email originated from outside of the organization. Do not click 
> links or open attachments unless you can confirm the sender and know the 
> content is safe.
>
>
>
> Hi Stefano,
>
>> On 19 Nov 2020, at 20:20, Stefano Stabellini <stefano.stabellini@xxxxxxxxxx> 
>> wrote:
>>
>> Hi Artem, Francesco, and all,
>>
>> I think we should come up with a shortlist of potential candidates. 
>> As we discussed during the last two calls, aside from price, the 
>> important parameters are:
>>
>> - completeness of the MISRAC checks
>> - ability of being called automatically as part of the CI-loop
>>    - availability of a remote API
>>    - OR a license that allows for headless invocations
>> - ability to take as input a list of deviations maintained together 
>> with  the source code (so that can we have different deviations for 
>> each Xen
>>  branch)
>>
>> Does this set of criteria seem reasonable?
> I think an important one is the ability to share the results of the tool.
> If the results are protected by some kind of license we will end up 
> having to fix all problems without an opportunity to share the tool 
> results on the mailing list for example if the CI loop can (and I 
> think it is critical that it does) be executed on pushes to the mailing list 
> before they are merged in staging.

We can run the Coverity analysis locally, so that would fix the "run on mailing 
list contributions" requirement. Next, we process the output of the command, 
and extract relevant information (introduced defects, their trace, 
fixed/dropped defects, ...) and format that into notifications of the relevant 
style (we have scripts for Coverity similarly to what we did for Infer, 
Fortify, CBMC, CppCheck, ... in 
https://urldefense.com/v3/__https://github.com/awslabs/one-line-scan__;!!GF_29dbcQIUBPA!iW5BfqLOXh9Q0qkIBYwp4F8hOzmxxvo6yHFb3V2Oq2vt_8DWXmMFnibdKsRMxx7uTw$
 [github[.]com]).

I am not a license expert, so I cannot tell what would be required to be 
allowed to share the results of email contributions back. This requirement 
should certainly be brought up early to select an appropriate tool.

Best,
Norbert

>
> Cheers
> Bertrand
>
>>
>> On Thu, 19 Nov 2020, Artem Mygaiev wrote:
>>> Hello all
>>>
>>> Using Coverity for MISRA was considered some time ago at the very first 
>>> days of FuSa SIG. Coverity indeed supports MISRA but there's a catch: Open 
>>> Source version of Coverity (Coverity Scan) does not support MISRA.
>>>
>>> BR,
>>> -- Artem
>>>
>>> -----Original Message-----
>>> From: Fusa-sig <fusa-sig-bounces@xxxxxxxxxxxxxxxxxxxx> On Behalf Of 
>>> Julien Grall
>>> Sent: четверг, 19 ноября 2020 г. 12:27
>>> To: Stefano Stabellini <stefano.stabellini@xxxxxxxxxx>
>>> Cc: David Ward <david.ward@xxxxxxxxxxxxxxx>; Francesco Brancati 
>>> <francesco.brancati@xxxxxxxxxxxxx>; pserwa@xxxxxxxxx; 
>>> mszczepankiewicz@xxxxxxxxx; fusa-sig@xxxxxxxxxxxxxxxxxxxx; Manthey, 
>>> Norbert <nmanthey@xxxxxxxxx>
>>> Subject: Re: Xen FuSa meeting tomorrow Tue 17 November
>>>
>>> (+Norbert)
>>>
>>> On 18/11/2020 19:15, Julien Grall wrote:
>>>>>> So you have enough data from coverity to provide some meaningful 
>>>>>> information.
>>>>>> :) I don't know whether this is based on just parsing e-mails or 
>>>>>> tools coverity may provide in a paid version.
>>>>> Excellent. Yeah, that is the kind of level of integration that we need.
>>>> I have asked the person in charge if he could provide more details 
>>>> how this was setup and the complexity to get it working. I will let 
>>>> you know when I have an answer.
>>> I have CCed Norbert who worked on the integration of coverity. @Stefano, 
>>> feel free to ask more details on the setup.
>>>
>>> Cheers,
>>>
>>> --
>>> Julien Grall
>>>



Amazon Development Center Germany GmbH
Krausenstr. 38
10117 Berlin
Geschaeftsfuehrung: Christian Schlaeger, Jonathan Weiss Eingetragen am 
Amtsgericht Charlottenburg unter HRB 149173 B
Sitz: Berlin
Ust-ID: DE 289 237 879