RE: Xen FuSa meeting tomorrow Tue 17 November

[Stefano Stabellini <stefano.stabellini@xxxxxxxxxx>]

We could certainly consider a commercial tool, we don't have a specific
constraint in that respect. Xen Project has a budget that could
potentially be used to cover the costs.

However, the issue is integration as part of the CI-loop. We need to run
it locally inside a server-side (no GUI) Linux container created on
purpose for every commit. So that it would be fully integrated in the
CI-loop: we could make the pipeline fail if MISRAC tests don't pass, and
we could even run the tests on a patch series before committing it.

Typically commercial tools cannot easily be deployed automatically on a
Linux container for licensing reasons. They require a human to accept a
licensing agreement, which cannot be done automatically in a CI-loop.

If the commercial tool is available "as a service", and its services can
be invoked automatically from the headless Linux container in the
CI-loop, that would work and might solve the issue of the licensing
agreement. Note that Coverity is available "as a service" but I don't
think it can be invoked automatically from a headless Linux container:
typically a human needs to login the web interface to get the results.
If a human has to login to know if tests pass, then we cannot
automatically block the CI-loop in case of failures.


In short, using a commercial tool is challenging, unless we find one
that could be used by a bot (not a human) both for triggering tests and
also to get the results back.



On Wed, 18 Nov 2020, David Ward wrote:
> I would agree that commercial tools are likely to have better and wider 
> coverage of MISRA C rules, hence why I commented yesterday that
> it’s important to evaluate how well the tools check rules, not just that they 
> claim to check a certain number of rules.
> 
>  
> 
>  
> 
> Best regards
> 
>  
> 
> David
> 
>  
> 
> From: Fusa-sig <fusa-sig-bounces@xxxxxxxxxxxxxxxxxxxx> On Behalf Of Francesco 
> Brancati
> Sent: 18 November 2020 11:48
> To: pserwa@xxxxxxxxx
> Cc: mszczepankiewicz@xxxxxxxxx; fusa-sig@xxxxxxxxxxxxxxxxxxxx
> Subject: Re: Xen FuSa meeting tomorrow Tue 17 November
> 
>  
> 
> Dear Piotr,
> 
> of course commercial tools offer much more features. Do you think their 
> licenses would allow the usage server side to support a community
> based development?
> 
> I guess we were focusing on free tools to overcome this issue.
> 
> @Stefano do we have any constraints in adopting a commercial tool?
> 
> Regards,
> 
> Francesco.
> 
> Il 18/11/2020 11:39, Piotr Serwa ha scritto:
> 
>       Hi Francesco,
> 
>        
> 
>       Why not taking a commercial tool, with a full MISRA 2012 support? Free 
> tools are far below the needs, I think. Commercial tools
>       typically come with qualification kits and safety certificates. I would 
> recommend investigating Axivion, Parasoft C test ,
>       Helix QAC, LDRA TBvision or Absint rule checker. Maybe you can 
> negotiate a free usage as Xen is open source. Coverity is doing
>       this approach (I mean free scan of Linux code).
> 
>        
> 
>       Regards
> 
>       Piotr
> 
>        
> 
>       From: Fusa-sig <fusa-sig-bounces@xxxxxxxxxxxxxxxxxxxx> On Behalf Of 
> Francesco Brancati
>       Sent: Tuesday, 17 November 2020 15:14
>       To: fusa-sig@xxxxxxxxxxxxxxxxxxxx
>       Cc: Lorenzo Falai <lorenzo.falai@xxxxxxxxxxxxx>
>       Subject: Re: Xen FuSa meeting tomorrow Tue 17 November
> 
>  
> 
> Dear all,
> 
> please find attached the result of our investigations on sonarQube and 
> alternatives.
> 
> the outcome can be summarized in this way:
> 
> SonarQube (sonarcloud) could be a valid tool for our needs because it 
> supports command line and serverside precommit operations and
> should support inline justification of violations. Unfortunately (as pointed 
> out also by Artem) MirsaC support is really limited.
> 
> we provide a list of alternatives (cppcheck is in the list) but support to 
> serverside precommit checks must be further investigated.
> 
> talk to you later,
> 
> Francesco and Lorenzo.
> 
> Il 17/11/2020 14:17, Bertrand Marquis ha scritto:
> 
> Hi Artem,
> 
>  
> 
> Here after you will find the current status I got on your tickets.
> 
>  
> 
> On 17 Nov 2020, at 11:24, Artem Mygaiev <Artem_Mygaiev@xxxxxxxx> wrote:
> 
>  
> 
> Hello all
> 
>  
> 
> Few notes on my AIs to discuss today:
> 
>  
> 
> - Sonar, unfortunately, support a very limited set of MISRA rules 
> (https://rules.sonarsource.com/c/tag/misra-c2012) so it is not suitable f
> or us. OTOH, cppcheck has a full set of MISRA-C-2012 143 rules supported via 
> public plugin (without publishing rules text, only giving an I
> D) https://github.com/danmar/cppcheck/blob/main/addons/misra.py. I think this 
> should be a great starting point so will try it now.
> 
>  
> 
> - We have created a migrated armclang Xen branch on top of current staging, 
> but I cannot check it without license for Arm DS safety compile
> r, unfortunately. Also I have re-checked Arm Safety Compiler issues after 
> migration to new support system (except one), here's the list:
> 
>  
> 
> I am working on that and will keep you informed.
> 
>  
> 
> CAS-138402-Y0Y9C3 --- 00195992
> 
>  
> 
> This is not considered a bug but a feature request and for now this is 
> considered.
> 
>  
> 
> CAS-137352-T7F4V1 --- 00192196
> 
>  
> 
> This a known limitation and there is a workaround for it.
> 
>  
> 
> CAS-138292-L5S0V0 --- cannot find it yet
> 
>  
> 
> There is a workaround for current compiler and this will be fixed in the new 
> compiler scheduled to be released around H2 2021 
> 
>  
> 
> CAS-137357-Z7W3B8 --- 00182044
> 
>  
> 
> This has been resolved in 6.6.4 version of the compiler.
> 
>  
> 
> CAS-137359-V7G6W6 --- 00118170
> 
>  
> 
> This is a limitation of the fromelf tool and is considered a feature request. 
> It has not been considered yet.
> 
>  
> 
> BR
> 
> Bertrand
> 
>  
> 
>  
> 
> BR,
> 
> -- Artem
> 
>  
> 
> -----Original Message-----
> 
> From: Fusa-sig <fusa-sig-bounces@xxxxxxxxxxxxxxxxxxxx> On Behalf Of Stefano 
> Stabellini
> 
> Sent: Tuesday, 17 November, 2020 00:49
> 
> To: Stefano Stabellini <stefano.stabellini@xxxxxxxxxx>
> 
> Cc: fusa-sig@xxxxxxxxxxxxxxxxxxxx; stefanos@xxxxxxxxxx
> 
> Subject: Xen FuSa meeting tomorrow Tue 17 November
> 
>  
> 
> Hi all,
> 
>  
> 
> I would like to remind you that tomorrow it is time for our Xen FuSa SIG 
> meeting. There are a number of outstanding actions, see below. Als
> o, David Ward kindly volunteered to present on the subject of the MISRA 
> Compliance 2020 document, which is extremely relevant as it provide
> s a framework to manage deviations.
> 
>  
> 
> Cheers,
> 
>  
> 
> Stefano
> 
>  
> 
>  
> 
>  
> 
> On Tue, 3 Nov 2020, Stefano Stabellini wrote:
> 
> Hi all,
> 
>  
> 
> These are the minutes of today's FuSa meeting.  Look for "ACTION" in 
> 
> the test to find the ACTION items.
> 
>  
> 
> Cheers,
> 
>  
> 
> Stefano
> 
>  
> 
>  
> 
>  
> 
> # Build Xen with ARMClang
> 
>  
> 
> Bertrand: ARM will internally build Xen with ARMClang to validate 
> 
> ARMClang against Xen. It is going to start in the next couple of months.
> 
>  
> 
> Artem: I have opened a bunch of issues against ARMClang. What is the 
> 
> status?
> 
>  
> 
> Bertrand: will check
> 
>  
> 
> ACTION(Bertrand): ARM to let us know when issues are going to be fixed 
> 
> and in which version of the compiler.
> 
>  
> 
> ACTION(Artem): send the ARMClang series for Xen again rebased on 
> 
> staging
> 
>  
> 
>  
> 
> # Resiltech presentation on MISRAC
> 
>  
> 
> First identify set of rules we have to comply to MISRAC. A subset of 
> 
> MISRAC, but which one?  Some rules are mandatory, some others are 
> 
> advisory?
> 
>  
> 
> Who is responsible for deciding which rules are mandatory (R1)? It is 
> 
> important to have the safety experts involved.
> 
>  
> 
> Once we identify the R1 rules, let's use static analysis to check for 
> 
> violations. For instance SonarCloud.
> 
>  
> 
> We need to device who is responsible for fixing the violations, and 
> 
> what happens when developers say that the solution is worse than the 
> 
> original code.  There is a need for a final pass by a safety expert 
> 
> after the developer's analysis. In case the safety expert team 
> 
> identifies that the justification cannot be accepted the code has to be fixed.
> 
>  
> 
>  
> 
> We need a tool able to process justifications for MISRAC violations 
> 
> inline with the code. It is important to maintain MISRAC violation 
> 
> justifications in sync with the code. Is there a tool that can do that 
> 
> today?
> 
>  
> 
> If the tool doesn't support it, we could add scripting to it, so that 
> 
> we could extract the justifications from the comments and populate the 
> 
> tool's database ourselves.
> 
>  
> 
>  
> 
> ACTION(Artem): work with Sonar and see how it handles justifications
> 
> ACTION(Francesco): do an analysis on the tools and justification 
> 
> handling
> 
>  
> 
>  
> 
> ACTION(Stefano): MISRAC justifiaction as incode comments, is it viable
> 
>                 from a community perspective? Start the discussion.
> 
> ACTION(Stefano): Diagram to describe the new contributor process
> 
>                 workflow
> 
>  
> 
>  
> 
> MISRAC document to provide a framework to manage deviations 
> 
> https://urldefense.com/v3/__https://www.misra.org.uk/forum/viewtopic.p
> 
> hp?f=241&t=1842__;!!GF_29dbcQIUBPA!lHt1TEb2koDpfOmJwNiV5B-0OQc3sCB429nx6W4sPDUdbXiz2C5WGp3Wh-tSKeddKg$
>  [misra[.]org[.]uk] ACTION(David Ward
> ): do a presentation on the topic during the next
> 
>                    FuSa meeting.
> 
>  
> 
>  
> 
> --
> 
> Francesco Brancati
> Innovation Manager and SW Solutions Expert
> Email: francesco.brancati@xxxxxxxxxxxxx
> Phone: +39 0587 21 24 65 (internal number: 104)
> Mobile: +39 333 48 52 041
> Skype: francesco.brancati
> [IMAGE]
> www.resiltech.com
> 
> 
> ___________________________________________________________________________________________________________________________________________
> 
> 
> This e-mail and related attachments are property of ResilTech S.r.l. and may 
> also be privileged. If you are not the intended
> recipient please delete it from your system and notify the sender.
> You shouldn't copy it or use it for any purpose nor disclose or distribute 
> its contents to any other person.
> Questa e-mail e tutti i suoi allegati sono proprietà di ResilTech S.r.l. e 
> possono essere soggetti a restrizioni legali. Se non siete
> l'effettivo destinatario o avete ricevuto il messaggio per errore siete 
> pregati di cancellarlo dal vostro sistema e di avvisare il
> mittente. E' vietata la duplicazione, l'uso a qualsiasi titolo, la 
> divulgazione o la distribuzione dei contenuti di questa e-mail a
> qualunque altro soggetto.
> 
> --
> 
> Francesco Brancati
> Innovation Manager and SW Solutions Expert
> Email: francesco.brancati@xxxxxxxxxxxxx
> Phone: +39 0587 21 24 65 (internal number: 104)
> Mobile: +39 333 48 52 041
> Skype: francesco.brancati
> [IMAGE]
> www.resiltech.com
> 
> 
> ___________________________________________________________________________________________________________________________________________
> 
> 
> This e-mail and related attachments are property of ResilTech S.r.l. and may 
> also be privileged. If you are not the intended recipient
> please delete it from your system and notify the sender.
> You shouldn't copy it or use it for any purpose nor disclose or distribute 
> its contents to any other person.
> Questa e-mail e tutti i suoi allegati sono proprietà di ResilTech S.r.l. e 
> possono essere soggetti a restrizioni legali. Se non siete
> l'effettivo destinatario o avete ricevuto il messaggio per errore siete 
> pregati di cancellarlo dal vostro sistema e di avvisare il
> mittente. E' vietata la duplicazione, l'uso a qualsiasi titolo, la 
> divulgazione o la distribuzione dei contenuti di questa e-mail a
> qualunque altro soggetto.
> 
> 
> HORIBA MIRA Ltd
> 
> Watling Street, Nuneaton, Warwickshire, CV10 0TU, England
> Registered in England and Wales No. 9626352
> VAT Registration  GB 100 1464 84
> 
> This email and any files transmitted with it are confidential and intended 
> solely for the use of the individual or entity to whom they are
> addressed. If you are not the named addressee you should not disseminate, 
> distribute or copy this e-mail. Please notify the sender
> immediately by e-mail if you have received this e-mail by mistake and delete 
> this e-mail from your system. If you are not the intended
> recipient you are notified that disclosing, copying, distributing or taking 
> any action in reliance on the contents of this information is
> strictly prohibited.
> 
>