Re: [PATCH 2/2] xen/misra: diff-report.py: add report patching feature

[Luca Fancellu <Luca.Fancellu@xxxxxxx>]

> On 17 May 2023, at 02:33, Stefano Stabellini <sstabellini@xxxxxxxxxx> wrote:
> 
> On Thu, 4 May 2023, Luca Fancellu wrote:
>> Add a feature to the diff-report.py script that improves the comparison
>> between two analysis report, one from a baseline codebase and the other
>> from the changes applied to the baseline.
>> 
>> The comparison between reports of different codebase is an issue because
>> entries in the baseline could have been moved in position due to addition
>> or deletion of unrelated lines or can disappear because of deletion of
>> the interested line, making the comparison between two revisions of the
>> code harder.
>> 
>> Having a baseline report, a report of the codebase with the changes
>> called "new report" and a git diff format file that describes the
>> changes happened to the code from the baseline, this feature can
>> understand which entries from the baseline report are deleted or shifted
>> in position due to changes to unrelated lines and can modify them as
>> they will appear in the "new report".
>> 
>> Having the "patched baseline" and the "new report", now it's simple
>> to make the diff between them and print only the entry that are new.
>> 
>> Signed-off-by: Luca Fancellu <luca.fancellu@xxxxxxx>
> 
> This is an amazing work!! Thanks Luca!
> 
> I am having issues trying the new patch feature. After applying this
> patch I get:
> 
> sstabellini@ubuntu-linux-20-04-desktop:/local/repos/xen-upstream/xen$ 
> ./scripts/diff-report.py
> Traceback (most recent call last):
>  File "./scripts/diff-report.py", line 5, in <module>
>    from xen_analysis.diff_tool.debug import Debug
>  File 
> "/local/repos/xen-upstream/xen/scripts/xen_analysis/diff_tool/debug.py", line 
> 4, in <module>
>    from .report import Report
>  File 
> "/local/repos/xen-upstream/xen/scripts/xen_analysis/diff_tool/report.py", 
> line 4, in <module>
>    from .unified_format_parser import UnifiedFormatParser, ChangeSet
>  File 
> "/local/repos/xen-upstream/xen/scripts/xen_analysis/diff_tool/unified_format_parser.py",
>  line 56, in <module>
>    class UnifiedFormatParser:
>  File 
> "/local/repos/xen-upstream/xen/scripts/xen_analysis/diff_tool/unified_format_parser.py",
>  line 57, in UnifiedFormatParser
>    def __init__(self, args: str | list) -> None:
> TypeError: unsupported operand type(s) for |: 'type' and 'type'
> 
> Also got a similar error elsewhere:
> 
> sstabellini@ubuntu-linux-20-04-desktop:/local/repos/xen-upstream/xen$ 
> ./scripts/diff-report.py --patch ~/p/1 -b /tmp/1 -r /tmp/1
> Traceback (most recent call last):
>  File "./scripts/diff-report.py", line 127, in <module>
>    main(sys.argv[1:])
>  File "./scripts/diff-report.py", line 102, in main
>    diffs = UnifiedFormatParser(diff_source)
>  File 
> "/local/repos/xen-upstream/xen/scripts/xen_analysis/diff_tool/unified_format_parser.py",
>  line 79, in __init__
>    self.__parse()
>  File 
> "/local/repos/xen-upstream/xen/scripts/xen_analysis/diff_tool/unified_format_parser.py",
>  line 94, in __parse
>    def parse_diff_header(line: str) -> ChangeSet | None:
> TypeError: unsupported operand type(s) for |: 'type' and 'NoneType'
> 
> My Python is 2.7.18
> 
> 
> Am I understanding correctly that one should run the scan for the
> baseline (saving the result somewhere), then apply the patch, run the
> scan again. Finally, one should call diff-report.py passing -b
> baseline-report -r new-report --patch the-patch-applied?

Hi Stefano,

Yes indeed, that procedure is correct, I think the error you are seeing comes 
from the python version,
I am using python 3, version 3.10.6.

The error seems to come from python annotations, I’m surprised you didn’t hit 
it when testing the first patch,
did you use python2 for that?

Is it a problem if I developed the tool having in mind its usage with python3?

Cheers,
Luca