Re: [PATCH] automation: restore CR filtering

[Stefano Stabellini <sstabellini@xxxxxxxxxx>]

On Tue, 20 Aug 2024, Anthony PERARD wrote:
> On Mon, Aug 19, 2024 at 06:56:47PM -0700, Stefano Stabellini wrote:
> > On Mon, 19 Aug 2024, Anthony PERARD wrote:
> > > On Mon, Aug 19, 2024 at 09:21:22AM +0200, Michal Orzel wrote:
> > > > On 17/08/2024 01:46, Stefano Stabellini wrote:
> > > > > diff --git a/automation/scripts/qemu-xtf-dom0less-arm64.sh 
> > > > > b/automation/scripts/qemu-xtf-dom0less-arm64.sh
> > > > > index 0666f6363e..ed44aab0f0 100755
> > > > > --- a/automation/scripts/qemu-xtf-dom0less-arm64.sh
> > > > > +++ b/automation/scripts/qemu-xtf-dom0less-arm64.sh
> > > > > @@ -65,4 +65,4 @@ export UBOOT_CMD="virtio scan; dhcp; tftpb 
> > > > > 0x40000000 boot.scr; source 0x4000000
> > > > >  export QEMU_LOG="smoke.serial"
> > > > >  export PASSED="${passed}"
> > > > > 
> > > > > -./automation/scripts/qemu-key.exp
> > > > > +./automation/scripts/qemu-key.exp | sed 's/\r//'
> > > > 
> > > > I compared 3 pipelines:
> > > > 1) one before c36efb7fcea6 
> > > > (https://gitlab.com/xen-project/hardware/xen/-/jobs/7566986885)
> > > > 2) one after c36efb7fcea6 
> > > > (https://gitlab.com/xen-project/hardware/xen/-/jobs/7603830706)
> > > > 3) one with this fix 
> > > > (https://gitlab.com/xen-project/people/sstabellini/xen/-/jobs/7603783403)
> > > > 
> > > > In 1), there is Xen log + Linux log in Gitlab web page
> > > > In 2), there is no log at all
> > > > In 3), there is only Xen log visible
> > > 
> > > It's nice that you can select uboot/Xen logs or Linux logs based on the
> > > number of '\r' at the end of a line (output cat -A):
> > >     U-Boot 2023.01+dfsg-2+deb12u1 (Apr 18 2024 - 22:00:21 +0000)^M^M$
> > >     (XEN) [    0.013864] Xen version 4.20-unstable (root@) (gcc (Alpine 
> > > 12.2.1_git20220924-r10) 12.2.1 20220924) debug=n Sat Aug 17 00:54:57 UTC 
> > > 2024^M^M$
> > >     [    0.000000] Booting Linux on physical CPU 0x0000000000 
> > > [0x411fd070]^M^M^M$
> > > 
> > > But to display to GitLab's job logs, we want: sed 's/\r\+$//'
> > > 
> > > Also, do you have to edit every single script to overcome a shortcoming
> > > from the "expect" script? Can't you write a bit of Tcl and edit the line
> > > in the script instead?
> > 
> > The sed route is not perfect but it works :-)
> > 
> > I did try using expect but the logs were mangled. I think I missed that
> > there can be multilple \r. I managed to get close to the wanted behavior
> > with the below, but the Xen logs are still missing and I don't know why. 
> > 
> > https://gitlab.com/xen-project/people/sstabellini/xen/-/jobs/7617161552
> 
> Well, it just looks like the output is duplicated, if you look at the
> raw output:
> https://gitlab.com/xen-project/people/sstabellini/xen/-/jobs/7617161552/raw
> 
> So I don't know if it's possible to edit the output before `expect`
> prints it. It probably is, but not easy to do.
> 
> I did try to edit the command line to change QEMU's output:
>     -eval spawn $env(QEMU_CMD)
>     +spawn sh -c "$env(QEMU_CMD) | sed s/\\\\r\\\\+//"
> 
> But then many failure, probably because expect can't interact with qemu
> anymore.
> 
> So I guess `sed` the output of `expect` will do. Maybe put that in a
> script that also call the expect script? (To avoid duplication, and help
> with maintainability of the whole thing.)

I tried a couple more times with expect but couldn't get it to work.
With sed it seems to behave correctly. Can you guys spot any issues with
it? If not, I'll send the full patch.

https://gitlab.com/xen-project/people/sstabellini/xen/-/jobs/7627376496

diff --git a/automation/scripts/qemu-smoke-dom0-arm64.sh 
b/automation/scripts/qemu-smoke-dom0-arm64.sh
index 0094bfc8e1..e0cea742b0 100755
--- a/automation/scripts/qemu-smoke-dom0-arm64.sh
+++ b/automation/scripts/qemu-smoke-dom0-arm64.sh
@@ -109,4 +109,4 @@ export QEMU_LOG="smoke.serial"
 export LOG_MSG="Domain-0"
 export PASSED="BusyBox"
 
-./automation/scripts/qemu-key.exp
+./automation/scripts/qemu-key.exp | sed 's/\r\+$//'