[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Xen-devel] [PATCH v7 03/11] scripts: add coccinelle script to use auto propagated errp

  • To: Markus Armbruster <armbru@xxxxxxxxxx>
  • From: Vladimir Sementsov-Ogievskiy <vsementsov@xxxxxxxxxxxxx>
  • Date: Tue, 25 Feb 2020 12:08:37 +0300
  • Arc-authentication-results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=virtuozzo.com; dmarc=pass action=none header.from=virtuozzo.com; dkim=pass header.d=virtuozzo.com; arc=none
  • Arc-message-signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=gFMQmqotNJjEGzKuU52R/Qq8mBIG7Z62QjPsFlGDiYU=; b=CTSLrRT/Sb1lfAhOWa9oJjeSyFSyKzHKPn3bsYgnBnJKRTB62XkslD4kpU8xXZ3VWDCj0etQJOTWTlwsQCCweAoSv8VfSVh1dcO0fLP06yjCKKVkGWpOn+TkqASnhuKn87w+oKR5Ck21Ewgm93n5FdVV/s+uCr6RL2hQDgjnS4F6IcbOTb0CjP7IbbZp3Cp7Lag320Xbk/40aTVqRBO99rYh2MybcXBeU4ksZthUhgQ7YU7hMJIFyvLp6Gy3i7/zG8C+tkags4HWrgdSEho/3epAxo0ryQ9BLg4OE0FiQQwmjAmy2PlxhgC0zMj6AOU7yowjpljh/wNAtGc6VjIXPw==
  • Arc-seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=PtBkEvs6SclKQirdRFxRsTdYmlW+fslORNq2Euq20uaNMyF5lfY7nSLALsAOPPYHz9RBhVvcS39dqCpvYKKLpX9/QKDmkoMFIyuW+45WSBAL+EIT4HdoNYe7Ix2eFF5EOt9Tu30hNSHfmY+H+J5Q6+2y2C/p1s37WKy32H3sqP/r0M1vm/sCig0URK9a571w/k3LoRbdBbfH2aG4kPdcEDZkRYyQcTJyH4k95Djq4VacT8uPVr737+JYamT9wBtHftaR9z3dnyr0cHpjPMvjGigNqbcI1Oj0nGva9pvttNTB3cKQ2Ygnb+zq+Bgy6IdJbFuO9AJEsOmLSIG61BWKfw==
  • Authentication-results: spf=none (sender IP is ) smtp.mailfrom=vsementsov@xxxxxxxxxxxxx;
  • Cc: Kevin Wolf <kwolf@xxxxxxxxxx>, Stefano Stabellini <sstabellini@xxxxxxxxxx>, Michael Roth <mdroth@xxxxxxxxxxxxxxxxxx>, qemu-block@xxxxxxxxxx, Paul Durrant <paul@xxxxxxx>, Laszlo Ersek <lersek@xxxxxxxxxx>, qemu-devel@xxxxxxxxxx, Greg Kurz <groug@xxxxxxxx>, Gerd Hoffmann <kraxel@xxxxxxxxxx>, Stefan Hajnoczi <stefanha@xxxxxxxxxx>, Anthony Perard <anthony.perard@xxxxxxxxxx>, xen-devel@xxxxxxxxxxxxxxxxxxxx, Max Reitz <mreitz@xxxxxxxxxx>, Philippe Mathieu-Daudé <philmd@xxxxxxxxxx>, Stefan Berger <stefanb@xxxxxxxxxxxxx>
  • Delivery-date: Tue, 25 Feb 2020 09:08:47 +0000
  • List-id: Xen developer discussion <xen-devel.lists.xenproject.org>

23.02.2020 11:55, Markus Armbruster wrote:
Vladimir Sementsov-Ogievskiy <vsementsov@xxxxxxxxxxxxx> writes:

Script adds ERRP_AUTO_PROPAGATE macro invocation where appropriate and
does corresponding changes in code (look for details in

Usage example:
spatch --sp-file scripts/coccinelle/auto-propagated-errp.cocci \
  --macro-file scripts/cocci-macro-file.h --in-place --no-show-diff \
  blockdev-nbd.c qemu-nbd.c {block/nbd*,nbd/*,include/block/nbd*}.[hc]

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@xxxxxxxxxxxxx>

CC: Eric Blake <eblake@xxxxxxxxxx>
CC: Kevin Wolf <kwolf@xxxxxxxxxx>
CC: Max Reitz <mreitz@xxxxxxxxxx>
CC: Greg Kurz <groug@xxxxxxxx>
CC: Stefano Stabellini <sstabellini@xxxxxxxxxx>
CC: Anthony Perard <anthony.perard@xxxxxxxxxx>
CC: Paul Durrant <paul@xxxxxxx>
CC: Stefan Hajnoczi <stefanha@xxxxxxxxxx>
CC: "Philippe Mathieu-Daudé" <philmd@xxxxxxxxxx>
CC: Laszlo Ersek <lersek@xxxxxxxxxx>
CC: Gerd Hoffmann <kraxel@xxxxxxxxxx>
CC: Stefan Berger <stefanb@xxxxxxxxxxxxx>
CC: Markus Armbruster <armbru@xxxxxxxxxx>
CC: Michael Roth <mdroth@xxxxxxxxxxxxxxxxxx>
CC: qemu-block@xxxxxxxxxx
CC: xen-devel@xxxxxxxxxxxxxxxxxxxx

  include/qapi/error.h                          |   3 +
  scripts/coccinelle/auto-propagated-errp.cocci | 158 ++++++++++++++++++
  2 files changed, 161 insertions(+)
  create mode 100644 scripts/coccinelle/auto-propagated-errp.cocci

diff --git a/include/qapi/error.h b/include/qapi/error.h
index b9452d4806..79f8e95214 100644
--- a/include/qapi/error.h
+++ b/include/qapi/error.h
@@ -141,6 +141,9 @@
   *         ...
   *     }
+ * For mass conversion use script
+ *   scripts/coccinelle/auto-propagated-errp.cocci
+ *
   * Receive and accumulate multiple errors (first one wins):
   *     Error *err = NULL, *local_err = NULL;

Extra blank line.

diff --git a/scripts/coccinelle/auto-propagated-errp.cocci 
new file mode 100644
index 0000000000..fb03c871cb
--- /dev/null
+++ b/scripts/coccinelle/auto-propagated-errp.cocci
@@ -0,0 +1,158 @@
+// Use ERRP_AUTO_PROPAGATE (see include/qapi/error.h)
+// Copyright (c) 2020 Virtuozzo International GmbH.
+// This program is free software; you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation; either version 2 of the License, or
+// (at your option) any later version.
+// This program is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// GNU General Public License for more details.
+// You should have received a copy of the GNU General Public License
+// along with this program.  If not, see <http://www.gnu.org/licenses/>.
+// Usage example:
+// spatch --sp-file scripts/coccinelle/auto-propagated-errp.cocci \
+//  --macro-file scripts/cocci-macro-file.h --in-place --no-show-diff \
+//  blockdev-nbd.c qemu-nbd.c {block/nbd*,nbd/*,include/block/nbd*}.[hc]
+// Add invocation to errp-functions where necessary
+// We should skip functions with "Error *const *errp"
+// parameter, but how to do it with coccinelle?
+// I don't know, so, I skip them by function name regex.
+// It's safe: if we did not skip some functions with
+// "Error *const *errp", ERRP_AUTO_PROPAGATE invocation
+// will fail to compile, because of const violation.

Not skipping a function we should skip fails to compile.

What about skipping a function we should not skip?

Then it will not be updated.. Not good but I don't have better solution.
Still, I hope, function called *error_append_*_hint will not return error
through errp pointer.

+identifier fn !~ "error_append_.*_hint";
+identifier local_err, ERRP;

A few of our coccinelle scripts use ALL_CAPS for meta-variables.  Most
don't.  Either is fine with me.  Mixing the two styles feels a bit
confusing, though.

+ fn(..., Error **ERRP, ...)
+ {
+    <+...
+        when != ERRP_AUTO_PROPAGATE();
+    error_append_hint(ERRP, ...);
+    error_prepend(ERRP, ...);
+    Error *local_err = NULL;
+    ...+>
+ }

Misses error_vprepend().  Currently harmless, but as long as we commit
the script, we better make it as robust as we reasonably can.

The previous patch explains this Coccinelle script's intent:

   To achieve these goals, later patches will add invocations
   of this macro at the start of functions with either use
   error_prepend/error_append_hint (solving 1) or which use
   local_err+error_propagate to check errors, switching those
   functions to use *errp instead (solving 2 and 3).

This rule matches "use error_prepend/error_append_hint" directly.  It
appears to use presence of a local Error * variable as proxy for "use
local_err+error_propagate to check errors".  Hmm.

We obviously have such a variable when we use "local_err+error_propagate
to check errors".  But we could also have such variables without use of
error_propagate().  In fact, error.h documents such use:

  * Call a function and receive an error from it:
  *     Error *err = NULL;
  *     foo(arg, &err);
  *     if (err) {
  *         handle the error...
  *     }

where "handle the error" frees it.

I figure such uses typically occur in functions without an Error **errp
parameter.  This rule doesn't apply then.  But they could occur even in
functions with such a parameter.  Consider:

     void foo(Error **errp)
         Error *err = NULL;

         if (err) {
             error_setg(errp, "completely different error");

Reasonable enough when bar() gives us an error that's misleading in this
context, isn't it?

The script transforms it like this:

     void foo(Error **errp)
    -    Error *err = NULL;

    -    bar(&err);
    -    if (err) {
    -        error_free(err);
    +    bar(errp);
    +    if (*errp) {
    +        error_free_errp(errp);
             error_setg(errp, "completely different error");


What is the problem with it? Updated code just use "new usual notation"
for handling error of sub-calls in function which reports errors through
errp pointer.

Now, if this script applied in just a few dozen places, we could rely on
eyeballing its output to catch unwanted transformations.  Since it
applies in so many more, I don't feel comfortable relying on reviewer

Can we make rule0 directly match error_propagate(errp, local_err)

I think it is possible, still I'm not sure we need it.

Another observation: the rule does not match error_reportf_err() and
warn_reportf_err().  These combine error_prepend(),
error_report()/warn_report() and error_free(), for convenience.  Don't
their users need ERRP_AUTO_PROPAGATE() just like error_prepend()'s

Right. These functions want to add information, which will not work
for error_fatal without wrapping.

+// Switch unusual (Error **) parameter names to errp
+// (this is necessary to use ERRP_AUTO_PROPAGATE).

Please put your rule comments right before the rule, i.e. before the
@-line introducing metavariable declarations, not after.  Same

+identifier rule0.fn;
+identifier rule0.ERRP != errp;
+ fn(...,
+-   Error **ERRP
++   Error **errp
+    ,...)
+ {
+     <...
+-    ERRP
++    errp
+     ...>
+ }

This normalizes errp parameter naming.  It matches exactly when rule0
matches (and inserts ERRP_AUTO_PROPAGATE()) and the Error ** parameter
is unusual.  Good.

+// We want to patch error propagation in functions regardless of
+// whether the function already uses ERRP_AUTO_PROPAGATE prior to
+// applying rule0, hence this one does not inherit from it.

I'm not sure I get this comment.  Let's see what the rule does.

+identifier fn !~ "error_append_.*_hint";
+identifier local_err;
+symbol errp;
+ fn(..., Error **errp, ...)
+ {
+     <...
+-    Error *local_err = NULL;
+     ...>
+ }

rule1 matches like rule0, except the Error ** parameter match is
tightened from any C identifier to the C identifier errp, and the
function body match tightened from "either use
error_prepend/error_append_hint or which use local_err+error_propagate
to check errors" to just the latter.

I figure tightening the Error ** parameter match has no effect, because
we already normalized the parameter name.

So rule1 deletes variable local_err where rule0 applied.  Correct?

The difference with rule0 is that rule0 contains
 "when != ERRP_AUTO_PROPAGATE()", so rule0 is not applied where
we already have macro invocation.

This is why we can't inherit from rule0.

No we believe that we have ERRP_AUTO_PROPAGATE invocation in all
corresponding places (added by rule0 or before script run) and want to
update all usage of local_err objects.

+// Handle pattern with goto, otherwise we'll finish up
+// with labels at function end which will not compile.
+identifier rule1.fn, rule1.local_err;
+identifier OUT;
+ fn(...)
+ {
+     <...
+-    goto OUT;
++    return;
+     ...>
+- OUT:
+-    error_propagate(errp, local_err);
+ }

This is one special case of error_propagate() deletion.  It additionally
gets rid of a goto we no longer want.  For the general case, see below.

The rule applies only where rule1 just deleted the variable.  Thus, the
two rules work in tandem.  Makes sense.

+identifier rule1.fn, rule1.local_err;

This rule also works in tandem with rule1.

+expression list args; // to reindent error_propagate_prepend

What is the comment trying to tell me?

Hmm, we can safely drop it. It's about the following:

instead of

 -    error_propagate_prepend(errp, local_err, args);
 +    error_prepend(errp, args);

we can use "...", like

 - error_propagate_prepend(errp, local_err
 + error_prepend(errp
   , ...);

but with metavar in use, coccinelle will correctly reindent the
whole call, which looks a lot better.

+ fn(...)
+ {
+     <...
+-    error_free(local_err);
+-    local_err = NULL;
++    error_free_errp(errp);


     static inline void error_free_errp(Error **errp)
         assert(errp && *errp);
         *errp = NULL;

Now let's examine the actual change.

The assertion's first half trivially holds, ERRP_AUTO_PROPAGATE()
ensures it.

The second half is new.  We now crash when we haven't set an error.  Why
is this safe?  Note that error_free(local_err) does nothing when

Hmm. Looks like we should tighten this restriction, and follow error_free
interface, which allows freeing unset errp.

The zapping of the variable pointing to the Error just freed is

+-    error_free(local_err);
++    error_free_errp(errp);

Here, the zapping is new.  Zapping dangling pointers is obviously safe.
Needed, or else the automatic error_propagate() due to
ERRP_AUTO_PROPAGATE() would propagate the dangling pointer.

+-    error_report_err(local_err);
++    error_report_errp(errp);

The only difference to the previous case is that we also report the

The previous case has a buddy that additionally matches *errp = NULL.
Why not this one?

Probably because no matches in code. But should be added here for
better case coverage.

+-    warn_report_err(local_err);
++    warn_report_errp(errp);


What about error_reportf_err(), warn_reportf_err()?

Up to here, this rule transforms the various forms of error_free().
Next: error_propagate().

+-    error_propagate_prepend(errp, local_err, args);
++    error_prepend(errp, args);
+-    error_propagate(errp, local_err);

rule0's adding of ERRP_AUTO_PROPAGATE() made error_propagate()

This is the general case of error_propagate() deletion.

I'd put the plain error_propagate() first, variations second, like you
do with error_free().

If neither of these two patterns match on a path from
ERRP_AUTO_PROPAGATE() to return, we effectively insert error_propagate()
where it wasn't before.  Does nothing when the local error is null
there.  Bug fix when it isn't: it's at least a memory leak, and quite
possibly worse.

Hmm. How can it be memory leak after any of error_free variants?

Identifying these bug fixes would be nice, but I don't have practical
ideas on how to do that.

Can we explain this in the commit message?

+     ...>
+ }
+identifier rule1.fn, rule1.local_err;
+ fn(...)
+ {
+     <...
+-    &local_err
++    errp
+-    local_err
++    *errp
+     ...>
+ }

Also in tandem with rule1, fixes up uses of local_err.  Good.

+identifier rule1.fn;
+ fn(...)
+ {
+     <...
+- *errp != NULL
++ *errp
+     ...>
+ }

Still in tandem with rule1, normalizes style.  Good.

Best regards,

Xen-devel mailing list



Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.