|
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [Xen-devel] [PATCH for-4.11 v3 1/1] Add new add_maintainers.pl script to optimise the workflow when using git format-patch with get_maintainer.pl
The tool covers step 2 of the following workflow
Step 1: git format-patch ... -o <patchdir> ...
Step 2: ./scripts/add_maintainers.pl -d <patchdir>
This overwrites *.patch files in <patchdir>
Step 3: git send-email -to xen-devel@xxxxxxxxxxxxxxxxxxxx <patchdir>/*.patchxm
I manually tested all options and the most common combinations
on Mac.
Changes since v1:
- Added RAB (indicated by Juergen on IRC that this is OK)
- Remove trailing whitespaces
- Renamed --prefix to --reroll-count
- Cleaned up short options -v, ... to be in line with git
- Added --tags|-t option to add AB, RAB and RB emails to CC list
- Added --insert|-i mode to allow for people adding CCs to commit message
instead of the e-mail header (the header is the default)
- Moved common code into functions
- Added logic, such that the tool only insert's To: and Cc: statements
which were not there before, allowing for running the tool multiple times
on the same <patchdir>
Changes since v2:
- Deleted --version and related infrastructure
- Added subroutine prototypes
- Removed AT and @lists declaration and used \@ in literals
- Changed usage message and options based on feedback
- Improved error handling
- Removed occurances of index() and replaced with regex
- Removed non-perl idioms
- Moved uniq statements to normalize and added info on what normalize does
- Read L: tags from MAINTAINERS file instead of using heuristic
- Fixed issues related to metacharacters in getmaintainers()
- Allow multiple -a | --arg values (because of this renamed --args)
- Identify tags via regex
- CC's from tags are only inserted in the mail header, never the body
- That is unless the new option --tagscc is used
- Added policy processing which includes reworking insert()
- Replaced -i|--insert with -p|--inspatch and -c|--inscover now using policies
- Added new policies to cover for all user requests
- Rewrote help message to center around usage of policies
- Reordered some code (e.g. help string first to make code more easily readable)
Cc: Andrew Cooper <andrew.cooper3@xxxxxxxxxx>
Cc: George Dunlap <George.Dunlap@xxxxxxxxxxxxx>
Cc: Ian Jackson <ian.jackson@xxxxxxxxxxxxx>
Cc: Jan Beulich <jbeulich@xxxxxxxx>
Cc: Julien Grall <julien.grall@xxxxxxx>
Cc: Konrad Rzeszutek Wilk <konrad.wilk@xxxxxxxxxx>
Cc: Stefano Stabellini <sstabellini@xxxxxxxxxx>
Cc: Tim Deegan <tim@xxxxxxx>
Cc: Wei Liu <wei.liu2@xxxxxxxxxx>
Signed-off-by: Lars Kurth <lars.kurth@xxxxxxxxxx>
Release-acked-by: Juergen Gross <jgross@xxxxxxxx>
---
scripts/add_maintainers.pl | 512 +++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 512 insertions(+)
create mode 100755 scripts/add_maintainers.pl
diff --git a/scripts/add_maintainers.pl b/scripts/add_maintainers.pl
new file mode 100755
index 0000000000..11ae60d888
--- /dev/null
+++ b/scripts/add_maintainers.pl
@@ -0,0 +1,512 @@
+#!/usr/bin/perl -w
+# (c) 2018, Lars Kurth <lars.kurth@xxxxxxxxxx>
+#
+# Add maintainers to patches generated with git format-patch
+#
+# Usage: perl scripts/add_maintainers.pl [OPTIONS] -patchdir <patchdir>
+#
+# Prerequisites: Execute
+# git format-patch ... -o <patchdir> ...
+#
+# ./scripts/get_maintainer.pl is present in the tree
+#
+# Licensed under the terms of the GNU GPL License version 2
+
+use strict;
+
+use Getopt::Long qw(:config no_auto_abbrev);
+use File::Basename;
+use List::MoreUtils qw(uniq);
+
+sub getmaintainers ($$$);
+sub gettagsfrompatch ($$$;$);
+sub normalize ($$);
+sub insert ($$$$);
+sub hastag ($$);
+
+# Tool Variables
+my $tool = $0;
+my $usage = <<EOT;
+USAGE: $tool [options] (--patchdir | -d) <patchdir>
+
+OPTIONS:
+--------
+ --reroll-count <n> | -v <n>
+ Choose patch files for specific version. This results into the
+ following filters on <patchdir>
+ 0: default - *.patch
+ >1: v<n>*.patch
+ --inspatch (top|ccbody|cc---|none) | -p (top|ccbody|cc---|none)
+ Insert email addresses into *.patch files according to the POLICY
+ See section POLICY:
+ --inscover (top|ccend|none) | -c (top|ccend|none)
+ Insert email addresses into cover letteraccording to the POLICY
+ See section PROCESSING POLICY:
+ --tags | -t
+ Read email addresses from tags and add to CC list.
+ Note that git send-email does not do this. It will add the senders
+ email adress to the CC list though
+ --tagscc
+ Same as tags, only that in this case CCs extracted from tags
+ are treated like CCs that have come from the *.patch file
+ --arg <argument> | -a <argument> ...
+ Arguments passed on to get_maintainer.pl
+ This option can be used multiple times, e.g. -a <a1> -a <a2> ...
+ --verbose
+ Show more output
+ --help | -h
+ Show this help information
+
+PROCESSING POLICY:
+------------------
+ *.patch files consist of several sections relevant to processing:
+ <top>: This is the email header containing email related information
+ It ends with the Subject: line
+ <body>: This is the body that ends up in the commit message
+ It ends with ---
+ <--->: This section contains the actual patches. CCs added here are
+ processed by git send-email, but are not stored in the commit
+ message. Some people add CCs into this section
+ <ccend>: It ends with '-- '
+
+ Note that cover letters do not have the <body> section.
+
+ The following options specifiy how CCs are insertied into *.patch files
+ top: Insert CCs into the email header
+ Insert CCs from *-by: tags and TOs from mailing lists into the header
+ (this is the default)
+ ccbody: Insert CCs into body
+ Insert CCs from *-by: tags and TOs from mailing lists into the header
+ unless specified otherwise (via --tagscc).
+ cc---: Insert CCs just after the --- line
+ Insert CCs from *-by: tags and TOs from mailing lists into the header
+ unless specified otherwise (via --tagscc).
+ ccend: Insert CCs before the '-- ' line
+ Insert CCs from *-by: tags and TOs from mailing lists into the header
+ unless specified otherwise (via --tagscc).
+ none: Neither insert TO, CCs from --tags nor other CCs
+
+WORKFLOW:
+---------
+ This script is intended to be used as part of the following workflow
+
+ Step 1: git format-patch ... -o <patchdir> ...
+ Step 2: ./scripts/add_maintainers.pl -d <patchdir>
+ This overwrites *.patch files in <patchdir> but makes a backup
+ Step 3: git send-email -to xen-devel\@lists.xenproject.org <patchdir>/*.patch
+EOT
+
+# Constants and functions related to policies
+
+# Constants for -p|--inspatch and -c|--inscover option processing
+my @ppolicies = ("top", "ccbody", "cc---", "none");
+my @cpolicies = ("top", "ccend", "none");
+
+# Hash is used to determine which mode value maps onto which search string
+my %inssearch = (
+ "top" => "Date:", # Insert before Date:
+ "ccbody" => "Signed-off-by:", # Insert before Signed-off-by:
+ "cc---" => "---", # Insert after ---
+ "ccend" => "-- ", # Insert before '-- '
+);
+
+# Hash is used to determine whether for a given mode we insert CCs after
+# the search string or before
+my %insafter = (
+ "top" => 0,
+ "ccbody" => 0,
+ "cc---" => 1,
+ "ccend" => 0,
+);
+
+# The following subroutines take a areference to arrays of
+# - @top: contains CCs from *-by: tags and TOs from mailing lists
+# - @cc: contains all other CC's
+# It will then apply the corect policies on the input file
+
+sub applypolicy_top ($$$) {
+ my ($file, $rtop, $rcc) = @_;
+ my $insert = join("\n", uniq (@$rtop, @$rcc));
+ insert($file , $insert, $inssearch{top}, $insafter{top});
+}
+
+sub applymixedpolicy ($$$$) {
+ my ($file, $rtop, $rcc, $mode) = @_;
+ my $top = join("\n", @$rtop);
+ my $cc = join("\n", @$rcc);
+ # Insert snippets into files
+ insert($file , $cc, $inssearch{$mode}, $insafter{$mode});
+ # The top
+ insert($file , $top, $inssearch{top}, $insafter{top});
+}
+
+sub applypolicy_ccbody($$$) {
+ my ($file, $rtop, $rcc) = @_;
+ applymixedpolicy($file, $rtop, $rcc, "ccbody");
+}
+
+# Use a different name to make sure perl doesn't throw a syntax error
+sub applypolicy_cc3minus ($$$) {
+ my ($file, $rtop, $rcc) = @_;
+ applymixedpolicy($file, $rtop, $rcc, "cc---");
+}
+
+sub applypolicy_ccend ($$$) {
+ my ($file, $rtop, $rcc) = @_;
+ applymixedpolicy($file, $rtop, $rcc, "ccend");
+}
+
+sub applypolicy_none ($$$) {
+ return;
+}
+
+# Hash for policy functions
+my %applypolicy = (
+ "top" => \&applypolicy_top,
+ "ccbody" => \&applypolicy_ccbody,
+ "cc---" => \&applypolicy_cc3minus,
+ "ccend" => \&applypolicy_ccend,
+ "none" => \&applypolicy_none,
+);
+
+# Arguments / Options
+my $help = 0;
+my $patch_dir = 0;
+my @get_maintainer_args = ();
+my $verbose = 0;
+my $rerollcount = 0;
+my $patch_prefix = "0"; # Use a 0, such that v* does not get picked up
+ # Obviously this will only work for series with
+ # < 999 patches, which should be fine
+my $tags = 0;
+my $tagscc = 0;
+my $ppolicy = "top";
+my $cpolicy = "top";
+
+# Constants
+# Keep these as constants, in case we want to make these configurable
+# in future
+my $CC = "Cc:"; # Note: git-send-mail requires Cc:
+my $TO = "To:";
+my $cover_letter = "0000-cover-letter.patch";
+my $get_maintainer = "./scripts/get_maintainer.pl";
+my $patch_ext = ".patch";
+my $maintainers = "MAINTAINERS";
+
+if (!GetOptions(
+ 'd|patchdir=s' => \$patch_dir,
+ 'v|reroll-count=i' => \$rerollcount,
+ 'p|inspatch=s' => \$ppolicy,
+ 'c|inscover=s' => \$cpolicy,
+ 't|tags' => \$tags,
+ 'tagscc' => \$tagscc,
+ 'a|arg=s' => \@get_maintainer_args,
+ 'verbose' => \$verbose,
+ 'h|help' => \$help,
+ )) {
+ die "$tool: invalid argument - use --help if necessary\n";
+}
+
+if ($help) {
+ print $usage;
+ exit 0;
+}
+
+if (!$patch_dir) {
+ die "$tool: Directory -d|--patchdir not specified\n";
+}
+
+if (! -e $patch_dir) {
+ die "$tool: Directory $patch_dir does not exist\n";
+}
+
+if ($rerollcount > 0) {
+ $patch_prefix = "v".$rerollcount;
+}
+
+if ( ! grep $_ eq $ppolicy, @ppolicies ) {
+ die "$tool: Invalid -p|--inspatch value\n";
+}
+if ( ! grep $_ eq $cpolicy, @cpolicies ) {
+ die "$tool: Invalid -c|--inscover value\n";
+}
+
+# Get the list of patches
+my $has_cover_letter = 0;
+my $cover_letter_file;
+my $pattern = $patch_dir.'/'.$patch_prefix.'*'.$patch_ext;
+
+$!=0;
+my @patches = glob($pattern);
+if ($!) {
+ die "$tool: Directory $patch_dir contains no patches\n";
+}
+if (!scalar @patches) {
+ die "$tool: Directory $patch_dir contains no matching patches\n";
+}
+
+# Do the actual processing
+my $file;
+my @combined_top;
+my @combined_cc;
+
+foreach my $file (@patches) {
+ if ($file =~ /\Q$cover_letter\E/) {
+ $has_cover_letter = 1;
+ $cover_letter_file = $file;
+ } else {
+ my @top; # To: lists returned by get_maintainers.pl
+ my @toppatch; # To: entries in *.patch
+ #
+ # Also includes CC's from tags as we do not want
+ # entries in the body such as
+ # CC: lars.kurth@xxxxxxxxxx
+ # ...
+ # Tested-by: lars.kurth@xxxxxxxxxx
+
+ my @cc; # Cc: maintainers returned by get_maintainers.pl
+ my @ccpatch; # Cc: entries in *.patch
+ my @extrapatch; # Cc: for AB, RB, RAB in *.patch
+
+ print "Processing: ".basename($file)."\n";
+
+ # Read tags from output of get_maintainers.pl
+ # Lists go into @top and everything else into @cc
+ getmaintainers($file, \@top, \@cc);
+
+ # Read all lines with CC & TO from the patch file (these will
+ # likely come from the commit message). Also read tags.
+ gettagsfrompatch($file, \@toppatch, \@ccpatch, \@extrapatch);
+
+ # With -t|--tags only add @extrapatch to @top and @combined_top
+ # With --tagscc treat tags as CC that came from the *.patch file
+ if ($tags & !$tagscc) {
+ # Copy these always onto the TO related arrays
+ push @top, @extrapatch;
+ push @combined_top, @extrapatch;
+ } elsif ($tagscc) {
+ # Treat these as if they came from CC's
+ push @ccpatch, @extrapatch;
+ push @combined_cc, @extrapatch;
+ }
+
+ # In this section we normalize the lists. We remove entries
+ # that are already in the patch, from @cc and @to
+ my @top_only = normalize(\@top, \@toppatch);
+ my @cc_only = normalize(\@cc, \@ccpatch);
+
+ # Apply the policy
+ $applypolicy{$ppolicy}($file, \@top_only, \@cc_only);
+ }
+}
+
+# Deal with the cover letter
+if ($has_cover_letter) {
+ my @toppatch; # Entries inserted at the top
+ my @ccpatch; # Cc: entries in *.patch
+
+ print "Processing: ".basename($cover_letter_file)."\n";
+
+ # Read all lines with CC & TO from the patch file such that subsequent
+ # calls don't lead to duplication
+ gettagsfrompatch($cover_letter_file, \@toppatch, \@ccpatch);
+
+ # In this section we normalize the lists. We remove entries
+ # that are already in the patch, from @cc and @to
+ my @top_only = normalize(\@combined_top, \@toppatch);
+ my @cc_only = normalize(\@combined_cc, \@ccpatch);
+
+ # Apply the policy
+ $applypolicy{$cpolicy}($cover_letter_file, \@top_only, \@cc_only);
+
+ print "\nDon't forget to add the subject and message to ".
+ $cover_letter_file."\n";
+}
+
+print "Then perform:\n".
+ "git send-email -to xen-devel\@lists.xenproject.org ".
+ $patch_dir.'/'.$patch_prefix."*.patch"."\n";
+
+exit 0;
+
+my $readmailinglists = 0;
+my @mailinglists = ();
+
+sub getmailinglists () {
+ # Read mailing list from MAINTAINERS file and copy
+ # a list of e-mail addresses to @mailinglists
+ if (!$readmailinglists) {
+ if (-e $maintainers) {
+ my $fh;
+ my $line;
+ open($fh, "<", $maintainers) or die $!;
+ while (my $line = <$fh>) {
+ chomp $line;
+ if ($line =~ /^L:[[:blank:]]+/m) {
+ push @mailinglists, $';
+ }
+ }
+ close $fh or die $!;
+ } else {
+ print "Warning: file '$maintainers' does not exist\n";
+ print "Warning: Mailing lists will be treated as CC's\n";
+ }
+ # Don't try again, even if the MAINTAINERS file does not exist
+ $readmailinglists = 1;
+ # Remove any duplicates
+ @mailinglists = uniq @mailinglists;
+ }
+}
+
+sub ismailinglist ($) {
+ my ($check) = @_;
+ # Get the mailing list information
+ getmailinglists();
+ # Do the check
+ if ( grep { $_ eq $check} @mailinglists) {
+ return 1;
+ }
+ return 0;
+}
+
+sub getmaintainers ($$$) {
+ my ($file, $rto, $rcc) = @_;
+ my $get_maintainer_args = join " ", @get_maintainer_args;
+ my $cmd = "$get_maintainer $get_maintainer_args <$file";
+ my $fh;
+
+ if (! -e $get_maintainer) {
+ die "$tool: The tool requires $get_maintainer\n";
+ }
+
+ open($fh, "-|", $cmd)
+ or die "Failed to open '$cmd'\n";
+ while(my $line = <$fh>) {
+ chomp $line;
+ # Keep lists and CC's separately as we dont want them in
+ # the commit message under a Cc: line
+ if (ismailinglist($line)) {
+ push @$rto, $TO." ".$line;
+ push @combined_top, $TO." ".$line;
+ } else {
+ push @$rcc, $CC." ".$line;
+ push @combined_cc, $CC." ".$line;
+ }
+ }
+ close $fh;
+}
+
+sub gettagsfrompatch ($$$;$) {
+ my ($file, $rto, $rcc, $rextra) = @_;
+ my $fh;
+
+ open($fh, "<", $file)
+ or die "Failed to open '$file'\n";
+ while(<$fh>) {
+ chomp;
+ my $line = $_;
+ my $nline;
+
+ if (hastag($line, $TO)) {
+ push @$rto, $line;
+ push @combined_top, $line;
+ }
+ if (hastag($line, $CC)) {
+ push @$rcc, $line;
+ push @combined_cc, $line;
+ }
+ # If there is an $rextra, then get various tags and add
+ # email addresses to the CC list
+ if ($rextra && $line =~ /^[-0-9a-z]+-by:[[:blank:]]+/mi) {
+ push @$rextra, $CC." ".$';
+ }
+ }
+ close $fh;
+}
+
+sub hastag ($$) {
+ my ($line, $tag) = @_;
+ if ($line =~ m{^\Q$tag\E}i) {
+ return 1;
+ }
+ return 0;
+}
+
+sub normalize ($$) {
+ # This function is used to normalize lists of tags or CC / TO lists
+ # - It removes duplicates in the input arrays
+ # - It ensures that elements in the second list are not in the first
+ my ($ra, $rb) = @_;
+ my @aonly = ();
+ my %seen;
+ my $item;
+
+ @$ra = uniq @$ra;
+ @$rb = uniq @$rb;
+ foreach $item (@$rb) {
+ $seen{$item} = 1;
+ }
+ foreach $item (@$ra) {
+ unless ($seen{$item}) {
+ # it's not in %seen, so add to @aonly
+ push @aonly, $item;
+ }
+ }
+
+ return @aonly;
+}
+
+sub readfile ($) {
+ my ($file) = @_;
+ my $fh;
+ my $content;
+ open($fh, "<", $file)
+ or die "Could not open file '$file' $!";
+ $content = do { local $/; <$fh> };
+ close $fh or die $!;
+
+ return $content;
+}
+
+sub writefile ($$) {
+ my ($content, $file) = @_;
+ my $fh;
+ open($fh, ">", $file)
+ or die "Could not open file '$file' $!";
+ print $fh $content or die $!;
+ close $fh or die $!;
+}
+
+sub insert ($$$$) {
+ my ($file, $insert, $delimiter, $insafter) = @_;
+ my $content;
+
+ if ($insert eq "") {
+ # Nothing to insert
+ return;
+ }
+ # Read file
+ $content = readfile($file);
+
+ # Split the string and generate new content
+ if ($content =~ /^\Q$delimiter\E/mi) {
+ if ($insafter) {
+ writefile($`.$delimiter."\n".$insert."\n".$', $file);
+
+ if ($verbose) {
+ print "\nInserted into ".basename($file).' after "'.
+ $delimiter."'"."\n-----\n".$insert."\n-----\n";
+ }
+ } else {
+ writefile($`.$insert."\n".$delimiter.$', $file);
+
+ if ($verbose) {
+ print "\nInserted into ".basename($file).' before "'.
+ $delimiter."'"."\n-----\n".$insert."\n-----\n";
+ }
+ }
+
+ } else {
+ print "Error: Didn't find '$delimiter' in '$file'\n";
+ }
+}
--
2.13.0
_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel
|
 |
Lists.xenproject.org is hosted with RackSpace, monitoring our |