Re: Golang Xen packages and the golang packaging system

[Nick Rosbrook <rosbrookn@xxxxxxxxx>]

On Thu, Apr 23, 2020 at 1:22 PM George Dunlap <George.Dunlap@xxxxxxxxxx> wrote:
>
>
> > On Apr 23, 2020, at 12:49 PM, George Dunlap <george.dunlap@xxxxxxxxxx> 
> > wrote:
> >
> >
> >
> >> On Apr 23, 2020, at 12:27 PM, Ian Jackson <ian.jackson@xxxxxxxxxx> wrote:
> >>
> >> Ian Jackson writes ("Re: Golang Xen packages and the golang packaging  
> >> system"):
> >>> This is quite unpleasant.  In particular, it makes a git tree out of
> >>> output files.  What will we do when someone sends us patches to the
> >>> bindings ?
> >>
> >> Also, anyone who redistributes your proposed golang package is
> >> violating our licence unless they ship a copy of xen.git[1] too, since
> >> the golang package is not source code.
> >>
> >> [1] Technically, a copy of the relevant parts will do.
> >
> > The “relevant parts” would primarily be gengotypes.py, right?  Oh, and I 
> > guess libxl_test.idl and friends.  libxl_test.idl isn’t included in the 
> > distribution either.
> >
> > I’m not an expert in the golang build system, but they generally seem to be 
> > trying to keep the functionality simple (which of course, means if you want 
> > to do anything non-basic, it’s incredibly complicated or completely 
> > impossible).
> >
> > There’s a command, `go generate`, which we could use to run gengotypes.py 
> > to generate the appropriate files.  But I’m not sure how to use that in a 
> > practical way for this sort of package: it might end up that people wanting 
> > to use the package would need to manually clone it, then manually run `go 
> > generate` before manually building the package.
> >
> > Checking in the generated files means that someone can simply add 
> > `golang.xenproject.org/xenlight` as a dependency (perhaps with a specific 
> > version tag, like v4.14), and everything Just Works.
> >
> > Nick may have some ideas on how to use the golang build system more 
> > effectively.
>
> So, the following seems to work quite well actually:
>
> mkdir vendor
> ln -s vendor/golang.xenproject.org /usr/share/gocode/src/golang.xenproject.org
> echo “golang.xenproject.org/xenlight” >> vendor/modules.txt
> go build -mod=vendor
>
> Using the above method, (say) redctl.git would build exactly the same on Xen 
> 4.14 as on Xen 4.15 (assuming redctl wasn’t using anything not available in 
> 4.14).
>
> I’m inclined to say we should start with just telling people to do that, and 
> look at doing something else if we discover that’s not suitable for some 
> reason.

If it's not viable to create another repo for the xenlight package, I
think we should should just initialize the go module, i.e. go.mod, at
xen.git/tools/golang. The downside is that tags cannot be independent
from the rest of xen.git, so users need to have `require <module
path>/xenlight@RELEASE-4.14.0` in their go.mod, but at least its `go
get`-able. And, this does not fetch the entire git tree.

This would also mean that we actually track the generated code (which
isn't really a big deal IMO, it's expected that people track their
generated gRPC code, for example).

I spent some time today trying to get `go get <vanity URL>/xenlight`
to work, after defining the go.mod (with <vanity URL>/xenlight as the
import path) in tools/golang. I *think* it's do-able with a meta-tag
like: `<meta name="go-import" content="<vanity URL>/xenlight mod
https://proxy.golang.org";>`, but I was having a hard time getting the
package to show up on pkg.go.dev or proxy.golang.org. I'm not totally
familiar with the module proxies yet, but it seems like it could be a
way to detach ourselves from the "package is defined at the root of a
git repo" problem.

-NR