Re: [Xen-devel] First release candidate for 3.2.0

[John Levon <levon@xxxxxxxxxxxxxxxxx>]

On Wed, Dec 12, 2007 at 01:36:38PM +0000, Keir Fraser wrote:

> >> Is Solaris easy to grab and build, or does it need a Solaris environment?
> >> Alternatively, can I grab a pre-built binary (with symbols) from somewhere?
> > 
> > Hi Keir, this was a false alarm: I failed to notice that physinfo got a
> > new 'out' handle and wasn't initialising it (yes, we use sysctl() in our
> > kernel - for both good and bad reasons). When I fixed that I booted up.
> > I'll do some more testing and report back either way.
> 
> If you tell us what you need it for (apart from the memory locking for tools
> operations, which is just unfortunate, and I wonder if there is perhaps a
> better way to address that problem) then perhaps we can extend platform_op()
> to provide the same functionality?

The first big one, as you say, is our privcmd driver.  We have to decode the
hypercall and essentially do a copy_from/to_user() on all the buffers passed
in. The best way to fix this is to do it in userspace: make xc_solaris.c use a
new ioctl() that passes in buffer address+size information as well as the
structs themselves. To do this cleanly means pushing a lot of do_domctl() etc.
into xc_$OS.c, and we just haven't had time - though I'd certainly be
interested to know how you felt about a change like that.

I actually want to extend this further for least privilege reasons and have the
ioctl()s be much more "semantic", but that's a much wider fix.

Then we have three remaining uses during boot. They all use
xen_sysctl_physinfo_t.

Number of total physical pages

        - we ask about this for the benefit of our Xen crash dump
          support. This could easily be a platform op I think?

Number of real CPUs

        - we use this in our errata checking code. It just seems plain
          wrong in the Xen case for us to be checking machine errata,
          but we've never found time to go through them and verify that
          the hypervisor does the same checks and fixes. If you're
          interested I list the errata that need this value below.

cpu_khz

        - this is an old, old change during bringup which is very
          possibly fixed; once again, we need to verify we can remove
          it. As the code says:

1023         /*
1024          * During dom0 bringup, it was noted that on at least one older
1025          * Intel HT machine, the hypervisor initially gives a 
tsc_to_system_mul
1026          * value that is quite wrong (the 3.06GHz clock was reported
1027          * as 4.77GHz)
1028          *
1029          * The curious thing is, that if you stop the kernel at entry,
1030          * breakpoint here and inspect the value with kmdb, the value
1031          * is correct - but if you don't stop and simply enable the
1032          * printf statement (below), you can see the bad value printed
1033          * here.  Almost as if something kmdb did caused the hypervisor to
1034          * figure it out correctly.  And, note that the hypervisor
1035          * eventually -does- figure it out correctly ... if you look at
1036          * the field later in the life of dom0, it is correct.
1037          *
1038          * For now, on dom0, we employ a slightly cheesy workaround of
1039          * using the DOM0_PHYSINFO hypercall.
1040          */
1041         if (DOMAIN_IS_INITDOMAIN(xen_info) && xpv_cpufreq_workaround) {
1042                 xen_sysctl_physinfo_t pi;
1043                 int ret;
1044
1045                 if ((ret = xen_get_physinfo(&pi)) != 0)
1046                         panic("xen_get_physinfo() failed: %d\n", ret);
1047
1048                 cpu_hz = 1000 * (uint64_t)pi.cpu_khz;
1049         } else {
1050                 cpu_hz = (UINT64_C(1000000000) << 32) / 
vti->tsc_to_system_mul;
1051
1052                 if (vti->tsc_shift < 0)
1053                         cpu_hz <<= -vti->tsc_shift;
1054                 else
1055                         cpu_hz >>= vti->tsc_shift;
1056         }

Errata:

http://src.opensolaris.org/source/xref/onnv/onnv-gate/usr/src/uts/i86pc/os/mp_startup.c#627

AMD 122: TLB Flush Filter May Cause Coherency Problem in Multiprocessor Systems
AMD 131: Multiprocessor Systems with Four or More Cores May Deadlock Waiting 
for a Probe Response
AMD: Disable C1-Clock ramping on multi-core/multi-processor K8 platforms to 
guard against TSC drift.
  (I do remember that Xen does this, am I right?)
Plus another one around lfence around line 1000-1040

regards,
john

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel