[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [PATCH v3 2/2] tools/xenstore: set open file descriptor limit for xenstored

Hi Andrew,

On 31/08/2021 13:37, Andrew Cooper wrote:
On 31/08/2021 13:30, Julien Grall wrote:
Hi Juergen,

On 31/08/2021 13:11, Juergen Gross wrote:
On 30.07.21 19:14, Julien Grall wrote:
Hi Ian,

On 30/07/2021 14:35, Ian Jackson wrote:
Juergen Gross writes ("[PATCH v3 2/2] tools/xenstore: set open file
descriptor limit for xenstored"):
Add a configuration item for the maximum number of domains xenstored
should support and set the limit of open file descriptors

For HVM domains there are up to 5 socket connections per domain (2 by
the xl daemon process, and 3 by qemu). So set the ulimit for
to 5 * XENSTORED_MAX_DOMAINS + 100 (the "+ 100" is for some headroom,
like logging, event channel device, etc.).
+## Type: integer
+## Default: 32768
+# Select maximum number of domains supported by xenstored.
+# Only evaluated if XENSTORETYPE is "daemon".

I approve of doing something about the fd limit.  I have some qualms
about the documentation.

The documentation doesn't say what happens if this limit is exceeded.
Also the default of 32758 suggests that we actually support that many
domains.  I don't think we do...

I didn't find anything in SUPPORT.md about how many guests we support
but I wouldn't want this setting here to imply full support for 32768

If you don't want to tackle this can of works, maybe add this:

    # This just controls some resource limits for xenstored; if the
    # limit is exceeded, xenstored will stop being able to function
    # properly for additional guests.  The default value is so large
    # that it won't be exceeded in a supported configuration, but
    # should not be taken to mean that the whole Xen system is
    # guaranteed to work properly with that many guests.

Julien, did you ask for this to be made configurable ?  Having written
the text above, I wonder if it wouldn't just be better to
unconditionally set it to "unlimited" rather than offering footgun
dressed up like a tuneable...

So in v1 (see [1]), Juergen wanted to raise the limit. I assumed
this meant that the default limit (configured by the system may not
be enough).

I felt this was wrong to impose an higher limit on everyone when an
admin may know the maximum number of domains.

By "unlimited", do you mean the calling "ulimit" (or whatever is
used for configuring FDs) with unlimited?

If so, I would be OK with that. My main was was to move the raising
the limit outside Xenstored because:
   1) This is easier for an admin to tweak it (in particular the OOM)
   2) It feels wrong to me that the daemon chose the limits
   3) An admin can enforce it

Coming back to this series, I'm puzzled now.

Julien, you didn't want me to raise the limit to a specific number
covering the maximum possible number of domains, because you thought
this might result in xenstored hogging huge numbers of file descriptors
in case of a bug. Why is unlimited better then? This will make the
possible number even larger.

I don't think I suggested the unlimited number is better... My main
objection in your original approach is you set an arbitrary limit you
in Xenstored (which may not apply at all) and don't offer a way to the
admin to tweak it.

If the limit is set outside of Xenstored, then it becomes much easier
for someone to just tweak the init script. I don't have a strong
opinion on whether the default limit should be "unlimited" or a fixed

xenstored is TCB.  It needs a large number of FDs, and can be trusted
with unlimited.

Also, like xenconsoled, we can calculate an upper bound, which is
derived from the ABI limit of 32k domids.

All you're haggling over is the error semantics in the case of:
1) the upper bound calculation is wrong, or
2) there is an fd leak

Personally, I think a fixed calculation is right, so fd leaks can be
spotted more obviously. >
An admin knob is not helpful - higher than the upper bound is just
wasteful, while lower will cause malfunctions.

A lot of users will never run more than a few handful (maybe hundreds?) number of domains... So setting the limit to allow 32K of domains sound really wasteful and wouldn't really help to spot FD leaks.

I don't really see the problem to allow an admin to say "I will never run more than N domains". So the limit can be lower and would still function normally. Do you mind explaining why you think otherwise?


Julien Grall



Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.