Re: storagedriver domain limited to 61 "shares" - Xen 4.16.6 pre2

[Juergen Gross <jgross@xxxxxxxx>]

On 12.02.24 07:57, J. Roeleveld wrote:
> On Monday, February 12, 2024 7:40:39 AM CET Juergen Gross wrote:
> > On 11.02.24 17:21, J. Roeleveld wrote:
> >
> > > Hi all,
> > >
> > > I am running into an issue with my storagedomain.
> > > I have the HBAs assigned to the domain,
> > > Added "driver_domain=1" to the config.
> > >
> > > And am accessing filesystems on this domain from other domains.
> > >
> > > This works as expected.
> > >
> > > However, I am only able to assign 61 filesystems to other domains. As soon
> > > as I attempt to assign a 62nd filesystem, it fails.
> > >
> > > I am completely at a loss where this limit of 61 is coming from and am
> > > unable to find anything with over 2 days of google searches.
> > >
> > > I also noticed that "xl devd" isn't actually running on my storagedomain
> > > anymore. It used to run in a previous version, but as the whole
> > > environment was booting succesfully I never bothered to keep checking.
> > >
> > > I do remember that it was running in the past. Currently, I think the
> > > init- script is stopping because "/proc/xen/capabilities" is empty.
> > >
> > > Please let me know which config-files are needed to troubleshoot this.
> >
> > My first guess would be that the driver domain is limited by the max number
> > of Xenstore nodes it is allowed to own.
> >
> > You can raise the default of 1000 nodes per domain (how to do that depends
> > on the Xenstore type you are running, xenstored or oxenstored).
> Hi Juergen,
>
> Many thanks for this. I forgot I already raised this in the past to 10000 on
> the host and the current amount of entries is quite close to this limit:
> # xenstore-ls | wc -l
> 9590
>
> I will increase this and try again when I have a chance to reboot the host.
Which version of Xen are you running?

In case it is not too old (>4.15), you could try to restart xenstored via
live update giving it the new parameters. Note that this feature is still
"Tech Preview", but there are no issues I'm aware of. In dom0 you'd do:

  xenstore-control live-update -c '<new parameters>' /usr/sbin/xenstored

> Is there any benefit of using "oxenstored" over "xenstored" ? I currently use
> "xenstored" and apart from this, don't seem to have any issues.
I'm biased, as I'm the maintainer of "xenstored". :-)

BTW, oxenstored has the same default of 1000 nodes per domain.


Juergen

Attachment: OpenPGP_0xB0DE9DD628BF132F.asc
Description: OpenPGP public key

Attachment: OpenPGP_signature.asc
Description: OpenPGP digital signature