Re: [Xen-devel] Lots of connections led oxenstored stuck

[Dave Scott <Dave.Scott@xxxxxxxxxx>]

On 8 Aug 2014, at 09:35, Liuqiming (John) <john.liuqiming@xxxxxxxxxx> wrote:

> In oxenstored it use "select" for incoming socket, so I don't think it can 
> handle more than 1024 socket connections. 

That’s true.

In the long term I’d like to use Lwt which internally uses libev and has a more 
scalable event loop.

In the short term I think Zheng Li (cc:d) may have a prototype patch to work 
around this issue. Is this right, Zheng?

Cheers,
Dave

> 
>> -----Original Message-----
>> From: xen-devel-bounces@xxxxxxxxxxxxx
>> [mailto:xen-devel-bounces@xxxxxxxxxxxxx] On Behalf Of Joe Jin
>> Sent: Friday, August 08, 2014 3:01 PM
>> To: David Scott; Luis R. Rodriguez; Ian Jackson
>> Cc: xen-devel
>> Subject: [Xen-devel] Lots of connections led oxenstored stuck
>> 
>> Hi,
>> 
>> During internal test on Xen-4.3-stable we found sometime when restarted
>> Xen, it stuck and does not response any request, xenstored.log filled
>> out below stuff:
>> [20140702T21:00:41.564Z|error|xenstored] caught exception
>> Unix.Unix_error(15, "accept", "")
>> 
>> I created reproducer which will create 2000 connections to oxenstored,
>> after
>> ran the reproducer, "xm list --long" will stuck, oxenstored does not
>> response anymore, same test case passed when use xenstored, any input
>> will appreciate!
>> 
>> /*
>> * This program used to test oxenstored connections stuck issue.
>> * please compile by below command:
>> *    gcc -o client client.c -lpthread
>> */
>> #include <stdio.h>
>> #include <sys/socket.h>
>> #include <sys/un.h>
>> #include <unistd.h>
>> #include <string.h>
>> #include <pthread.h>
>> #include <stdlib.h>
>> #include <errno.h>
>> 
>> 
>> void *main_thread(void *arg)
>> {
>>      struct sockaddr_un address;
>>      int socket_fd, nbytes;
>>      char buffer[256];
>>      int i;
>>      extern int errno;
>> 
>>      memcpy(&i, arg, sizeof(i));
>>      socket_fd = socket(PF_UNIX, SOCK_STREAM, 0);
>>      if (socket_fd < 0) {
>>              fprintf(stderr, "socket() %dth failed, errno=%d\n", i, errno);
>>              return;
>>      }
>>      fprintf(stderr, "socket() %dth ok!\n", i);
>> 
>>      /* start with a clean address structure */
>>      memset(&address, 0, sizeof(struct sockaddr_un));
>> 
>>      address.sun_family = AF_UNIX;
>>      snprintf(address.sun_path, 1024, "/var/run/xenstored/socket");
>> 
>>      if (connect(socket_fd,
>>                  (struct sockaddr *) &address,
>>                  sizeof(struct sockaddr_un)) != 0) {
>>              fprintf(stderr, "connect() %d failed, error=%d", i, errno);
>>              return;
>>      }
>>      fprintf(stderr, "connec() %dth ok!\n", i);
>> 
>>      while (1)
>>              sleep(1);
>>      if (arg) {
>>              free(arg);
>>              arg = NULL;
>>      }
>> 
>>      return;
>> }
>> 
>> int main(void)
>> {
>>      int i;
>>      for (i = 0; i < 2000; i++) {
>>              void *arg = malloc(sizeof(i));
>>              memset(arg, 0, sizeof(i));
>>              memcpy(arg, &i, sizeof(i));
>>              pthread_t thread;
>>              if (pthread_create(&thread, NULL, main_thread, arg) != 0) {
>>                      perror("pthread_create:");
>>                      break;
>>              }
>>      }
>>      /* Wait all children exit */
>>      sleep(3);
>>      return 0;
>> }
>> /* end */
>> 
>> Thanks,
>> Joe
>> 
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@xxxxxxxxxxxxx
>> http://lists.xen.org/xen-devel


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel