On 2014/10/20 13:32, Wen Congyang wrote: > On 10/20/2014 12:48 PM, Linhaifeng wrote: >> >> >> On 2014/10/20 10:12, Wen Congyang wrote: >>> On 10/18/2014 11:20 AM, Linhaifeng wrote: >>>> >>>> >>>> On 2014/10/17 21:26, Daniel P. Berrange wrote: >>>>> On Fri, Oct 17, 2014 at 04:57:27PM +0800, Linhaifeng wrote: >>>>>> >>>>>> >>>>>> On 2014/10/17 16:33, Daniel P. Berrange wrote: >>>>>>> On Fri, Oct 17, 2014 at 04:27:17PM +0800, haifeng....@huawei.com wrote: >>>>>>>> From: linhaifeng <haifeng....@huawei.com> >>>>>>>> >>>>>>>> The VM start with share hugepage should close the hugefile fd >>>>>>>> when exit.Because the hugepage fd may be send to other process >>>>>>>> e.g vhost-user If qemu not close the fd the other process can >>>>>>>> not free the hugepage otherwise exit process,this is ugly,so >>>>>>>> qemu should close all shared fd when exit. >>>>>>>> >>>>>>>> Signed-off-by: linhaifeng <haifeng....@huawei.com> >>>>>>> >>>>>>> Err, all file descriptors are closed automatically when a process >>>>>>> exits. So manually calling close(fd) before exit can't have any >>>>>>> functional effect on a resource leak. >>>>>>> >>>>>>> If QEMU has sent the FD to another process, that process has a >>>>>>> completely separate copy of the FD. Closing the FD in QEMU will >>>>>>> not close the FD in the other process. You need the other process >>>>>>> to exit for the copy to be closed. >>>>>>> >>>>>>> Regards, >>>>>>> Daniel >>>>>>> >>>>>> Hi,daniel >>>>>> >>>>>> QEMU send the fd by unix domain socket.unix domain socket just install >>>>>> the fd to >>>>>> other process and inc the f_count,if qemu not close the fd the f_count >>>>>> is not dec. >>>>>> Then the other process even close the fd the hugepage would not freed >>>>>> whise the other process exit. >>>>> >>>>> The kernel always closes all FDs when a process exits. So if this FD is >>>>> not being correctly closed then it is a kernel bug. There should never >>>>> be any reason for an application to do close(fd) before exiting. >>>>> >>>>> Regards, >>>>> Daniel >>>>> >>>> Hi,daniel >>>> >>>> I don't think this is kernel's bug.May be this a problem about usage. >>>> If you open a file you should close it too. >>> >>> If you don't close it, the kernel will help you when the program exits. >>> >> Yes,when the hugepage is only used for qemu,the kernel will free the file >> object.If the hugepage shared for other process,when qemu exit the kernel >> will not free the file. > > Even if the hugepage is shared with the other process, the kernel will auto > close the fd when qemu > exits. If the kernel doesn't do it, it is a kernel bug. > Kernel supply close to fix this bug.If you call open you must call close. If not, the result is unpredictability. >>>> >>>> This is <<linux man page>>about how to free resource of file. >>>> http://linux.die.net/man/2/close >>>> >>>> >>>> I'm trying to describe my problem. >>>> >>>> For example, there are 2 VMs run with hugepage and the hugepage only for >>>> QEMU to use. >>>> >>>> Before run VM the meminfo is : >>>> HugePages_Total: 4096 >>>> HugePages_Free: 4096 >>>> HugePages_Rsvd: 0 >>>> HugePages_Surp: 0 >>>> Hugepagesize: 2048 kB >>>> >>>> Run the two VMs.QEMU deal with hugepage as follow steps: >>>> 1.open >>>> 2.unlink >>>> 3.mmap >>>> 4.use memory of hugepage.After this step the meminfo is : >>>> HugePages_Total: 4096 >>>> HugePages_Free: 0 >>>> HugePages_Rsvd: 0 >>>> HugePages_Surp: 0 >>>> Hugepagesize: 2048 kB >>>> 5.shutdown VM with signal 15 without close(fd).After this step the meminfo >>>> is : >>>> HugePages_Total: 4096 >>>> HugePages_Free: 4096 >>>> HugePages_Rsvd: 0 >>>> HugePages_Surp: 0 >>>> Hugepagesize: 2048 kB >>>> >>>> Yes,it works well,like you said the kernel recycle all resources. >>>> >>>> For another example,there are 2 VMs run with hugepage and share the >>>> hugepage with vapp(a vhost-user application). >>> >>> The vapp is your internal application? >>> >> Yes vapp is a application to share the QEMU's hugepage.So threr are two >> process use the hugepage. >> >>>> >>>> Before run VM the meminfo is : >>>> HugePages_Total: 4096 >>>> HugePages_Free: 4096 >>>> HugePages_Rsvd: 0 >>>> HugePages_Surp: 0 >>>> Hugepagesize: 2048 kB >>>> >>>> Run the first VM.QEMU deal with hugepage as follow steps: >>>> 1.open >>>> 2.unlink >>>> 3.mmap >>>> 4.use memory of hugepage and send the fd to vapp with unix domain >>>> socket.After this step the meminfo is: > > Do you modify qemu? > >>>> HugePages_Total: 4096 >>>> HugePages_Free: 2048 >>>> HugePages_Rsvd: 0 >>>> HugePages_Surp: 0 >>>> Hugepagesize: 2048 kB >>>> >>>> Run the second VM.After this step the meminfo is: >>>> HugePages_Total: 4096 >>>> HugePages_Free: 0 >>>> HugePages_Rsvd: 0 >>>> HugePages_Surp: 0 >>>> Hugepagesize: 2048 kB >>>> >>>> Then I want to close the first VM and run another VM.After close the first >>>> VM and close the fd in vapp the meminfo is : >>>> HugePages_Total: 4096 >>>> HugePages_Free: 0 >>>> HugePages_Rsvd: 0 >>>> HugePages_Surp: 0 >>>> Hugepagesize: 2048 kB >>> >>> Does the qemu still run after you close the first VM? If the qemu exits, >>> the fd will be closed by the kernel, so this >>> bug is very strange. >>> >> qemu is not run when close the first VM.If other process used the file will >> be closed by kernel too? > > If qeum doesn't run after the first vm is closed, the fd should be closed > even if another process uses the file. > >> >>>> >>>> So failed to run the third VM because the first VM have not free the >>>> hugepage.After apply this patch the meminfo is: >>>> HugePages_Total: 4096 >>>> HugePages_Free: 2048 >>>> HugePages_Rsvd: 0 >>>> HugePages_Surp: 0 >>>> Hugepagesize: 2048 kB >>>> So i can run the third VM success. >>>> >>> >>> >>> . >>> >> > > > . >
-- Regards, Haifeng
