Well, further developments :-). I never got round to specifically creating a verbose log under the conditions I suspected were causing the reboot. Instead it happened again today, but this time I noticed that the MsiInstaller outputted a list of files that were locked and what processes were locking them into the event logs. This is where it gets interesting. Now I am not sure if this is my misunderstanding of .NET or it is a bug with the MSI installer (which I would have thought would be found by now so I think that's unlikely).
There are quite a few dll's shared between the web application and it's services (such as a DAL dll and some third party's like log4net). These dlls are not installed to one location and referenced by each application (like they probably should be, using something like a merge module), instead they are installed with each application. So for example there is a log4net dll in the same directory as every service and in the web\bin directory. According to the event logs every service (and dllhost, which I bet is the COM with some .NET COM dlls, so that's another challenge to solve later) has a lock on the log4net dll inside the web\bin directory, I assume instead of its own. This confuses me a lot, why would the services lock the web applications log4net.dll instead of the one inside their own directories? To further investigate this I installed unlocker, then for each instance of log4net on the server I found out what applications had a lock on the file. Locker gave me the results I was expecting, each service had a lock on only its copy of log4net. The way I see it there are 2 options: 1. Unlocker is lieing to me and .NET does some really weird locking when it comes to a dll existing on a system multiple times. 2. Msi Installer is lieing to itself and see's the other applications locks on an identical file in different locations as the same file, which to me would be a bug in msiinstaller. The obvious work around is to not only stop IIS and COM before running the upgrade, but to also stop all the services to prevent them accessing the file. Of course I didn't get to test this either, but again this seems excessive. Any thoughts on this behaviour, work-arounds or even a proper solution? Thanks, Adam On Thursday 22 January 2009 00:24:47 Michael Osmond wrote: > Adam, > > Good luck!! > > Michael > > -----Original Message----- > From: Adam Burton [mailto:adz...@googlemail.com] > Sent: Thursday, 22 January 2009 9:53 AM > To: wix-users@lists.sourceforge.net > Subject: Re: [WiX-users] Prevent reboot. > > ::Rolls eyes and bows head:: oh I really hate COM .... it's always being > difficult with me lol. I hadn't thought about the COM stuff, we run it > as a server process so I bet that's still chilling out somewhere in the > processes. I bet that when I run a verbose log tomorrow it will confirm > that. > > Unfortunately as if COM didn't have enough of "a life of its own" the > code we have in VB6 isn't really well maintained in terms of keeping > binary compatibility etc so we don't keep the same GUIDs for components > and so on, therefore my control of the COM isn't exactly ... clean. I > suppose I may need to write a script that searches for our COM > applications by name and performs a shutdown after stopping IIS, > shouldn't be too hard. Although I guess it depends when the MSI performs > its checks for file locks whether this is a cause of the problem. As > (again ... unfortunately) I am forced to run a custom VBS script that > registers the COM components (although I have to say due to the lack of > proper maintaining of our COM components binary compatibility etc its > made using WiX easier to use our custom scripts than the COMPlus > extension, "solve" evil with evil I guess) and at the start of the > install I run another VBS script that removes our COM applications > before performing any file operation. At the moment though I am at home > so I can't remember where I put the custom action in the sequence, but I > would presume it would be before the lock checks (I am guessing they are > done as it looks at each individual file on the fly not checks each file > for locks then removes it or whatever) so removing the COM application > should stop COM from locking its DLLs. > > Regardless I see 2 options, IIS is doing something funky with its file > locks or I need to remove my COM applications earlier to prevent MSI > bumping into COMs file locks. I guess I'll have to see tomorrow :-) ... > pending I get the time. > > Thanks for your help so far. > > Adam > > On Wednesday 21 January 2009 22:59:01 Michael Osmond wrote: > > Adam, > > > > I agree (and our experience bears it out), IIS reset should kill off > > any thing that the web app is doing (actually we rarely even need to > > do that. Unless is some of the web application code is in COM+, that > > can have a life of its own. > > > > Michael > > > > -----Original Message----- > > From: Adam Burton [mailto:adz...@googlemail.com] > > Sent: Thursday, 22 January 2009 8:28 AM > > To: wix-users@lists.sourceforge.net > > Subject: Re: [WiX-users] Prevent reboot. > > > > Michael, > > > > Well yea that will be sorta the case. Some people will probably be in > > the middle of using the site (if the poor souls are sad enough or > > getting close to dead line to feel the need to work during lunch :-) > > ), not had chance to tell it to send out a warning email yet ... > > although they should know by now when it is going to happen, when > > suddenly the process stops IIS and copies the MSI to a place > > accessible by the server then installs it (the MSI is about 30MB for > > the website, with our poor network at this time we are talking several > > > (10-20) seconds before its copied and started the install which I feel > > > is enough time for IIS to release the locks on any dlls). At least > > with the services I can understand Windows Installer maybe being too > > quick to start installing before we its let the service settle > > (althought like i said I would think its smarter than that) but the > website I am a bit stumped with. > > > > Adam > > > > > > On Wednesday 21 January 2009 22:10:55 Michael Osmond wrote: > > > Adam, > > > > > > I suspect there is something more going on here. We have been > > > following a test deployment process like yours for several years for > > > > a > > > > > suite of large web applications, and the only time we have file lock > > > > issues is if someone is actively testing (hitting the pages) when > > > the > > upgrade occurs. > > > > > > > > > Michael > > > > > > -----Original Message----- > > > From: Adam Burton [mailto:adz...@googlemail.com] > > > Sent: Thursday, 22 January 2009 5:53 AM > > > To: General discussion for Windows Installer XML toolset. > > > Subject: Re: [WiX-users] Prevent reboot. > > > > > > I don't know about that. I can understand why it gets the locks etc, > > > > but its the releasing that seems to mess up for me, or behave in a > > > manor I would not except. For example the services I install ... why > > > > are files still locked when the msi stops the service ... the > > > service is the only thing that will use those file ... I can't see > > > why I should need to stop the service for the msi before hand. Same > > > with the > > > > > IIS files, IIS is stopped, it should release those DLLs that are > > > only accessed by IIS (they live in the website 'bin' directory, > > > surely once > > > > > IIS is stopped it has no use for them and should release them). I > > > have > > > > > just always assumed that it is a separate service/process/thread > > > that releases the locks and its just too slow off the mark for the > > > MSI. If that is the case though then you'd think there'd be stuff in > > > > Windows Installer that would take that into account so I'd at least > > > only be rebooting every so often rather than a majority of the time. > > > > > > Also its not uncommon for me to delete a directory and the directory > > > > wont go away. With some inspection using stuff like process explorer > > > > it appears "System" has a lock on it (presumably for shi**z and > > > giggles), soon as you kill the handle the directory goes. While the > > > lock idea is sane, Windows implementation doesn't seem to work for > > > me, > > > > > therefore in its current state it's not sane for me :-). > > > > > > Before anyone thinks I am Windows bashing this is just one of the > > > gripes I have with Windows, I have plenty of gripes with other OS's > > too. > > > Although I have to say on the plus side I am yet to encounter the > > > issue in Vista, but that being said I don't tend to delete > > > directories > > > > > often or install already running services in Vista so maybe I've > > > just not trodden the path :-). > > > > > > Regardless I must be doing something else wrong because even if I > > > got the file's name from the log file I still wouldn't know where to > > start. > > > Anything that is supposed to use the files, in the case of IIS, is > > > shut down and for the services it is shut down by Windows installer > > > which I would think is smart enough to handle a situation like > > > shutting down the service correctly so any files locked only by it > > > will be released sometimes preventing the requirement of a reboot. > > > > > > On Wednesday 21 January 2009 18:03:05 Rob Mensching wrote: > > > > MSI verbose log file should point out the file being locked. Then > > > > > you > > > > > > > just need to figure out how to get the thing to be unlocked. > > > > That's > > > > > > how Windows works. It's perfectly sane, just not always very > > > > convenient. <smile/> > > > > > > > > -----Original Message----- > > > > From: Adam Burton [mailto:adz...@googlemail.com] > > > > Sent: Wednesday, January 21, 2009 06:00 > > > > To: WiX-users@lists.sourceforge.net > > > > Subject: [WiX-users] Prevent reboot. > > > > > > > > Hi, > > > > I have created some WiX based installers for some services and a > > > > web > > > > > > application I am working on at work. We use the installers as part > > > > > of our test rigs, so we patch our test changes, build the new > > > > version and > > > > > > > install it. Since this whole process is automated if the MSI > > > > determines it needs to reboot, it will. > > > > > > > > Previously when I installed the services it would cause a reboot, > > > > I assume due to file locks even though the installer would stop > > > > the service. My resolution was to stop the service before > > > > performing the > > > > > > upgrade, this seems to have solved that issue, although it seems > > > > unnecessary. I am now getting the same problem with the Web > > > > Application. It automatically deploys a new test build 2 times > > > > during the day, once in early morning before anyone is here and > > > > once > > > > > > at lunch. The morning build runs fine (I would guess this is > > > > because > > > > > > no ones use the application for hours so IIS has released its file > > > > locks) but the afternoon builds during lunch forces a reboot most > > > > of > > > > > > the time (I would guess because IIS has file locks due to recent > > > > activity), which is not only annoying because its rebooting the > > > > server, but also the server hosts other software (such as virtual > > > > servers) taking that down. I have tried stopping IIS before it > > > > begins the install but it still causes a reboot most of the time. > > > > Is > > > > > > there a switch that could solve this or some way to make windows > > > > locking more > > > sane or something? I am just stumped. > > > > > > > > Thanks in advance, > > > > Adam. > > > > ------------------------------------------------------------------ > > > > -- > > > > -- > > > > -------- > > > > This SF.net email is sponsored by: > > > > SourcForge Community > > > > SourceForge wants to tell your story. > > > > http://p.sf.net/sfu/sf-spreadtheword > > > > _______________________________________________ > > > > WiX-users mailing list > > > > WiX-users@lists.sourceforge.net > > > > https://lists.sourceforge.net/lists/listinfo/wix-users > > > > > > > > > > > > ------------------------------------------------------------------ > > > > -- > > > > -- > > > > -------- > > > > This SF.net email is sponsored by: > > > > SourcForge Community > > > > SourceForge wants to tell your story. > > > > http://p.sf.net/sfu/sf-spreadtheword > > > > _______________________________________________ > > > > WiX-users mailing list > > > > WiX-users@lists.sourceforge.net > > > > https://lists.sourceforge.net/lists/listinfo/wix-users > > > > > > > > > > -------------------------------------------------------------------- > > > -- > > > -- > > > ------ > > > This SF.net email is sponsored by: > > > SourcForge Community > > > SourceForge wants to tell your story. > > > http://p.sf.net/sfu/sf-spreadtheword > > > _______________________________________________ > > > WiX-users mailing list > > > WiX-users@lists.sourceforge.net > > > https://lists.sourceforge.net/lists/listinfo/wix-users > > > > > > > ---------------------------------------------------------------------- > > -- > > ------ > > This SF.net email is sponsored by: > > SourcForge Community > > SourceForge wants to tell your story. > > http://p.sf.net/sfu/sf-spreadtheword > > _______________________________________________ > > WiX-users mailing list > > WiX-users@lists.sourceforge.net > > https://lists.sourceforge.net/lists/listinfo/wix-users > > > > ------------------------------------------------------------------------ > ------ > This SF.net email is sponsored by: > SourcForge Community > SourceForge wants to tell your story. > http://p.sf.net/sfu/sf-spreadtheword > _______________________________________________ > WiX-users mailing list > WiX-users@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/wix-users > ------------------------------------------------------------------------------ This SF.net email is sponsored by: SourcForge Community SourceForge wants to tell your story. http://p.sf.net/sfu/sf-spreadtheword _______________________________________________ WiX-users mailing list WiX-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wix-users
