I figure I might as well add to this discussion. I'm a daily WinDbg with
GFlags user/addict, to the point where I can go from a vanilla system to
a full debug system in a few minutes (it'd be faster if it weren't for
the delay of downloading symbols from the Microsoft Symbol Server).
Following is a tutorial I've passed around our Engineering department a
few times.
I thought I'd add a "little" with how we use WinDbg (commonly referred
to as "windbag" by Windows developers). The latest version can always be
downloaded from
http://www.microsoft.com/whdc/devtools/debugging/default.mspx. The
default installation options are best, and are used in paths in these
instructions.
Pageheap and GFlags
http://support.microsoft.com/default.aspx?scid=kb;en-us;286470
While these settings can help you in configuring a process to start in
the debugger with options to catch exceptions earlier, we have an easier
way to do that. This page is an excellent resource on what the debugger
does to your process with these options.
Our easier way of enabling these options comes in form of the attached
.REG file. As you can see, this will merge settings into the registry
for our service, SomeService.exe. I'd do this for every process you want
to run in the debugger. It instructs Windows to start this process in
the debugger with full pageheap and gflag options.
If you're doing this for a service running under the Local System
account, you must go in to the Service Control Manager and set the
service properties to "Interact with the desktop". For services that run
as a specific user, you cannot use the Service Control Manager. Start
those types of services by opening Windbg and navigating to File -> Open
Executable. Be aware of services that depend on other services or make
RPC calls into services, as these will start them up when you might not
want them to. Once Windbg is open for any process, you should enter "g"
in the WinDbg command line to get the process going. If you wait too
long to do this for services, they will time-out. Non-service processes
don't need to worry about any of this, other than pressing "g" once you
want to begin.
The first thing I do on a new system is download that system's Windows
symbols from the Microsoft Symbol Server to the C:\websymbols directory
so I don't have to dynamically download each symbol when its image is
referenced. This can be done with symchk.exe, which comes with Debugging
Tools for Windows. I point it at the system32 (or equivalent) directory
recursively, and go about configuring and debugging as it can take an
hour:
symchk /r c:\windows\system32 /s
SRV*c:\websymbols\*http://msdl.microsoft.com/download/symbols
If you are using a process in WinDbg for the first time, you need to
configure the Workspace and save it. I do the following for each
process:
1. Navigate to File -> Symbol File Path.
2. Add
"c:\SomeProcess;c:\websymbols;SRV*c:\websymbols*http://msdl.microsoft.co
m/download/symbols", where "c:\SomeProcess" represents the path where
you've added the symbols of your process.
3. Press OK.
4. Navigate to File -> Image File Path.
5. Add "c:\SomeProcess;c:\windows\system32", where
"c:\SomeProcess" represents the path where your process is located.
6. Press OK.
7. Navigate to File -> Save Workspace.
Once you've saved the workspace, each time you start this process from
its location, all of these settings will be loaded.
When an unhandled exception occurs, the "analyze -v" commad is every
developer's best friend. It'll give you a very robust stack trace and
line information where your problem is (if symbols are properly loaded).
The "lm" command will show you loaded symbols and the "ld *" command
will force all symbols to be loaded. You can save a dump file for later
by using the ".dump /mfa c:\mydump.dmp" command.
Almost everything else you'd want to know can be found in the help file
for Debugging Tools for Windows. It has a lot of useful information.
Enjoy.
Joel Peterson
Quality Assurance Engineer
[EMAIL PROTECTED] <mailto:[EMAIL PROTECTED]>
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of Rob
Mensching
Sent: Tuesday, December 05, 2006 12:27 PM
To: Michael Osmond; wix-users@lists.sourceforge.net
Subject: Re: [WiX-users] ConfigureSQL Bug
You might try running the install under a debugger after using GFlags (
http://technet2.microsoft.com/WindowsServer/en/library/b6af1963-3b75-42f
2-860f-aff9354aefde1033.mspx?mfr=true) to turn on PageHeap. PageHeap
might be able to catch the memory corruption when it happens instead of
waiting for the second corruption to hit. I would also suggest try
using the command-line debuggers instead of VS. The command-line
debuggers are a lot less invasive (and insanely more difficult to use
<smile/>). I've had much more luck catching these kinds of problems
with the command-line debuggers + PageHeap.
It sounds like you have a SQL Script in exactly the right "shape" to
reproduce the problem. If this is the first script in your install,
maybe you could share the Script/MSI with me and I could try to repro as
well. I don't have time right now but maybe next week if you don't find
it first.
From: Michael Osmond [mailto:[EMAIL PROTECTED]
Sent: Tuesday, December 05, 2006 11:36
To: Rob Mensching; wix-users@lists.sourceforge.net
Subject: RE: Re: [WiX-users] ConfigureSQL Bug
That's what the other developers here said (re being caused elsewhere).
Anything specific you would like us to try?
-----Original Message-----
From: Rob Mensching [mailto:[EMAIL PROTECTED]
Sent: Wed 6/12/2006 5:09 AM
To: Michael Osmond; wix-users@lists.sourceforge.net
Cc:
Subject: RE: Re: [WiX-users] ConfigureSQL Bug
Wow, wickedly awesome! This is fantastic... thank you so much.
I've had nightmares about this bug.
More likely than not memory got corrupted somewhere before that
call and StrAllocStringAnsi() just happened to touch it the second time
thus blowing up. At least, that was my conjecture before, your analysis
will hopefully help us nail this bug. Thanks so much.
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of Michael
Osmond
Sent: Tuesday, December 05, 2006 03:01
To: wix-users@lists.sourceforge.net
Subject: Re: [WiX-users] ConfigureSQL Bug
Rob,
Some more information. I have spent some time adding log
messages into both ConfigureSQL and ScaSqlStrsReadScripts (in
scasqlstr.cpp).
We have been able to narrow down that the crash occurs in the
function "StrAllocStringAnsi" but not where or why. In this case it
appears to be while processing the first script file in the install
sequence.
It appears as though we don't get a return from
StrAllocStringAnsi.
We hope to spend a little more time on it tomorrow.
Michael
-----Original Message-----
From: Michael Osmond
Sent: Thu 30/11/2006 8:27 AM
To: wix-users@lists.sourceforge.net
Cc:
Subject: RE: [WiX-users] ConfigureSQL Bug
Dana, Rob,
Thanks for the info.
A few stats from me. We have been using WIX to ship 10
production systems (these are server based installs) for almost 2 years.
Most have been built on Wix 2.3220 as it was stable and working at the
time we started. Until recently I did all of our upgrades as minor
upgrades - and never saw this issue other than on the mailing list. So
I concur this happens only in a Major upgrade scenario.
At first I thought it was something to do with the
contents of the SQL scripts. Because I now have 2 systems using the
major upgrade and one does exhibit this problem and the other doesn't.
>From an MSI point of view there is nothing different between the two
installer packages - they use the same file names and install sequence,
its just the contents of the SQL scripts that are different.
But (there's always a but) ... looking at a log where
it fails and a log that is good (see the bottom of my first email) it
appears that the script crashes at the start before it starts reading
the script tables.
I was just starting to put some log messages into the
code yesterday afternoon a la Rob's suggestion. So I will spend some
more time on that today. I need to nail this one if we can. Please
bear with me though I am not a developer by trade so things like
pageheap don't mean a great deal.
Michael
________________________________
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of Dana
Gutride
Sent: Thursday, 30 November 2006 2:58 AM
To: wix-users@lists.sourceforge.net
Subject: Re: [WiX-users] ConfigureSQL Bug
This problem has just become a bigger deal where I work.
Maybe those of us that have experienced this problem could put our heads
together to try to figure out what we are doing in common that is
causing this to happen.
During database upgrades, we connect to an existing
database, run 3-4 sql strings, and approximately 100 sql scripts (loaded
from binary references) that total about 1.2 MB in size. This happens
with much greater consistency on the upgrade and almost never on a new
install where we create the database.
Dana
On 11/29/06, Rob Mensching < [EMAIL PROTECTED]
<mailto:[EMAIL PROTECTED]> > wrote:
Based on previous reports, I believe that there is a bug
in the SQL CustomActions that has been near impossible to track down
where somewhere memory gets corrupted and causes the CustomAction to
crash. You see leaked handles because MSI catches the exception and
tells you that your code did not clean up correctly (duh, it crashed).
I've never seen a consistent repro and never under a
debugger. Normally it was failing like 5% so your 50% is amazing. You
might try using pageheap/gflags + a debugger. Pageheap may force the
crash even when the debugger normally hides it. You also *might* try
adding a bunch more logging statements to the code to try and narrow
down how far the code gets before crashing. At the moment there is a
lot of code that could be causing the problem.
Killing this bug would be very, very good.
From: [EMAIL PROTECTED] [mailto:
[EMAIL PROTECTED]
<mailto:[EMAIL PROTECTED]> ] On Behalf Of Dana
Gutride
Sent: Wednesday, November 29, 2006 05:40
To: wix-users@lists.sourceforge.net
Subject: Re: [WiX-users] ConfigureSQL Bug
I have also noticed this bug. Oddly enough it is
happening during an upgrade only. During our initial install when we
create the database, this doesn't happen. During the upgrade, the user
can select the database to upgrade, 10-15 SqlScripts are all run against
it and I see the same error. It is tough to track down because it does
happen periodically, and a reboot almost always clears the problem up.
Dana
On 11/29/06, Michael Osmond <[EMAIL PROTECTED]>
wrote:
Hello Rob and All,
I have a sort of reproducible case of ConfigureSql
MSIHandle leaking error. I understand this one has not been found
The error shows up as the following log output:
MSI (s) (BC:48) [10:05:24:840]: Doing action:
ConfigureSql
Action 10:05:24: ConfigureSql. Configuring SQL Server
Action start 10:05:24: ConfigureSql.
MSI (s) (BC:6C) [10:05:24:900]: Invoking remote custom
action. DLL: C:\WINDOWS\Installer\MSI48.tmp, Entrypoint: ConfigureSql
MSI (s) (BC:6C) [10:05:25:593]: Leaked MSIHANDLE (393)
of type 790531 for thread 1228
MSI (s) (BC:6C) [10:05:25:593]: Leaked MSIHANDLE (392)
of type 790531 for thread 1228
MSI (s) (BC:6C) [10:05:25:593]: Leaked MSIHANDLE (381)
of type 790540 for thread 1228
MSI (s) (BC:6C) [10:05:25:593]: Leaked MSIHANDLE (380)
of type 790540 for thread 1228
MSI (s) (BC:6C) [10:05:25:593]: Leaked MSIHANDLE (318)
of type 790541 for thread 1228
MSI (s) (BC:6C) [10:05:25:593]: Note: 1: 2769 2:
ConfigureSql 3: 5
DEBUG: Error 2769: Custom Action ConfigureSql did not
close 5 MSIHANDLEs.
Internal Error 2769. ConfigureSql, 5
Action ended 10:05:25: ConfigureSql. Return value 3.
I am using WIX 2.0.4415.0
What can I tell you about it:
1. In my case its occuring during a major upgrade
during the installation of the new version.
2. It is not a hard problem. On our main test server
(where we needed to get the install completed so the testers could work)
it was occuring 95% of the time. On a second test server about 50% of
the time and on a development PC (with IDE etc) only ever once.
3. I cannot get a consistent pattern to bringing the
problem on. Some times a reboot will clear the problem, other times it
occurs straight after a reboot.
4. On the 2nd test server it was more likely to occur
if the MSI was accessed from a UNC path rather than local. But again
this was not consistent.
5. I tried to debug the custom action (I was using the
Remote debug tools in VS2003 to attach to the test server). But when I
used a DLL that allowed debugging the error would not occur. Its
almost as if the problem is to do with timing (the test servers are both
Virtual Machines).
However. It does appear to be happening in the inital
part of the Custom Action, before it begins scripting the deferred
custom actions. Next is the relevant snippet of the log if the error
does not occur:
MSI (s) (4C:18) [08:50:43:250]: Doing action:
ConfigureSql
Action 8:50:43: ConfigureSql. Configuring SQL Server
Action start 8:50:43: ConfigureSql.
MSI (s) (4C:CC) [08:50:43:311]: Invoking remote custom
action. DLL: C:\WINDOWS\Installer\MSI102.tmp, Entrypoint: ConfigureSql
MSI (s) (4C!08) [08:50:56:163]: PROPERTY CHANGE:
Adding CreateDatabase property. Its value is
'DBSQL2?PERM-WIN2K3??CINS?1?1???0?0'.
MSI (s) (4C!08) [08:50:56:183]: Doing action:
CreateDatabase
Action 8:50:56: CreateDatabase. Creating Databases
Action start 8:50:56: CreateDatabase.
1: Creating Databases
Action ended 8:50:56: CreateDatabase. Return value 1.
MSI (s) (4C!08) [08:50:56:244]: PROPERTY CHANGE:
Adding ExecuteSqlStrings property. Its value is
'sqlserver_Master?PERM-WIN2K3??master?-2147483648?
1???SingleUser_RI?17?alter database CINS set SINGLE_USER With Rollback
Immediate?backup_RI?17?backup database CINS to
disk='C:\DOCUME~1\ADMINI~1.TES\LOCALS~1\Temp\CINS.bak' with INIT'.
MSI (s) (4C!08) [08:50:56:254]: Doing action:
ExecuteSqlStrings
Action 8:50:56: ExecuteSqlStrings. Executing SQL
Strings
Action start 8:50:56: ExecuteSqlStrings.
1: Executing SQL Strings
Action ended 8:50:56: ExecuteSqlStrings. Return value
1.
MSI (s) (4C!08) [08:50:56:305]: PROPERTY CHANGE:
Adding RollbackExecuteSqlStrings property. Its value is
'sqlserver_Master?PERM-WIN2K3??master?-
2147483648?1???restore_singleUser_RI?25?alter database CINS set
SINGLE_USER With Rollback Immediate?restore_drop_RI?25?drop database
CINS?restore_cmd_RI?25?restore database CINS from
disk='C:\DOCUME~1\ADMINI~1.TES\LOCALS~1\Temp\CINS.bak'?multiUserRB_RI?25
?alter database CINS set MULTI_USER'.
MSI (s) (4C!08) [08:50:56:326]: Doing action:
RollbackExecuteSqlStrings
Action 8:50:56: RollbackExecuteSqlStrings. Rolling
back SQL Strings
Action start 8:50:56: RollbackExecuteSqlStrings.
1: Rolling back SQL Strings
Action ended 8:50:56: RollbackExecuteSqlStrings.
Return value 1.
I am happy to help further in getting to the bottom of
this, but am unsure what you need further.
Michael
------------------------------------------------------------------------
-
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the
chance to share your
opinions on IT & business topics through brief surveys -
and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDE
V
_______________________________________________
WiX-users mailing list
WiX-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/wix-users
ÿþW i n d o w s R e g i s t r y E d i t o r V e r s i o n 5 . 0 0
[ H K E Y _ L O C A L _ M A C H I N E \ S O F T W A R E \ M i c r o s o f t \ W i n d o w s
N T \ C u r r e n t V e r s i o n \ I m a g e F i l e E x e c u t i o n
O p t i o n s \ S o m e P r o c e s s . e x e ]
" G l o b a l F l a g " = " 0 x 0 2 0 0 0 0 0 0 "
" P a g e H e a p F l a g s " = " 0 x 0 0 0 0 0 0 0 3 "
" D e b u g g e r " = " C : \ \ P r o g r a ~ 1 \ \ D e b u g g ~ 1 \ \ w i n d b g . e x e "
-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
WiX-users mailing list
WiX-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/wix-users
