You have a managed application crash dump and you would like to load sos.dll, to use the powerful commands it provides to help with managed debugging, but the load of sos.dll always fails. The command that you are executing for loading sos.dll is…

0:015> .loadby sos clr
Unable to find module ‘clr’

On enter you see the above error and you are not sure what is going on. So in order to understand the error message it is important to understand what .loadby command does.

“.loadby sos clr” means load sos.dll from where clr.dll is loaded. So as you might have guessed by now if clr.dll is not loaded then sos.dll cannot be found. This is what the error means when it says “Unable to find module ‘clr’”. All it means is clr.dll is not in the loaded module list which means there is no path to locate clr.dll.

So how can I fix this error? In order to fix this error you must understand that clr.dll has been introduced into .net applications from .net 4.0 and prior to that we used to have mscorwks.dll. So if your .net application is not a 4.0 app then clr.dll won’t be loaded, yes you will have to use mscorwks.dll instead…

0:015> .loadby sos mscorwks

But you are damn sure that the application that crashed is a .net 4.0 application but you still see the error! All I can say is clr.dll is not loaded. Wait for it to load or break on its load and then execute .loadby sos clr.

So how to break on a module load…
0:015> sxe ld clr

This will for sure help!

Share
 

Found a nice article which describes how to take native/managed crash dumps automatically using CDB as soon as a crash takes place. We know that for CLR 2.0 we’ve got to configure crash handler settings at two places in the registry for a native and managed application.

For native

  1. HKLM\Software\Microsoft\Windows NT\Current Version\AeDebug\Debugger
  2. HKLM\Software\Microsoft\Windows NT\Current Version\AeDebug\Auto

For managed

  1. HKLM\Software\Microsoft\.NETFramework\DbgManagedDebugger
  2. HKLM\Software\Microsoft\.NETFramework\DbgJITDebugLaunchSetting

This article from MSDN explains in detail on how to configure registry entries to take automatic crash dumps using CDB…

http://blogs.msdn.com/b/clrteam/archive/2009/10/15/automatically-capturing-a-dump-when-a-process-crashes.aspx

Share
 





Share
 

This blog entry deals with user mode dumps only. Kernel mode dump files is not dealt with here but should be quite similar.

Define dump file

It is the memory snapshot of a process. The dump file saves all information pertaining to a process. The information include, loaded modules/dlls, handles, executing threads and other stuffs. Optionally we can include heap information.

How to open a dump file?

Dump files can be opened in a debugger as you would normally open a process or attach to a process. All debuggers provide options for opening a dump file. Here is the menu option in WinDbg which opens a dump file…

Open dump file option in windbg

Open dump file option in windbg

A similar but, kinda hidden, version of open dump file option in Visual Studio, its not so visible unless you dig a bit. See below screenshot…

How to create a dump file?

A quick option which is quite is limited is to use the task manager “Create Dump File” option. Which looks like…

Task manager create dump file option
Task manager create dump file option

With windbg use the .dump /ma D:\DumpFile.dmp command. With visual studio you can attach to a process and use Debug->Save dump as or while you are debugging use Debug->Save dump as.

Another tool that comes along with ”Debugging tools for windows” is ADPlus, a VBScript based tool, which automates the process of creating a dump file. This tool works by using cdb.exe (console debugger) command line options. Very powerful. I’ll blog about the tool in another post.

Limitations of dump files

You cannot execute the dump file, just like you would normally do with a process, or set breakpoints; a dump is a ‘snapshot’ of a process at a particular point of time. So all stuff in memory up till that particular point of time will be available but not after that particular point of time. Also if you have a wrong dump file then you could end up wasting time, so in order to prevent such cases we have tools like ADPlus which when setup properly automates the process of capturing dumps.

Different kinds of dumps
  • Crash dump – automatically taken during a crash
  • Hang dump – automatically taken (using tools like ADPlus)
  • High CPU dump – automatically taken (using tools like ADPlus)
Dump file tips in windbg
  1. I normally execute the following command in WinDbg: !analyze -v. This command gives a summary of the stuff in dump file. So it shows you the stack of the thread in which the exception occurred.
  2. Also when you open a dump file in WinDbg you’ll get a brief summary of the exception that occurred and also a view of the registers.
  3. If you have a hang dump or a high CPU dump you will love the following command: !analyze -hang. This shows us the name of the function which is eating up most of CPU time.
  4. You will love the command !runaway which shows the excecution time for each thread, so this way you can identify the thread that’s taking most of CPU time.
  5. To know the type of dump file you are working on enter “||” into WinDbg. This command will tell you whether you are dealing with a full dump or a mini dump. A sample output on a dump of MsPaint.exe looks like…
    0 Full memory user mini dump: D:\MsPaint.dmp
  6. To know the name and id of the process for which this dump was made use the single pipe command “|”. Sample output looks like…
    0 id: 11e4 examine name: C:\Windows\System32\mspaint.exe
  7. .ecxr gives you exception record along with registers display.
  8. .lastevent shows you the last exception that occurred.
  9. You can create a mini dump out of a full dump by executing a .dump filename command on the existing dump.
  10. To view last error status of all threads use !gle -all
  11. Its very important to have the correct pdb files. Public symbol path and downstream store path should be set in WinDbg or by using _NT_SYMBOL_PATH environment variable. This is how mine looks -> SRV*C:\DebugSymbols*http://msdl.microsoft.com/download/symbols
  12. Don’t forget you are doing “post mortem” debugging. The dead doesn’t walk and talk. :)
  13. ~* kpn shows stack of all threads
  14. !locks displays locks held by threads
  15. !lmvm ModuleName display information about a module. !lmvm User32 display information about user32.dll.
    • lm sm display all modules loaded/unloaded in sorted order, ’when’ the dump was taken.
    • lm or lmvi display all modules loaded/unloaded in their load/unload order
  16. dt command displays type information of a type passed in.
  17. dv command displays all local variables
  18. x to examine a symbol. For e.g. to see all symbols in kernel32 starting LoadLibrary use following command: x kernel32!LoadLibrary*. The output looks like…
    00000000`77906640 kernel32!LoadLibraryExWStub =
    00000000`778fe3b0 kernel32!LoadLibraryExAStub =
    00000000`77907058 kernel32!LoadLibraryExA =
    00000000`77906f80 kernel32!LoadLibraryW =
    00000000`77907070 kernel32!LoadLibraryA =
    00000000`77906634 kernel32!LoadLibraryExW = 

Conclusion

Have fun debugging dump files and help others too. Spread word about this article. If you’ve got comments let me know. :)

Share
 

WinDbg rocks. :) Setting breakpoints is very easy in WinDbg. The command to set a breakpoint is ‘bp’. So if you want to break whenever a dll is loaded into a process then type in following command…

bp kernel32!LoadLibraryW

So to trigger this breakpoint attach ‘notepad.exe’ to the debugger and then type in this command. Now let the app run (press F5). Goto File->Open (this will trigger a definite LoadLibrary :)). Now have a look in WinDbg which will have following output…

Breakpoint 0 hit
kernel32!LoadLibraryW:
00000000`76e50420 4533c0          xor     r8d,r8d

 To view call stack, type in ‘kpn’. I’ll blog more on breakpoints as and when I get time. Happy debugging. :)

Share
 

What’s a breakpoint?

A breakpoint is defined as the location where a debugger breaks execution to allow the user to have a look or to modify the execution context.

What’s new with breakpoints?

With visual studio 2005 and 2008 behavior of breakpoint has changed. Some features that were added are as follows…

  1. Know hit count of a break point, no more need to keep a temp debugger variable for counting hits. For e.g. you can set a conditional breakpoint and then enable hit count. You’ll see how many times the condition was satisfied, you can also disable breaking of execution so that the program keeps running.
  2. Trace local variables/function name and more to the visual studio immediate/output window.
  3. Run a macro when a breakpoint is hit.
  4. We can disable breaking of execution which means there will only be tracing going on and no breaking of execution.

So to access these features, after adding a breakpoint, open the breakpoint window and right click on this breakpoint. Lower half of the dropdown contains these features, here is a screenshot of the context menu…

Breakpoint Context Menu

Breakpoint Context Menu

There are different types of break points available in visual studio. There are four in my knowledge…

  1. File breakpoint – Breaks at a location in a file
  2. Address breakpoint – Breaks at an address
  3. Function breakpoint – Breaks at a function
  4. Data breakpoint – Breaks at specific byte locations

Hit count

Let me show you how to count the number of even numbers from 1 to 100, I know it’s quite easy, it’s fifty, but via debugger? Here is a small function with a loop from 1 to 100.

void CountEven( const int From, const int To )
{
   for( int Index = From; Index < To; ++Index )
   {
      // Set a conditional breakpoint to get the count of odd numbers,
      // Dummy code to allow set a breakpoint
      ::SendMessage( AfxGetMainWnd()->GetSafeHwnd(), WM_NULL, 0, 0 );
   }
}

Now set a break point inside the for loop (Press F9) and right click and select “Breakpoint->Condition” item. You’ll get the following dialog…

Setting a Conditional breakpoint

Setting a Conditional breakpoint

So here I’ve set the condition that whenever Index%2 is zero then we have an even number. So when this happens I’m asking the debugger to break execution. So this works as expected but our aim is to count the number of even numbers using debugger, so for that again right click on the breakpoint line and select “Breakpoint->Hit Count” item, following dialog pops up…

Enable hit count in the debugger

Enable hit count in the debugger

You can see that I’ve selected an option called “break when the hit count is a multiple of”. I’ve given 10 as the option, so that frequency of breaking of execution is less. When I run the code this is what I get in the breakpoints window…

Hit count result

Hit count result

See the “Hit Count” item?  Execution broke 5 times this means for every 10 hits, hence count is 50. Isn’t this fun?

Tracing

Remember the times when we had to trace statements just for debugging purpose, for e.g. we wanted to trace a certain variable’s value when it satisfies a condition. Let’s take the above example, we’ve already enabled a conditional breakpoint such that it’s breaks execution whenever “Index” is an even number. Right click on the breakpoint line and select “Breakpoint” and then “When Hit” option. Following dialog pops up…

When Hit Dialog

When Hit Dialog

Check the first option Print a message”, this is the option that prints a message either to your “immediate window” or to your “Output window” based on the options that you’ve set. Now run the program to have some fun, this is the output in my computer…

Output of "when hit"

Output of "when hit"

All even index values are traced to the output window, isn’t this cool?

Filter

This one is even more cool. It allows us to set breakpoint filters. This is kind of a conditional but with a larger scope. For e.g. if you want to break execution only for a certain thread or only for a certain process. So for this right click on the breakpoint line and select “Breakpionts->Filter”. Following dialog pops up…

Breakpoint filter

Breakpoint filter

Pretty easy to use. So this means for a function that is called from multiple threads you can explicitly tell the debugger for which threads to break execution.

Tid bits

  1. Did you know that you can set a breakpoint in the call stack window by press F9?
  2. Did you know that you can delete all breakpoints by press Ctrl + Shift + F9?
  3. Did you know that you can run to cursor by pressing Ctrl + F10?
  4. Did you know that you display breakpoints window by pressing Alt + F9?
  5. Did you know that you can use “Set Next Statement” by pressing Ctrl + Shift + F10?

Yeap that’s it from me for now, all the best. :)

Share
 

It’s always irritating to debug paint messages and sometimes expensive too. I’ve tried different methods to try debug painting issues, always ended up disappointed.

Why is it such a pain to debug WM_PAINT messages?

Main reason is, we are not in control. Paint messages comes via a window framework. Painting is done whenever a window gains focus or is activated this means our application will never get focus if we set a breakpoint in an OnPaint function because as soon as our application comes to focus it again sends a repaint request which again breaks into the OnPaint function.

It’s can be expensive too because we’ll need some kind of a dual monitor setup. So that our application runs on one monitor and the debugger on the other. For professional game/UI developers it’s ok because that’s their job and they will always need such a setup. But guys like me cannot affort such a setup.

Any workaround?

Yes, that’s the good news. Windows explorer has a cute feature called “Tile Horizontally” and “Tile Vertically”. It also has another feature to select taskbar items using control key.

So this is how a standar taskbar looks, as you already know that at a time, in windows, only one window can be active.

taskbar_oneapp_selected

The tab having a dark shade is the one that’s active, and if you right click on this tab you’ll get a context menu which will correspond to the active application’s system menu.

Now here is the interesting part use ‘Ctrl’ key to select multiple tabs. Press ‘Ctrl’ and then use your mouse to click on tabs you wanna select and then right click on any of these selected tabs and see what you get! Here is a screenshot of what I get…

taskbar_multipleapp_selected

These menu items are interesting, most windows users are unaware of this option, it’s a quick way to arrange your windows. You can try them one by one. Note that there is an option to close a group too. :)

So returning back to our original discussion, we now apply this trick to our application. We are going to select visual studio and an application that’s being debugged. I’ve set a break point in the OnPaint function.

So we select both applications in the taskbar using ‘Ctrl’ key and mouse and then right click and select ‘Tile vertically’ or ‘Tile Horizontally’. I would select ‘Tile vertically’ as it would give me enough vertical space but you can decide what’s best for you. Eventually what I get is a screen like this with two windows that doesn’t overlap each other. One of them is visual studio and the other one is my application that’s being debugged, also note that I’ve set a breakpoint in OnPaint function. Both applications behave independently without interfering with each other.

vertical_tile

Now my desktop acts like a dual monitor setup. ;) It’s kinda cool for debugging purpose and yeah I agree this is not the ultimate solution but it’s quite useful. Hope this helps you. :)

Share
 

Why?

Well debugging exe is quite easy, but ever wondered how to debug a dll. Well why? Simple, you write a dll and you wanna debug a bug just like you would do with an executable.

Can you give us some examples?

Well so what is a dll? I think of it as a parasite because just like a parasite it’s dependent on an executable to get it running. Some examples follow…

  1. A good example to think of will be rundll32.exe. Ever thought about that name “rundll32.exe”, it just means “run a 32 bit dll using this exe”, so how does this achieve this, remember the command line that we pass to it, i.e. name of the dll followed by a procedure name exported from our dll.
  2. Another example to think of is regsvr32.exe, does the same stuff i.e. registers a COM dll but this time with a difference, we don’t have to give a procedure name. Because COM programming protocol defines mandatory functions to be exported from a COM dll and regsvr32.exe knows exactly which function to call based on the command line switches passed in.
  3. Another example that comes to mind is the windows surrogate process “dllhost.exe”, I’ve heard people say that it’s a virus ;) . Naah it’s a COM surrogate process which helps in hosting a COM dll to make the dll behave like a process hence providing the flexibility to behave like an executabable and at the same time like a dll, vice versa is not possible. Well it can at times be hosting a malicious dll too. It’s quite rare though.

What has this got to do with debugging a DLL?

The reason why I’ve given above examples is to help you understand that a DLL cannot run by itself, it needs a host process to help it run.

Exactly! same for debugging it needs a process to get it running so that execution reaches our breakpoint that we’ve set. Aha! so there must be some option to tell the debugger about the host process. Oh yeah, goto Project settings->Debugging->Executable for Debug Session. Select any process which is using your dll.

Think that you are writing a window’s shell extension, which are dlls, think on how to debug them. So for this case your target process will be explorer.exe. So you will specify explorer.exe as your process. Or imagine writing an activex control, so for this case the target process will be an activex test container.

This is a shot from VC6, it’s quite similar in VC7, VC8 and VC9, instead of tabs there you have a tree menu holding the “Debugging” item right at the top of the tree.

Well now just press F5 and see what happens, you will see your target process running and just try doing a button or menu click that could invoke the function in which you’ve set a breakpoint.  If you are writing a shell extension for e.g. extending shell context menu, then you would right click on a file in the explorer window.

Also note that if your active project is a ‘dll project’ then you’ll get prompted to enter a target process name to get the dll hosted as soon as you press F5. So no need to go to project properties.

Hope this helps you?

Share
 

I will explain very briefly, how to fix crashes in release builds of VC applications.

Difference between debug and release builds

Well so what’s the difference between release and debug builds? The only major difference is that debug builds are for debugging purpose and release builds are for end users so for this purpose debug builds contain additional debug information, this is why there is considerable size differences in binaries for debug and release.

Common reasons for crashes in release builds

Well not too many steady reasons for crashes in release builds, but some to think of I will list here…

  1. Uninitialized variables
  2. Uninitialized pointers
  3. Always set a pointer to NULL after deletion
  4. Invalid iterators (std::iterator)
  5. Crap code
  6. Array index out of bounds for array and other stl classes.
  7. Too much optimization.
  8. RTTI project settings.

Check out above reasons if there is a crash in your release builds.

So tell us how to fix issues in release builds

  • Learn assembly. :P Joking! Naah not everyone has time to learn assembly but hey it’s good to have basic knowledge on this topic.
  • Enable debug information for your release binaries. Ha, but will this increase the size of my release binaries. Nope! An extra file called a pdb (Program database) is created, which helps the JIT or any debugger to list out symbols used in your applications.
  • Always keep an updated set of symbols for all binaries; debuggers do download them automatically if you set _NT_SYMBOL_PATH to point to a symbol server. More information on how to set a symbol path is here.
  • Stop writing crap code.

Make my release build a debug build

If you are in the process of development and then you notice a crash while testing release build and you don’t know where the crash is taking place, do the following…

  1. Enable debugging for your release builds
  2. Go to C-C++/Optimizations/ and select Disable(Debug). If you don’t enable this bit you will see weird behavior while debugging. For e.g. addresses show as NULL even for valid objects on stack. By the way this might even be the cause of those crashes so initially you can leave it as it is but watch out for weird behaviors, don’t be surprised.
  3. Go to C-C++/Debug Info/ and select Program Database, this will generate a pdb file.
  4. If the output for a particular project is of .lib type then this much will suffice.
  5. Now for dlls and exes, follow steps 1 – 3.
  6. Go to Link tab/section and enable “Generate debug info”, this will add further debugging information to your exe or dll. Will show real deep stack traces with values of passed in parameters.

Yup now your release exe/dll/lib is now in full fledged debugging form. An additional thing to note is that _DEBUG is not enabled so AfxTrace, TRACE, TRACE0, TRACE1, TRACE2, ASSERTs, and VERIFYs won’t work.

Hope this helps you guys! :)

Share
 

Ever felt the need to watch over return values from functions! Well I did always but unfortunately couldn’t come up with something! I was reading through one of John Robbins’ Bugslayer articles in MSDN and found an interesting tip related to seeing function parameters passed in by adding a watch statement over the ESP stack pointer variable.

So that got me thinking like “why not EAX”, huh that was easy! So this tip happens due to the tip given by one of the readers of John’s article in Bugslayer. I will put a link to that tip once I find it along with the author’s name.

Now let me tell you how the watch statement for our tip should look like…

  1. To watch string values returned by functions…
    (char*)EAX
  2. To watch integer values returned by functions…
    (int*)EAX
  3. To watch objects returned by functions…
    (CMyObject*)EAX

Now you may ask why do I need this, I can always use the auto variable debug window. Well don’t forget that we are adding watch to a register (EAX), so this means even raw function call’s return value is shown here.

Let me show you a live case where you may need this. See here I’ve got four function calls which does some strange operations but return values doensn’t matter hence they are not stored. It’s just useless to store return values in temporary var just for the purpose of debugging, so above watches come handy.

DoOperationReturnBool(); // Returns bool, but returned values not stored
DoOperationReturnEnum(); // Cast EAX to appropriate enum type or int
DoOperationReturnCharPtr(); // Cast EAX to char pointer  or wide char ptr
DoOperationReturnObject(); // Cast EAX to appropriate object type, I haven’t tested this but should work

I am using this these days, yet to see a drawback, tell me if you find any! Have fun debugging :)

Share
© 2012 bits and bytes Suffusion theme by Sayontan Sinha