Rick Strahl's Weblog  

Wind, waves, code and everything in between...
.NET • C# • Markdown • WPF • All Things Web
Contact   •   Articles   •   Products   •   Support   •   Advertise
Sponsored by:
Markdown Monster - The Markdown Editor for Windows

EXE COM Server Invokation leaking Handles


:P
On this page:

Last week I started looking into a problem that causes my IIS Worker processes to have errors on shutdown. After some mind numbing debugging I finally seem to have traced the problem down to COM object invokation and a handle leak that results because of it. What I ran into here though is very odd as it appears to be not specific to my code but a general handle leak when instantiating EXE servers.

First off let me say C++ and COM are not my forte especially now after having been away from it for a long time. But I've boiled down the following problem to its bare minimum and I'm completely stumped as to why the code would leak handles because it's such fundamental COM code.

Ok so the scenario is as follows: I have a COM client (in my 'real' app it's the ISAPI extension) that is instantiating EXE COM servers and then shortly after releasing them all happening on a single thread. This scenario occurs for certain long running processes that go off and run outside of the normal thread pool. However, the problem is reproducible without any special thread related issues interfering.

In any case the operation of the servers works perfectly fine - servers load and are properly unloaded when inst->Release() is called. The problem is that there is a Windows Handle leak.

As I was debugging my app I noticed that the code that loads EXE COM instances was effectively leaking a Windows handle when I checked in ProcessExplorer. So I kept simplifying the code more and more until I ended up with nothing more than a call to CoCreateInstance() and a inst->Release() call in COM, which in effect leaked a handle. It leaks a handle when inst->Release() is called and it does this only on EXE servers, not on DLL servers.

I managed to boil the code down in a simple C++ Console application:

#include "stdafx.h"

int _tmain(int argc, _TCHAR* argv[])
{

    // COM Instance Handle Leak Bug
    IUnknown *pComObj = NULL;
    //IDispatch *pComObj = NULL;
    CLSID clsid;
    
    HRESULT hr = CLSIDFromProgID( L"VisualFoxPro.Application",&clsid);

    CoInitialize(NULL);
    char input[80];

    for(int x=1; x< 10; x++)
    {
        // *** Commenting the next two lines out results in no handle leaks
        // *** Running these two lines will leak one handle per request
        hr = CoCreateInstance(clsid, NULL, CLSCTX_SERVER, IID_IUnknown, (LPVOID *) &pComObj);
        //hr = CoCreateInstance(clsid, NULL, CLSCTX_SERVER, IID_IDispatch, (LPVOID *) &pComObj);

        DWORD count = pComObj->Release();
        
        printf("%d: %d\r\n",x,count);    
        scanf("%c",input);

    }
    printf("Completed.");
    scanf("%c",input);

    CoUninitialize();

    return 0;
}

When I run this code and check the handle count in process Explorer I see the handle count increase on every single call to  pComObj->Release().

ProcExplorer

It doesn't matter what EXE server I use here. I'm using a custom COM server of mine, or a more generic server above. Tried a tiny little ATL server, tried a few others like SnagIt, Word, Excel - the behavior is exactly the same - they all leak a handle.

Taking a closer look at the handles that are leaked on Release in Process Explorer (View | Lower Pane | Handles then Show unnamed handles) results in a <Unknown handle> created:

 InvalidHandle

 

So as I said at the outside I'm not a C++ wizard, but this seems like a pretty fundamental problem given how basic this code is. Create an instance and release it - can't get any easier right? I've never noticed this before because I never debugged this all the way, but in testing I'm now seeing that the same behavior occurs on XP, Windows 2003 server and on my dev setup on Vista (as well as on Server 2008). I also tried the more round about DCOM mechanism using CoCreateInstanceEx() but the results there are the same.

I just can't make sense of what's happening here as the code is about as fundamental as it comes - make a call to CoCreateInstance() which ups the ref count, then call Release() to unload the server and all resources associated with it. The actual COM server is behaving properly, but its the damn handles that are being left behind.

What could possibly be wrong with this code? What else could I possibly do to release a handle that I don't even hold in my code? Is there some other way to release resources? It would appear that there's some issue in the DCOM proxy clean up that occurs when the server is released, but I'm not sure what that would be.

Are there any COM  wonks that can point at the error in my ways?

I've attached the C++ project and EXE if anybody's adventurous - if you check this out you may have to change the server name to an EXE server installed on your machine (Word.Application or whatever that exists - although Word is problematic because it doesn't actually unload unless visibility is set off or Close() is called).

Posted in C++  

The Voices of Reason


 

AC
August 25, 2008

# re: EXE COM Server Invokation leaking Handles

Yuck, I feel your pain and remember the horrors. After reading your post I can't see what might be wrong, so I went to my old collection of COM+ books. After so many pleasant years getting stuff done with C# it pains me to read C++ COM+ again, remembering all the ways that you could shoot yourself in the foot with a cruise missle.

Anyway, musn't grumble. A few things to try while looking at these aged texts are:
- use CoInitializeEX as CoInitialize is the obsolete version for clients that don't know better and are oblivious to threading. Might also want to dig in the CLSID's to check out what the server you're using is using and the threading model it supports.
CoInitializeEX(NULL, COINIT_APARTMENTTHREADED)


- in the test code, check your HResults. It's pedantic, but maybe it's telling you something.
if(FAILED(hr)) 
    cout << "CoInitializeEx failed" << endl;


- in CoCreateInstance, instead of using CLSCTX_SERVER which is a bitmask, use single values to ensure you're getting what you need / want. CLSCTX_INPROC_SERVER or CLSCTX_LOCAL_SERVER, etc.

- ditch the 2nd call to CoCreateInstance and call QueryInterface on your pUnknown to get your IDispatch pointer. You don't need to create another object for that. Maybe it's the code you wrote, but if both lines are uncommented, you're overwriting your IUnknown pointer with an IDispatch pointer, which is kinda bad. It's hard to tell if that's what you meant to do, or it's an error that snuck in while you were debugging.

IUnknown* pUnknown = 0;
IDispatch* pDisp = 0;
HRESULT hr =0;

hr = CoInitializeEX(NULL, COINIT_APARTMENTTHREADED);
if(FAILED(hr)) 
    cout << "CoInitializeEx failed" << endl;
    
hr = CoCreateInstance(clsid, NULL, CLSCTX_INPROC_SERVER, IID_IUnknown, (void**)&pUnknown);
if(FAILED(hr))
    cout << "CoCreateInstance failed" << endl;

hr = pUnknown->QueryInterface(&pDispatch);
if(FAILED(hr))
    cout << "QueryInterface failed for IDispatch .. not supported" << endl;
    
pUnknown->Release(); // can release IUnknown pointer ... no longer needed.

// do something via IDispatch    
pDispatch->Release();

CoUninitialize();


Finally, my smart book says you can replace calls to CoCreateInstance with CoGetClassObject (not directly of course) since you have greater control of how and when objects are instantiated. Maybe this is something you need to do.

It says that a basic version of CoCreateInstance is:
IClassFactory* pClassFactory = 0;
hr = CoGetClassObject(clsid,CLSCTX_INPROC_SERVER,NULL,IID_IClassFactory, (void**)&pClassFactory);
if(FAILED(hr))
    cout << "CoGetClassObject failed" << endl;
pClassFactory->CreateInstance(NULL, IID_IUnknown, (void**)pUnknown);
if(FAILED(hr))
    cout << "pClassFactory->CreateInstance failed" << endl;
pClassFactory->Release();


blah blah blah. I have no idea if any of this will help you. I hope so, but I certainly can't remember any of it. I just remember it wasn't easy. This guy has some COM tips (http://www.devguy.com/fp/Tips/COM/)

I'd post you the books, but I'm in NZ at the moment. Sorry. :)

Rick Strahl
August 25, 2008

# re: EXE COM Server Invokation leaking Handles

@AC - thanks for posting. I've actually tried all of this already. InProc servers are not leaking - it's only Out of Proc servers that are causing the problem, but unfortunately that's exactly what I need to call in this code.

It just seems very odd that this is happening - it seems like a fundamental flaw unless I'm doing something silly, but given that really it's just CoCreateInstance and inst->Release() and the actual servers are releasing I can't see that there's anything wrong in my code.

As to CoInitialize() - IIS sets up threads with CoInitialize() so effectively I have no control over the thread's COM environment but I'm pretty sure it's default multi-threaded mode. But - switching to Apartment threaded initialization in the test code also doesn't seem to make any difference either and it IS working with Apartment threaded DLL servers.

I'm stumped. Anybody have a recommendation for a good forum to ask questions on this sort of thing?

Kevin Pirkl
August 26, 2008

# re: EXE COM Server Invokation leaking Handles

You might try Adam Nathan at Microsoft. He is quite the Guru of COM and Interop, etc.. He might be able to help. I your entry and noted my bookshelf and if anyone can give you some direction he probably can.

Kevin Pirkl
August 26, 2008

# re: EXE COM Server Invokation leaking Handles

I asked my twin brother to run some tests. He used used InternetExplorer.Application and Work.Application and ran the loop 80,000 times and did not see any leaks at all. He made three suggestions. 1> Dont run test in a debug build version, 2> With OLE Automation Server Activations use IID_IDispatch, 3> Run your test outside the debugger.

He did notice that Word.Application remained running even after Release

I dont know what that all means not being a C++ person like my brother is so I cant say what all the above would me.

Cheers

Rick Strahl
August 26, 2008

# re: EXE COM Server Invokation leaking Handles

Thanks @Kevin. Internet Explorer may not exhibit this behavior because it doesn't release unless you explicitly call it's Quit method. Otherwise it'll stick around and keep reusing a single instance into which tabs are thrown. So this might explain why the handle count doesn't change with it. Same deal with Word.

Now, I did create a simple ATL COM server and tried that and lo and behold that DOESN'T leak handles. I suspect it has something to do with the DCOM proxies which the simple ATL object doesn't implement.

For kicks I also tried the following inside of Visual FoxPro (or you could use a VBS for) on the command window:

o = CREATEOBJECT("VisualFoxPro.Application")
o=.f.
o = CREATEOBJECT("VisualFoxPro.Application")
o=.f.
o = CREATEOBJECT("VisualFoxPro.Application")
o=.f.


and there too there is a handle leak even though the servers are going away. So the issue is a bit deeper than a bug in my code I think.

Any other suggestions for COM servers to try? I'm curious if others are seeing similar behavior.

AC
August 26, 2008

# re: EXE COM Server Invokation leaking Handles

I wonder if you're waiting long enough. COM fakes, but doesn't really release for something like 10 minutes to allow all other cleanup code to run and go away before really unloading the dll.
http://archives.neohapsis.com/archives/microsoft/various/dcom/2001-q4/0812.html

Or maybe the COM object you're using just has broken reference counting. Take a look at some comments at http://blogs.msdn.com/oldnewthing/archive/2004/03/26/96777.aspx - Then again, maybe just ask Raymond Chen what's going on.

Rick Strahl
August 26, 2008

# re: EXE COM Server Invokation leaking Handles

@AC - yes DCOM has a 8 minute life time lease which is meant to keep servers alive in true DCOM disconnected scenarios. But a call to Release() should clean up those holds - otherwise the server wouldn't actually release (ie. stay loaded) and that's not actually the case. The server is releasing from the process list.

FWIW, I've been able to duplicate this behavior from ANY COM client including .NET Interop clients so it's not the client code that's the problem.

As to ref counting - I would say it's possible but it's commercial applications that are in question here and so I'm inclined to think that their internal housekeeping is Ok for ref-counting. Release() is returning 0, so the ref count according to the COM system is clean. I've experimented a bit and adding extra ref counts will indeed only release the EXE server when it goes to 0. So I think that ref counts are OK.

To me this looks there's a problem in the COM proxy stack. Unfortunately I've never really checked this in the past so I'm not sure if this has always been the case or whether this is some new behavior. But I'm pretty sure that during my ISAPI debugging sessions when I originally built my component years ago I very carefully checked for memory and handle leaks. Something this obvious I think I would have noticed at the time.

I started down this path because I've had shut down exceptions in IIS worker processes that are running these COM objects. When these objects are loaded and unloaded a few times with these left over handles, that's when I get the shutdown errors and this is definitely something that is new for about the last year or so. The problem does occur on what looks like all WIndows platforms so it's gotta be some core RPC issue.

I have somebody at Microsoft I trust checking into this and he's duplicated the issue and is following up internally. Hopefully there be some clearing up of what's happening. It sure seems like a bug, but with DCOM and RPC any craziness is not impossible...

Andy Bruce
September 12, 2008

# re: EXE COM Server Invokation leaking Handles

I'm enjoying your blog--your hours are crazy but you love the code...

I've run a number of tests using Excel as an automation server along with several other server objects using VS2005 on W2K3 SP2. I am *not* running into an unnamed handle leak, but rather a "Synchronization Event" handle leak. This is usually happening one per call to Release (occasionally two).

Taking out the call to Release has the effect of: 1) leaving Excel running; 2) the synchronization event handle leak does not appear.

I also tried deleting one of the "extra" event handles (but not when immediately created, waited an iteration). This causes the application to terminate immediately (along with any created Excel instances).

If you want the code that I worked with, just let me know. It's basically yours, calling thru IDispatch and using some code M$ provided to handle invoking QUIT in Excel using generic IDispatch (which didn't have an effect on the "extra" synchronization event handles).

I'll keep up with your entries to see what you find. Despite your claim to non-C++ worthiness, your DCOM knowledge is far superior to mine...

Fernando D. Bozzo
November 23, 2008

# re: EXE COM Server Invokation leaking Handles

Hi Rick, I've tested what you write:

o = CREATEOBJECT("VisualFoxPro.Application")
o=.f.
o = CREATEOBJECT("VisualFoxPro.Application")
o=.f.
o = CREATEOBJECT("VisualFoxPro.Application")
o=.f.

and I have too handles not closed inmediately, buy few minutes later they dissapear, like if they was auto garbage collected.

Rick Strahl
November 23, 2008

# re: EXE COM Server Invokation leaking Handles

@Fernando - Yes probably about 8 minutes later which is the DCOM server ping timeout. The problem is that this only happens if the calling application is still running. Kill the source app and those servers never go away.

This is the problem in IIS which runs and doesn't unload those refs that are left hanging. In Web Connection I actually have code that will physically kill servers after a short timeout to get around this and that works to get rid of the servers, but these pseudo references that are still active cause the IIS worker process to throw an exception at the very final shutdown. It doesn't hurt anything but there's an event log entry which is disturbing some folks.

Annoying. Whatever the issue is nobody at Microsoft seems to care much...

West Wind  © Rick Strahl, West Wind Technologies, 2005 - 2024