Rick Strahl's Weblog  

Wind, waves, code and everything in between...
.NET • C# • Markdown • WPF • All Things Web
Contact   •   Articles   •   Products   •   Support   •   Advertise
Sponsored by:
Markdown Monster - The Markdown Editor for Windows

ASP.NET Caching under Memory Pressure


:P
On this page:

I’ve been struggling over the last few days with an application that is using the ASP.NET cache quite  bit. As it turns out the programmatic cache in ASP.NET in most of my applications is not keeping items in them under almost all circumstances.

 

When I’m talking about the programmatic cache, I mean the Context.Cache object and manually adding items into the cache. I’m a big fan of HTML fragment caching for rendering portions of a page either via code based rendering or from control based rendering, capturing the output and then storing it in cache. I use this commonly for portions of sidebars that are semi-dynamic to avoid excessive use of user controls which gets messy and often serves no good purpose if the use is one off. To create a user control just to get caching seems like overkill.

 

I’ve noticed recently as I was debugging my RSS problem on my live server that the cache in most of my applications is empty! Completely empty – and that should not be… items are added and while they are typically added with relatively short timeouts of a few minutes or less sometimes, they are long enough that I would notice them.

 

To check this out I just created a simple page and simply iterate through the cache and echo items out like this:

 

protected void Page_Load(object sender, EventArgs e)

{

    foreach (DictionaryEntry d in this.Cache)

    {

        Response.Write(d.Key);

        Response.Write(" - ");

        Response.Write(d.Value);

        Response.Write("<hr>");

       

    }

}

 

If you’re looking for something way more sophisticated you can also check out Steve Smith’s CacheManager tool here:

 

http://aspalliance.com/cachemanager/

 

So when I look at the cache when I start my application right from the start I usually see a bunch of items popping into the cache as I would expect. But as the application keeps for a while cache items start disappearing out of the cache even though items are definitely written to it. In some cases I can see the items show up very briefly (seconds or less) and almost immediatedly disappear.

 

What’s going on here? I suspect that the problem has to do with memory. The server I run this site on is pretty old and it has only 512megs of memory. It’s running 5 different ASP.NET AppPools and a number of older FoxPro West Wind Web Connection apps. Some of the ASP.NET apps are pretty resource intensive. The server has no real performance problems but memory is definitely tight and I’m wary to update this old machine at this point.

 

But it’s an eye opener to see that the ASP.NET Cache is not actually caching in most application level situations in this low memory environment. Apparently the memory pressure on the box is keeping the cache from even storing tiny little entries. Some of the values I’m storing are things like two or three line value summaries that only need to refresh every so often. Even these things which are less than 80 string characters won’t stay in the cache.

 

It’s disconcerting because, unless you actually check your cache contents occasionally you really have no idea of what’s getting cached. I had happily assumed that my entries where getting cached when in fact they weren’t.

 

It doesn’t break the application (in most cases) but it certainly affects performance. I say 'in most cases' - I actually started looking into this because I had one request that is an image handler and uses the cache to pass values between the executing request and an HttpHandler. In this particular scenario I need to absolutely make sure that the value makes it into the cache and stays there. Even though this value is a single small ID value string (a guid) and the value was retrieved almost instantly this code failed intermittently. ASP.NET’s cache was dumping even this short value!

 

Luckily there’s a workaround:

 

Cache.Add(CacheKey,IdValue, null,

          DateTime.UtcNow.AddSeconds(30), TimeSpan.Zero,

          CacheItemPriority.NotRemovable,null);

 

Which ensures that the cache manager doesn’t remove the item from the cache. Only an explicit Context.Cache.Remove() or a timeout will remove the item.

 

So yeah, I know the solution to this problem is to upgrade the machine and add more memory <s>. I wasn’t planning on doing any updates to this box as I probably replace it when LongHorn server becomes available. In the meantime I guess I have to live without manual caching.

 

OutputCache

Incidentally, Page OutputCaching probably has a lot to do with the memory usage of these applications as well. I do notice that requests that are cached with OutputCache at the page level seem to cache fine. I can see this happening by looking at my internal request logging and seeing entries that come back with a 0 request time length. So that OutputCaching seems to work reasonably well.

 

The OutputCache is sort of a mystery black box to me <s>. It seems there’s no easy way to tell just what is cached and how it’s cached.  Does anybody know if there’s a way to get at the OutputCache entries in anyway? Is ASP.NET even storing the cached content in the same cache (and hiding the entries from external iteration) or is it using a completely separate cache. And then there’s Kernel Caching where some content is cached by IIS. But again there’s no way to tell that I know of whether an item actually made it into the Kernel Cache. <shrug>

 

More common than you might think

The moral of this entry <g> is that this low memory and cache refusal might actually much more common than you expect. ISPs for example often run in tight memory environments as they keep the AppPool memory limits really low. And even with a lot of memory Caching can very quickly overrun memory. If you have large pages that get cached memory goes quick. Caching localized content - another quick way to load up cache memory excessively as each locale gets different cache entries.

 

Today for kicks I checked several of my customer's applications that are running in a hosted environments and 2 out of 4 exhibited the same behavior with the Context.Cache basically showing empty. Eeek...

 

 

Posted in ASP.NET  

The Voices of Reason


 

Jeff Atwood
February 24, 2007

# re: ASP.NET Caching under Memory Pressure

This is another reason servers should ALWAYS, ALWAYS be running a 64-bit OS, and stuffed to the gills with memory.

http://www.codinghorror.com/blog/archives/000435.html

Hardware is so ridiculously cheap these days there's no reason not to upgrade.. if I was building a server, I'd go for a cheap x64 capable dual-core (probably AMD, as the pricing is better) and 4gb of off-the-shelf memory.. and slap Win2k3 64-bit edition on there.

Steve from Pleasant Hill
February 24, 2007

# re: ASP.NET Caching under Memory Pressure

At 512K no doubt you had no available chip memory left, with the OS, IIS, various app pools -- and were on to the swapfile "virtual memory".

Rick Strahl
February 24, 2007

# re: ASP.NET Caching under Memory Pressure

Jeff, if you are really running 64 bit, aren't you're halving your memory usage? With 64 bit pointers and integers everything's twice as big so your 4 gig acts really like 2 gig in the 32 bit space...

Steve, I'm not really complaining, I'm just stating what I found out and didn't consider previously <g>. It's very easy to assume that the cache will 'just work'. But even in this light I am very surprised that even these really small entries are failing. I can understand loading up large chunks, but small snippets that may amount to 1k total in the entire app aren't making it in either. That seems a bit odd. Then again the cache probably looks at some sort of performance/memory metric and just flat out refuses to add anything regardless of what it actually is you're adding.

Bertrand Le Roy
February 24, 2007

# re: ASP.NET Caching under Memory Pressure

mmh, isn't it an abuse of the cache to use it as a communication device? Shouldn't you be using an application variable for this? The application should never assume that a piece of data will be in the cache.

Steve from Pleasant Hill
February 24, 2007

# re: ASP.NET Caching under Memory Pressure

Bertand, while it is never safe to assume something will be in cache, it is probably very common that cache is being used for such things.

Rick Strahl
February 24, 2007

# re: ASP.NET Caching under Memory Pressure

Bertrand, yes absolutely agree.

However, the reason I use Cache is that it can expire. You wouldn't want to orphan an Application entry if say the user never actually goes there... The scenario where this is used is for a CAPTCHA, so often times you display the Captcha but it doesn't release until the user submits. In some cases he never does. Application won't work, neither will Session because the timeout's too long. The only other alternative then would be to stuff it in the database somewhere.

Jeff Atwood
February 25, 2007

# re: ASP.NET Caching under Memory Pressure


It's good that you're publicizing this issue.

We recently had the exact same problem at Vertigo on a website project. Every time it happens, it's mystifying to developers. Understandably so. When everything you put in the cache disappears after a few milliseconds, that's not expected behavior. It's downright unnatural.

But it does happen a lot more often than people realize, and since most cache code recreates the objects when they can't be pulled from cache, the behavior isn't noticed.

Same root cause though, overloaded web server with tons of AppDomains and not enough memory.

Michael Reynolds
February 27, 2007

# re: ASP.NET Caching under Memory Pressure

My experience with this is that even with ample memory available ASP.NET would remove items that were anything more than a trivial size.

Raising the CachePriority to AboveNormal or High will solve the problem in most cases. I would try that before using NotRemovable, so that ASP.NET can still manage the cache to some degree.

It would be interesting to know what criteria ASP.NET uses when deciding to remove an item.

AndrewClark
February 28, 2007

# re: ASP.NET Caching under Memory Pressure

"It would be interesting to know what criteria ASP.NET uses when deciding to remove an item."

I'm curious, would the following link help answer this question?

CacheItemRemovedCallback delegate

http://msdn.microsoft.com/library/default.asp?url=/library/en-us/cpguide/html/cpconnotifyingapplicationswhenitemisdeletedfromcache.asp

Jeff Atwood
March 01, 2007

# re: ASP.NET Caching under Memory Pressure

> I would try that before using NotRemovable, so that ASP.NET can still manage the cache to some degree.

Oh, believe me, we tried this. Rick uses my CAPTCHA control (which stores a tiny captcha object in the cache, which it uses to render the image later) and it took us WEEKS to figure this out. We tried everything we could think of. And then some.

http://www.codeproject.com/aspnet/CaptchaControl.asp

NotRemovable is the ONLY thing that worked. It also makes sense in this case, the captcha data really has to be there for the user to enter it after the postback. The only downside is for robots and so forth who generate a lot of page views but never complete a captcha. So they have to expire out of the cache. But then again, as I said, the CAPTCHA object itself is teeny, and you can set the expiration fairly aggressively if you want (it also serves as an upper bound on the time allowed to enter the CAPTCHA, to prevent CAPTCHA farms and so forth).

Jeff Atwood
March 01, 2007

# re: ASP.NET Caching under Memory Pressure

> CacheItemRemovedCallback delegate

Yep, that's why they came to me at Vertigo when they ran into that. It is good to look at this if you're curious why the cache keeps expiring in milliseconds, before you can even get any use out of the items IN the damn cache.

Guess what it told them: "removed due to memory pressure". Exactly what you would expect.

Rick Strahl
March 01, 2007

# re: ASP.NET Caching under Memory Pressure

Jeff re- the CAPTCHA... The thing to do with that is not show the CAPTCHA and make it invisible until you click a button or something else to show Comments. I found that BECAUSE of the not removable I was really clobbering the cache originally. We talked about this before - your control needs an update to not stick stuff into the cache if not Visible which wasn't working with the last release I used but fixed

IOW, if NotRemovable is used one has to be really careful about what to gets put in the cache and carefully set the timeout to ensure this stuff doesn't clutter up the cache.


re- Delegate. It's a great feature, but for debugging pretty worthless <s>. What else can you think of that would actually trigger the item to be released early other than memory pressure? <s>. The thing that would be interesting is a metric and I'm guessing the metric ASP.NET uses is based entirely on some system status regardless of the item itself stored in the cache. After all how could the cache know how big an object is in memory (other than simple values like strings etc.).

Emre
April 10, 2007

# re: ASP.NET Caching under Memory Pressure

I had the same problem on ASP.NET 2.0. I tried using sliding expiration, giving high priority to cache data. However none of them worked. Cache items had been automatically removed in very short times, say seconds. Only thing that worked is setting to notremovable.

Jordan Rieger
April 13, 2007

# re: ASP.NET Caching under Memory Pressure

I found the same behavior on my fairly powerful development machine (P4 2.4 w/ 1G). I was worried that it would affect my application in production but this solution helped. Thank you very much for this article and discussion.

Vicky
May 30, 2007

# re: ASP.NET Caching under Memory Pressure

Hi everyone,

I am new to ASP.NET cache. I found this article when I was looking for helpful resource about cache user control. I am not sure if my problem actually is similar to most problems you discussed above, since I didn't use programmatic cache.

Basically, I have 3 dropdownlists, ddlAccountGroupType, ddlGroupName and ddlAccountID in a user control. The reason I use cache varybycontrol because the ddlAccountID could contain over 8000 accounts. So I have <%@OutputCache duration="600" varybycontrol="ddlAccountGroupType;ddlGroupName;ddlAccountID" %> on my .ascx file.

Once selected index is changed in ddlAccountID, it will raise event handler to trigger the parent page to retrieve Order list and allow user to edit the order details. However, sometimes, when parent page do a postback, the second dropdownlist ddlGroupname become empty when it was supposed to have one single item "--ALL--". I could figure out why it happens. Do you think it's the same memory issue? Is there a way to set fragement cache NotRemovable?

By the way, I read something about Cache does not work with Web Farms.
Although my problem happens on my local development server, but my production server does use web farm. Do you have any thoughts or suggestions about this? Does your application running on web farm?

Thank you in advance.

# DotNetSlackers: ASP.NET Caching under Memory Pressure


Jacob
July 24, 2007

# re: ASP.NET Caching under Memory Pressure

"The OutputCache is sort of a mystery black box to me. It seems there’s no easy way to tell just what is cached and how it’s cached. Does anybody know if there’s a way to get at the OutputCache entries in anyway? "

Looks like they're stored in HttpRuntime._cacheInternal which is different than the user one (HttpRuntime._cachePublic) but doesn't seem to be exposed through any public interface. However, the page cache does update a few stats in the NT performance monitor under ASP.Net Applications/Output Caches Entries etc...

Mark Simmons
February 06, 2008

# re: ASP.NET Caching under Memory Pressure

Thank you for posting this detail - this is a nebulous item to troubleshoot, especially when you're running the VS 2005 Developer's WP for an ASP.Net app and not using IIS.

I had a user control presenting some statistics, some of it user-oriented and kept in session state and another portion of it global for the application but only to live for up to 3 hours between refreshes. Cache continually came up empty, so I put a break point in the OnRemoved callback to see why it kept getting killed - continually the reason was "underused" but the footprint of the object was so small I had no idea why it would get dropped after only 10 - 15 seconds.

I added this line in to the caching statement using HttpRuntime.Cache and voila - 3 hours later I see the callback routine getting called only because the item is now expired.

Thank you!

Les Lewicki
March 19, 2008

# re: ASP.NET Caching under Memory Pressure

I found this one very interesting and I think it explains what's going on with cache under memory pressure in your case.

http://blogs.msdn.com/praveeny/archive/2006/12/11/asp-net-2-0-cache-objects-get-trimmed-when-you-have-low-available-memory.aspx

Edmartinez
November 19, 2008

# re: ASP.NET Caching under Memory Pressure

Hi Rick! There is added pressure on memory due to ASP.NET cache’s in-process and standalone nature. I suggest you look into distributed caching because it not only relieves the pressure on memory, but it is also scalable and highly reliable.

krish
November 26, 2008

# re: ASP.NET Caching under Memory Pressure

Hi,
IS there any way to limit the size of each individual cache item put into the cache?
I don't want to cache big items basically.

Thank you

Rick Strahl
November 26, 2008

# re: ASP.NET Caching under Memory Pressure

@Krish - no, but you can create a generic wrapper function for cache access and put the logic inside there.

Problem is how to determine size if it's none string data. How do figure out what the size of an object is?

krish
November 28, 2008

# re: ASP.NET Caching under Memory Pressure

I did something like this:
System.IO.MemoryStream ms = new System.IO.MemoryStream();
BinaryFormatter bf = new BinaryFormatter();
bf.Serialize(ms, obj);
size = ms.Length;
break;

But the serialize operation is timing out most of the times because of the huge size objects...

Eric Berens
May 04, 2009

# re: ASP.NET Caching under Memory Pressure

"The OutputCache is sort of a mystery black box to me <s>. It seems there’s no easy way to tell just what is cached and how it’s cached. Does anybody know if there’s a way to get at the OutputCache entries in anyway? Is ASP.NET even storing the cached content in the same cache (and hiding the entries from external iteration) or is it using a completely separate cache. And then there’s Kernel Caching where some content is cached by IIS. But again there’s no way to tell that I know of whether an item actually made it into the Kernel Cache. <shrug>"


OutputCache is the bain of my existence. I spent an entire day running through reflector trying to figure out where OutputCache is being stored and why it differs from System.Web.Caching.Cache. From what I gathered it is stored internally differently then the global static cache that you can enumerate through and remove items, etc.

In my usage I wanted to skip the overhead of running through the page life cycle checking if cache items already existed and retrieving them if they did...that's what output cache was intended for...at least my grasp on it did. However, when I wanted to remove the output cach I ran into some problems since i was using VaryByParam and VaryByCustom. The public RemoveOutputCacheItem method only accepts one parameter in its signature...the URL of the page. If you dig deeper into the lib the private method it relies on actually accepts a URL and KEY! Now why on earth would M$ do this? I boggles my mind. Not to mention they seal the class so you cannot expose the underlying method to get around this.

So I *hope* in future releases they 1: either expose the internal cache storage used for page output cache or 2: all us to pass in a dependency key to the RemoveOutputCacheItem method.

/me shakes head....sigh.

Joe Rattz
February 26, 2010

# re: ASP.NET Caching under Memory Pressure

Thanks Rick. I just realized I had some Cache objects not remaining cached. It is quite puzzling when this happens. Even odder is that for me, it happened on my production server which has plenty of memory available. I had 4 Gb remaining out of 8 Gb and my objects were getting flushed. Odd. I had just blogged about using Cache and have added a comment about this problem. You can read about it here:

http://www.linqdev.com/PublicPortal/publicportal/blog.aspx?EntryID=54

Thanks!

William
March 17, 2010

# re: ASP.NET Caching under Memory Pressure

although ASP.NET is wonderful tool of caching but it is good for small web farms only once your app started to grow then after a certain point (something like more then 2 servers), it started to give you a lot of problems most important being performance and scalable problems along with one which is discussed in this article. in my opinion the best way to overcome these issues is the use of a distributed .NET caching solution. there are some very good distributed caching solution are available which really are doing the tricks for the developers for example NCache, Velocity etc

http://www.alachisoft.com/ncache/index.html

# re: ASP.NET Caching under Memory Pressure

Hi Everyone,

Does anyone know how to control the usercontrol output cache programatically ? As Strahl said "Does anybody know if there’s a way to get at the OutputCache entries in anyway? Is ASP.NET even storing the cached content in the same cache (and hiding the entries from external iteration) or is it using a completely separate cache. And then there’s Kernel Caching where some content is cached by IIS. But again there’s no way to tell that I know of whether an item actually made it into the Kernel Cache." ?

How can we read them and tweak them ? Also if we use file dependancy will it be polling the file frequently for modification date ?

How can we know all these hidden tasks done by Mr asp.net ?

Cheers
Karthik

Raja Lakshman
March 15, 2013

# re: ASP.NET Caching under Memory Pressure

Using Enterprise Library 5.0 Caching Application Block is the way to go. No stress on memory with this.

Rick Strahl
March 15, 2013

# re: ASP.NET Caching under Memory Pressure

@Raja - Enterprise library? Just say "no" :-)

West Wind  © Rick Strahl, West Wind Technologies, 2005 - 2024