w3wp high memory and cpu usage in SharePoint

This week we had a huge performance issue with SharePoint at a customer. The w3wp process took a huge amount of memory for itself (1,4 - 1,5 gigabyte) without any user activity. Another symptom was that while performing a full / incremental crawl, the server suffered from very high CPU usage. Since I already installed the Infrastructure update and latest hotfixes, it was really hard troubleshooting the problem.

Some things to check if you experience the same symptoms:

1. Windows Server 2003 Service Pack 2 =>
   
   
2. Services 3.0 and MOSS 2007 Service Pack 1 => and
   
   Before SP1, there were some issues regarding the search crawl. For example, the memory usage doesn't increase as much as before while the crawl is running.
   
   
3. Check your SharePoint permissions: in my case, I solved the issue with setting the permissions on the following key HKEY_LOCAL_MACHINE\Software\Microsoft\Shared Tools\web server extensions\12\WSS and HKEY_LOCAL_MACHINE\Software\Microsoft\Shared Tools\web server extensions\12\Search. I added the Everyone group and gave them "Read" rights.

Very strange, but the latter seemed to solve my problems! Web applications reached sizes of 650 MB till 900 MB max, which was a big difference with the 1,4 gigabyte we used to encounter. I will try to research which permissions for which group or user needs te be set. The Everyone group is too general. However, the problem is very difficult to reproduce :(