Question Insufficient system resources on 64-bit Windows RSS Sear

[Guest CrashHunter]

One of our customers has a 64-bit Windows 2003 Ent.SP2 domain controller on a

VMWare Esx server. The VM has 1 CPU and 1 GB RAM allocated (adding up to 4 GB

did not make a difference).

Our application tries to dump the registry hives to files and it gets the

1450 error ("insufficient system resources...") on the SOFTWARE hive. The

hive is not extremely big (exporting it from registry editor results in a

~75MB file).

I have created a very basic sample application that just opens the registry

key and calls the RegSaveKey API and we get the same error.

Strangely, the "reg.exe" tool, which I would expect to use the same APIs, is

successful and it results in a ~35MB file; I have used the save parameter,

not the export which would generate a similar file with the one resulted when

exporting from Registry Editor.

I have tried to set the PoolUsageMaximum and the PagedPoolSize according to

some KBs but it did not make any difference. Moreover, the theoretical limit

for paged pool memory on 64-bit Windows is 128GB, while I can see that, when

trying to save the SOFTWARE hive, the Total Kernel Memory grows up to less

than 100 MB before returning the insufficient resources error.

If I use the 32-bit version of my sample tool, I get the error even when

trying to only dump the SOFTWARE\Wow6432Node\Microsoft key (although it

worked on all subkeys); the 64-bit version is able to save the

SOFTWARE\Wow6432Node but it fails with the same error on the SOFTWARE alone.

There are many customers that use our application without any such problem

so I am pretty sure that it is only a Windows configuration issue, but I

can't get it working.

The sample application that I use to repro the problem just calls:

 

EnablePrivilege(sComputer, SE_BACKUP_NAME);

LONG rc = ERROR_SUCCESS;

HKEY hKey = HKEY_LOCAL_MACHINE, hKeyToSave = NULL;

REGSAM regMask = KEY_READ;

rc = RegCreateKeyEx(hKey, sRegKey, 0, NULL, REG_OPTION_BACKUP_RESTORE,

regMask,NULL,&hKeyToSave, NULL);

if (rc != ERROR_SUCCESS) {

_tprintf(_T("RegCreateKeyEx error %u (%s)\n"), rc, sRegKey);

break;

}

rc = RegSaveKey(hKeyToSave, sFile, NULL);

 

Does anyone have any idea what can be wrong?