VS2003 C++ 中异常的堆大小限制

问题描述

我有一个使用大量数据的 C++ 应用程序，并且在测试时注意到它的内存不足，但仍有足够的可用内存.我已将代码简化为示例测试用例，如下所示；

I have a C++ app that uses large arrays of data, and have noticed while testing that it is running out of memory, while there is still plenty of memory available. I have reduced the code to a sample test case as follows;

void    MemTest()
{
    size_t Size = 500*1024*1024;  // 512mb
    if (Size > _HEAP_MAXREQ)
        TRACE("Invalid Size");
    void * mem = malloc(Size);
    if (mem == NULL)
        TRACE("allocation failed");

}

如果我创建一个新的 MFC 项目，包括这个函数，并从 InitInstance 运行它，它在调试模式下工作正常(内存按预期分配)，但在释放模式下失败(malloc 返回 NULL).单步执行释放到 C 运行时，我的函数被内联我得到以下内容

If I create a new MFC project, include this function, and run it from InitInstance, it works fine in debug mode (memory allocated as expected), yet fails in release mode (malloc returns NULL). Single stepping through release into the C run times, my function gets inlined I get the following

// malloc.c

void * __cdecl _malloc_base (size_t size)

{
        void *res = _nh_malloc_base(size, _newmode);

        RTCCALLBACK(_RTC_Allocate_hook, (res, size, 0));

        return res;
}

调用_nh_malloc_base

Calling _nh_malloc_base

void * __cdecl _nh_malloc_base (size_t size, int nhFlag)
{
        void * pvReturn;

        //  validate size
        if (size > _HEAP_MAXREQ)
            return NULL;
'
'

并且 (size > _HEAP_MAXREQ) 返回 true，因此我的内存没有被分配.关注大小会返回预期的 512MB，这表明该程序正在链接到一个具有小得多的 _HEAP_MAXREQ 的不同运行时库.Grepping _HEAP_MAXREQ 的 VC++ 文件夹显示预期的 0xFFFFFFE0，所以我无法弄清楚这里发生了什么.任何人都知道会导致此问题的任何 CRT 更改或版本，还是我遗漏了一些更明显的东西?

And (size > _HEAP_MAXREQ) returns true and hence my memory doesn't get allocated. Putting a watch on size comes back with the exptected 512MB, which suggests the program is linking into a different run-time library with a much smaller _HEAP_MAXREQ. Grepping the VC++ folders for _HEAP_MAXREQ shows the expected 0xFFFFFFE0, so I can't figure out what is happening here. Anyone know of any CRT changes or versions that would cause this problem, or am I missing something way more obvious?

根据 Andreas 的建议，在此装配视图下查看此显示以下内容；

As suggested by Andreas, looking at this under this assembly view shows the following;

--- f:vs70builds3077vccrtbldcrtsrcmalloc.c ------------------------------
_heap_alloc:
0040B0E5  push        0Ch  
0040B0E7  push        4280B0h 
0040B0EC  call        __SEH_prolog (40CFF8h) 
0040B0F1  mov         esi,dword ptr [size] 
0040B0F4  cmp         dword ptr [___active_heap (434660h)],3 
0040B0FB  jne         $L19917+7 (40B12Bh) 
0040B0FD  cmp         esi,dword ptr [___sbh_threshold (43464Ch)] 
0040B103  ja          $L19917+7 (40B12Bh) 
0040B105  push        4    
0040B107  call        _lock (40DE73h) 
0040B10C  pop         ecx  
0040B10D  and         dword ptr [ebp-4],0 
0040B111  push        esi  
0040B112  call        __sbh_alloc_block (40E736h) 
0040B117  pop         ecx  
0040B118  mov         dword ptr [pvReturn],eax 
0040B11B  or          dword ptr [ebp-4],0FFFFFFFFh 
0040B11F  call        $L19916 (40B157h) 
$L19917:
0040B124  mov         eax,dword ptr [pvReturn] 
0040B127  test        eax,eax 
0040B129  jne         $L19917+2Ah (40B14Eh) 
0040B12B  test        esi,esi 
0040B12D  jne         $L19917+0Ch (40B130h) 
0040B12F  inc         esi  
0040B130  cmp         dword ptr [___active_heap (434660h)],1 
0040B137  je          $L19917+1Bh (40B13Fh) 
0040B139  add         esi,0Fh 
0040B13C  and         esi,0FFFFFFF0h 
0040B13F  push        esi  
0040B140  push        0    
0040B142  push        dword ptr [__crtheap (43465Ch)] 
0040B148  call        dword ptr [__imp__HeapAlloc@12 (425144h)] 
0040B14E  call        __SEH_epilog (40D033h) 
0040B153  ret              
$L19914:
0040B154  mov         esi,dword ptr [ebp+8] 
$L19916:
0040B157  push        4    
0040B159  call        _unlock (40DDBEh) 
0040B15E  pop         ecx  
$L19929:
0040B15F  ret              
_nh_malloc:
0040B160  cmp         dword ptr [esp+4],0FFFFFFE0h 
0040B165  ja          _nh_malloc+29h (40B189h) 

寄存器如下；

EAX = 009C8AF0 EBX = FFFFFFFF ECX = 009C8A88 EDX = 00747365 ESI = 00430F80EDI = 00430F80 EIP = 0040B160 ESP = 0013FDF4 EBP = 0013FFC0 EFL = 00000206

EAX = 009C8AF0 EBX = FFFFFFFF ECX = 009C8A88 EDX = 00747365 ESI = 00430F80 EDI = 00430F80 EIP = 0040B160 ESP = 0013FDF4 EBP = 0013FFC0 EFL = 00000206

所以比较似乎是针对正确的常数，即@040B160 cmp dword ptr [esp+4],0FFFFFFE0h，还有 esp+4 = 0013FDF8 = 1F400000 (my 512mb)

So the compare does appear to be against the correct constant, i.e. @040B160 cmp dword ptr [esp+4],0FFFFFFE0h, also esp+4 = 0013FDF8 = 1F400000 (my 512mb)

第二次根据 Andreas 的帖子，问题实际上出在 HeapAlloc 中.使用 HeapCreate & 为大对象更改为新的单独堆HeapAlloc 并没有帮助缓解问题，也没有尝试使用具有各种参数的 VirtualAlloc.一些进一步的实验表明，在分配一大段连续内存失败的情况下，两个较小的块产生相同的总内存是可以的.例如在 300MB malloc 失败的情况下，2 x 150MB malloc 可以正常工作.所以看起来我需要一个新的数组类，它可以存在于许多较大的内存片段中，而不是单个连续块中.不是什么大问题，但在当今这个时代，我本来希望 Win32 多一点.

Second edit: Problem was actually in HeapAlloc, as per Andreas' post. Changing to a new seperate heap for large objects, using HeapCreate & HeapAlloc, did not help alleviate the problem, nor did an attempt to use VirtualAlloc with various parameters. Some further experimentation has shown that where allocation one large section of contiguous memory fails, two smaller blocks yielding the same total memory is ok. e.g. where a 300MB malloc fails, 2 x 150MB mallocs work ok. So it looks like I'll need a new array class that can live in a number of biggish memory fragments rather than a single contiguous block. Not a major problem, but I would have expected a bit more out of Win32 in this day and age.

上次以下内容产生了 1.875GB 的空间，尽管不连续

Last edit: The following yielded 1.875GB of space, albeit non-contiguous

#define TenMB 1024*1024*10

void    SmallerAllocs()
{

    size_t Total = 0;
    LPVOID  p[200];
    for (int i = 0; i < 200; i++)
    {
        p[i] = malloc(TenMB);
        if (p[i])
            Total += TenMB; else
            break;
    }
    CString Msg;
    Msg.Format("Allocated %0.3lfGB",Total/(1024.0*1024.0*1024.0));
    AfxMessageBox(Msg,MB_OK);
}

推荐答案

可能是调试器在发布模式中欺骗了你?在发布模式下，单步执行和变量值都不可靠.

May it be the cast that the debugger is playing a trick on you in release-mode? Neither single stepping nor the values of variables are reliable in release-mode.

我在发布模式下在 VS2003 中尝试了您的示例，当单步执行时，它起初看起来像是代码落在 return NULL 行上，但是当我继续单步执行时，它最终会继续进入 HeapAlloc，我猜是这个函数失败了，查看反汇编 if (size > _HEAP_MAXREQ) 显示以下内容:

I tried your example in VS2003 in release mode, and when single stepping it does at first look like the code is landing on the return NULL line, but when I continue stepping it eventually continues into HeapAlloc, I would guess that it's this function that's failing, looking at the disassembly if (size > _HEAP_MAXREQ) reveals the following:

00401078  cmp         dword ptr [esp+4],0FFFFFFE0h 

所以我不认为 _HEAP_MAXREQ 有问题.

so I don't think it's a problem with _HEAP_MAXREQ.

这篇关于VS2003 C++ 中异常的堆大小限制的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持html5模板网！