Note that there are some explanatory texts on larger screens.

plurals
  1. POFunction pointer incorrect in Visual Studio 2005, code starts at 1 byte offset
    primarykey
    data
    text
    <p>The code in question hooks into explorer.exe but was crashing on entry to the callback function:</p> <pre>Unhandled exception at 0x60055b50 (redacted.dll) in explorer.exe: 0xC0000005: Access violation writing location 0x548b0cca.</pre> <p>Callstack:</p> <pre> > redacted.dll!myCallWndProcRetCallback(int nCode=0x00000000, unsigned int wParam=0x00000000, long lParam=0x015afa58) Line 799 C++ user32.dll!_DispatchHookW@16() + 0x31 bytes user32.dll!_fnHkINLPCWPRETSTRUCTW@20() + 0x5e bytes user32.dll!___fnDWORD@4() + 0x24 bytes ntdll.dll!_KiUserCallbackDispatcher@12() + 0x13 bytes user32.dll!_NtUserMessageCall@28() + 0xc bytes user32.dll!_SendMessageW@16() + 0x49 bytes explorer.exe!CTaskBand::_FindIndexByHwnd() + 0x21 bytes explorer.exe!CTaskBand::_HandleShellHook() + 0x48 bytes explorer.exe!CTaskBand::v_WndProc() + 0x660 bytes explorer.exe!CImpWndProc::s_WndProc() + 0x3f bytes </pre> <p>Visual Studio 2005 gave the following disassembly:</p> <pre> --- c:\projects\redacted.cpp ------------------------- //------------------------------------------------------------------------------ LRESULT CALLBACK myCallWndProcRetCallback(int nCode, WPARAM wParam, LPARAM lParam) { 60055B50 inc dword ptr [ebx+548B0CC4h] 60055B56 and al,18h 60055B58 mov eax,dword ptr [g_callWndProcRetHook (600B9EE8h)] 60055B5D push esi </pre> <p>and the memory around 0x548B0CC4 is all ?????? so it is not mapped memory, hence the crash.</p> <p>The machine code at the start of myCallWndProcRetCallback is this:</p> <pre> 0x60055B50: ff 83 c4 0c 8b 54 24 18 a1 e8 9e 0b 60 56 52 57 50 ff 15 8c a6 09 60 5f 5e 83 c4 08 c2 0c 00 cc 8b 4c 24 04 8b 01 8b 50 </pre> <p>But Visual Studio also sometimes gives the following disassembly for this function:</p> <pre> --- c:\projects\redacted.cpp ------------------------- 60055B51 add esp,0Ch if ( nCode == HC_ACTION && lParam != NULL) { 60055B54 mov edx,dword ptr [esp+18h] 60055B58 mov eax,dword ptr [g_callWndProcRetHook (600B9EE8h)] 60055B5D push esi </pre> <p>This looks like the correct disassembly but it is from 1 byte later than the disassembly above! You can see the instructions are the same from 0x60055B58 onward.</p> <p>So, it looks like the linker says the function is at 0x60055B50 but the code actually starts at 0x60055B51. I have confirmed the former is the callback set into the Windows hook. So when Windows calls back into the function it executes bad code.</p> <p>The question I have is how the linker could get this wrong? I did a rebuild and the problem went away, it seems random. At the time the /FORCE:MULTIPLE linker option was in effect but without it no link error is reported for this callback.</p> <p>A late addition: Could this be related to the relocation or rebasing of a DLL? If the relocation was off by 1 byte, this could perhaps cause the problem?</p>
    singulars
    1. This table or related slice is empty.
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
 

Querying!

 
Guidance

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload