Bing Xia's Blog

How to load OpenSSL source code into Visual Studio

2010-03-03T15:10:00.004-05:00

I tried to step into OpenSSL source code to resolve some issue. I built the debug version of the library. However, as I tried to step into the source code in the Visual Studio, the source code could not be found even if I had specified the Debug Source Files in the Visual Studio. The same thing happened in WinDBG. Then I realized that it might have something to do with the compiler options. From the MSDN, one of compiler options is

/ZI
Produces a program database, as described above, in a format that supports the Edit and Continue feature. If you want to use Edit and Continue debugging, you must use this option. Because most optimizations are incompatible with Edit and Continue, using /ZI disables any #pragma optimize statements in your code.

Then I added this option to the OpenSSL makefile and built the debug version of the library again. I could load the OpenSSL source code into the Visual Studio since then.

How to verify OpenSSL DSA signature using .NET Crypto

2010-02-19T14:21:00.003-05:00

The DSA signature generated by OpenSSL cannot be verified using .NET crypto API. The error message indicates the DSA signature length is not 40 bytes. The DSA signature generated by .NET crypto API cannot be verified using OpenSSL, either. The error has something to do with ASN.1 encoding routines. Obviously, both errors are due to the different encoding schemes used to encode the generated signature. From the error message, it seems that OpenSSL uses ASN.1 encoding. There is no mention in MSDN regarding to the DSA signature encoding scheme used in .NET crypto. Thank to the article by Jeffrey Walton -- Cryptographic Interoperability: Digital Signatures. .NET crypto uses P1363 to encode the DSA signature. Besides, the article also provides very useful AnsKeyBuilder and AsnKeyParser classes, which make the conversion from P1363 to ASN.1, and vise versa pretty trivial.

Learn PLINQ and Rx

2010-01-27T17:46:00.003-05:00

I am learning some new features in .NET 4.0 -- PLINQ and Reactive Extensions these days. The following links are used in my reading. As I keep learning, I would add more links as long as I consider the article is useful.

PLINQ:

Rx:

http://rxwiki.wikidot.com/

How to edit memory in WinDBG (Cont)

2010-01-22T14:22:00.004-05:00

I just ran into another good blog by Roberto Farah regarding to how to edit memory using WinDBG and its usefulness -- WinDbg Live Programming: Fazendo um log das chamadas à MessageBox. Unfortunately, I do not understand the language used in the blog. However, the blog is just too good to ignore. I redo it on my own and add my own comment on it.

The blog shows how to capture the message text for the MessageBoxW and write it to the debugger console by editing the instruction in the memory.

Allocate 1000 bytes of memory. It would return the starting address of the allocated memory.
0:000> .dvalloc 1000
Allocated 1000 bytes starting at 00340000

Add CRLF in unicode to the beginning of the memory.
0:000> r $t0=0x00340000
0:000> ezu $t0 "\r\n"

Check the memory does start with CRLF.
0:000> db $t0 L0xF
00340000 0d 00 0a 00 00 00 00 00-00 00 00 00 00 00 00 ...............

Start editing the memory right after CRLF. Since the string is in unicode, 6 bytes are required. First push EAX onto the stack and call OutputDebugStringW, then push CRLF onto the stack and call OutputDebugStringW, finally add a debugbreak.
0:000> r $t1=$t0+0x6
0:000> a $t1
00340006 push eax
00340007 call kernel32!OutputDebugStringW
0034000c push 0x00340000
00340011 call kernel32!OutputDebugStringW
00340016 int 3
00340017

Check the memory after editing
0:000> u $t1
00340006 50 push eax
00340007 e8f9b3517c call kernel32!OutputDebugStringW (7c85b405)
0034000c 6800003400 push 340000h
00340011 e8efb3517c call kernel32!OutputDebugStringW (7c85b405)
00340016 cc int 3
00340017 0000 add byte ptr [eax],al
00340019 0000 add byte ptr [eax],al
0034001b 0000 add byte ptr [eax],al

0:000> db $t0
00340000 0d 00 0a 00 00 00 50 e8-f9 b3 51 7c 68 00 00 34 ......P...Q|h..4
00340010 00 e8 ef b3 51 7c cc 00-00 00 00 00 00 00 00 00 ....Q|..........
00340020 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 ................
00340030 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 ................
00340040 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 ................
00340050 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 ................
00340060 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 ................
00340070 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 ................

Set a breakpoint at the beginning of MessageBoxExW. At the breakpoint, store EIP to $t2, the second parameter to EAX, and the new address to EIP.
0:000> bp user32!MessageBoxExW "r $t2=@eip;r eax=poi(@esp+8);r eip=$t1;g"

Set a breakpoint at the debugbreak. When the new instructions have been executed, restore the original EIP.
0:000> bp $t0+0x16 "r eip=$t2;g"
0:000> g

The following is the source code.
int _tmain(int argc, _TCHAR* argv[])
{
wchar_t wzBuffer[128];
DWORD dwInterval = 1000, dwCount = 3;

for(DWORD a = 0 ; a < dwCount ; a++)
{
wsprintf(wzBuffer, L"This is message box%d", a);
MessageBox(NULL, wzBuffer, L"LOG", MB_OK);
Sleep(dwInterval);
}
return 0;
}

How to edit memory in WinDBG

2010-01-21T11:11:00.004-05:00

The blog by Roberto Farah -- Special Command—Editing memory with a, eb, ed, ew, eza, ezu provides a pretty good example on how to edit memory using WinDBG. Of course, the same techniques could be used in reverse engineering as well.

Understand Pool Consumption

2010-01-06T12:29:00.003-05:00

The blog -- Understanding Pool Consumption and Event ID: 2020 or 2019 by Tate provides pretty good information on the kernel pool and how to trouble-shoot its consumption.

Heap corruption in managed code

2009-10-24T12:07:00.020-04:00

It has been reported that the SSH server would crash during download when the client changed the max packet size from 30000 to 50000. It seemed to be a classic example of buffer overflow.

My initial suspect was the heap corruption due to the buffer overflow.

The SSH server is written in C# with its data access and crypto library in native C++. When I used the Application Verifier with full page heap enabled, the windbg showed the following error.

(1244.1004): Access violation - code c0000005 (first chance)
First chance exceptions are reported before any exception handling.
This exception may be expected and handled.
eax=21630953 ebx=00000000 ecx=02f2b024 edx=00000000 esi=02efae5c edi=09f6f0ec
eip=09b5b6eb esp=09f6efdc ebp=09f6f0fc iopl=0 nv up ei pl zr na pe nc
cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00010246
Missing image name, possible paged-out or corrupt data.
+0x9b5b6ea:
09b5b6eb ff505c call dword ptr [eax+5Ch] ds:0023:216309af=????????

0:019> kb
ChildEBP RetAddr Args to Child
WARNING: Frame IP not in any known module. Following frames may be wrong.
09f6f0fc 09b5a602 02f02ba0 02f68a50 02f2b024 +0x9b5b6ea
*** WARNING: Unable to verify checksum for C:\WINDOWS\assembly\NativeImages_v2.0.50727_32\System\3de5bd01124463d7862bd173af90bc83\System.ni.dll
09f6f410 7a57ee09 0148c220 02efad80 02f30b08 +0x9b5a601
09f6f448 7a581eba 02f689f8 792f5681 00000000 System_ni+0x13ee09
09f6f460 79e71b4c 09f6f480 0147fb6a 09f6f4f0 System_ni+0x141eba
09f6f470 79e821b9 09f6f540 00000000 09f6f510 mscorwks!CallDescrWorker+0x33
09f6f4f0 79e96531 09f6f540 00000000 09f6f510 mscorwks!CallDescrWorkerWithHandler+0xa3
09f6f630 79e96564 79241ff0 09f6f764 09f6f684 mscorwks!MethodDesc::CallDescr+0x19c
09f6f64c 79e96582 79241ff0 09f6f764 09f6f684 mscorwks!MethodDesc::CallTargetWorker+0x1f
09f6f664 79f6a259 09f6f684 80acc23f 02a3a6b8 mscorwks!MethodDescCallSite::CallWithValueTypes_RetArgSlot+0x1a
09f6f830 79f6a3ae 09f6f8c0 80acc2ef 02f68a40 mscorwks!ExecuteCodeWithGuaranteedCleanupHelper+0x9f
*** WARNING: Unable to verify checksum for C:\WINDOWS\assembly\NativeImages_v2.0.50727_32\mscorlib\7124a40b9998f7b63c86bd1a2125ce26\mscorlib.ni.dll
09f6f8e0 792f5577 09f6f884 02f30b6c 02f689d8 mscorwks!ReflectionInvocation::ExecuteCodeWithGuaranteedCleanup+0x10f
09f6f8fc 792e01c5 00000000 02f30b6c 02f30b08 mscorlib_ni+0x235577
09f6f914 7a5825b1 00000000 00000000 00000000 mscorlib_ni+0x2201c5
09f6f930 7a57ed70 02f30b08 00000000 00000000 System_ni+0x1425b1
09f6f95c 7a5824b4 00000000 02f30b08 00000000 System_ni+0x13ed70
09f6f994 7928cdc4 02f2c39c 00000060 00000000 System_ni+0x1424b4
09f6f9b4 79e71b4c 02f2c39c 09f6f9d8 0147fb6a mscorlib_ni+0x1ccdc4
09f6f9c8 79e821b9 09f6fb64 00000001 09f6fb58 mscorwks!CallDescrWorker+0x33
09f6fa48 79e8281f 09f6fb64 00000001 09f6fb58 mscorwks!CallDescrWorkerWithHandler+0xa3
09f6fa68 79e82860 09f6fb60 00000001 09f6fb58 mscorwks!DispatchCallBody+0x1e

0:019> !clrstack
OS Thread Id: 0x1004 (19)
ESP EIP
09f6efdc 09b5b6eb SSHServerAPI.Transport.Core._ProcessReadPacket(SSHCommonAPI.SSH2BufferStream, Byte[], Byte[])
09f6f10c 09b5a602 SSHServerAPI.Transport.Core._OnPacketRecv(System.IAsyncResult)
09f6f418 7a57ee09 System.Net.LazyAsyncResult.Complete(IntPtr)
09f6f450 7a581eba System.Net.ContextAwareResult.CompleteCallback(System.Object)
09f6f458 792f5681 System.Threading.ExecutionContext.runTryCode(System.Object)
09f6f884 79e71b4c [HelperMethodFrame_PROTECTOBJ: 09f6f884] System.Runtime.CompilerServices.RuntimeHelpers.ExecuteCodeWithGuaranteedCleanup(TryCode, CleanupCode, System.Object)
09f6f8ec 792f5577 System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object)
09f6f908 792e01c5 System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object)
09f6f920 7a5825b1 System.Net.ContextAwareResult.Complete(IntPtr)
09f6f938 7a57ed70 System.Net.LazyAsyncResult.ProtectedInvokeCallback(System.Object, IntPtr)
09f6f968 7a5824b4 System.Net.Sockets.BaseOverlappedAsyncResult.CompletionPortCallback(UInt32, UInt32, System.Threading.NativeOverlapped*)
09f6f9a0 7928cdc4 System.Threading._IOCompletionCallback.PerformIOCompletionCallback(UInt32, UInt32, System.Threading.NativeOverlapped*)
09f6fb40 79e71b4c [GCFrame: 09f6fb40]

It indicated an access violation exception at 09b5b6eb. It seemed that EAX had been overwritten somewhere before. The stack trace did not provide any very useful clue on the location where the buffer overflow occurred. However, from the stack trace, the access violation seemed to happen in the SSH server managed code layer.

Then I used the SOS command - VerifyHeap to verify the integrity of the managed heap.

0:019> !VerifyHeap
-verify will only produce output if there are errors in the heap
object 02efae5c: bad member 02f2b024 at 02efae84
curr_object : 02efae5c
Last good object: 02efae2c
----------------

0:019> !do 02efae5c
Name: SSHServerAPI.Transport.Core
MethodTable: 09205efc
EEClass: 09b43250
Size: 180(0xb4) bytes
(C:\ftp.server.7.5\debug\SSHServerAPI.dll)
Fields:
MT Field Offset Type VT Attr Value Name
...
79333470 40001df 20 System.Byte[] 0 instance 02f22760 m_buffer
79333470 40001e0 24 System.Byte[] 0 instance 02f19e9c m_ReadBuf
09206af8 40001e1 28 ....SSH2BufferStream 0 instance 02f2b024 m_ReadStream
...

0:019> !do 02f2b024
Invalid object

It indicated that m_ReadStream had been corrupted.

0:019> dd 02f2b024 L1
02f2b024 21630953

0:019> !dumpheap -type SSH2BufferStream
Address MT Size
02efde10 09206af8 60
02f05340 09206af8 60
02f14eec 09206af8 60
02f15074 09206af8 60
02f15304 09206af8 60
02f154a0 09206af8 60
02f15840 09206af8 60
02f15ae0 09206af8 60
object 02f2b024: does not have valid MT
curr_object : 02f2b024
Last good object: 02f22760
----------------
total 8 objects
Statistics:
MT Count TotalSize Class Name
09206af8 8 480 SSHCommonAPI.SSH2BufferStream
Total 8 objects

0:019> !do 02f15ae0
Name: SSHCommonAPI.SSH2BufferStream
MethodTable: 09206af8
EEClass: 09b45adc
Size: 60(0x3c) bytes
(C:\ftp.server.7.5\debug\SSHCommonAPI.dll)
Fields:
MT Field Offset Type VT Attr Value Name
7933061c 400018a 4 System.Object 0 instance 00000000 __identity
...

0:019> dd 02f15ae0 L1
02f15ae0 09206af8

Internally, a managed object starts with 8 bytes of metadata -- first 4 bytes for the sync block index, and the next 4 bytes for the method table address. It seems that the address of a managed object in the windbg points to its method table address. So by comparing the method table addresses between an invalid object and valid one, it confirmed that the method table address of m_ReadStream at 02f2b024 had been overwritten.

Since right above m_ReadStream is m_ReadBuf. It happens to be a byte array. Even more, the array size has been hard-coded to 35000. When I changed its size to 40000, the SSH server no longer crashed during download. It looked like that the buffer overflow of m_ReadBuf was the culprit. Then I tried to understand where and how in the code the buffer overflow happened. I found that m_ReadBuf was solely used to read client responses. During download, each client response would be much smaller that 35000. So m_ReadBuf should not be overflowed.

Now the question is why there was no crash in the server when m_ReadBuf size was changed from 35000 to 40000.

One possible explanation could be that when the m_ReadBuf size was 35000, the overflow happened to overwrite the next object; when the m_ReadBuf size is 40000, the next object was moved back by 5000 bytes, so when the overflow happened, it just overwrote some unused memory, and therefore there was no crash!

Since a managed object in the managed heap starts with its method table address, which would not be changed throughout its lifetime, so a data breakpoint could be used to break the code when the method table address is overwritten. By this way, the stack trace would display the exact location when the overflow occurs.

The following steps were used to set a data breakpoint.

0:022> !dumpheap -type SSHServerAPI.SFTP.Subsystem
Address MT Size
02f1af4c 0920af7c 84
total 1 objects
Statistics:
MT Count TotalSize Class Name
0920af7c 1 84 SSHServerAPI.SFTP.Subsystem
Total 1 objects

0:022> !dumpmt -md 0920af7c
EEClass: 0a1d64b4
Module: 02534164Name: SSHServerAPI.SFTP.Subsystem
mdToken: 02000033 (C:\ftp.server.7.5\debug\SSHServerAPI.dll)
BaseSize: 0x54
ComponentSize: 0x0
Number of IFaces in IFaceMap: 1Slots in VTable: 72
--------------------------------------
MethodDesc Table
Entry MethodDesc JIT Name
...
06ad4691 0920ad10 NONE SSHServerAPI.SFTP.Subsystem.FXP_Read()
...

0:022> !bpmd -md 0920ad10
MethodDesc = 0920ad10
Adding pending breakpoints...
sxe -c "!bpmd -notification;g" clrn

First, set a managed breakpoint at SSHServerAPI.SFTP.Subsystem.FXP_Read, which is a core function for download.

0:022> g
Closing _RecordsetPtr (m_rset)Closing _RecordsetPtr (m_rset)ModLoad: 605d0000 605d9000 C:\WINDOWS\system32\mslbui.dll
(111c.f2c): CLR notification exception - code e0444143 (first chance)
JITTED SSHServerAPI!SSHServerAPI.SFTP.Subsystem.FXP_Read()
Setting breakpoint: bp 0A446030 [SSHServerAPI.SFTP.Subsystem.FXP_Read()]
bp 0A446030
Breakpoint: JIT notification received for method SSHServerAPI.SFTP.Subsystem.FXP_Read().
Breakpoint 0 hit
eax=0920ad10 ebx=00000000 ecx=02f05e9c edx=00000002 esi=09e6ef64 edi=02f001f4
eip=0a446030 esp=09e6ecec ebp=09e6edd8 iopl=0 nv up ei pl nz na pe nc
cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00000206
+0xa44602f:
0a446030 55 push ebp

0:018> !dumpheap -type SSHServerAPI.Transport.Core
Address MT Size
02efae04 09205efc 180 ThinLock owner 6 (0132efa0) Recursive 0
02f224d8 09205efc 180
total 2 objects
Statistics:
MT Count TotalSize Class Name
09205efc 2 360 SSHServerAPI.Transport.Core
Total 2 objects

0:018> !do 02efae04
Name: SSHServerAPI.Transport.Core
MethodTable: 09205efc
EEClass: 09b42cac
Size: 180(0xb4) bytes
(C:\ftp.server.7.5\debug\SSHServerAPI.dll)
Fields:
MT Field Offset Type VT Attr Value Name
...
79333470 40001df 20 System.Byte[] 0 instance 02f1064c m_buffer
79333470 40001e0 24 System.Byte[] 0 instance 02f07d88 m_ReadBuf
09206af8 40001e1 28 ....SSH2BufferStream 0 instance 02f18f10 m_ReadStream
...

0:018> !do 02f18f10
Name: SSHCommonAPI.SSH2BufferStream
MethodTable: 09206af8
EEClass: 09b45538
Size: 60(0x3c) bytes
(C:\ftp.server.7.5\debug\SSHCommonAPI.dll)
Fields:
MT Field Offset Type VT Attr Value Name
7933061c 400018a 4 System.Object 0 instance 00000000 __identity
...

0:018> ba w4 02f18f10

When the breakpoint hit, retrieved the address of m_ReadStream, checked whether the object was still valid, and then set a data breakpoint on its first four bytes, which contained the address of its method table.

There were two instances of SSHServerAPI.Transport.Core, the same steps could be used to set a data breakpoint.

0:018> g
Breakpoint 1 hit
eax=00000058 ebx=02f18f04 ecx=7ae6d4c1 edx=0000000c esi=02f18ef4 edi=041429f8
eip=0fb76a99 esp=09e6e604 ebp=00000010 iopl=0 nv up ei pl nz na po nc
cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00000202
*** WARNING: Unable to verify checksum for C:\ftp.server.7.5\debug\IPSLIBEAY32.dll
IPSLIBEAY32!AES_cbc_encrypt+0xc9:
0fb76a99 42 inc edx

0:018> !do 02efae04
Name: SSHServerAPI.Transport.Core
MethodTable: 09205efc
EEClass: 09b42cac
Size: 180(0xb4) bytes
(C:\ftp.server.7.5\debug\SSHServerAPI.dll)
Fields:
MT Field Offset Type VT Attr Value Name
...
79333470 40001df 20 System.Byte[] 0 instance 02f1064c m_buffer
79333470 40001e0 24 System.Byte[] 0 instance 02f07d88 m_ReadBuf
09206af8 40001e1 28 ....SSH2BufferStream 0 instance 02f18f10 m_ReadStream
...

0:018> !do 02f18f10
Invalid object

When the breakpoint hit, checked that m_ReadStream was indeed no longer valid.

0:018> kb
ChildEBP RetAddr Args to Child
09e6e600 79e71bb0 0000000e 0148c220 0fbc98b0 IPSLIBEAY32!AES_cbc_encrypt+0xc9
09e6e62c 0fb95fb7 0413a158 02f10664 000088b0 mscorwks!GetThreadGeneric+0xe
09e6e64c 0fb96bfa 05c97af0 02f10654 05c97b20 IPSLIBEAY32!EVP_aes_192_ecb+0x37
09e6e65c 0fb96b79 05c97af0 02f10664 0413a158 IPSLIBEAY32!EVP_EncryptUpdate+0x12a
09e6e670 79e71d71 09e6e6a8 00000008 0132efa0 IPSLIBEAY32!EVP_EncryptUpdate+0xa9
09e6e68c 79e71d8b 05c97af0 02f10664 09e6e730 mscorwks!PInvokeCalliWorker+0x35
09e6e7b8 7c929fef 0148c220 02fd788c 02fd78c8 mscorwks!PInvokeCalliReturnFromCall
09e6e818 02fd76f8 02fd76f8 02fd7358 02fd7358 ntdll!RtlAcquireResourceShared+0x120
WARNING: Frame IP not in any known module. Following frames may be wrong.
09e6e824 02fd7358 02fd7358 00000000 00000000 +0x2fd76f7
09e6e828 02fd7358 00000000 00000000 02fd76f8 +0x2fd7357
09e6e82c 00000000 00000000 02fd76f8 02fd71a4 +0x2fd7357

0:018> !clrstack
OS Thread Id: 0xf2c (18)
ESP EIP
09e6e6fc 0fb76a99 [PInvokeCalliFrame: 09e6e6fc]
09e6e71c 09b59311 UtilAPI.SymmetricTransform.TransformBlock(Byte[], Int32, Int32, Byte[], Int32)
09e6e75c 09b587d6 SSHServerAPI.Transport.Core.SendPacket(SSHCommonAPI.Transport.PacketOut)
09e6e9c8 0a1ebc7e SSHServerAPI.Transport.Channel.SendSSHPackets(SSHCommonAPI.SSH2BufferStream ByRef)
09e6ea7c 0a1eb6c1 SSHServerAPI.Transport.Channel.SendPacket(SSHCommonAPI.SFTP.SFTPPacket)
09e6eb70 0a4465ed SSHServerAPI.SFTP.Subsystem.FXP_Read()
09e6ecf0 0a1eabb0 SSHServerAPI.SFTP.Subsystem.DispathPacket()
09e6ede0 0a1ea5c7 SSHServerAPI.SFTP.Subsystem.ProcessData(SSHCommonAPI.SSH2BufferStream)
09e6ef0c 0a1ea0fa SSHServerAPI.Transport.Channel.ChannelData(SSHCommonAPI.SSH2BufferStream)
09e6ef94 0a1e9e79 SSHServerAPI.Transport.PacketDispatch.HandleChannelData()
09e6efcc 0a1e7a89 SSHServerAPI.Transport.PacketDispatch._ChannelDispatch()
09e6f01c 09b5bb59 SSHServerAPI.Transport.PacketDispatch.Process(SSHCommonAPI.SSH2BufferStream, Byte[], Byte[])
09e6f064 09b5b7cb SSHServerAPI.Transport.Core._ProcessReadPacket(SSHCommonAPI.SSH2BufferStream, Byte[], Byte[])
09e6f18c 09b5a72a SSHServerAPI.Transport.Core._OnPacketRecv(System.IAsyncResult)
09e6f498 7a57ee09 System.Net.LazyAsyncResult.Complete(IntPtr)
09e6f4d0 7a581eba System.Net.ContextAwareResult.CompleteCallback(System.Object)
09e6f4d8 792f5681 System.Threading.ExecutionContext.runTryCode(System.Object)
09e6f904 79e71b4c [HelperMethodFrame_PROTECTOBJ: 09e6f904]
System.Runtime.CompilerServices.RuntimeHelpers.ExecuteCodeWithGuaranteedCleanup(TryCode, CleanupCode, System.Object)
09e6f96c 792f5577 System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext,
System.Threading.ContextCallback, System.Object)
09e6f988 792e01c5 System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object)
09e6f9a0 7a5825b1 System.Net.ContextAwareResult.Complete(IntPtr)
09e6f9b8 7a57ed70 System.Net.LazyAsyncResult.ProtectedInvokeCallback(System.Object, IntPtr)
09e6f9e8 7a5824b4 System.Net.Sockets.BaseOverlappedAsyncResult.CompletionPortCallback(UInt32, UInt32, System.Threading.NativeOverlapped*)
09e6fa20 7928cdc4 System.Threading._IOCompletionCallback.PerformIOCompletionCallback(UInt32, UInt32, System.Threading.NativeOverlapped*)
09e6fbc0 79e71b4c [GCFrame: 09e6fbc0]

From the stack trace, it seemed that the overwrite happened during AES encryption. Now I have the exact location where the overwrite occurred!

0:018> uf IPSLIBEAY32!AES_cbc_encrypt
IPSLIBEAY32!AES_cbc_encrypt:
0fb769d0 55 push ebp
0fb769d1 57 push edi
0fb769d2 56 push esi
0fb769d3 53 push ebx
0fb769d4 83ec1c sub esp,1Ch
0fb769d7 837c244401 cmp dword ptr [esp+44h],1 ; a flag
0fb769dc 8b7c2430 mov edi,dword ptr [esp+30h] ; input buffer
0fb769e0 8b5c2434 mov ebx,dword ptr [esp+34h] ; output buffer
0fb769e4 8b6c2438 mov ebp,dword ptr [esp+38h] ; length
0fb769e8 8b742440 mov esi,dword ptr [esp+40h]
0fb769ec 0f848f000000 je IPSLIBEAY32!AES_cbc_encrypt+0xb1 (0fb76a81)
...
IPSLIBEAY32!AES_cbc_encrypt+0xc0:
0fb76a90 8a0432 mov al,byte ptr [edx+esi]
0fb76a93 32043a xor al,byte ptr [edx+edi]
0fb76a96 88041a mov byte ptr [edx+ebx],al
0fb76a99 42 inc edx
0fb76a9a 83fa0f cmp edx,0Fh
0fb76a9d 76f1 jbe IPSLIBEAY32!AES_cbc_encrypt+0xc0 (0fb76a90)

IPSLIBEAY32!AES_cbc_encrypt+0xcf:
0fb76a9f ebc5 jmp IPSLIBEAY32!AES_cbc_encrypt+0x96 (0fb76a66)
...

0:018> r
eax=00000058 ebx=02f18f04 ecx=7ae6d4c1 edx=0000000c esi=02f18ef4 edi=041429f8
eip=0fb76a99 esp=09e6e604 ebp=00000010 iopl=0 nv up ei pl nz na po nc
cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00000202
IPSLIBEAY32!AES_cbc_encrypt+0xc9:
0fb76a99 42 inc edx

From the assembly, EDX was the counter, EDI was the input buffer, and EBX was the output buffer. From the instruction at 0fb76a96, (EDX+EBX) = (0000000c+02f18f04) = 02f18f10. And 02f18f10 happened to be the address for m_ReadStream. So somehow during the encryption, the output buffer had been written out of its boundary.

With those information, I traced the code again and found during the AES CBC mode encryption, the output ciphertext sometimes could be longer than the input plaintext. However, the code assumed that the input plaintext and output ciphertext would always have the same length. So when the ciphertext was longer than the input plaintext, the output buffer would be written out of its boundary.

Debug a child process

2009-10-16T09:43:00.006-04:00

Sometimes the debugee is a child process, which could only be launched by its parent process.

The following two cases need to be considered.
First, the parent process could be launched directly, such as the command prompt. In that case, windbg -o processname could be used to debug its all child process. For instance, Why did DllRegisterServer return 80070006? provides an example on how to debug a specific child process launched by the command prompt.

Second, the parent process could not be launched directly, such as services. In that case, the windbg meta-command .childdbg (Debug Child Processes) could be used.

For instance, to debug aspnet_wp.exe -- an IIS worker process from its startup, the following steps could be used.
1. iisreset
It will kill the aspnet_wp.exe.
2. attach windbg to inetinfo.exe
3. .childdbg 1
It will enable child process debugging in the windbg.
4. send an asp .net request to IIS
IIS will launch aspnet_wp.exe when the first asp .net request comes in.

Pseudo-registers for Visual Studio IDE

2009-10-08T14:02:00.003-04:00

The post by Gregg -- Whidbey Debugger pseudo-register - $user introduces a new pseudo-register -- $user. The pseudo-register provides "loads of information about the debuggee user" and is helpful when it comes to debugging security related issues. Besides, '@err,hr' will print the last win32 error formatted as an error message. It is also very handy.

The post by Kenny Kerr -- X64 Debugging With Pseudo Variables And Format Specifiers provides a more comprehensive list of pseudo-registers.

WCF Service application hangs

2009-09-30T10:42:00.009-04:00

Ad Hoc Transfer (AHT) is a WCF ASP .NET application. When performing multiple downloads -- about 10 files from the FireFox, sometimes a transfer would pause ~30 seconds or more before download begins or in the middle of the download.

There could be various reasons for a WCF service application to hang.

From the size of memory dump, there was nothing out of ordinary, so the hang might not be related to the memory issue.

One possible reason could be WCF connection pool exhausion described in the post -- Troubleshooting WCF Service application hangs.

0:033> !dumpheap -type ServiceThrottle -short
02ebe87c
02f2b6e0
02f2d4ac

0:033> !do 02ebe87c
Name: System.ServiceModel.Dispatcher.ServiceThrottle
MethodTable: 0f1ea548
EEClass: 0ee51f20
Size: 36(0x24) bytes
(C:\WINDOWS\assembly\GAC_MSIL\System.ServiceModel\3.0.0.0__b77a5c561934e089\System.ServiceModel.dll)
Fields:
MT Field Offset Type VT Attr Value Name
0f1ea58c 400371d 4 ...cher.FlowThrottle 0 instance 02ebe93c calls ; maxConcurrentCalls
0f1ea58c 400371e 8 ...cher.FlowThrottle 0 instance 02ebea34 sessions ; maxConcurrentSessions
0f1a6020 400371f c ...her.QuotaThrottle 0 instance 00000000 dynamic
0f1ea58c 4003720 10 ...cher.FlowThrottle 0 instance 00000000 instanceContexts
0f1e9bb0 4003721 14 ...l.ServiceHostBase 0 instance 02ebe674 host
793044cc 4003722 1c System.Boolean 1 instance 1 isActive
7933061c 4003723 18 System.Object 0 instance 02ebe8a0 thisLock

0:033> !do 02ebea34
Name: System.ServiceModel.Dispatcher.FlowThrottle
MethodTable: 0f1ea58c
EEClass: 0ee51f8c
Size: 36(0x24) bytes
(C:\WINDOWS\assembly\GAC_MSIL\System.ServiceModel\3.0.0.0__b77a5c561934e089\System.ServiceModel.dll)
Fields:
MT Field Offset Type VT Attr Value Name
79332c4c 40034f5 18 System.Int32 1 instance 10 capacity
79332c4c 40034f6 1c System.Int32 1 instance 8 count
7933061c 40034f7 4 System.Object 0 instance 02ebea58 mutex
7931e960 40034f8 8 ...ding.WaitCallback 0 instance 02ebea14 release
00000000 40034f9 c 0 instance 02ebea64 waiters
79330a00 40034fa 10 System.String 0 instance 02ebe99c propertyName
79330a00 40034fb 14 System.String 0 instance 02ebe9d8 configName

0:033> !do 02ebe674
Name: System.ServiceModel.ServiceHost
MethodTable: 0f1ef2b4
EEClass: 0ee55db4
Size: 140(0x8c) bytes
(C:\WINDOWS\assembly\GAC_MSIL\System.ServiceModel\3.0.0.0__b77a5c561934e089\System.ServiceModel.dll)
Fields:
MT Field Offset Type VT Attr Value Name
...
System.Object 0 instance 00000000 singletonInstance 79331840 40032cd 7c System.Type 0 instance 02ebe660 serviceType
0f1ea7dc 40032ce 80 ...ontractCollection 0 instance 02ebed1c reflectedContracts
7932c8e0 40032cf 84 System.IDisposable 0 instance 00000000 disposableInstance

0:033> !do 02ebe660
Name: System.RuntimeType
MethodTable: 79331b80
EEClass: 790eddd8
Size: 20(0x14) bytes
Type MethodTable: 10fe7104
Type Name: WSFTPWebService.WSFTPWebService
(C:\WINDOWS\assembly\GAC_32\mscorlib\2.0.0.0__b77a5c561934e089\mscorlib.dll)
Fields:
...

Since the capacity is still larger than the count, so WCF connection pool exhaustion should not be the reason.

Since the hang only happens in multiple downloads, another possible reason could be the thread synchronization issue. From the code, it seems that in multiple downloads, the same WCF proxy is used for all the download requests among different threads. The MSDN site does not mention that the ClientBase class is thread-safe. However, in the code, the thread-safe assumption has been made and no calls to WCF proxy API have been synchronized among threads. So the possible fix would be to synchronize the call to WCF proxy API.

Counterfactual Debugging: Data Ordering

2009-09-25T11:25:00.006-04:00

The post -- Counterfactual Debugging: Data Ordering by Dmitry Vostokov provides an interesting exercise on how to understand Dereference Fixpoints issue from the assembly code level.

Mixed DLL Loading Problem

2009-09-24T12:30:00.004-04:00

The post -- NET Hang Case Study: The GC-Loader Lock Deadlock (a story of mixed mode dlls) by Tess Ferrandez provides a good study case for mixed dll loading problem. Besides, it also shows some useful WinDBG tips, such as how to detect the deadlock caused by the mixed dll, how to tell whether GC has been triggered and which thread triggered it.

ETW Introduction and Overview

2009-09-21T12:46:00.004-04:00

With the advent of Vista and Windows 7, ETW (Event Tracing for Windows) would become more and more widely used in debugging and instrumenting the applications. The following links provide some useful information regarding to what ETW is and how to enable ETW for troubleshooting,

Enable WCF debugging trace and message logs

2009-09-17T14:10:00.008-04:00

WCF debugging trace and message logs are sometimes very crucial in resolving WCF related issues. To enable the trace and message logs, the application configuration would be modified as following,

within [system.serviceModel] and [/system.serviceModel] tags, add [diagnostics] [messageLogging maxMessagesToLog="30000"
logEntireMessage="true"
logMessagesAtServiceLevel="true"
logMalformedMessages="true"
logMessagesAtTransportLevel="true"]
[/messageLogging]
[/diagnostics]
within [configuration] and [/configuration] tags, add [system.diagnostics]
[sources]
[source name="System.ServiceModel" switchValue="Verbose, ActivityTracing" propagateActivity="true" ]
[listeners]
[add name="xml" /]
[/listeners]
[/source]
[source name="System.ServiceModel.MessageLogging" switchValue="Verbose"]
[listeners]
[add name="xml" /]
[/listeners]
[/source]
[/sources]
[sharedListeners]
[add name="xml" type="System.Diagnostics.XmlWriterTraceListener" initializeData="WCFTracing.svclog" /]
[/sharedListeners]
[trace autoflush="true" /]
[/system.diagnostics]

After modifying the configuration file, make sure the process hosting the WCF proxy or service has the write permission to the folder where the trace and message log file will reside.

After the trace and message log file is generated, the Service Trace Viewer could be used to analyze the trace and log file.

Here is a tutorial on how to use the Service Trace Viewer.

Function keys for a command-prompt ninja

2009-09-01T11:03:00.003-04:00

The following function keys might improve the usability of the command prompt.

F1 retypes the previous command one character at a time
F2 brings up a dialog and asks “Enter the char to copy up to:”
F3 retypes the last command in full
F4 brings up a dialog and asks “Enter char to delete up to:”
F5 as for F3
F6 Print EOF character (Ctrl+Z)
F7 brings up a dialog of all the recent command history
F8 brings up each of the most recent commands, one at a time
F9 brings up a dialog and asks “Enter command number:”

WCF web service failed with 0x8007007E

2009-07-31T16:43:00.016-04:00

The WCF web service I worked on failed with the following error message when I tried to launch it from IIS:

The specified module could not be found. (Exception from HRESULT: 0x8007007E).

From the previous encounter, I was pretty sure that it must have something to do with WrapperUtilAPI.dll. The library written in managed C++ serves as the bridge between the managed and unmanaged code.

Since the web service is in the managed code, I first tried the Fusion log viewer to see whether there had been any assembly binding error. Except that the web service module itself did not show in the log viewer, there had been no assembly binding error for all other dependent modules, such as WrapperUtilAPI.dll:

The operation was successful.
Bind result: hr = 0x0. The operation completed successfully.

It seemed that the culprit was from the unmanaged code. Then I used Process Monitor to record the activities from aspnet_wp.exe when I tried to launch the web service through it.

For a DLL to load successfully, the following operation sequence should take place,

QueryOpen C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727\Temporary ASP.NET Files\wsftp.webservicehost\89b3cfc4\ab5fbeff\assembly\dl3\f9b47a83\007d695c_3011ca01\ftpaccess.dll NAME NOT FOUND
QueryOpen C:\WINDOWS\system32\ftpaccess.dll NAME NOT FOUND
QueryOpen C:\WINDOWS\system\ftpaccess.dll NAME NOT FOUND
QueryOpen C:\WINDOWS\ftpaccess.dll NAME NOT FOUND
QueryOpen C:\WINDOWS\system32\ftpaccess.dll NAME NOT FOUND
QueryOpen C:\Perl\site\bin\ftpaccess.dll NAME NOT FOUND
QueryOpen C:\Perl\bin\ftpaccess.dll NAME NOT FOUND
QueryOpen C:\WINDOWS\system32\ftpaccess.dll NAME NOT FOUND
QueryOpen C:\WINDOWS\ftpaccess.dll NAME NOT FOUND
QueryOpen C:\WINDOWS\system32\wbem\ftpaccess.dll NAME NOT FOUND
QueryOpen C:\Program Files\Microsoft SQL Server\90\Tools\Binn\ftpaccess.dll NAME NOT FOUND
QueryOpen C:\Program Files\Ipswitch\Common\ftpaccess.dll SUCCESS
QueryOpen C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727\Temporary ASP.NET Files\wsftp.webservicehost\89b3cfc4\ab5fbeff\assembly\dl3\f9b47a83\007d695c_3011ca01\ftpaccess.dll NAME NOT FOUND
QueryOpen C:\WINDOWS\system32\ftpaccess.dll NAME NOT FOUND
QueryOpen C:\WINDOWS\system\ftpaccess.dll NAME NOT FOUND
QueryOpen C:\WINDOWS\ftpaccess.dll NAME NOT FOUND
QueryOpen C:\WINDOWS\system32\ftpaccess.dll NAME NOT FOUND
QueryOpen C:\Perl\site\bin\ftpaccess.dll NAME NOT FOUND
QueryOpen C:\Perl\bin\ftpaccess.dll NAME NOT FOUND
QueryOpen C:\WINDOWS\system32\ftpaccess.dll NAME NOT FOUND
QueryOpen C:\WINDOWS\ftpaccess.dll NAME NOT FOUND
QueryOpen C:\WINDOWS\system32\wbem\ftpaccess.dll NAME NOT FOUND
QueryOpen C:\Program Files\Microsoft SQL Server\90\Tools\Binn\ftpaccess.dll NAME NOT FOUND
QueryOpen C:\Program Files\Ipswitch\Common\ftpaccess.dll SUCCESS
CreateFile C:\Program Files\Ipswitch\Common\ftpaccess.dll SUCCESS
CloseFile C:\Program Files\Ipswitch\Common\ftpaccess.dll SUCCESS
Load Image C:\Program Files\Ipswitch\Common\ftpaccess.dll SUCCESS

QueryOpen will try to locate a module using the DLL search path.

While looking through the log from Process Monitor, I noticed the following lines,

QueryOpen C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727\Temporary ASP.NET Files\wsftp.webservicehost\89b3cfc4\ab5fbeff\assembly\dl3\f9b47a83\007d695c_3011ca01\zlib1.dll NAME NOT FOUND
QueryOpen C:\WINDOWS\system32\zlib1.dll NAME NOT FOUND
QueryOpen C:\WINDOWS\system\zlib1.dll NAME NOT FOUND
QueryOpen C:\WINDOWS\zlib1.dll NAME NOT FOUND
QueryOpen C:\WINDOWS\system32\zlib1.dll NAME NOT FOUND
QueryOpen C:\Perl\site\bin\zlib1.dll NAME NOT FOUND
QueryOpen C:\Perl\bin\zlib1.dll NAME NOT FOUND
QueryOpen C:\WINDOWS\system32\zlib1.dll NAME NOT FOUND
QueryOpen C:\WINDOWS\zlib1.dll NAME NOT FOUND
QueryOpen C:\WINDOWS\system32\wbem\zlib1.dll NAME NOT FOUND
QueryOpen C:\Program Files\Microsoft SQL Server\90\Tools\Binn\zlib1.dll NAME NOT FOUND
QueryOpen C:\Program Files\Ipswitch\Common\zlib1.dll NAME NOT FOUND
QueryOpen C:\Program Files\Support Tools\zlib1.dll NAME NOT FOUND
QueryOpen C:\Program Files\Ipswitch\Common\zlib1.dll NAME NOT FOUND
QueryOpen C:\MinGW\bin\zlib1.dll NAME NOT FOUND
QueryOpen C:\Program Files\QuickTime\QTSystem\zlib1.dll NAME NOT FOUND
QueryOpen C:\WINDOWS\system32\windowspowershell\v1.0\zlib1.dll NAME NOT FOUND
QueryOpen C:\WINDOWS\WinSxS\x86_Microsoft.VC90.CRT_1fc8b3b9a1e18e3b_9.0.30729.1_x-ww_6f74963e\zlib1.dll NAME NOT FOUND
CloseFile C:\WINDOWS\WinSxS\x86_Microsoft.VC90.CRT_1fc8b3b9a1e18e3b_9.0.30729.1_x-ww_6f74963e SUCCESS
CloseFile C:\WINDOWS\WinSxS\x86_Microsoft.VC90.CRT_1fc8b3b9a1e18e3b_9.0.30729.1_x-ww_6f74963e SUCCESS
CloseFile C:\WINDOWS\WinSxS\x86_Microsoft.VC90.CRT_1fc8b3b9a1e18e3b_9.0.30729.1_x-ww_6f74963e SUCCESS
CloseFile C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727\Temporary ASP.NET Files\wsftp.webservicehost\89b3cfc4\ab5fbeff\assembly\dl3\f9b47a83\007d695c_3011ca01\WrapperUtilAPI.DLL SUCCESS

It seemed that the application tried to load zlib1.dll. However, the module could not be found in the DLL search path. Therefore, instead of loading zlib1.dll, the application closed WrapperUtilAPI.dll, which had been loaded early.

Since I was pretty sure that zlib1.dll had to be loaded so that WrapperUtilAPI.dll could be loaded successfully, and its location was not in the PATH environment variable and the DLL search path. So at the point, the root cause seemed to be the failure to load zlib1.dll. Then I copied it to the location on the DLL search path, the web service could be launched from IIS.

How to Bind to a custom App.Config file?

2009-07-29T19:41:00.006-04:00

One of questions I ever had when I worked on the ISAPI extension was where to store the configuration file. The ISAPI extension is in unmanaged code hosted in IIS via dllhost.exe. It will interact with the backend managed code by COM interop. The manged code will create WCF proxies to communicate to the WCF web service. Therefore, the configuration file will be required by the WCF code to load the information regarding to the contract, binding, endpoint, and client behavior. A naive solution is to use dllhost.exe.config in the Windows system directory. Fortunately, the article by Ohad -- Binding to a custom App.Config file provides a useful information to solve the problem.

Build and debug ISAPI extension

2009-07-22T17:39:00.004-04:00

I have built and debugged an ISAPI extension DLL before. However, it still gave me a little bit hard time today when I tried to debug an ISAPI extension DLL which I built by myself. The article by Mehdi Mousav -- What is an ISAPI Extension? provies a pretty good coverage on this topic. The setup of ISAP extension DLL in IIS is straightforward -- copy the DLL to the folder which can be accessed from the URL, such as http://Domain/DLLPath. For the author of the ISAPI extension DLL, one caveat is the module definition file (.def). The ISAPI extension DLL needs to be built with it; otherwise, the ISAPI extension APIs will not be visible to an IIS worker process.

Obtain stack limit of WOW64 process

2009-07-07T16:27:00.004-04:00

A WOW64 process is a 32-bit process running on x64 Windows. The article -- Raw Stack Dump of WOW64 Process by Dmitry Vostokov explains several ways to retrieve the stack limit of a thread from TEB for a WOW64 process.

Hunt for parameters of a function

2009-07-07T00:21:00.003-04:00

The article -- Hunting for a Driver by Dmitry Vostokov illustrates a useful approach to discover the parameters of a function.

MSIL tutorials

2009-06-03T10:15:00.003-04:00

The following is some useful links to MSIL tutorials.

Introduction to MSIL by Kenny Kerr
ILDASM is Your New Best Friend by John Robbins
MSIL Tutorial by Alex F

Calling conventions

2009-05-15T16:04:00.017-04:00

The calling convention of a function answers the following questions,

How parameters are passed into a function
Who cleans up the stack on function return

The following lists common x86 function calling conventions.

Win32 (Stdcall): The parameters are pushed onto the stack from right to left. The called function cleans up the stack.

01003540 push esi
01003541 lea eax,[ebp-62Ch]
01003547 push eax
01003548 call module!func (01001058)

Native C++ (Thiscall): "this" pointer is passed via ECX; the rest of parameters are pushed onto the stack from right to left. The called function cleans up the stack.

00413586 mov eax,dword ptr [ebp+0Ch]
00413589 push eax
0041358a mov ecx,dword ptr [ebp+8]
0041358d push ecx
0041358e mov ecx,dword ptr [ebp-14h] ; store "this" to ECX
00413591 call module!func (00411505)

COM (Stdcall for C++): The parameters are pushed onto the stack from right to left, include "this" pointer, i.e. "this" pointer is pushed onto the stack as the first parameter. The called function cleans up the stack.

01002ffe mov ecx,dword ptr [eax] ; ecx="this"->lpvtbl
01003000 push ebx
01003001 push offset 01002c7c
01003006 push eax ; "this" as 1st parameter
01003007 call dword ptr [ecx] ; [ecx]=QueryInterface, [ecx+4]=AddRef, [ecx+8]=Release

Fastcall: First two parameters are passed in via ECX and EDX; the rest are pushed onto the stack from right to left. The called function cleans up the stack.

0100248e mov edx,eax ; 2nd parameter
01002490 mov ecx,edi ; 1st parameter
01002492 call module!func (01002445)

Cdecl: The parameters are pushed onto the stack from right to left. The calling function cleans up the stack.

01002490 push eax
01002491 push offset 0100118c
01002496 call module!func (010026c5)
0100249b pop ecx ; pop 1st parameter
0100249c pop ecx ; pop 2nd parameter

.NET Framework: The .NET Framework uses the fastcall calling convention. The first two paramters are passed via ECX and EDX. For an instance method, "this" pointer is passed via ECX as the first paramter.

There are three methods to dispatch a method call.

Interface-based dispatch:
mov ecx,edi ; move "this" pointer into ecx
mov eax,dword ptr [ecx] ; move "TypeHandle" into eax
mov eax,dword ptr [eax+0Ch] ; move IVMap into eax at offset 12
mov eax,dword ptr [eax+30h] ; move the ifc impl start slot into eax
call dword ptr [eax] ; call method

Direct dispatch:
mov ecx,esi ; move "this" pointer into ecx
cmp dword ptr [ecx],ecx ; compare and set flags
call dword ptr ds:[009552D8h] ; directly call method

Virtual dispatch:
mov ecx,esi ; move "this" pointer into ecx
mov eax,dword ptr [ecx] ; acquire the MethodTable address
call dword ptr [eax+44h] ; dispatch to the method at offset 0x44

(Excerpted from Drill Into .NET Framework Internals to See How the CLR Creates Runtime Objects)

Debug a service application from startup

2009-05-09T14:03:00.010-04:00

The article -- Debugging a Service Application provides pretty detailed information on debugging a service application. However, besides adding a ProgramName registry key under HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Image File Execution Options, where ProgramName is the name of the service application's executable file, and adjusting the service application timeout by adding or modifying a DWORD value called ServicesPipeTimeout under HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control, in my recent experience, I have found that the following two items should be aware of as well.

For WinDBG GUI to pop up, Allow service to interact with desktop option should be enabled for the service application.

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\ServicesPipeTimeout could affect the Windows startup time. For instance, when I changed its value to 86,400,000 (24 hours), it took my XP box about one day to fully start up! When I changed its value to 600,000 (10 minutes), it took my XP box about 10 minutes to fully start up.

Static data members in inheritance

2009-05-06T13:13:00.004-04:00

A static data member means that one copy of data would be shared among all instances of the same class. It is true even when inheritance is involved.

class base
{
public: static wchar_t * s_pName;
};

wchar_t * base::s_pName = L"base";

class derived1 : public base { };

class derived2 : public base { };

int _tmain(int argc, _TCHAR* argv[])
{
derived1 d1;
d1.s_pName = L"derived1";

derived2 d2;
d2.s_pName = L"derived2";

wprintf(L"%s\n", d1.s_pName);
wprintf(L"%s\n", d2.s_pName);

return 0;
}

Since s_pName is a static data member in the base class, all the instances of the derived classes share one copy of the data member. Therefore, the output of the above program would be as following,

derived2
derived2.

Obviously, it is not a desirable output for the instances of derived1 class. To overcome this limitation, we could use a so-called "mixin-style" base class. I replace the angle brackets for a template with square brackets.

template[class T]
class base
{
public: static wchar_t * s_pName;
};

template[class T] wchar_t * base[T]::s_pName = L"base";

class derived1 : public base[derived1] { };

class derived2 : public base[derived2] { };

int _tmain(int argc, _TCHAR* argv[])
{
derived1 d1;
d1.s_pName = L"derived1";

derived2 d2;
d2.s_pName = L"derived2";

wprintf(L"%s\n", d1.s_pName);
wprintf(L"%s\n", d2.s_pName);

return 0;
}

The base class is turned into a template. The template part of the base class ensures that each derived class gets a different s_pName. The output from the above program would be as following,

derived1
derived2.

Why did PEM_read_bio_PrivateKey fail to load a key in FIPS mode?

2009-05-06T10:29:00.008-04:00

My former manager told me that PEM_read_bio_PrivateKey failed to load a private key generated via OpenSSL commandline in FIPS mode. So I tried to load the key he sent to me using a simple test application. When not in FIPS mode, PEM_read_bio_PrivateKey worked fine. However, when in FIPS mode, I got the error -- "error:0608008D:digital envelope routines:EVP_DigestInit:disabled for fips". The error was consistent to what he had told me. Then I asked him whether he used any non-FIPS algorithm during the key generation. The answer was no.

Then I set a breakpoint at PEM_read_bio_PrivateKey and stepped into the function. After several rounds of code tracing, I had the following stacktrace.

0:000> kb
ChildEBP RetAddr Args to Child
0012f0d8 0fb43d0c 0fbc9ab0 0fbbef38 0012f5ec LIBEAY32!EVP_BytesToKey+0x10
0012f5cc 0fb43503 0012f5e8 0091ee40 0012f610 LIBEAY32!PEM_do_header+0xcc
0012f614 0fb46281 0012fa50 0012fa58 0012fa4c LIBEAY32!PEM_bytes_read_bio+0xe3
0012fa60 0041146d 0091c258 00000000 00000000 LIBEAY32!PEM_read_bio_PrivateKey+0x51
0012ff68 00411b18 00000001 003d4c58 003d59d8 openssl_loadkey!wmain+0x7d
0012ffb8 0041195f 0012fff0 7c817077 0115f6f2 openssl_loadkey!__tmainCRTStartup+0x1a8
0012ffc0 7c817077 0115f6f2 0115f77e 7ffdf000 openssl_loadkey!wmainCRTStartup+0xf
0012fff0 00000000 00411082 00000000 78746341 kernel32!BaseProcessStart+0x23

EVP_BytesToKey would return error. It caused PEM_read_bio_PrivateKey to fail to load the key. The second parameter of EVP_BytesToKey is a pointer to EVP_MD.

0:000> ln 0fbbef38
(0fbbef38) LIBEAY32!md5_md | (0fbbef80) LIBEAY32!sha_md
Exact matches:
LIBEAY32!md5_md = struct env_md_st

For some reason, MD5 was passed into the function. Since MD5 is a non-FIPS algorithm, in FIPS mode, it would be disabled in OpenSSL library. Then the question was where MD5 came from. When looking at the OpenSSL source code, I found that when a key file contains a header, PEM_read_bio_PrivateKey would read the header and retrieve the encryption cipher, then it would try to generate the key using EVP_BytesToKey by passing MD5 as its second parameter. Since MD5 is hard-coded as the second parameter, any key file containing a header which specifies an encryption cipher would fail to be loaded in FIPS mode. It is definitely a defect in the OpenSSL code. Until the fix, the workaround is not to generate a key with a header containing an encryption cipher. The OpenSSL versions we used in testing are openssl-0.9.7m and openssl-0.9.8j.