Tutorial 20: Window Subclassing

In this tutorial, we will learn about window subclassing, what it is and how to use it to your advantage.

Theory:

If you program in Windows for some time, you will find some cases where a window has nearly the attributes you need in your program but not quite. Have you encountered a situation where you want some special kind of edit control that can filter out some unwanted text? The straightforward thing to do is to code your own window. But it's really hard work and time-consuming. Window subclassing to the rescue.
In a nutshell, window subclassing allows you to "take over" the subclassed window. You will have absolute control over it. Let's take an example to make this clearer. Suppose you need a text box that accepts only hex numbers. If you use a simple edit control, you have no say whatsoever when your user types something other than hex numbers into your text box, ie. if the user types "zb+q*" into your text box, you can't do anything with it except rejecting the whole text string. This is unprofessional at least. In essence, you need the ability to examine each character the user typed into the text box right at the moment he typed it.
We will examine how to do that now. When the user types something into a text box, Windows sends WM_CHAR message to the edit control's window procedure. This window procedure resides inside Windows itself so we can't modify it. But we can redirect the message flow to our own window procedure. So that our window procedure will get first shot at any message Windows sends to the edit control. If our window procedure chooses to act on the message, it can do so. But if it doesn't want to handle the message, it can pass it to the original window procedure. This way, our window procedure inserts itself between Windows and the edit control. Look at the flow below: Now we put our attention on how to subclass a window. Note that subclassing is not limited to controls, it can be used with any window.
Let's think about how Windows knows where the edit control's window procedure resides. A guess?......lpfnWndProc member of WNDCLASSEX structure. If we can replace this member with the address of our own window procedure, Windows will send messages to our window proc instead.
We can do that by calling SetWindowLong. hWnd = handle of the window to change the value in the WNDCLASSEX structure
nIndex == value to change. dwNewLong = the replacement value.
So our job is easy: We code a window proc that will handle the messages for the edit control and then call SetWindowLong with GWL_WNDPROC flag, passing along the address of our window proc as the third parameter. If the function succeeds, the return value is the previous value of the specified 32-bit integer, in our case, the address of the original window procedure. We need to store this value for use within our window procedure.
Remember that there will be some messages we don't want to handle, we will pass them to the original window procedure. We can do that by calling CallWindowProc function. lpPrevWndFunc = the address of the original window procedure.
The remaining four parameters are the ones passed to our window procedure. We just pass them along to CallWindowProc.

Code Sample:

____________________________________________________________________________________

[WindowClass:
 style: 3  lpfnWndProc: MainWindowProc   cbClsExtra: 0  cbWndExtra: 0
 hInstance: 0  hIcon: 0  hCursor: 0  hbrBackground: &COLOR_APPWORKSPACE
 lpszMenuName: 0  lpszClassName: ClassName]

[WindowHandle: 0]

[FirstMessage: fAdressee: 0  fMessage: 0  fwParam: 0  flParam: 0 0 0 0 0 0 0]

____________________________________________________________________________________________
____________________________________________________________________________________________

Main:

    call 'Kernel32.GetModuleHandleA' 0      | mov D§hInstance eax

    call 'User32.LoadIconA'  0  &IDI_APPLICATION | mov D§hIcon eax
 
    call 'User32.LoadCursorA' 0  &IDC_ARROW  | mov D§hCursor eax
 
    call 'USER32.RegisterClassA' WindowClass
 
    call 'User32.CreateWindowExA' &WS_EX_CLIENTEDGE ClassName  AppName,
                                 &WS_OVERLAPPEDWINDOW+&WS_VISIBLE,
                                 &CW_USEDEFAULT  &CW_USEDEFAULT  350  200  0,
                                 &NULL  D§hInstance  0
      mov D§WindowHandle eax
 
    call 'User32.ShowWindow'  D§WindowHandle &SW_SHOW
    call 'User32.UpdateWindow'  D§WindowHandle

 
L1: call 'User32.GetMessageA' FirstMessage 0 0 0 | cmp eax 0 | je L9>
        call 'User32.TranslateMessage'  FirstMessage
        call 'User32.DispatchMessageA'  FirstMessage
    jmp L1<

L9: call 'Kernel32.ExitProcess' 0

 _________________________________________________________________________________________

[hwndEdit: 0  OldWndProc: 0
 ClassName:    'SubclassWinClass' 0
 AppName:      'Subclassing Demo' 0
 EditClass:    'EDIT' 0
 EnterMessage: 'You pressed the Enter key in the text box!' 0]
____________________________________________________________________________________________
____________________________________________________________________________________________

Proc MainWindowProc:
    Arguments @Adressee, @Message, @wParam, @lParam
 
    pushad

      mov eax D@Message
      .If eax e &WM_CLOSE
          call 'USER32.DestroyWindow' D§WindowHandle
      .Else_If eax e &WM_DESTROY
          call 'User32.PostQuitMessage' 0
      .Else_If eax e &WM_CREATE
          call 'User32.CreateWindowExA' &WS_EX_CLIENTEDGE EditClass &NULL,
                                     &WS_CHILD+&WS_VISIBLE+&WS_BORDER,
                                       20 20 300 25  D@Adressee  &NULL,
                                       D§hInstance &NULL
                mov D§hwndEdit eax
          call 'User32.SetFocus' eax
_________________________________________

; Subclass it!
_________________________________________

          call 'User32.SetWindowLongA' D§hwndEdit &GWL_WNDPROC EditWndProc
                mov D§OldWndProc eax

      .Else
            popad
            call 'User32.DefWindowProcA' D@Adressee D@Message D@wParam D@lParam
            Exit
 
      .End_If
 
      popad | mov eax &FALSE
EndP

____________________________________________________________________________________________
____________________________________________________________________________________________
 

Proc EditWndProc:
    Arguments @Adressee, @Message, @wParam, @lParam

    pushad
 
    .If D@Message e &WM_CHAR
        mov eax D@wParam
          On al e &VK_BACK, jmp L1>
            On al b '0', jmp L9>
              On al be '9', Jmp L1>
                or al 32
                On al b 'a', jmp L9>
                  On al a 'f', jmp L9>
                    and al 0DF
L1:                 call 'User32.CallWindowProcA' D§OldWndProc  D§hwndEdit  D@Message,
                                                eax  D@lParam
L9:                  popad | Exit
 
    .Else_If D@Message e &WM_KEYDOWN
        mov eax D@wParam
        If al e &VK_RETURN
           call 'User32.MessageBoxA'  D@Adressee  EnterMessage  AppName  &MB_ICONINFORMATION
           call 'User32.SetFocus'  D§hwndEdit
 
        Else
           call 'User32.CallWindowProcA' D§OldWndProc  D§hwndEdit  D@Message,
                                        D@wParam  D@lParam
           popad | Exit
 
        End_If
    .Else
        call 'User32.CallWindowProcA' D§OldWndProc  D§hwndEdit  D@Message,
                                      D@wParam  D@lParam
        popad | Exit
 
    .endif
 
    popad | mov eax &FALSE
Endp
 

Analysis:

After the edit control is created, we subclass it by calling SetWindowLong, replacing the address of the original window procedure with our own window procedure. Note that we store the address of the original window procedure for use with CallWindowProc. Note the EditWndProc is an ordinary window procedure. Within EditWndProc, we filter WM_CHAR messages. If the character is between 0-9 or a-f, we accept it by passing along the message to the original window procedure. If it is a lower case character, we convert it to upper case by adding it with 20h. Note that, if the character is not the one we expect, we discard it. We don't pass it to the original window proc. So when the user types something other than 0-9 or a-f, the character just doesn't appear in the edit control. I want to demonstrate the power of subclassing further by trapping Enter key. EditWndProc checks WM_KEYDOWN message if it's VK_RETURN (the Enter key). If it is, it displays a message box saying "You pressed the Enter key in the text box!". If it's not an Enter key, it passes the message to the original window procedure.
You can use window subclassing to take control over other windows. It's one of the powerful techniques you should have in your arsenal.

[Iczelion's Win32 Assembly Homepage]