Demystifying Subclassing

 

By Fritz Onion
Published in C++ Report, July 1997 issue.

Subclassing is a standard technique in Windows programming for customizing the behavior of a window. However, with the advent of MFC to wrap subclassing into virtual function overriding, it is quickly becoming an obscure technique unknown to a large population of Windows programmers. This is unfortunate, because there are many cases where using subclassing is the most elegant and clean way to solve a problem. In addition, to fully understand how MFC encapsulates windows, it is critical to understand the mechanism of subclassing. In this column, we will demystify the art of Windows subclassing, and explain how it is used in the context of MFC. To do this, it will first be necessary to understand what vanilla Windows subclassing is and how it works. With that knowledge under our belts, we can then look behind the scenes in MFC and understand how it masks most MFC users from having to ever deal with subclassing. Finally, we will also look into MFC抯 support for performing a hybrid of Windows/C++ subclassing. We will focus mainly on subclassing child window controls embedded in dialogs, but all discussion will also be relevant to generic window subclassing.

WINDOWS SUBCLASSING

The procedure of subclassing a window is possible because of the way that Windows routes messages. Windows messages are passed to the window procedure associated with the window to which the message was sent. For example, when the user clicks the left mouse button in a window, the window procedure associated with that window receives the WM_LBUTTONDOWN message. Figure 1 shows the relationship between an edit control window in a dialog and its window procedure (here called EditWndProc).

Figure 1

Traditional Windows subclassing is performed by substituting a different window procedure for a particular window, and then calling the old window procedure from within the new one, after adding supplemental functionality. One of the most common places for subclassing to occur is within a dialog box. In order to customize the behavior of a control in a dialog, one provides a new window procedure for that control, intercepting the subset of messages that are to change, and passing the remaining messages along to the old window procedure. This procedure is analogous in many ways to overriding virtual functions in a C++ class, as we shall see in MFC抯 version of subclassing.

Suppose we wanted to change the behavior of an edit control in a dialog box, perhaps to respond to a right mouse button click. Edit controls do not do anything in response to right mouse clicks by default, so we would have to subclass the edit control to intercept the WM_RBUTTONDOWN message. The first step is to define a new window procedure to do this:

	WNDPROC gOldEditWndProc;	// Global variable to store old
								// window procedure

	LRESULT CALLBACK RtClickWndProc(HWND hWnd, UINT iMsg, WPARAM wParam, LPARAM lParam)
	{
		if (iMsg == WM_RBUTTONDOWN)
			// Perform special handling here to respond to 
			// right mouse click
			return 0;
		else
			return CallWindowProc(gOldEditWndProc, hWnd, iMsg, wParam, lParam);
	}

Notice that if the message is anything but WM_RBUTTONDOWN, we call the function CallWindowProc with gOldEditWndProc and the other parameters passed into our window procedure. This global variable will store the old window procedure associated with the edit control when we do the actual subclassing. The effect of calling CallWindowProc is simply to invoke the function pointer passed as the first parameter, so this statement calls the original window procedure. This is how we augment the functionality of the original window procedure, by explicitly calling it in our new window procedure once we have handled all the messages that we want to handle differently.

The actual subclassing occurs when the dialog box first comes up, typically in response to the WM_INITDIALOG message. The code to perform the subclassing of the edit control whose identifier is IDC_MYEDIT and whose parent dialog window handle is DlgWnd would be:

	HWND ctrl = GetDlgItem(DlgWnd, IDC_MYEDIT);
	GOldEditWndProc = (WNDPROC)SetWindowLong(ctrl, GWL_WNDPROC, (LONG) RtClickWndProc);

Now, when the edit control in the dialog box receives a right mouse button click, it will be processed by our custom window procedure. Any other messages will be passed along to the base window procedure. The way messages are routed to the edit control in our dialog box once the subclassing has been performed is shown in Figure 2.

Figure 2

Windows subclassing is similar in concept to virtual function overriding where messages correspond to virtual functions. A window that has been subclassed will pass all of its messages to the new window procedure. That new window procedure will handle whatever messages it is 'overriding' and will pass the remaining messages along to the previously attached window procedure. In other words, a message is dynamically bound to the window procedure that is farthest down the chain of subclassed window procedures. Windows subclassing is different from virtual functions in several ways. First of all, it is syntactically clumsy, especially when compared to virtual function overriding where the binding is implicit when a simple keyword is added to a function prototype. And secondly, the process of subclassing itself is dynamic. That is, windows are subclassed at runtime not at compile time. This second difference means that subclassing is more generic and flexible than virtual functions (but of course, it only works for windows and messages).

MFC SUBCLASSING

MFC goes to great lengths to merge the two concepts of windows subclassing and C++ subclassing. A common base class for all windows (CWnd) implements a virtual function for every possible message. To create a custom window, you simply derive a new class from CWnd, and override the subset of message handlers you want to customize.

Notice that there was no mention of a window procedure in MFC's subclassing. This is because MFC hides the window procedure and relies entirely on C++ subclassing to provide the capability to create custom windows. How does MFC do this?

The first step is to remove the window procedure from the picture. To do this, MFC defines a single global window procedure called AfxWndProc(). It is the task of this window procedure to delegate messages to the appropriate CWnd-derived objects. Whenever a CWnd-derived object allocates its window handle (by calling Create()) it associates AfxWndProc() with that window as its window procedure. This means that every MFC window shares the same window procedure, effectively eliminating the window procedure from the picture.

The next task is to route any message coming into this global window procedure to the appropriate CWnd-derived object. Recall that messages are always sent to a specific window, and the handle to that window is passed into a window procedure as a parameter:

LRESULT CALLBACK AfxWndProc(HWND hWnd, ...);

Given this handle, there needs to be a way to retrieve the class associated with that handle. This implies that there is a one-to-one mapping between window handles and CWnd-derived objects, and this mapping is strictly maintained by MFC. If you try to associate a CWnd-derived object with a window handle that is already associated with another CWnd-derived object, it will generate an assertion failure. This mapping is stored in a global hash table called the perm map. Any time a window handle is created by calling a CWnd-derived object's Create() function, an entry is made into the perm map associating the created window handle with the pointer to the CWnd object. Using this perm map, an HWND can be mapped to a CWnd-derived object by simply looking up the association in the perm map, and our task of removing the window procedure is nearly complete.

The final step is to actually invoke the appropriate virtual function message handler of the CWnd-derived object. This is done by calling another CWnd member function called WindowProc().

	LRESULT CWnd::WindowProc(UINT message, WPARAM wParam, LPARAM lParam)
	{
		LRESULT lResult = 0;
		if (!OnWndMsg(message, wParam, lParam, &lResult))
			lResult = DefWindowProc(message, wParam, lParam);
		return lResult;
	}

The WindowProc() function does two things. First, it tries to handle the message by calling CWnd::OnWndMsg(). If that function fails to handle the message, it calls CWnd::DefWindowProc(). The OnWndMsg() function basically does a giant switch statement on the incoming message ID, and invokes the appropriate virtual message handler. The DefWindowProc() function typically just calls the SDK version of DefWindowProc, a default window procedure supplied by the system to perform standard handling for all windows messages. Figure 3 shows the path a message takes when an MFC window class is the recipient.

Figure 3

MFC/WINDOWS HYBRID SUBCLASSING

MFC抯 elimination of window procedures and use of virtual functions for message overriding makes the process of subclassing a very natural and easy one for C++ programmers. However, in providing this nice interface, it eliminates one of the most powerful features of subclassing ?its dynamic nature. With plain Windows subclassing it is possible to dynamically alter the behavior of any window by simply replacing its window procedure. There is one case in particular where the dynamic nature of subclassing is essential ?control subclassing in dialogs. Because controls are created automatically in dialogs from a dialog resource, subclassing is the simplest way to change the behavior of a control.

In order to retain this ability to subclass dialog controls, MFC provides a hybrid technique between standard Windows subclassing and MFC subclassing in addition to its static subclassing. Suppose, for example, we wanted to subclass an edit control to prevent the user from entering the character 慳.?We want the edit control to behave exactly like standard edit controls do for all cases, except when the character 慳?is typed, in which case nothing should happen. To do this, we would derive a new class from CEdit:

	class CNoAEdit : public CEdit
	{
	protected:
		void OnChar(UINT nChar, UINT nRepCnt, UINT nFlags);
	};

Where the implementation of OnChar() would be:

	void CNoAEdit::OnChar(UINT nChar, UINT nRepCnt, UINT nFlags) 
	{
		if (nChar != 'a')	
			CEdit::OnChar(nChar, nRepCnt, nFlags);
	}

Now to use our new class in a dialog box (wrapped by a class called CMyDlg) we would add a new data member of type CNoAEdit (called m_NoAEdit). We would add a handler for the WM_INITIDIALOG message to our dialog class and subclass the control. Once the dialog box is invoked, in the WM_INITDIALOG message handler, we would subclass it by calling the CWnd function SubclassDlgItem().

	BOOL CMyDlg::OnInitDialog()	// Handler for WM_INITDIALOG
	{
		CDialog::OnInitDialog();
		m_NoAEdit.SubclassDlgItem(IDC_EDIT, this);
		return true;
	}

Once the dialog box is invoked, all WM_CHAR messages sent to our edit control will now be routed to our m_NoAEdit variable first. If the character coming in is a lower case 'a,' we will swallow the message by not calling the base class version of OnChar(). If it is any other character, we will let it pass through to the base class and resume normal behavior associated with edit controls.

The key to this procedure is that the MFC class needs to place itself dynamically in the chain of window procedure calls. To do this, we have to subclass the control in the standard Windows way, and then let AfxWndProc do its work, simply acting as the window procedure farthest down in the hierarchy. In order to subclass a non-MFC window, we will need to keep a pointer to the old window procedure to chain to if the incoming message is not handled. This is exactly what MFC does in its CWnd class.

	class CWnd : public CCmdTarget
	{
		...
	protected:
		WNDPROC m_pfnSuper; // for subclassing of controls
	};

When SubclassDlgItem is called, it uses Windows subclassing to set the window procedure of the edit control to be AfxWndProc. This will cause the messages to be first routed to our CNoAEdit class. It also saves the result of SetWindowLong() in the m_pfnSuper member variable of CWnd. This will act as the link to chain the extra messages back to the original window procedure associated with the edit control when it was first created.

	BOOL CWnd::SubclassDlgItem(UINT nID, CWnd* pParent)
	{
		HWND hWndControl = ::GetDlgItem(pParent->m_hWnd, nID);
		if (hWndControl != NULL)
			return SubclassWindow(hWndControl);

		return FALSE;   // control not found
	}

	BOOL CWnd::SubclassWindow(HWND hWnd)
	{
		// now hook into the AFX WndProc
		m_pfnSuper = (WNDPROC)::SetWindowLong(hWnd, GWL_WNDPROC, (DWORD)AfxWndProc);

		return TRUE;
	}

There is now only one link missing in this subclassing, and that is, how does the original window procedure of the edit control actually get invoked? If you recall the behavior of CWnd::WindowProc(), if its call to OnWndMsg() failed, it proceeded to call CWnd::DefWindowProc(). It is in CWnd's implementation of DefWindowProc() that the missing subclassing link is located.

	LRESULT CWnd::DefWindowProc(UINT nMsg, WPARAM wParam, LPARAM lParam)
	{
		if (m_pfnSuper != NULL)
			return ::CallWindowProc(m_pfnSuper, m_hWnd, nMsg, wParam, lParam);
		else
			return ::DefWindowProc(m_hWnd, nMsg, wParam, lParam);
	}

If DefWindowProc is invoked, it means that there were no virtual function overrides for that message, and the default behavior should occur. If this is a subclassed control, the default behavior will be to invoke the window procedure of the control before it was subclassed. If it is not a subclassed control, it simply calls the SDK version of DefWindowProc which defines standard behavior for all windows messages.

This technique allows the MFC developer to subclass dialog controls without having to resort to plain Windows subclassing. It has the limitation, however, of being used only one level deep. For example, you could not subclass a control with our CNoAEdit class, and then subclass it again using with another CEdit-derived class (perhaps CNoBEdit). MFC subclassing must always occur at the lowest level.

Conclusions

MFC masks its users from the details of window procedures and Windows subclassing by using C++ classes to provide similar behavior. By adding the capability to dynamically subclass controls in dialogs, a large percentage of the techniques which used Windows subclassing before can now be done in MFC without any knowledge of window procedures or Windows subclassing. The major limitation of MFC抯 implementation is that it is not possible to dynamically subclass a window more than one level deep. If one understands how Windows subclassing works, and how MFC encapsulates it in its CWnd class, it is possible to work around this limitation by using standard Windows subclassing in addition to MFC抯 classes.