Implementing TextBox with on-screen touch keyboard – part 2

In the previous article we discussed our motivation and another possible approach (which I can’t recommend) and saw how to set up the environment. Now it’s time to write some code.

MSAA and UI Automation

To tell Windows to show the on-screen keyboard, we must inform it that the active control is a TextBox control (or, being more general, an editable control with the ability to display a text). This is done by using the Microsoft Active Accessibility (MSAA) or UI Automation.

MSAA is a legacy API that was introduced in Windows 95, and was designed to make Windows applications accessible. The core of MSAA is implementing the IAccessible interface. The important thing is that MSAA support is included in the .net framework for all the built-in controls by default (you can provide your own implementation by overriding the Control.CreateAccessibilityInstance() method), but as we have already seen in the previous article, the on-screen keyboard doesn’t work for built-in .net controls, so we need to use UI Automation instead of MSAA.

UI Automation is newer, more powerful and slightly more complicated API based on the Client-Server model. The server represents the UI element which provides information to a client, usually an accessibility application. The client is represented by Windows itself in our case, but it could be a narrator app or any kind of an application helping people with disabilities. The implementation details will be explained in the following paragraph.

Implementing the TextBoxProvider

As we mentioned above, we need to implement a server-side provider which communicates with a client-side provider. In the simplest scenario (which is our case, luckily) the provider implements the IRawElementProviderSimple interface. This is the essential part, but it is not the only thing we have to do. Our control contains a text value, so we have to implement the ITextProvider control pattern interface and because the text value can be further modified, we should implement IValueProvider as well.

Declaring the interfaces

UI Automation is a COM-based framework, so we need to declare .net interface representing the COM one. If you target .net framework 3.0 or higher, you are lucky, because all the required interfaces are already declared in the System.Windows.Automation.Provider namespace in the UIAutomationProvider.dll assembly. However, our code sample targets .net 2.0, so we need to declare the interfaces by ourselves. To do this, you may use a tool like Tlbimp.exe or to open the UIAutomationProvider.dll in Reflector or ILSpy and copy the required interfaces. Please be aware while extracting COM interfaces from ILSpy, because the functions or properties have to be in a correct order, and both ILSpy or Reflector may list them jumbled.

Putting it all together

As we mentioned above, IRawElementProviderSimple interface is the core one, so we will describe it in more detail.

In the ProviderOptions property we specify that our provider is a server-side provider and that it expects to be called according to COM threading rules. The second property, HostRawElementProvider, should return a provider of the hosting control, so the client applications may create an accessibility tree – we get it using the UiaHostProviderFromHwnd function.

This first function we need to implement is GetPatternProvider. It returns the provider for an UIAutomation control pattern. Our control displays text which can be edited, so we need to return provider for a TextPattern (provider implementing ITextProvider interface) and ValuePattern (IValueProvider interface).  In our case, all these interfaces are for simplicity implemented by a single class, so we return this.

The second function is GetPropertyValue where we should return values for properties describing our control. In our case we specify that the control can obtain focus by the keyboard, it is an editable control and we also provide its name.

Let’s stay for a while at providing the name of a control. In Windows it is a common practice that we should include a Winforms control Label before controls such as list boxes, edit boxes, or other controls that do not have a useful name on the control itself to describe what the control is intended for. When getting the accessibility name of these controls, we should return the Text of the Label preceding the control, because it describes the control better than any of the control’s properties. To do this, we would need to somehow find the Label before our TextBox (e.g. iterate the child controls of the parent control and looking for a control which TabOrder is by 1 less than the TextBox‘ TabOrder). Luckily we don’t have to write the code, guys at Microsoft have already done it for the MSAA.

Our implementation of IRawElementProviderSimple thus looks like following

class TextBoxAutomationProvider : IRawElementProviderSimple, ITextProvider, IValueProvider
{
    [return: MarshalAs(UnmanagedType.IUnknown)]
    public override object GetPatternProvider(int patternId)
    {
        if (patternId == UiaControlPatternIds.UIA_TextEditPatternId || patternId == UiaControlPatternIds.UIA_ValuePatternId || patternId == UiaControlPatternIds.UIA_TextPatternId)
            return this;
 
        return null;
    }
 
    public override object GetPropertyValue(int propertyId)
    {
        if (propertyId == UiaElementPropertyIds.UIA_NamePropertyId)
        {
            if (Owner.AccessibilityObject != null)
                return Owner.AccessibilityObject.Name;
        }
        if (propertyId == UiaElementPropertyIds.UIA_ControlTypePropertyId)
            return UiaControlTypeIds.UIA_EditControlTypeId;
        if (propertyId == UiaElementPropertyIds.UIA_IsKeyboardFocusablePropertyId)
            return true;
 
        return null;
    }
The code sample demonstrates the complete example with implemented all the required interfaces.
 Download example