android - 如何使用 "Taking Action for Users"的辅助功能服务？

Question

背景

返回few years ago ，我询问了 TeamViewer 如何允许用户在不与设备进行正常交互的情况下控制设备。有人告诉我，这是制造商专门为该应用程序允许的特殊“后门”，并且只有对其他应用程序使用 root 权限才有可能。

看到像 "Airplane Mode Shortcut" 这样的应用程序允许切换飞行模式，通过自动导航到它的屏幕并切换开关，它让我意识到这种情况已经改变。

问题

据说在the docs :

Starting with Android 4.0 (API Level 14), accessibility services can act on behalf of users, including changing the input focus and selecting (activating) user interface elements. In Android 4.1 (API Level 16) the range of actions has been expanded to include scrolling lists and interacting with text fields. Accessibility services can also take global actions, such as navigating to the Home screen, pressing the Back button, opening the notifications screen and recent applications list. Android 4.1 also includes a new type of focus, Accessibilty Focus, which makes all visible elements selectable by an accessibility service.

These new capabilities make it possible for developers of accessibility services to create alternative navigation modes such as gesture navigation, and give users with disabilities improved control of their Android devices.

但是没有更多关于如何使用它的信息。只有我找到的示例位于底部，但这些示例非常古老并且是 apiDemos 包的一部分。

问题

如何制作可以查询、聚焦、点击、输入文本以及执行其他 UI 相关操作的服务？

Answer 1

通过实现 AccessibilityService ( https://developer.android.com/training/accessibility/service.html )，您可以访问该功能。

您可以检查用户最后交互的元素或对其执行操作，或者检查当前处于 Activity 状态的整个应用程序。

通过实现onAccessibilityEvent(AccessibilityEvent event)拦截用户事件，在这里你可以用event.getSource()获取虚拟 View (代表原始 View )，然后用getClassName() 或 getText() 或您在文档中找到的任何内容。

通过调用 getRootInActiveWindow() 检查整个应用程序并使用 getRootInActiveWindow().getChild(index) 遍历虚拟 View 树。

getRootInActiveWindow() 和event.getSource() 都返回AccessibilityNodeInfo，您可以在其上调用performAction(action)并执行类似单击、设置文本等操作。

示例:Play 商店

打开 Play 商店应用后，搜索“facebook”应用并在 Play 商店上打开它的页面。

    @Override
    public void onAccessibilityEvent(final AccessibilityEvent event) {

        AccessibilityNodeInfo rootInActiveWindow = getRootInActiveWindow();
        //Inspect app elements if ready
        if (rootInActiveWindow != null) {
            //Search bar is covered with textview which need to be clicked
            List<AccessibilityNodeInfo> searchBarIdle = rootInActiveWindow.findAccessibilityNodeInfosByViewId("com.android.vending:id/search_box_idle_text");
            if (searchBarIdle.size() > 0) {
                AccessibilityNodeInfo searchBar = searchBarIdle.get(0);
                searchBar.performAction(AccessibilityNodeInfo.ACTION_CLICK);
            }
            //Check is search bar is visible
            List<AccessibilityNodeInfo> searchBars = rootInActiveWindow.findAccessibilityNodeInfosByViewId("com.android.vending:id/search_box_text_input");
            if (searchBars.size() > 0) {
                AccessibilityNodeInfo searchBar = searchBars.get(0);
                //Check is searchbar have the required text, if not set the text
                if (searchBar.getText() == null || !searchBar.getText().toString().equalsIgnoreCase("facebook")) {
                    Bundle args = new Bundle();
                    args.putString(AccessibilityNodeInfo.ACTION_ARGUMENT_SET_TEXT_CHARSEQUENCE, "facebook");
                    searchBar.performAction(AccessibilityNodeInfo.ACTION_SET_TEXT, args);
                } else {
                    //There is no way to press Enter to perform search, so find corresponding suggestion and click
                    List<AccessibilityNodeInfo> searchSuggestions = rootInActiveWindow.findAccessibilityNodeInfosByViewId("com.android.vending:id/suggest_text");
                    for (AccessibilityNodeInfo suggestion : searchSuggestions) {
                        if(suggestion.getText().toString().equals("Facebook")) {
                            //We found textview, but its not clickable, so we should perform the click on the parent
                            AccessibilityNodeInfo clickableParent = suggestion.getParent();
                            clickableParent.performAction(AccessibilityNodeInfo.ACTION_CLICK);
                        }
                    }
                }
            }


        }
   }

---

编辑:完整代码如下:

MyAccessibilityService

public class MyAccessibilityService extends AccessibilityService {

    @Override
    public void onCreate() {
        super.onCreate();
        Log.d("MyAccessibilityService", "onCreate");
    }

    @Override
    public void onAccessibilityEvent(final AccessibilityEvent event) {
        Log.d("MyAccessibilityService", "onAccessibilityEvent");
        AccessibilityNodeInfo rootInActiveWindow = getRootInActiveWindow();
        //Inspect app elements if ready
        if (rootInActiveWindow != null) {
            //Search bar is covered with textview which need to be clicked
            List<AccessibilityNodeInfo> searchBarIdle = rootInActiveWindow.findAccessibilityNodeInfosByViewId("com.android.vending:id/search_box_idle_text");
            if (searchBarIdle.size() > 0) {
                AccessibilityNodeInfo searchBar = searchBarIdle.get(0);
                searchBar.performAction(AccessibilityNodeInfo.ACTION_CLICK);
            }
            //Check is search bar is visible
            List<AccessibilityNodeInfo> searchBars = rootInActiveWindow.findAccessibilityNodeInfosByViewId("com.android.vending:id/search_box_text_input");
            if (searchBars.size() > 0) {
                AccessibilityNodeInfo searchBar = searchBars.get(0);
                //Check is searchbar have the required text, if not set the text
                if (searchBar.getText() == null || !searchBar.getText().toString().equalsIgnoreCase("facebook")) {
                    Bundle args = new Bundle();
                    args.putString(AccessibilityNodeInfo.ACTION_ARGUMENT_SET_TEXT_CHARSEQUENCE, "facebook");
                    searchBar.performAction(AccessibilityNodeInfo.ACTION_SET_TEXT, args);
                } else {
                    //There is no way to press Enter to perform search, so find corresponding suggestion and click
                    List<AccessibilityNodeInfo> searchSuggestions = rootInActiveWindow.findAccessibilityNodeInfosByViewId("com.android.vending:id/suggest_text");
                    for (AccessibilityNodeInfo suggestion : searchSuggestions) {
                        if (suggestion.getText().toString().equals("Facebook")) {
                            //We found textview, but its not clickable, so we should perform the click on the parent
                            AccessibilityNodeInfo clickableParent = suggestion.getParent();
                            clickableParent.performAction(AccessibilityNodeInfo.ACTION_CLICK);
                        }
                    }
                }
            }
        }
    }

    @Override
    public void onInterrupt() {
    }
}

AndroidManifest.xml

<manifest xmlns:android="http://schemas.android.com/apk/res/android"
          package="com.example.findfacebookapp">

    <application
        android:allowBackup="true"
        android:icon="@mipmap/ic_launcher"
        android:label="@string/app_name"
        android:supportsRtl="true"
        android:theme="@style/AppTheme">
        <activity android:name=".MainActivity">
            <intent-filter>
                <action android:name="android.intent.action.MAIN"/>

                <category android:name="android.intent.category.LAUNCHER"/>
            </intent-filter>
        </activity>
        <service
            android:name=".MyAccessibilityService"
            android:label="@string/accessibility_service_label"
            android:permission="android.permission.BIND_ACCESSIBILITY_SERVICE">
            <intent-filter>
                <action android:name="android.accessibilityservice.AccessibilityService"/>
            </intent-filter>

            <meta-data
                android:name="android.accessibilityservice"
                android:resource="@xml/accessibility_service_config"/>
        </service>
    </application>

</manifest>

res/xml/accessibility_service_config.xml

<?xml version="1.0" encoding="utf-8"?>
<accessibility-service
    xmlns:android="http://schemas.android.com/apk/res/android"
    android:accessibilityEventTypes="typeAllMask"
    android:accessibilityFeedbackType="feedbackAllMask"
    android:accessibilityFlags="flagDefault"
    android:canRequestEnhancedWebAccessibility="true"
    android:canRetrieveWindowContent="true"
    android:description="@string/app_name"
    android:notificationTimeout="100"/>

主要 Activity

public class MainActivity extends AppCompatActivity {

    public void onEnableAccClick(View view) {
        startActivityForResult(new Intent(Settings.ACTION_ACCESSIBILITY_SETTINGS), 1);
    }

}