Android 可点击屏幕文本

Question

我很好奇，前几天我看到这个应用程序允许它打开其他应用程序并自动为您设置某些功能。我开始意识到它一定是在使用某种屏幕点击功能，但我似乎找不到任何类似这样的文档。例如，如果我们知道来自其他应用程序的屏幕文本是“就绪”，有没有办法读取该文本并可能执行类似的操作:

protected void processText(String text)
{
  if (text.contains("Ready"))
      // click the ready text
}

Answer 1

我使用 AccessibilityService 完成了此操作.不过，它只能在 API 级别 >= 16 上正常工作。

您需要扩展 AccessibilityService .例如，此类将获取 USSD 响应的文本并关闭对话框。

// ....
public class UssdAccessibilityService extends AccessibilityService {
    public UssdAccessibilityService() {
    }

    @TargetApi(16)
    @Override
    public void onAccessibilityEvent(AccessibilityEvent event) {
        if (!"com.android.phone".equalsIgnoreCase((String)event.getPackageName())){
            // In this example we are only interested in events comming
            // from "com.android.phone" package
            event.recycle();
            return;
        }

        String className = (String)event.getClassName();
        if (className == null || (!className.contains("AlertDialog") && !className.contains("AlertDialog"))){
            // Class is not an USSD dialog
            event.recycle();
            return;
        }

        AccessibilityNodeInfo source = event.getSource();
        if (source == null) {
            // getSource() is annotated @Nullable, so we do this to be
            // safe just in case
            event.recycle();
            return;
        }

        AccessibilityNodeInfo acceptButton = null;
        String ussdText = null;

        int childCount = source.getChildCount();
        for (int i = 0; i < childCount; i++){
            AccessibilityNodeInfo current = source.getChild(i);
            if (current == null)
                continue;

            String currentText = (String)current.getText();
            if (current.isClickable()){
                // In the case of USSD dialogs, there is only one clickable.
                // May be necessary to do more robust search in other scenarios
                acceptButton = current;
                continue;
            }

            ussdText = currentText;

            current.recycle();
        }

        if (ussdText!= null) {
            if (acceptButton != null)
                acceptButton.performAction(AccessibilityNodeInfo.ACTION_CLICK);

        }

        source.recycle();
        event.recycle();
    }

    // ....
}

您必须在 <application> 下的 list 中声明无障碍服务

<service
        android:name=".UssdAccessibilityService"
        android:enabled="true"
        android:label="Read USSD codes and dismiss"
        android:permission="android.permission.BIND_ACCESSIBILITY_SERVICE">
        <intent-filter>
            <action android:name="android.accessibilityservice.AccessibilityService" />
        </intent-filter>

        <meta-data
            android:name="android.accessibilityservice"
            android:resource="@xml/accessibility_service_config" />
</service>

下res/xml创建 accessibility_service_config.xml

<accessibility-service xmlns:android="http://schemas.android.com/apk/res/android"
    android:description="@string/accessibility_service_description"
    android:packageNames="com.android.phone,com.ats.android.activationcodebot"
    android:accessibilityEventTypes="typeWindowStateChanged|typeWindowContentChanged"
    android:accessibilityFlags="flagDefault"
    android:accessibilityFeedbackType="feedbackSpoken"
    android:notificationTimeout="100"
    android:canRetrieveWindowContent="true"
    />

当然，您必须根据自己的需要调整此代码。

最后，您必须在 Android 的设置 > 辅助功能 中手动启用辅助功能服务(或要求用户这样做)。

阅读更多... Developing an Accessibility Service