Some usage issues and optimization suggestions

[k-w840]

Hello, now mobilecli can take screenshots, obtain the view hierarchy, and then provide these to the LLM. The LLM can use this data to operate the simulator through the web driver agent for automated testing. During the use of this, some issues were found. For apps with complex pages, it consumes a lot of context because, although the view hierarchy data is filtered, the filtered content is still substantial, resulting in high context usage. Can this be optimized? I have thought of a possible solution:

1. Obtain specific view information based on name or other details (for this step, should only the basic information be returned, filtering out extra fields like position to reduce context usage?)
2. If there are duplicates, the LLM can determine which one it is based on position or other information.
   Can this issue be optimized? Looking forward to your reply, thank you.

[gmegidish]

Hey @k-w840 , thanks for the question.

We also implemented https://github.com/mobile-next/mobile-mcp and I think you'll find a lot of the LLM trouble solved there.

However, this issue is very interesting and nobody else reported about it before. mobilecli will dump the information as is, what would be a good implementation for your LLM using mobilecli, is to tell it to dump it to a temporary file. then it can use 'jq' or 'python' for further processing.

But it's a legit question for mobile-mcp, and I'll see that we have a solution there as well.

Can you please try telling your LLM to write to disk the result?