Skip to content

Commit 356d7b3

Browse files
authored
Merge pull request #212 from NetherlandsForensicInstitute/204-create-workshop-files
204 Create new tutorial
2 parents d1ddbe1 + 58c67c7 commit 356d7b3

37 files changed

Lines changed: 627 additions & 2749 deletions

RELEASE_NOTES

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,12 +1,13 @@
11
Version 5.1.0
22
-------------
3+
204. Tutorial: Create new tutorial files
34
213. Teleguard: Fix typo in variable name
45
199. WhatsApp: Add view contact profile picture functionality
56
154. Add citation section to README
67
196. Add demo gif and link to video
78
210. Allow uppercase letters in package name check
89
208. Teleguard: support newest version 4.0.9, add send_picture and clear_history
9-
202. Google Chrome: Updater support to 145.0.7632.159
10+
202. Google Chrome: Update support to 145.0.7632.159
1011
206. Google Maps: support newest version 26.10.01
1112
[Dependabot] Bump Pillow version to 12.1.1
1213

puma/apps/android/google_chrome/google_chrome.py

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -62,7 +62,7 @@ def __init__(self, device_udid):
6262
)
6363

6464
@action(current_tab_state)
65-
def visit_url(self, url_string: str, tab_index: int):
65+
def visit_url(self, url_string: str, tab_index: int=None):
6666
"""
6767
Visits a url in an existing tab.
6868
Note that if you supply a tab index that is a new tab, the action will fail. Use go_to_new_tab instead.
@@ -94,7 +94,7 @@ def visit_url_incognito(self, url_string: str):
9494
self._enter_url(url_string, URL_BAR)
9595

9696
@action(current_tab_state)
97-
def bookmark_page(self, tab_index: int):
97+
def bookmark_page(self, tab_index: int=None):
9898
"""
9999
Bookmarks the current page.
100100
:param tab_index: Index of the tab to bookmark.
@@ -117,7 +117,7 @@ def load_first_bookmark(self, folder_name: str):
117117
self.driver.click(FIRST_BOOKMARK)
118118

119119
@action(current_tab_state)
120-
def delete_bookmark(self, tab_index: int):
120+
def delete_bookmark(self, tab_index: int=None):
121121
"""
122122
Delete the current bookmark.
123123
:param tab_index: Index of the tab to delete the bookmark from.

puma/apps/android/google_maps/google_maps.py

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -43,6 +43,9 @@ def _ensure_at_start(self):
4343
if not self.app_open():
4444
self.activate_app()
4545

46+
def get_route_simulator(self) -> RouteSimulator:
47+
return self.route_simulator
48+
4649
@log_action
4750
def search_place(self, search_string: str):
4851
self._ensure_at_start()

puma/apps/android/teleguard/README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -60,7 +60,7 @@ phone = TeleGuard("emulator-5444")
6060
# Send Bob a message
6161
phone.send_message("Hi Bob!", "Bob")
6262
# Alternatively, use:
63-
phone.send_message("Hi Charlie", conversation="Charlie")
63+
phone.send_message(message="Hi Charlie", conversation="Charlie")
6464
# A second message can be sent without supplying the conversation again:
6565
phone.send_message("Hi Charlie, please reply!")
6666
```

puma/apps/android/teleguard/teleguard.py

Lines changed: 57 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,11 @@
1+
from typing import List
2+
3+
from appium.webdriver import WebElement
14
from appium.webdriver.common.appiumby import AppiumBy
25

36
from puma.apps.android.teleguard.xpaths import *
47
from puma.state_graph.action import action
5-
from puma.state_graph.puma_driver import PumaDriver, supported_version
8+
from puma.state_graph.puma_driver import PumaDriver, supported_version, PumaClickException
69
from puma.state_graph.state import ContextualState, SimpleState, compose_clicks
710
from puma.state_graph.state_graph import StateGraph
811

@@ -111,25 +114,72 @@ def add_contact(self, teleguard_id: str):
111114
self.driver.click(CONVERSATION_STATE_INVITE)
112115

113116
@action(conversations_state)
114-
def accept_invite(self):
117+
def invite_received(self, scroll: bool = False, max_swipes: int = 10) -> bool:
118+
"""
119+
Checks whether an invite has been received. By default this method does not scroll to look for an invite.
120+
"""
121+
if scroll:
122+
try:
123+
self.driver.swipe_to_find_element(CONVERSATION_STATE_YOU_HAVE_BEEN_INVITED, max_swipes=max_swipes)
124+
return True
125+
except PumaClickException:
126+
return False
127+
return self.driver.is_present(CONVERSATION_STATE_YOU_HAVE_BEEN_INVITED)
128+
129+
@action(conversations_state)
130+
def accept_invite(self) -> str:
115131
"""
116-
Accepts an invite from another user.
132+
Accepts an invite from another user and returns the name of the accepted invite.
117133
118134
If there are multiple invites, only the topmost invite in the UI will be accepted.
135+
136+
If no invite was sent, an exception is raised.
119137
"""
120138
self.driver.swipe_to_click_element(CONVERSATION_STATE_YOU_HAVE_BEEN_INVITED)
121-
self.driver.click(CONVERSATION_STATE_ACCEPT_INVITE)
139+
name = self.driver.get_element(CONVERSATION_STATE_INVITATION_NAME).get_attribute('content-desc')
140+
self.driver.click(CONVERSATION_STATE_INVITATION_ACCEPT)
141+
return name
142+
143+
def _extract_name(self, conversation_row: WebElement) -> str:
144+
"""
145+
Given a conversation row, extracts the name of the conversation.
146+
For users with an avatar, the first line of the content description is the conversation name.
147+
For users without an avatar, there is an extra first line with just the first letter of the name.
148+
We try to detect this pattern and return the first or second line based on this.
149+
150+
:param conversation_row: The conversation row to extract a name from.
151+
"""
152+
content_description = conversation_row.get_attribute('content-desc')
153+
lines = content_description.split('\n')[:2]
154+
if len(lines[0])>1:
155+
return lines[0]
156+
elif lines[1][0] == lines[0]:
157+
return lines[1]
158+
else:
159+
return lines[0]
160+
161+
@action(conversations_state)
162+
def conversations_with_unread_messages(self) -> List[str]:
163+
"""
164+
Returns a list of names of conversations that have unread messages.
165+
"""
166+
try:
167+
elements = self.driver.get_elements(CONVERSATION_STATE_UNREAD_MESSAGES)
168+
names = [self._extract_name(e) for e in elements]
169+
return names
170+
except PumaClickException:
171+
return []
122172

123173
@action(chat_state)
124-
def send_message(self, msg: str, conversation: str = None):
174+
def send_message(self, message: str, conversation: str = None):
125175
"""
126176
Sends a message in the current chat conversation.
127177
128-
:param msg: The message to send.
178+
:param message: The message to send.
129179
:param conversation: The name of the conversation to send the message in.
130180
"""
131181
self.driver.click(CHAT_STATE_TEXT_FIELD)
132-
self.driver.send_keys(CHAT_STATE_TEXT_FIELD, msg)
182+
self.driver.send_keys(CHAT_STATE_TEXT_FIELD, message)
133183
self.driver.click(CHAT_STATE_SEND_BUTTON)
134184

135185
@action(chat_options_state, end_state=chat_state)

puma/apps/android/teleguard/xpaths.py

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,19 +1,22 @@
11
CONVERSATION_STATE_TELEGUARD_HEADER = '//android.view.View[@content-desc="TeleGuard"]'
2-
CONVERSATION_STATE_HAMBURGER_MENU = '//android.widget.FrameLayout[@resource-id="android:id/content"]/android.widget.FrameLayout/android.widget.FrameLayout/android.view.View/android.view.View/android.view.View/android.view.View/android.view.View[1]/android.view.View[2]/android.view.View[3]'
2+
CONVERSATION_STATE_HAMBURGER_MENU = '//android.view.View[@content-desc="TeleGuard"]/following-sibling::*[last()]'
33
CONVERSATION_STATE_SETTINGS_BUTTON = '//android.widget.ImageView[@content-desc="Settings"]'
44
CONVERSATION_STATE_ABOUT_BUTTON = '//android.widget.ImageView[@content-desc="About"]'
55
CONVERSATION_STATE_TELEGUARD_STATUS = '//android.view.View[@content-desc="Online"]|//android.view.View[contains(@content-desc, "Connection to server")]'
66
CONVERSATION_STATE_ADD_CONTACT = '//android.widget.ImageView[@content-desc="Add contact"]'
77
CONVERSATION_STATE_EDIT_TEXT = '//android.widget.EditText'
88
CONVERSATION_STATE_INVITE = '//android.widget.Button[@content-desc="INVITE"]'
99
CONVERSATION_STATE_YOU_HAVE_BEEN_INVITED = '//android.view.View[contains(@content-desc, "You have been invited")]'
10-
CONVERSATION_STATE_ACCEPT_INVITE = '//android.widget.Button[@content-desc="ACCEPT INVITE"]'
10+
CONVERSATION_STATE_INVITATION_ID = '//android.view.View[starts-with(lower-case(@content-desc), "teleguard id:")]'
11+
CONVERSATION_STATE_INVITATION_NAME = '//android.view.View[starts-with(lower-case(@content-desc), "teleguard id:")]/preceding-sibling::*[last()]'
12+
CONVERSATION_STATE_INVITATION_ACCEPT = '//android.widget.Button[@content-desc="ACCEPT INVITE"]'
13+
CONVERSATION_STATE_UNREAD_MESSAGES = '//*[@content-desc and matches(@content-desc, "\\n\\d$")]'
1114

1215
CHAT_STATE_CONVERSATION_NAME = ('//android.widget.FrameLayout[@resource-id="android:id/content"]/android.widget.FrameLayout/android.widget.FrameLayout/android.view.View/android.view.View/android.view.View/android.view.View/android.view.View[1]/android.view.View[2]/android.widget.ImageView[2][@content-desc]|'
1316
'//android.widget.FrameLayout[@resource-id="android:id/content"]/android.widget.FrameLayout/android.widget.FrameLayout/android.view.View/android.view.View/android.view.View/android.view.View/android.view.View[1]/android.view.View[2]/android.view.View[1][@content-desc]')
1417
CHAT_STATE_TEXT_FIELD = '//android.widget.EditText[@hint="Send a message"]'
1518
CHAT_STATE_SEND_MEDIA_BUTTON = '//android.widget.FrameLayout[@resource-id="android:id/content"]/android.widget.FrameLayout/android.widget.FrameLayout/android.view.View/android.view.View/android.view.View/android.view.View/android.widget.ImageView[2]'
16-
CHAT_STATE_SEND_BUTTON = '//android.widget.FrameLayout[@resource-id="android:id/content"]/android.widget.FrameLayout/android.widget.FrameLayout/android.view.View/android.view.View/android.view.View/android.view.View/android.widget.ImageView[3]'
19+
CHAT_STATE_SEND_BUTTON = '//android.widget.FrameLayout[@resource-id="android:id/content"]/android.widget.FrameLayout/android.widget.FrameLayout/android.view.View/android.view.View/android.view.View/android.view.View/android.widget.ImageView[last()]'
1720
CHAT_STATE_MICROPHONE_BUTTON = '//android.widget.FrameLayout[@resource-id="android:id/content"]/android.widget.FrameLayout/android.widget.FrameLayout/android.view.View/android.view.View/android.view.View/android.view.View/android.widget.ImageView[4]'
1821
CHAT_STATE_THREE_DOTS = '//android.widget.FrameLayout[@resource-id="android:id/content"]/android.widget.FrameLayout/android.widget.FrameLayout/android.view.View/android.view.View/android.view.View/android.view.View/android.view.View[1]/android.view.View[2]/android.widget.ImageView[last()]'
1922

Lines changed: 169 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,169 @@
1+
{
2+
"cells": [
3+
{
4+
"metadata": {},
5+
"cell_type": "markdown",
6+
"source": [
7+
"# Welcome to the Puma Tutorial\n",
8+
"\n",
9+
"Puma — *Programmable Utility for Mobile Automation* — is a Python library that lets you control Android apps through simple, high‑level actions.\n",
10+
"It supports a variety of popular applications such as **Telegram**, **TeleGuard**, **WhatsApp**, **Google Maps**, and more (see the README.md)\n",
11+
"\n",
12+
"In this tutorial you will learn how to use Puma to interact with the a few of these apps: Sending messages with TeleGuard, navigating using Google Maps and conduct web searches using Google Chrome. Finally, you will build a full scenario, using multiple applications simultaneously to mimic real user behavior.\n",
13+
"\n",
14+
"The workshop is primarily aimed at _using_ Puma. If you’re interested in extending Puma itself by extending or improving features, or by adding support for new apps, check out bonus **Exercise 5**. Please refer to the [CONTRIBUTING.md](../../CONTRIBUTING.md) for guidelines, or ask one of us for help."
15+
],
16+
"id": "64ab1c9aa8a8e397"
17+
},
18+
{
19+
"metadata": {},
20+
"cell_type": "markdown",
21+
"source": [
22+
"# Exercise 1: Send us a message with TeleGuard\n",
23+
"In this exercise, you will learn to use Puma through a few simple actions.\n",
24+
"We will add a contact and send a message using the chat app TeleGuard. TeleGuard does not require personal information such as a phone number, making it the perfect candidate for a tutorial!\n",
25+
"\n",
26+
"> ❔ New to jupyter notebooks? You need to run all executable cells sequentially in order to initialize the variables. Weird behaviour and/or crashes? Restart the kernel and rerun the code blocks, top of screen: Kernel -> Restart Kernel...\n",
27+
"\n",
28+
"Your TeleGuard ID is displayed in a \"chat\" with TeleGuard, and can be shared with other users to message one another.\n",
29+
"\n",
30+
"Fill in your device udid and initialize TeleGuard.\n",
31+
"\n",
32+
"The device udid can be found by running\n",
33+
"```bash\n",
34+
"adb devices\n",
35+
"```\n",
36+
"from your terminal. If you use an emulator, the udid has already been filled in, as emulators have default udids. For physical devices, the udid is unique for the device and always needs to be checked.\n"
37+
],
38+
"id": "465be2cf0878a0b4"
39+
},
40+
{
41+
"metadata": {},
42+
"cell_type": "code",
43+
"source": [
44+
"from puma.apps.android.teleguard.teleguard import TeleGuard\n",
45+
"\n",
46+
"# This initializes an object through which we can execute TeleGuard actions. Initialization can take a while. The device udid can be found by running `adb devices` in your terminal\n",
47+
"teleguard = TeleGuard(device_udid=\"emulator-5554\")"
48+
],
49+
"id": "4594520f1f70d68",
50+
"outputs": [],
51+
"execution_count": null
52+
},
53+
{
54+
"metadata": {},
55+
"cell_type": "markdown",
56+
"source": [
57+
"You probably got an error stating Appium is not running. To establish a connection to the Appium driver, we need to start Appium first.\n",
58+
"In a terminal, run the command\n",
59+
"```bash\n",
60+
"appium\n",
61+
"```\n",
62+
"Leave this terminal open and do not stop the Appium process, then rerun the above code."
63+
],
64+
"id": "5494fac7991d6161"
65+
},
66+
{
67+
"metadata": {},
68+
"cell_type": "markdown",
69+
"source": [
70+
"### Appium\n",
71+
"Appium is a framework for testing applications, based on Selenium. In Puma, we use Appium to execute actions on the device.\n",
72+
"In this context, we define functions in Python that exist of one or more `Appium actions`. An Appium action is an action on the phone, for instance tapping on an element. In our Python code, we define functions executing one or more\n",
73+
"Appium actions, for instance `send_message()` where the message box is tapped, text is added to the box and the send button is clicked."
74+
],
75+
"id": "800446abe5fe54c2"
76+
},
77+
{
78+
"metadata": {},
79+
"cell_type": "markdown",
80+
"source": [
81+
"## 1. Add a contact\n",
82+
"To send messages, you need to have someone to talk to. This can be done by adding a contact.\n",
83+
"If you are attending a workshop, the TeleGuard ID of the workshop host is visible on screen, add them! If you are doing the tutorial by yourself, create an additional TeleGuard ID on another device, or ask someone else to do it. Enter the ID in the cell below and execute the code:"
84+
],
85+
"id": "733fe69884e9c1f8"
86+
},
87+
{
88+
"metadata": {},
89+
"cell_type": "code",
90+
"source": [
91+
"contact_id = '' # TeleGuard ID of the person you want to add\n",
92+
"teleguard.add_contact(contact_id)"
93+
],
94+
"id": "32e97c439cef4e75",
95+
"outputs": [],
96+
"execution_count": null
97+
},
98+
{
99+
"metadata": {},
100+
"cell_type": "markdown",
101+
"source": [
102+
"## 2. Send a message\n",
103+
"Send a message to the contact you just added. Enter their username and the message below."
104+
],
105+
"id": "da3e4a385a663d3f"
106+
},
107+
{
108+
"metadata": {},
109+
"cell_type": "code",
110+
"source": [
111+
"recipient_username = '' # Username or group name you will send a message to\n",
112+
"message = '' # Message to send\n",
113+
"\n",
114+
"teleguard.send_message(message=message, conversation=recipient_username)"
115+
],
116+
"id": "9b0e8b4d27d7e2e0",
117+
"outputs": [],
118+
"execution_count": null
119+
},
120+
{
121+
"metadata": {},
122+
"cell_type": "markdown",
123+
"source": [
124+
"## 3. Send a picture with TeleGuard\n",
125+
"Apart from sending messages, Puma also supports sending pictures. Each application has a `README.md`, listing all functionality of an application and the options. Sending a picture can be done in several ways, e.g. sending a picture from the gallery or taking a new picture with the camera. Lastly, an optional caption can be added.\n",
126+
"_Note_:\n",
127+
"`picture_id` is the index of the picture shown in the Android media picker (the first picture has index 1).\n",
128+
"If `picture_id` is omitted, TeleGuard will open the camera, take a picture, and send it.\n",
129+
"`caption` is optional – you can omit it to send the image without text."
130+
],
131+
"id": "6ff03781ea8d1e41"
132+
},
133+
{
134+
"metadata": {},
135+
"cell_type": "code",
136+
"source": [
137+
"teleguard.send_picture(conversation=\"\",\n",
138+
" #optional parameters, you can also remove them\n",
139+
" picture_id=1, caption=\"\")"
140+
],
141+
"id": "a7e4d7bbfb65737a",
142+
"outputs": [],
143+
"execution_count": null
144+
},
145+
{
146+
"metadata": {},
147+
"cell_type": "markdown",
148+
"source": [
149+
"#### What to observe\n",
150+
"After running the cell, the TeleGuard app on the device should open the media picker (or camera) and automatically send the selected image.\n",
151+
"The caption (if provided) appears in the chat bubble alongside the picture.\n",
152+
"If you omitted `picture_id`, a new photo will be captured and sent.\n",
153+
"Feel free to experiment with different picture_id values or captions to see how the app behaves."
154+
],
155+
"id": "af07003adf5f9961"
156+
},
157+
{
158+
"metadata": {},
159+
"cell_type": "code",
160+
"source": "",
161+
"id": "65597861fba26725",
162+
"outputs": [],
163+
"execution_count": null
164+
}
165+
],
166+
"metadata": {},
167+
"nbformat": 4,
168+
"nbformat_minor": 5
169+
}

0 commit comments

Comments
 (0)