Network Related Faults (NRFs) refer to unexpected app behaviors under abnormal network conditions (only faults explicitly confirmed by the developers are called bugs in our context).
Nowadays, Android apps are becoming much more network-dependent, with increasingly network-related faults being observed, severely undermining user experience. Given that such faults are scattered in modern apps and require complex network patterns to trigger, their detection is challenging.
To date, we still lack a general and in-depth understanding of such faults. To fill this gap, we conduct the first empirical study on 154 real-world network-related bugs filtered from 42 diverse representative Android apps to investigate their characteristics, influences, triggering patterns, and origins.
While existing Android testing approaches struggle with detecting such faults due to a lack of effective network events and efficient injection, we propose NATE on the basis of our study, a network-aware testing enhancer to empower existing general Android testing approaches to detect network-related faults. Based on curiosity-driven reinforcement learning, NATE conducts network-aware guidance to inject effective network events, enabling testing approaches to explore network-related extra app functionalities and detect network-related faults.
Please refer to our ASE 2025 paper "NATE: A Network-Aware Testing Enhancer for Network-Related Fault Detection in Android Apps" for more details. The paper will be published soon.
Here is the full list of the 154 identified real-world network-related bugs in our empirical study.
NATE is a network-aware testing enhancer guided by RL-based curiosity-driven exploration, to empower existing general Android testing approaches to detect network-related faults.
- Python: Tested on Python 3.11
- Android: The normal functioning of ADB commands in the command line.
Integrating NATE with a general Android testing approach is very straightforward and could be achieved both intrusively and non-intrusively.
To directly integrate NATE with a general Android testing approach:
- Create a Python project. (If the approach itself is Python-based, skip this step.)
- Put
NATEandandroid_testing_utilsunder the root directory of your Python project to avoid import issues. (If you put it elsewhere, you need to adjust the imports yourself.) - For each testing step of the general testing approach, call NATE and pass the required App State and Selected App Event.
Very similar to the workflow in 3.3.1. The only difference is that the required App State and Selected App Event by NATE is obtained non-intrusively, rather than passed by direct function calls. For example, by capturing the real-time log stream from the general testing tools, the required step-wise App State and Selected App Event could be extracted in time.
Before the test starts, initialize a NATE_MAIN instance from NATE/nate_main.py:
- device_id: The device ID of the target device (e.g., "emulator-5554")
- package_name: The package name of the App Under Test (e.g., org.mozilla.fenix.debug)
- use_pure_random: To randomly inject network events without guidance, enable this option
- auto_action: Whether the number of actions in each state is known when testing; if not, enable this option
- netflow_strategy: As the network-related APIs in different Android Versions are different, we need different strategies. Currently, we have implemented:
NetFlowStrategy.mAppUidStatsMapTested on Android 14 EmulatorsNetFlowStrategy.xt_qtaguidTested on Android 9 Emulators
NOTING!!! Before starting NATE, it is suggested to try the network monitoring adb commands on your target Android Version in advance. You may need to implement yours for your Android Version.
At each test loop step t+1, call NATE_MAIN.nate_step:
- raw_app_state: The state identifier on step t
- selected_action: The app event on step t, if using auto_action, this is the action identifier; if not, this is the action index
- target_app_state: The state identifier on step t+1, this may be None when an end state is met
- target_selected_action: The app event on step t+1, if using auto_action, this is the action identifier; if not, this is the action index
- raw_action_count: If not auto_action, provide the number of actions on step t, else None
- target_action_count: If not auto_action, provide the number of actions on step t+1, else None
NATE will automatically learn and inject network events when appropriate.
Call NATE_MAIN.persist_history to persist data.
- NATE/guidance/q_table: The learned Q-Table.
- NATE/network_injector/injection_record: The injection history.
- NATE/network_monitor/network_monitor_record: Network monitoring details.