The performance of mobile devices directly affects billions of people worldwide. Yet despite memory management be- ing key to their responsiveness, energy efficiency, and cost, mobile devices are understudied in the literature. A paucity of suitable methodologies and benchmarks is likely both a cause and a consequence of this gap. It also reflects the chal- lenges of evaluating mobile devices due to: i) their inherently multi-tenanted nature, ii) the scarcity of widely-used open source workloads suitable as benchmarks, iii) the challenge of determinism and reproducibility given mobile devices’ ex- tensive use of GPS and network services, iv) the complexity of mobile performance criteria. We study this problem using the Android Runtime (ART), which is particularly interesting because it is open sourced, garbage collected, and its market extends from the most ad- vanced to the most basic mobile devices available, with a com- mensurate diversity of performance expectations. Our study makes the following contributions: i) we identify pitfalls and challenges to the sound evaluation of garbage collection in ART, ii) we describe a framework for the principled perfor- mance evaluation of overheads in ART, iii) we curate a small benchmark suite comprised of widely used real-world appli- cations, and iv) we conduct an evaluation of these real-world workloads as well as some DaCapo benchmarks and a mi- cro-benchmark. For a modestly sized heap, we find that the lower bound on garbage collection overheads vary consider- ably among the benchmarks we evaluate, from 2 % to 51 %, and that overall, overheads are similar to those identified in recent studies of Java workloads running on OpenJDK. We hope that this work will demystify the challenges of studying memory management in the Android Runtime. By doing so, we hope to open up research and lead to more innovation in this highly impactful and memory-sensitive domain.