The Making of Switch - Part 3: Movement Tracking

The real world is messy, and this post is all about taming location and activity data for the purposes of train journey discovery.

For starters, location data tells us where the user is geographically at any given time, and their activity tells us what they are most likely doing, such as driving, cycling or walking.

Requirements

The goal hasn’t changed. What I set out to build was an autonomous train journey tracking system, which for me meant an app that is set up once and works in the background without my intervention, or any distraction:

• Non-intrusive: no permanent notification on the screen
• Always on: no need to open, or check in
• Battery friendly: very limited battery drain

For the sole purpose of movement tracking, I wanted to replicate what Google Maps Timeline already had to offer, and use it as a reference to validate my results.

Location Data

I started off by collecting location data, since this seemed the most straightforward problem to be solved, given that there are already so many location-based apps out there. Every major platform and framework offered some way to give the app developer the ability to collect location data. With the focus on Android, building on top of React Native, the Expo framework made it easy to get started.

For an Android app to have access to location data, even when running in background, the following rules have to be followed:

1. The Android manifest must declare the ACCESS_BACKGROUND_LOCATION permission
2. The user has to grant the Allow all the time permission to give the app the ability to collect location data in the background. This is a two-step process, where at first the user grants the app permission to access location data while running in the foreground. Once granted, the app redirects the user to their Android permissions-settings for a final confirmation to allow this all the time.

From a UX and privacy perspective, this step may create a lot of friction, rightfully so, simply because a lot of bad actors collect location data for no apparent reason. This is why it’s so important to educate the user on why this level of permission is required. For Switch, I carefully crafted a permissions screen explaining each step they have to carry out and why it matters.

More information on accessing background location can be found in the official Android documentation: Access location in the background

Background work module

My initial implementation was based on the expo-location module, providing multi-platform access to background location. During my first field-tests, I started experiencing intermittent issues, and then eventual failures that never recovered. After digging around in the expo codebase, I found that the abstraction around the Android API was getting in the way. This is when I decided to switch gears, and build my own expo native module to get full access to Android’s location API.

My knowledge around the Android system was very limited, requiring me to learn about what it means for an app to operate in the background. Simply put, an app is only truly active when it’s in the foreground, and visible to the user. Once swiped away, or another app becomes active, there’s absolutely no guarantee that your app will continue to run. This is why Android provides additional primitives to get work done in the background, while the app is suspended. Those primitives are scheduled by the OS, to limit battery drain and avoid long-running tasks locking up resources.

I followed the recommended approach and registered a PendingIntent pointed to a BroadcastReceiver, using the requestLocationUpdates method provided by the FusedLocationProviderClient.

BroadcastReceiver Behavior

Another issue I discovered during field testing was a problem where the app just stopped tracking entirely, until I had re-started the app. My investigation revealed that my phone was restarted during the night due to an automated Android security update, dropping all registered PendingIntents. This would be fine for some apps, but I wanted Switch to continue tracking, even after a device restart or OS update. Luckily Android offers a way to listen to system level events, such as BOOT_COMPLETED and MY_PACKAGE_REPLACED. All it takes is a declaration within the Android manifest and an action reacting on said event.

Now when the app receives a BOOT_COMPLETED event, it automatically re-registers the location and activity tracking modules, ensuring a fully hands-off experience, even after a device restart.

Background Location Limitations

Getting access to location data is just half of the battle. During the implementation I learned just how restrictive that API was. The official Android documentation summarizes the limitations on their Background Location Limits page, highlighting the use of the FusedLocationProviderClient:

the location system service computes a new location for your app only a few times each hour.

This was a hard pill to swallow, making me question the overall feasibility of this project. Such infrequent location updates (10-20min apart) make it almost impossible to reliably determine departure and arrival stations, a key requirement in this whole endeavor. Imagine arriving at the station just 5 minutes before the train’s departure, the likelihood of capturing that location is rather low. Besides the low frequency, the the accuracy of the location readings was yet another reason for concern.

Location Accuracy

When requesting location updates, using the Fused Location Provider (FLP), you are required to set the desired location accuracy based on its impact on power usage. The options are: High, Balanced, Low. After some real-world testing, I decided to set accuracy to Balanced, giving me average results for little impact on battery life. The more accurate High setting attempts to use GPS every single time, drawing much more energy. Considering the scenario this is used in, getting a GPS lock inside a train, while in tunnels, or underground, is almost impossible. GPS still requires a line-of-sight with the satellites to work reliably. This is hardly the case in the real-world, where a phone might be stored in a bag, in pockets or a behind a window tinted with metal, blocking radio signals.

The FLP, as the name already suggests, relies on cell-towers and WiFi to triangulate and estimate the position. This method isn’t very accurate, but good enough, except when it isn’t. Having collected location data for the past few month, I’ve come across many readings that are far from reality. 90% of the readings are within the 100-200 m range of accuracy, but the remaining 10 % are all over the place. Some of the outliers may even be caused by OS level caching, where older readings are resurfaced minutes later, some even a day later. Other factors, such as low cellular coverage and density are also contributing to the lack of accuracy.

To address some of these shortcomings, I started off by clustering location readings in a 200m radius, with the benefit of having fewer readings in roughly the same place. In addition, the majority of the caching problem was fixed by removing duplicate location entries that occured within a 1h window.

The following two screenshots demonstrate this filtering mechanism. The blue line represents the actual train tracks, and the red one connects the dots of our location readings. In an ideal scenario, those two lines would overlap. As seen on the first screenshot, the line goes back and forth, caused by the Android caching problem. After the fix, the line continues in direction of travel, as visualized in the second screenshot.

One of the biggest limitations was still the low frequency of location updates. One can live with 2 - 10 updates an hour, while on a long-distance train, but capturing boarding and exit stations remained an issue. Of course this can be solved with a foreground service, like navigation or sports-tracking apps often do, but I wanted a non-intrusive, always on experience.

As mentioned in the previous chapter, to determine the train we are on, we needed to know at least where the train has left the station. If that data point was missing, the entire recording was rendered useless.

Geofencing to the rescue, or not?

This is the kind of problem geofences are supposed to solve, defined by a location and a radius. Once registered, the app gets notified when the device enters or exits that region. Exactly what I needed.

I had already prepared a database of all stations and their corresponding geo-locations (see Part 4 - GTFS Schedule), ready to power the geofences. As it turned out, Android limits the maximum number of simultaneous geofences to 100, whereas iOS only allows 20, at any given moment. I worked around this limitation by updating geofences with every new location reading. The following screenshot shows a map of geofences created around my location in Vienna, each one representing a train station.

With renewed hope to have solved the issue of skipping departure, or arrival stations, I started a new test run. To my surprise, the results did not improve at all. After some initial troubleshooting and double-checking the geolocation update mechanism, I was not able to pinpoint the problem. This was especially frustrating because the app received occasional geofence events, and sometimes didn’t.

After some further research I learned that geofencing is simply a different way of accessing background location updates. The same limits apply, therefore geofences are easily to be missed. According to the documentation on Background Location Limits:

Background apps can receive geofencing transition events more frequently than updates from the Fused Location Provider.

Where “can” is a very misleading term in an API documentation. This was the moment I decided to abandon geofences, and find other avenues to improve detection of departure and arrival stations.

Activity Recognition

Given the low accuracy and frequency of location readings, it seemed almost impossible to track train journeys reliably by just using this single source of data. Android also offers a very promising Activity Recognition API, offering a high-level abstraction of the phone’s movement patterns. This on-device capability offers insight into the physical activity of its user. It reports whether the user is currently on foot, driving or even cycling. My idea was to fill the gaps in location readings with this activity data, if possible. Looking at the patterns of a regular departure from a station, one walks toward the platform and boards the train, which eventually rides off. If I were able to keep track of those activity transitions, from walking to driving, I had a higher chance of catching that moment.

Similar to location updates, physical activity data is restricted, and the user needs to explicitly grant permissions to the app. Once the ACTIVITY_RECOGNITION permission is granted, the app can start collecting data.

The Activity Recognition API offers two primary modes of operation: First, an API focused on delivering updates of all possible activity types, and their corresponding likelihood. This is useful for scenarios where the probability of a single activity type over time needs to be tracked. The second mode is focused on activity transitions, reporting on transitions from one activity type to another. For example when a person walks to the train station, gets on the train and departs, then the activity transition mechanism would emit an event where walking ends, and driving starts.

The first screenshot captures a visualized view of activity updates, where each entry represents a snapshot of every activity’s likelihood (0 - 100). The second one demonstrates activity transition behavior, with start and end times attached. There’s a clear correlation between these two data-views, where the IN_VEHICLE transition lasts from 08:04:20 to 08:07:35 and the activity update reports high likelihood at 08:04:21 and 08:05:21.

As demonstrated above, the activity updates provide more granular data compared to activity transitions. The first iteration of Switch relied on the activity-updates data-source, where I manually transformed the data to activity transitions. Due to the increased complexity of data filtering, and random data snafus, I switched to activity transitions, providing additional benefits. With this data at hand, I was determined that detecting departures was inching a step closer.

Catching Departure Locations

Keeping track of activity transitions turned out to be very useful, because using this API unlocked another key element that significantly improved departure and arrival station detection. As mentioned above, the frequency of location readings can only be increased by running a foreground service, which Switch doesn’t offer. During my research I learned about the existence of foreground services, services that, when running just a few seconds, won’t be reported to the user. However starting foreground services from background apps, such as Switch, is prohibited by Android. Fortunately there are some Exemptions from background start restrictions listed in the documentation. One of them caught my immediate attention:

Your app receives an event that’s related to geofencing or activity recognition transition.

With renewed excitement to finally be able to improve location readings around train stations, I set out and implemented a foreground service that is triggered by activity transitions. The only purpose of the foreground service was to ping the current location, store it, and shut down again. From that moment on, when walking up to the train station, and then standing still, or sitting down, this activity transition would then start the foreground service and record the current location.

The following screenshots illustrate this behavior, where the change in activity to WALKING (16:47:26) triggered a location reading, which was stored just a second later (16:47:27). When I stopped walking (16:52:50), a new location reading was triggered as well, but isn’t visible due to clustering (200m).

After some more field-testing, I was pleased to see the results improve dramatically. There were still a few edge cases not covered by this implementation, but overall this improvement seemed to work. I captured one of the edge cases where I was running to catch a train which departed right after boarding. In this case, the usual 1 min delay in activity transition was enough to miss the station. Of course, this can be mitigated with more clever tracking techniques, but for the time being I decided to start leaving home just 1-2 min earlier, to avoid those close calls entirely.

Determining departing trains

Why is it so important to detect the departure station in the first place? As mentioned in the previous chapter, I needed a reference point for a potential train departure, to match that with the schedule at large. Another advantage of this approach is to be prepared to support just-in-time information, like upcoming connections based on the current journey.

Next Steps

With location and activity tracking in place, I had just reached another milestone on the journey towards train travel matching. Despite the numerous opportunities to improve the current implementation, I decided to move on and focus on the second part of my data requirements, the train schedule itself.

Read on: Part 4 - GTFS Timetable