Building a PWA on Google, part 1

This is the first in a series of blog posts on the lessons the Google Bulletin team learned from creating an external PWA. In these posts we will share some of the challenges we face, the approaches we take to overcome them, and general tips to avoid cheating. This is by no means a complete overview of the PWAs. The objective is to share the learnings from the experience of our team.

For this first post, we'll cover a little general information first, and then dive into everything we learned about service workers.

Background

Bulletin was in active development from mid-2017 to mid-2019.

Why we chose to build a PWA

Before delving into the development process, let's examine why creating a PWA was an attractive option for this project:

• Ability to quickly iterate. Especially valuable as the Bulletin would be tested in multiple markets.
• Single code base. Our users were roughly evenly split between Android and iOS. A PWA meant that we could create a single web application that would work on both platforms. This increased the speed and impact of the team.
• Updated quickly and regardless of user behavior. PWAs can be automatically updated, reducing the number of outdated clients in the wild. We were able to drive major changes to the backend with a very short migration time for clients.
• It is easily integrated with your own and third-party applications. Such integrations were a requirement for the application. With a PWA, it often meant simply opening a URL.
• The friction of installing an app has been removed.

Our framework

For Bulletin, we use Polymer, but any modern, well-supported framework will do.

What we learned about service workers

You can't have a PWA without one service worker. Service workers give you a lot of power such as advanced caching strategies, offline capabilities, background syncing, and more. While service workers add some complexity, we found that their benefits outweigh the additional complexity.

Generate it if you can

Avoid writing a service worker script by hand. Writing service workers by hand requires manually managing cached resources and rewriting the logic that is common to most service worker libraries, such as Workbox.

That said, due to our internal tech stack, we were unable to use a library to generate and manage our service worker. Our learnings below will at times reflect that. Go to Issues for non-generated service workers to read more.

Not all libraries are compatible with service workers

Some JS libraries make assumptions that don't work as expected when run by a service worker. For example, assuming window or document are available, or using an API not available to service workers (XMLHttpRequest, local storage, etc.). Make sure that the critical libraries you need for your application are supported by service workers. For this particular PWA, we wanted to use
gapi.js for authentication, but they were unable to do so because it did not support service workers. Library authors should also reduce or eliminate unnecessary assumptions about JavaScript context whenever possible to support service worker use cases, such as avoiding incompatible service worker APIs and avoiding global state.

Avoid accessing IndexedDB during initialization

Do not read IndexedDB while initializing your service worker script, or you may get into this unwanted situation:

1. The user has a web application with IndexedDB (IDB) version N
2. A new web application is included with the N + 1 version of the IDB
3. User visits PWA, which triggers the download of a new service worker
4. The new service worker reads from the IDB before registering install event handler, activating a BID update cycle to go from N to N + 1
5. Since the user has an old client with version N, the service worker update process hangs as active connections are still open to the previous version of the database
6. Service worker hangs and never installs

In our case, the cache is invalidated on the service worker installation, so if the service worker was never installed, the users never received the updated application.

Make it tough

Although service worker scripts run in the background, they can also be terminated at any time, even when they are in the middle of I / O operations (network, IDB, etc.). Any long running process should be able to resume at any time.

In the case of a sync process that uploaded large files to the server and saved to the BID, our solution for interrupted partial uploads was to take advantage of the resumable system from our internal upload library, saving the resumable upload URL to the BID before upload and use that URL to resume an upload if it didn't complete the first time. Also, before any long I / O operation, the state was saved in IDB to indicate where we were in the process for each record.

Don't depend on the global state

Because service workers exist in a different context, many symbols that you might expect to exist are not present. Much of our code ran on both window context, as well as a service worker context (such as registration, flags, sync, etc.). The code should be defensive about the services you use, such as local storage or cookies. You can use
globalThis

to refer to the global object in a way that works in all contexts. Also use data stored in global variables sparingly, as there is no guarantee when the script will terminate and state will be evicted.

Local development

An important component of service workers is caching resources locally. However, during development this is the opposite than you want, especially when updates are done lazily. You still want to install the server worker so you can debug problems with it or work with other APIs such as background sync or notifications. In Chrome, you can achieve this through Chrome DevTools by enabling the Bypass for network checkboxRequest panel> Service workers panel) in addition to enabling the Disable cache check box in the Net panel to disable cache memory as well. To cover more browsers, we opted for a different solution by including a flag to disable caching in our service worker, which is enabled by default in developer builds. This ensures that developers always get their latest changes without caching problems. It is important to include the Cache-Control: no-cache header also to prevent the browser from caching any assets.

Lighthouse

Lighthouse provides a number of useful debugging tools for PWAs. It scans a site and generates reports covering PWA, performance, accessibility, SEO, and other best practices.
We recommend running Lighthouse in continuous integration to let you know if you don't meet any of the criteria for being a PWA. This actually happened to us once, where the service worker wasn't installing and we didn't notice a production boost before. Having Lighthouse as part of our CI would have prevented it.

Embrace continuous delivery

Because service workers can update automatically, users lack the ability to limit updates. This significantly reduces the number of outdated customers in the wild. When the user opened our app, the service worker served the old customer while lazily downloading the new customer. Once the new client is downloaded, it will ask the user to refresh the page to access new features. Even if the user ignored this request, the next time he refreshed the page, he would receive the new version of the client. As a result, it is quite difficult for a user to reject updates in the same way that it can for native applications.

We were able to drive major changes to the backend with a very short migration time for clients. We generally give users one month to upgrade to newer clients before making major changes. Since the application would work while it was out of date, it was possible that older clients would exist in the wild if the user had not opened the application for a long time. On iOS, service workers are
evicted after a couple of weeks
so this case does not happen. For Android, this problem could be mitigated if it is not published while it is out of date or if the content is manually expired after a few weeks. In practice, we never encounter problems with outdated clients. How strict a team wants to be given here depends on your specific use case, but PWAs provide significantly more flexibility than native apps.

Get cookie values in a service worker

Sometimes it is necessary to access cookie values in the context of a service worker. In our case, we needed to access the cookie values to generate a token to authenticate the originating API requests. In a service worker, synchronous APIs like document.cookies Are not avaliables. You can always send a message to active (windowed) clients from the service worker to request cookie settings, although the service worker may run in the background with no windowed clients available, such as during a synchronization in background. To fix this, we created an endpoint on our frontend server that simply returned the value of the cookie to the client. The service worker made a network request to this endpoint and read the response to get the cookie values.

With the launch of the
Cookie store API, this workaround should no longer be necessary for browsers that support it as it provides asynchronous access to browser cookies and can be used directly by the service worker.

Pitfalls for non-generated service workers

Make sure the service worker script changes if you change any static cached files

A common PWA pattern is for a service worker to install all static application files during their
install phase, which allows clients to directly access the caching API cache for all subsequent visits. Service workers are only installed when the browser detects that the service worker script has changed in some way, so we had to make sure that the service worker script file changed in some way when changed a cached file. We did this manually by embedding a hash of the set of static resource files within our service worker script, so each version produced a different service worker JavaScript file. Service worker libraries such as
Workbox Automate this process for you.

Unit Exam

The service worker APIs work by adding event listeners to the global object. For example:

self.addEventListener('fetch', (evt) => evt.respondWith(fetch('/foo')));

This can be difficult to test because you need to simulate the event trigger, the event object, wait for the respondWith () callback, and then wait for the promise, before finally asserting the result. An easier way to structure this is to delegate the entire implementation to another file, which is easier to test.

import fetchHandler desde './fetch_handler.js';
self.addEventListener('fetch', (evt) => evt.respondWith(fetchHandler(evt)));

Due to the difficulties of unit testing a service worker script, we kept the core service worker script as basic as possible, dividing most of the implementation into other modules. Since those files were just standard JS modules, they could be more easily unit tested with standard test libraries.

Stay tuned for parts 2 and 3

In parts 2 and 3 of this series we will talk about media management and issues specific to iOS. If you want to ask us more about creating a PWA on Google, visit our author profiles to find out how to contact us: