So What’s a PWA?

Briefly, PWAs are essentially just your typical modern website designed with HTML, CSS, and JavaScript, much like what we’re already creating with our TMS WEB Core projects. The thinking behind PWA, in part, is that apps should try to support as many devices as possible in order to reach the largest numbers of users.  And to degrade gracefully on devices that lack the latest standards, while offering the best experiences possible when device support is available.  Some of the same thinking behind responsive web design. In practical terms, there are a few reasons why having your app upgraded to function as a PWA may be of interest to you.

• Some App Stores allow PWAs directly, rather than native apps.  Notably, this isn’t necessarily the case for Apple’s App Store.  Some have made it through using various PWA app-builder-type tools, so it is conceivably possible, but not really their thing. And then there’s Microsoft, who might very well add your PWA app to their App Store without even telling you.  Strange times!
• PWA apps include functionality that makes it possible for them to work offline to some degree.  Whether this makes sense for your project naturally depends on the kinds of data your app is dealing with.  Storing reference information is a good use of this kind of function, and perhaps collecting data while not connected may also be feasible.
• Because PWA apps contain additional information about themselves, it is also often possible for the app to appear as a regular app in desktop environments, and as pseudo-apps in mobile devices, even without App Store support.  Some of this you can just do separately anyway, but as a PWA app this support becomes an integral part of the project.

• A PWA App must be served up from a secure domain. This means HTTPS and SSL. So if you’ve been putting this off, well, time to get on it. The good news is that SSL certificates are readily available for free from several solid sources. I’m partial to LetsEncrypt, for example. This wasn’t always the case!
• A service worker must be present.  This is a bit of JavaScript that in some ways serves a function that is a bit like a proxy server or a caching server. Your app asks it for stuff, and it goes about providing whatever has been asked for, deciding whether to serve up a local copy from its cache or go out and get it if a network connection is available, for example.
• A manifest JSON file must be present. This is kind of a catch-all in that a manifest file can contain many important things.  For example, it’s what tells the service worker what files it really should be caching and which ones perhaps it really should not be caching.  Various icons and other things are also defined here.  

Some of these things you might already include in your project.  For example, if you’re not already securing your site with SSL, it is already advertising this fact by showing various non-secure icons and warnings.  Perhaps not very intrusively at the moment, but no doubt there are many who would very much like to make this a requirement. And if you’re doing any kind of work to support Android devices, you’re probably already well-versed when it comes to fiddling with manifest files.  

Creating a PWA App.

When creating a new TMS WEB Core project, there is a “TMS WEB PWA Application” template ready to go. There aren’t any options to pick from or anything, it just creates a new blank project that looks a lot like a regular non-PWA TMS WEB Core project.  But there are a few extra files over in the Project Manager that aren’t normally there.  The ServiceWorker.js is there, along with the Manifest.json file and some icon files.

 

New TMS WEB Core PWA Application

The rest of the project looks the same, and if you just add a button or two and run the app, everything appears to work the same as well. In fact, you can run the app right away and then go and look in Google Chrome’s developer tools and see that, indeed, everything is already in place to qualify as a PWA app, right out of the gate.  We’ll try not to break it, but we’re off to a good start already. Note that it isn’t really saying anything about the security aspects of our project. An exercise for another day might be to investigate the other numbers at the top and why they aren’t all 100%.

 

Google Chrome’s Lighthouse Results

What is actually different now, compared to a non-PWA app?  Let’s have a look at a regular Project.html file and then a PWA Project.html file and see what’s different. Here’s what a normal run-of-the-mill TMS WEB Core Project.html contains when it is first created.

And here is what our new PWA app’s Project.html looks like.

Hmm… not really much different aside from an extra pair of icon definitions.  So what does the initial output of our pair of projects look like, then?  First, the non-PWA project has the usual suspects.  Our Project.html file and the JS file that is created when the project is compiled, as well as a JavaScript MAP file used for debugging, and an HTML file for our Unit1 Delphi Unit.  

Default Non-PWA Output

If we look at the output of our PWA version of the project, we see all of these, but also the Manifest.json, ServiceWorker.js and three icon files.  And not really much else.  

Default PWA Output

As with other TMS WEB Core projects, it should be noted that all of these files will need to be included when the project is deployed within its target web server.  But where can we tell that something different is going on?  Let’s have a look at our Project Options in the IDE.  For a regular TMS WEB Core project, the defaults might look like this.

Default Non-PWA TMS WEB Project Options

But in our PWA project, it looks like this.  A pile of PWA options that default to exactly what we should expect – a Manifest.json file, a ServiceWorker.js file and again with the icons. There are some others there that are interesting as well.  The “Start URL” for when you want to launch with specific parameters for your project, or with a specific page. And “Automatically copied file extensions” can be useful. Remember that the Service Worker can behave a bit like a proxy server or a caching server, so here you can tell it to cache a few things right away.  Then they’re always available.  Have to be mindful that you don’t copy too many files, but there are plenty of instances where having a select few on-hand will be beneficial.

Default PWA TMS WEB Project Options

Behind the scenes, then, TMS WEB Core compiles your project and does the necessary wiring to make either a PWA app or a non-PWA app, including (or not) the necessary ServiceWorker.js and other files as needed. Nothing for the developer to worry about at all, honestly.  It works remarkably well.  So that solves one mystery (where’s the difference) and creates another (how, then, do we upgrade an existing project?).  So let’s quickly tackle that one next.

Upgrading an Existing Project.

What to do if you’ve been working hard on your TMS WEB Core project, and you’ve just now learned about PWAs?  Or maybe you knew before, but were more interested in the Bootstrap template rather than the PWA template.  Well, not a big deal.  Let’s give it a try.  The basic idea is to create a new empty PWA project using the template, and then copy over the units from the existing project, and maybe add in whatever changes have been made to the Project.html or to the Manage JavaScript Libraries list for your project.  

For the Actorious project that materialized out of our Tabulator miniseries, this is exactly the case.  It was created using the Bootstrap template, but now we’d like it to be a PWA app as well.  The following steps were performed, and voila! it now shows up in Lighthouse as a PWA app.

• Created a new empty project using the PWA template, in an ActoriousClient project folder.
• Renamed the project to be ActoriousClient. So we’ll have ActoriousClient.html and ActoriousClient.js in the output folder.
• Copied the contents of the original Project.html file over first, and added in the two lines that were different in the default PWA project file that we identified previously (the icon references).
• Copied over the CSS file for the project into the new folder and added it to the project.
• Tested everything so far.  I had to add a button to the default Unit1 form as otherwise nothing would display and Lighthouse would complain.
• Also had to remove the fancy form fade-in that was added or again Lighthouse would complain that there was nothing to display.
• No mystery then that we’re now at a blank page with a ton of CSS. That passes the PWA test. And scores even worse in the other numbers.

For the form itself, the original project has a Main.pas and is referenced as MainForm.  But this is a single-form application with nothing going on at all in Main.html.  And I’ve been curious about this new-ish DirectForm capability.  So the next steps involved changing this as well.

• Unit1 was removed from the new project entirely (no Unit1.pas or Unit1.dfm).
• A new Direct Form was added using the template. It was renamed to Main.pas and to MainForm in the form itself, and then closed.
• Main.pas from the original project was then copied over the Main.pas that we just created.
• Main.pas was then opened up in the new project. Sort of a sleight of hand, could probably just add it to the project easily enough if it weren’t for this DirectForm two-step.
• Copied over the img folder from the old project to the new project, and then added all the files to the project in the Project Manager.
• Bonus: In the latest version of TMS WEB Core, you can remove an entire folder at a time if needed.
• Adjust other Project Options to match, like ECMA version and project version information.

The project runs fine just like that.  I hadn’t made any changes to the “program” source code, just to the main form, so not really anything else to do.  And the DirectForm capability seems to work exactly as advertised – no more pesky Main.html file to be found anywhere.  Curiously, I thought that CreateForm would have been replaced with something else, but perhaps if there is no HTML file connected to the form, it doesn’t try? Mysteries for another day. Now, if you’re using forms with templates, then by all means keep those around.  But in this case it was an extra download being made when the application was being initialized, without any actual benefit.  

Running the ‘desktop’ version of the Lighthouse checks (as I’ve not done anything to make it mobile-ready as yet) gets us to our PWA confirmation.  And a much worse score everywhere else, but that’s somewhat expected given that its doing more than drawing a button on a page. 

Actorious Upgraded to PWA

Deploying a PWA app isn’t really any different.  Just be mindful of the extra new files and things should be off to a good start.  We haven’t really changed anything else in the Actorious project, so not really any changes needed anywhere else so far.  We’ll be making some changes momentarily, but just getting it out the door is job number one.  No problems here. After running through the usual deployment steps for the project, the new version is up and running, with PWA links showing up in Chrome and Edge.  Firefox for the desktop isn’t really playing the PWA game at the moment, though they claim to be doing so on Android.  Something to be mindful of if you use Firefox. Note also that you can run the same Lighthouse test against the live version of your website (or anyone’s website for that matter) just as easily as you can for the development version.  

About that SSL Requirement.

In order to actually be a full-blown PWA app, it is important that the HTTPS/SSL situation is sorted out.  That’s a pretty substantial topic in its own right and when it comes to TMS WEB Core projects, it is actually likely to be largely outside of the domain of the developer. For relatively simple projects, for example, there are no code changes to support SSL or HTTPS.  Nothing in the JS or CSS or HTML files that need to be adjusted really either.  This is largely the domain of the web server itself.  It alone controls the keys, literally. For projects I’ve been working on, like Actorious, they get hosted on a VPS that I use that is managed with a server control panel product called VirtualMin.  There are more than a few other control panel products that offer similar functionality, like Plesk, cPanel or ISPConfig.  While I’ve not used all of them, I’d imagine they all have the same suite of tools available in some form for acquiring an SSL certificate and applying it to your web server, if that’s how you’re hosting is setup. It is one of the key features of these tools, after all. These tools usually also take care of automatic renewals.  And once you have an SSL certificate and the rest of it setup, you can serve up many different projects from the same certificate (myserver1.com/project1, myserver1.com/project2 etc.), typically, and you only need to acquire new certificates when you want to also register new domain names for your projects. And that’s what I’ve done for Actorious.  And while SSL certificates are free, domain names are not, sadly.  Hopefully enough pizza money comes in to cover the domain fees!

Note that none of this applies to XData because it is in effect a web server in its own right (well, Sparkle, technically) and so the SSL situation is very different there.  There’s nothing really that stipulates that a TMS WEB Core client app must use SSL to talk to an XData server, though it is probably a very good idea to do so.  There is the requirement that everything that a PWA app sends to the browser is done over SSL however. No mixing SSL and non-SSL content.

About that Service Worker Requirement.

As we’ve seen so far, well, this is pretty well satisfied at this point.  Whether starting a new project or converting an existing project, getting the ServiceWorker.js in place and operating is reasonably straight-forward and I don’t think I’ve encountered problems with it so far. What it is doing is enormously complex relative to most of the things we’ll ever have to deal with in a TMS WEB Core project, but thankfully we don’t have to mess with it too much at all.  But we should be at least aware of what its capable of.  

When a PWA app is running on, say, a mobile device, any network requests for pages or images or other content passes from the app through the Service Worker and then outbound.  The Service Worker keeps tabs on whether the internet connection is available or not, and can send this information to your TMS WEB Core application. For example, if you were working on a restaurant ordering app, maybe the app is generally available offline for browsing menus and so on.  But if offline, you’d want to disable the “order” button, and then re-enable it when the internet connection becomes available again.  

procedure TForm1.WebFormCreate(Sender: TObject);
begin
  Application.OnOnlineChange := AppOnlineChange;
end;
procedure TForm1.AppOnlineChange(Sender: TObject; AStatus: TOnlineStatus);
begin
  if (Application.isOnline) then
  begin
    WebButton1.Caption := 'PWA Connection ONLINE';
    WebButton1.Enabled := True;
  end
  else
  begin
    WebButton1.Caption := 'PWA Connection OFFLINE';
    WebButton1.Enabled := False;
  end;
end;

If the restaurant menus are available as a handful of PDFs that the TMS WEB Core app just links to directly, then the Service Worker can automatically download them to a local cache even before anyone looks at any of them.  This can be setup just by specifying ‘pdf’ in the Project Options that we were looking at earlier.  The app doesn’t even have to change the links its using – the Service Worker will pickup on the fact that the user wants the PDF files, and will likely serve them the local copy even if the internet connection is available, if it has already cached them.  The only catch here is to be somewhat mindful of how much data you’re going to be copying over to every client.  For a handful of menus it probably isn’t a big deal.  Likewise for the images that make up the UI of your project, also not a big deal.  But if you have an image repository of some kind, then you probably don’t want to copy the entire thing over via the Service Worker. 

About that Manifest Requirement.

This then is where we’re going to spend the rest of our time in our PWA introduction.  The Manfiest.json file is, well, a JSON file which we should all be familiar with by now.  Lots of different things can go into a Manifest.  The idea is that each environment that supports PWAs will pick out things that it needs from this file. If you happen to know where your PWA app will be running, you can likely get away with leaving out a few things, but that kind of goes against the point of PWA in the first place – providing an app that works in every environment.  So what do we have to work with when just starting out?  The default Manifest.json looks like this.

{
   "short_name":"$(ShortName)",
   "name":"$(Name)",
   "description":"$(Description)",
   "start_url":"$(StartURL)",
   "display":"fullscreen",
   "theme_color":"$(ThemeColor)",
   "background_color":"$(BackgroundColor)",
   "icons":
   [
    {
      "src": "$(LowResFileName)",
      "type": "image/png",
      "sizes": "64x64",
      "purpose": "any maskable"
    },
    {
      "src": "$(MidResFileName)",
      "type": "image/png",
      "sizes": "256x256",
      "purpose": "any maskable"
    },
    {
      "src": "$(HighResFileName)",
      "type": "image/png",
      "sizes": "512x512",
      "purpose": "any maskable"
    }
   ]
}

And you can probably immediately figure out that all those variables are automatically filled in from properties that you can specify in either Delphi itself or in the TMS WEB Core project settings. And this really is about the minimum that you’d need to fill out to support some basic devices.  There are plenty more entries that can further refine how specific devices work.  One particularly pesky area is the list of icons.  Apple devices, for example, have dozens of different icons that can be defined for nearly every device and orientation they have.  There are also images that can be displayed while PWA apps are loading.  And permutations and combinations not only for sizes and orientations but also for dark vs. light device settings.  It all gets to be a bit overwhelming sometimes, particularly when you want things to just work without having to tediously tinker so much. 

To help with such things, there are numerous online icon generators.  Many started out life as favicon generators – those little icons beside the URL in the browser address bar, or shown beside bookmark icons. Or as icons in browser tabs, once browsers had tabs, that is.  Over time these little icon generators have grown into tiny little monster apps that can create dozens and dozens of variations of icons and placeholders and themes and the rest of it.  The approach we’re going to use here is to create a huge version of an icon that we’d like to use, 1024×1024, and then feed it into one of these online icon generators.  We can then pick out the icons we want and stick them where they need to be stuck.

The icon we’re starting with is just a large five-point start with a border, and a stylized A, with a black background inside the star and a transparent background otherwise.  Five minutes with any image editing app. If I recall I used an outdated version of PixelMator on an outdated Mac, so nothing special about it.  Just that it is a PNG file at 1024×1024.  Scaled down, it looks like this.

In the Delphi IDE, under the Project options, it is then trivial to select the newly minted icons, which will then get renamed and included as part of the project, all referenced by the Manifest.json file.  While we’re at it, we can also set some of those other Manifest properties by updating the TMS WEB Project Options for name and description and so on.  And we can also edit the Manifest.json file directly and add in other icons or other properties as needed.

Now, to be fair, this is covering what might end up being a day-long adventure with icons into just a few minutes.  Icons are sometimes a manifestation of an organization’s brand, so it is important to get them right, and to manually adjust as needed so they look as great as they possibly can.  And this can take time and trial and error.  And a lot of patience.  Some devices (iOS is notorious for this) will cache images you send them, so even though you update new icons, they won’t necessarily appear right away.  Likewise for website favicons and the rest of the lot.  It can be a bit trying at times, with many devices having a personality all their own, or even different OS releases, when it comes to these things.  But eventually it is possible to get everything sorted and looking lovely, so if you’re struggling, keep at it.  Plenty of resources around with hints and tips for every circumstance, I’m sure.  At least that has been my experience so far.

So, when you’re done, the result is that you’ll have a website with a new icon for its favicon, and you’ll be able to add it to your desktop using Chrome, where you’ll also see an icon and your app name.  And if you visit the website using Safari on an iPad, you can click the “add to home page” button and it should come up with yet another icon and the name of your website.  The cool thing is that if you then open that homepage icon, you get a fullscreen app (even with a splash screen if you’ve added in all the right images) and nothing at all in the way of Safari menus getting in the way.  Almost like a real iOS app.  Which was kind of the point.

About Those Updates.

One final topic before we finish up our introduction to PWAs. Once an app has been installed on a device, naturally we’d like it to automatically update when the website is updated.  But in the land of PWAs, this has been historically rather difficult to accomplish. Most people are familiar with clicking the refresh button on a browser.  Some are aware that there are different kinds of refreshes that clear out data.  And a few know that you can remove everything and start fresh if necessary.  But in a PWA app, there often isn’t a refresh button.  Particularly when you’ve added it to your iPad home screen.  There are no buttons of any kind. And the person using the app might not even know it is out of date to begin with.  

This can be a real problem, and I’m not about to say I have the solution for every device and every OS version.  That’d be a little impractical.  But we can at least try to do a few things to help out with this.  The first thing, though, is to have a way to know what version we’re running.  And if we can get a date to go with that version, that’d be great too. 

In the TMS WEB Project Options, there’s an option for “Version” and “Auto-Increment Version”.  If we use those, we can then get the project version as a value to help us try and see what is happening. However, this is a bit of a tricky thing for the Delphi developer who has been around for awhile.  Because we’re accustomed to using the Delphi project version information.  Been around forever, right?  Well, this isn’t that.  But it is similar enough for our purposes.  When enabled, the version number is used when naming the underlying JS file.  Which is super-helpful as it ensures that each new build references a unique JS filename.  Something similar can be setup if you find yourself constantly editing CSS files as well, but hopefully that’s just at the start of your project.  What we’d like though is to have that version number appearing somewhere in our project.

Another thing that we’d like is to have the date.  There are lots of considerations here, but what might work is to use the date that comes along with the files from the web server.  So whenever you update your production website files, for example, their new timestamps can be used to provide a release date that we can then display.  This works for Apache, but not sure if it is universally workable in the same way.  For formatting the date, let’s also just sneak over and use Luxon because, well, we’ve covered it plenty already.  The resulting code looks like this.

procedure TForm1.WebFormCreate(Sender: TObject);
var
  Version: String;  // Version from project
  Release: String; // Date from webserver
begin
  // Version information
  asm
    Version = ProjectName;
    Release = new luxon.DateTime.fromISO(new Date(document.lastModified).toISOString()).toFormat('yyyy-MMM-dd');
  end;
  console.log(Copy(Version,1,Pos('_',Version)-1)+' Version '+RightStr(Version,4)+' Released '+Release);
end;

Note that somewhere in Project.html we have to sneak in this line so that we can use the value later.  It could also very likely be that this is defined somewhere more accessible.

ProjectName = "$(ProjectName)";

The result should be a line in the console that looks somethign like this.

ActoriousClient Version 1511 Released 2022-Jul-07

But naturally there are plenty of ways to further customize this.  For our current problem, let’s just make sure that it is visible somewhere in the app.  There’s a Support section in the Actorious app, so it can just be added there by changing a TWebLabel.  Something like the following would work.

  lblSupport.HTML := 'Support and discussions about Actorious are managed through Discord, a free service.'+
   '
'+
   '
'+
   Copy(Version,1,Pos('_',Version)-1)+
   '
'+
   'Version: '+RightStr(Version,4)+
   '
'+
   ' Released: '+Release;

With that in place, now whenever we release a new version of the app, we can at least see what it is in the client without having to look for whatever it was that was changed in the new version. Like, you know, the version information being added to the project 🙂  In any event, with this added and then the update deployed, we should expect to see it everywhere, right?

• Checking desktop Firefox, visiting the page shows that it is already updated. No trouble here.
• Checking iPad Safari, after a simple refresh, it seems to be updated already. 
• Loading the app via the home page icon, it is not updated.
• Force-quitting the app and restarting it doesn’t fix the problem either.
• Deleting the home page icon and adding it again via Safari does seem to work consistently.

So that’s a bit of a nuisance, to say the least.  Honestly, though, sometimes you can make a change in the project, run it, and even Google Chrome won’t bother to reload the project with the new JS file. To get at the issue more directly, let’s consider two problems.  First, the app needs some way of knowing there’s an update available.  Having the version information in the project doesn’t do us much good if the project isn’t updated, but at least we can compare that to something else. And then, once we know the version is out of date, we need a way to actually flush the cache and perform a reload to get the new version.  

 if (MainForm.VersionCheck = 0) then
  begin
    try
      Response := await(MainForm.Client.RawInvokeAsync('IActorInfoService.GetClientVersion', []));
      Data := string(TJSObject(Response.Result)['value']);
      console.log('Server Version: '+Data+' Current Version: '+IntToStr(MainForm.VersionNum));
      if StrToIntDef(Data,0) > MainForm.VersionNum then
      begin
        console.log('Version is out of date.');
        asm
          if ('serviceWorker' in navigator) {
            navigator.serviceWorker.getRegistrations().then(function (registrations) {
              for (let registration of registrations) {
                registration.update()
          }})}
          window.location.reload(true);
        end;;
      end
      else
      begin
        console.log('Version is up to date.');
      end;
    except on E: Exception do
      begin
      // don't care
      end;
    end;
  end;

Some caveats with this approach.  If you set the value returned by GetClientVersion to a number greater than what the client can update to, it will loop endlessly.  Also, deciding when to do the check is important.  Having it run when the app starts makes sense and might not even be noticed, but “start” means different things on mobile devices. So having another way to trigger it would help.  In the Actorious app, as an experiment,  VersionCheck is set to 0 when the user checks the version number, which will then lead to a reload.  And Walter’s code seems to do a good job of flushing out enough that the reload actually works. This indeed leads to an updated iPad app version without having to remove and re-add the home screen icon.  

What Do You Think?

That about covers our introduction to PWAs.  What do you think?  There’s always more to the story.  Is there an aspect of PWAs that needs to be explored further?  Are there other TMS WEB Core topics that came to mind while reading through this that warrant more attention? Lots of topics to cover so let me know what you’re interested in and we can head in that direction!

Andrew Simard.