Taking the web offline – Service Worker (Death of the dinosaur)

If you are not interested in the full story watch the result here.

If I look at my work for parleys.com it seems like the circle is finally closing. When I started working for parleys.com back in 2007 one of my first tasks was to build a desktop application to download presentations so that our users where able to watch presentations on a plane, train or wherever connectivity was an issue. Even though this application was build from the same code base as the website (Flex/AIR) it were still two separate applications. It required us to provide downloads and the user to perform an installation. We needed to care about updating and about platform compatibility issues.

Still at that time it was a really nice use case and felt like „Yeah that`s it!“.

8 years later (8 years!!!) things have changed. Who is downloading applications from the internet anymore? Yeah it still exists but it feels somehow clumsy. The good news, today in 2015 we have all the tools to make these kind of „desktop“ apps obsolete.

Let`briefly look what we need to make a web application run offline and what options we have today.

I would divide the requirements into two groups. First we need a way to make our application shell – so the code that we need to actually run our application available offline. I mean things like static assets, your index.html, js files, images…  And second we need the possibility to store the content data the user needs inside of the application (in our case video, slides and course assets).

Let`s first check the options for storing the data:

Local Storage

Local storage is really only meant as a cookie replacement. It is easy to use but also the options are limited but the biggest problem which ruled it out for our needs is the maximum storage capacity of only around 3-10Mb. Not enough to store videos which combined might be several gigabytes.

IndexedDB

Indexed DB fits ours needs quite well as it allows you to store large data amounts (using quota management and user permissions) and can also store blobs. It is also quite good supported among the ever green browsers. While in theory you can store large data in practice there are known issues when you want to store large blobs in your db. Basically it`s not really possible to append data to an existing blob without reading the existing chunk into memory. So your only option is to have the whole file in memory and save it as one piece. Something which you do not want to do with files several hundred megabytes or even gigabytes large.

Filesystem/File API

So here comes the Filesystem API. The filesystem api is not a W3C standard by now but just a proposal from Google. I have not followed it closely but i have the feeling that it ,might never become a standard. https://hacks.mozilla.org/2012/07/why-no-filesystem-api-in-firefox/comment-page-1/

That`s really unfortunate because it basically solves all our problems with the other apis. As the name implies you have read, write and create access to a sandboxed part of the local filesystem which makes it a perfect fit, if we want to store files. Used with the FileReader API you get random access to files so you can easily manipulate them and append data efficiently. So even with it not being a standard yet, i have decided to go with this option for now. Yes, it is chrome only right now and yes, it is not a standard but you can see it as a progressive enhancement technique. Users on chrome will benefit from the extra functionality and the others can still enjoy the web of yesterday. Also the way I implemented it makes it easy to switch later the filesystem API with Indexed DB (once it deals better with large files) so eventually it will work in more and more browsers.

No let`s examine the options we have for storing our application shell.

Application Cache

On paper application cache gives you exactly what we need and it is around for quite a while. You can define assets in a manifest and those will be cached by the browser but there are many challenges with app cache. It is difficult to manage which assets to cache and when the cache should update. Check out this article for more info.
We actually had a version of parleys for offline use in beta stadium but never released it. I just never felt really confident with the app cache solution we had. You can check out the Devoxx 2012 keynote where we demoed this app if you`re interested in history.

Service Worker + Cache API

What sparked my interest in this web app/offline topic was the introduction of the Service Worker API last year. Service Worker gives you really fine control over network requests. You are able to intercept and transform request all with a pretty simple API. Together with the Cache API you really have full fine grained control over which assets you want to store or what alternative content you want to serve in case e.g. the network is down. There are many approaches how to use Service Workers and Jake Archibald (the man behind the API himself) has a nice collection of usage patterns. The way I am using the SW API for parleys.com really does not do it justice as i am only using it to cache the app shell, but you can go further and improve overall caching for all kind of assets, also content assets and provide fallbacks and improve the overall responsiveness of your application.

Usage on Parleys.com

For parleys.com I have used the Filesystem API to build a full fledge download manager. Files are downloaded in chunks and when a download is interrupted you can always pick it up again when you have connectivity again without restarting the download from scratch.

The Service Worker API is used to serve the application shell so once the app is cached you can shut down wifi go to parleys.com and it still works – it is really a groundbreaking change in my opinion. Well so groundbreaking that it is also kind of an issue. Who knows this? The user is not used to this functionality and if he is sitting in a plane without connectivity he might just not even try to load a web page as he does not expect it to be working. This is really something new where new patterns need to be established. Not yet sure how this can be approached, but in my opinion some thought needs to be put into this. Any suggestions?

After this long introduction let`s have a look at the result:

Issues

Because we were a bit afraid that user behavior and expectation (or lack of expectation) might actually be a problem and because of the chrome only „issue“ we also still investigated other solutions and build a version which can be downloaded as a desktop client. It uses the awesome Electron project which is, like node webkit, based on chromium.

The nice thing is that with the download manager in place without changing one line of code (yeah ok it were 1 or 2) and with the help of this great build packager we have alternative desktop apps (mac/win).

Still I am happy that we will not use them and go with the web version instead. I think this new technology can only be pushed if we use and provide it. Then in 2 years offline web apps will be the norm, maybe like responsive design is today.

Related links:

Parleys Desktop Prototype 2013: https://www.youtube.com/watch?v=RtypJAykq74 (Node-Webkit based)

Flex 4 is Awesome!

Started my first serious app with Flex 4 and i can`t tell how much I enjoy it.
Have not heard so many AHHHS, WOW, F$%CK Yipiiiiiiiee from myself while developing for quite some time.
Don`t really want to add to the fake HTML 5/Flash is dead discussion (have not yet seen a year without this talk :-))
Only thing I can say is that great tooling and proved frameworks really matters to me…

Flex 4 – I love you!

Screenrecording app with AIR 2.0 Beta

Yeah yeah it`s been a while….

But here is something I just have to show. I guess you`ve all played with the awesome AIR 2.0 Beta so you know about the NativeProcess class and all the options we have now.
Basically the only thing you have to do is google for some cool cross platform command line apps and build a nice AIR user interface and you can build beautifull and powerfull desktop apps.
So one of the first things I tried was something that was on my wishlist for the flash platform for a long time – SCREENRECORDING!

This app records the users screen and encodes it to flash friendly mp4 files with h264 video.

Check out this screenrecording of my screenrecording app :-)

I really plan to write an article about how to do this but i`ll never promise anything again :-)

Berlin finally gets a Flex Camp

badge_vertical_highJune 14th (My Birthday by the way ;-)) Berlin (Germany) will get a completely free Flex Camp organized by the Berlin Flex User Group (Flex Labs).
Whether you want to enjoy free presentations and hand on coding session or just come and celebrate my birthday this is definitely the place to be.
Remember we don`t have a Max this year in europe so this is something you should not miss.

Confirmed speakers are: Andre Michelle, Thomas Reppa, Dirk Eismann, Cedric Madelaine, Duane Nickull…

You can`t imagine how much energy the Berlin Flex User Group puts into this event so please show up and make it a success.
I can no longer accept that the Belgiums laugh about our small community :-)

Get the details here: http://www.flex-labs.de/flashcamp

Thanks to Flex Lab for organizing this!

Ahhhhh! And very short notice but well worth it. Tomorrow May 27th they have Andrew Shorten talk about Catalyst and Dirk Eismann about BlazeDAss.
Check out the details here: http://www.flex-labs.de/blog/2009/05/2705-designdevelop-meeting-update/

Loading/Playing mp3s without round-tripping to the server

One of the cool new features of Flash Player 10 is that you can load files from the client without uploading them to the server.
The only limitation is that you`ll only be able to access the raw bytes of the file and there is no way to find out the filepath (see problem discussion here). The problem is that until now we cannot pass mp3 bytes to the sound object or the netstream class so there is no easy way to play mp3s locally on the client (without first uploading them to the server).
So here is a solution that uses the old trick of generating a SWF file in memory which contains the mp3 associated with a linkageId.

Check out my solution here
bild-7Note that some mp3s might not work as the Mp3Reader class is not very optimized but it should show the general steps to chieve the goal.
Let me know if it`s useful and if you could make any enhancements.

Next we`ll look into how to load two mp3s and mix them and save them as a wav (PCM) file.

Thanks Adobe! I`m now a community expert :-)

I got a nice surprise in my email and i`m really honored to be a Adobe community expert now!

Developer Logo
While i`m not yet sure what this really means i`m know it means i should kick my ass and get this blog running again.
I was really lazy with posting here the last months because of a lack of time and the frustration of this blog being hacked.
However i`ll do my best to post a bit more in the coming months.

RichApps has been hacked :-(

Oh well now it happened to me too :-( Richapps was hacked by some turkish political group.
Can anyone explain to me why they do this? So the current status is that i could save the blogs entries and comments but all attachments are gone.
I`ll try to fix some of them but I`ll not be able to recover everything. Let me know in this post if you need something urgently and i`ll try to fix it.

To all the aggregator readers a big apologize! I`m not sure but i`m afraid all the reimported posts will show up again and waste your aggregators.
I`m really sorry for this but i`m out of control on this :-(

For all you Hackers go get a live – for gods sake do your games with big corporates but not with me! There are better targets.
Maybe if you loosers stop doing this you`ll even find a girlfriend or some other joy in your life. I only feel sorry for you bastards.

JavaFX released = what a disappointment

Today Sun oficially released JavaFX 1.0. I was really looking forward for this release but I`m really shocked how bad it actually is. This is purely from a user point of view because i haven`t done any development so far.
One of the biggest problems of Java has always been the install experience so I thought they would do it perfect this time but not so. I think I have never experienced a worse install experience as for the new JavaFX. I went to the sample page http://javafx.com/samples/ and selected one of the samples. First there was nothing happening just a blank screen, then machine freeze for about half a minute then nothing. Went back selected a different example – machine freeze and after some minutes i finally got a window telling me i need to install something and that it`s untrusted content WTF. Ok installed JavaFX and thought now it should be all fine. Select a sample again and now they finally load. It takes unbelievable long to load. I always thought the Flex SDK is big but if this is just unbelievable. Ok when the app finally, after some screen flicking loads it`s unbelievable unresponsive. I really expected JavaFX to be blazing fast compared to Flash Player but it`s just sluggish and feels like good old applets.  After that bad experience I wanted to finally see something cool and checked out the more promising examples. First video playback. What`s cool about this is that it can handle flv files! Really nice! But it`s slow. The video playback stutters extremely on my machine (New MacBook Pro). Went on to the next killer feature example. Dragging an mp3 player from the website to the desktop. Well I couldn`t figure out how to do it :-( Tried to grab and drag every pixel but it just did nothing :-(
The other strange thing is that everytime you scroll a page with an JavaFX applet the applet just disappears.
Seems like they have to disable the applets while scrolling for performance or something but this is just insane.

This really looks like a JFX flame post but it`s all just out of disappointment. I really had high expectations but now they are all gone. I think if you missed the RIA train like SUN did and then take so long for the release you have to make it perfect. Well this is not the perfect release they would have needed IMO.
I really really wanted SUN to succeed with JFX because to me it seemed the only possible alternative to the Flash Platform but seems like I just have to rely on Adobe for a long time (which is not too bad ;-)).

BTW seeing the JavaFX logo and the FlexBuilder logo on my desktop side by side I really hope SUN gets sued for this. It`s just shameless and maybe hints at how innovative they are (not).