Ephemeral Exchange

Sometime you need to exchange transient data between platforms, but they don’t talk to each other very well. For example you could be launching a webapp with some data generated from Excel, but you want to get feedback data back into Excel again. You could use a database, or push/pull solutions, but it can be complex to set up, and the authentication is often difficult to get right.

NOTE – as of 20 Nov 2017, Ephemeral exchange is migrating to a Firestore/ Firebase cloud function back end.

  • Migrating from App Engine to Cloud Functions

What is Ephemeral Exchange

It’s a JSON API to exchange ephemeral data that doesn’t need each platform to authenticate, but relies on using keys that are known by each collaboration party, and issued by the API. Apps knowing the item id, and appropriately authorized access keys can use them to retrieve data written by an entirely different platform (even using a web browser)

Demo 

and here’s another showing how to use streetview and generate slides with selected views.

Demo of the JSON editor

Demo of push notification

 

Hosted or open source

The code is open source, but I’m also providing a hosted solution on Google Cloud Platform. All you need to do is register, get a boss key, and you are ready to go. The hosted solution supports CORS so you can use it even from client side JavaScript without worrying about Cross Origin request issues.

Code

Most of the demos and libraries mentioned in these pages are available in their own github repo

Architecture

NOTE – as of 20 Nov 2017, Ephemeral exchange is migrating to a Firestore/ Firebase cloud function back end – this is the original App Engine/Redis back end which will be retired shortly

 

 

 

Demo for push notification on Google Sheets add-ons

NOTE – as of 20 Nov 2017, Ephemeral exchange is migrating to a Firestore/ Firebase cloud function back end – this is the original App Engine/Redis back end which will be retired shortly. These are legacy links for info

Security

The hosted solution is of course public, but the data is transient – It only allows data to persist for a short period of time. There is a lot of security built into the access solution, some of which is described below. The key thing is that permissions are on a single item level, so you can allow an app to access one item in your store, but not another.

Access Keys

Keys are used to identify yourself and to enable you to access the exchange. They expire after some period of your choosing, and are associated with a particular registered account, and different kinds of keys can do different things. 

Boss key

It looks like this, and you can use it to generate writer and reader keys for your account. You might want to generate multiple keys so you could give them different expiry times and use different keys to allow access to specific items or types of items.bab-2f8-15c6get53b36

Here’s an example of generating 2 reader keys that will expire after 2 days, plus the response

 

Looks like this, and you can generate them to share with apps and others who you are allowing to read  data items.

 

This example is using a write key to write some data, and is also assigning some keys that can read that data (Note that in these example I’m using GET to write small amounts of data. This makes it possible to use the Browser or other platforms that don’t allow POST, but normally data is written with POST)

and here is one of these keys issuing a read request for that item

 

Writer key

Looks like this, and you can generate them to share with apps and others who you are allowing to update data items.

wak-h4l-b1235bf5f3d6

and here is the item being updated by that other write key

Item keys

When an item is written to the store it generates a unique id, which is associated with your account. You can also specify a lifetime for that item, after which it expires. Every item has a default lifetime of 30 minutes. Before an item can be accessed, you need the item id, along with the access key that created it plus the lock if there is one, and also the key has to be one that is associated with your account. The only key that can read an item by default is the one that created it. However, you can add a list of reader keys that can access an item, or writer keys that can update that item. A public item key looks like this.

 

dx1fa-ex4-bdasea1fpak1

Alias

Sometimes it’s useful to use a consistent name to access some data, even though the underlying data may change. To do this you can assign an alias. An alias is actually something that belongs to a specific key, for example a reader key, and different physical items can be ‘slotted into that name’. Imagine that you had a task that regularly needed to find some data written by some other process – let’s call it stock-price. 


The task that had the responsibility for creating that data item regularly would do something like this

and would get this response

Assuming that the receiving task already knew the updater key as part of a standard configuration, the problem here is that it won’t know the item key. In some cases, where the receiving task is initiated by the creating task, passing the transient item key would be ok, but in other cases, where there is no direct communication, an alias can be used. 

First the creating task assigns a standard alias that is known by the receiving task (in our case stock-price) to the newly created data item. For this he needs the writer key to be able to write to the store, the updater key who is going to use the alias and the item id key to assign to the alias.

 

Notes for Apps Script developers, and comparison with Apps Script CacheService

If you are Apps Script developer, you probably use the CacheService. You can use Effex instead of that if you want, but here are the main differences. 

  Apps Script Effex
Scope To a user, document, or script. Cannot be used across multiple scripts Can be used across any platforms, so has no scope boundaries.
 Performance
 
Fast – about 30ms, integrated into Apps Script platform Depending on your location, about 200ms. Runs on Google infrastructure, but is not Apps Script.
Rate limiting The CacheService seems to be free of rate limits Since you would use UrlFetchApp to access it from Apps Script, you will be using UrlFetch quota and would hit UrlFetchApp rate limits well before the Effex rate limits kicked in.
Maximum item size 100k max The free tier , 512k. Other tiers not yet available
Lifetime 6 hours max  The free tier, 12 hours max. Other tiers not yet available
Analytics none Full usage data by operation and size to the individual key level

In summary, for small caching requirements that are limited to within a single Apps Script, the apps script caching is a better choice. If you need to extend across multiple scripts, or share with other platforms you could consider efx.

For more like this, see Google Apps Scripts snippets

Subpages: