Stream content to Google Cloud Storage

In this project, I need to get videos from Vimeo to Google Cloud Storage in order to run the Video Intelligence API on them.  There are a number of ways to do that, but I’d like to simply stream them directly using a cloud function.
This article looks at the Node implementation for streaming directly to cloud storage, and we’ll look at how to covert that to a Cloud run function in a later one. Note that although I’m streaming videos, this technique could work with any kind of file.

Credentials

First we need a service account, a bucketname and a folder that will be the target for the video files loaded from Vimeo. I’ve covered getting service accounts elsewhere so let’s assume you already have a downloaded service account .json file. I keep all my configs on one place, organized like this, with different configurations for different scenarios. These are referenced in the code so here’s a redacted version

Vimeo download files

I have a note of these in my database, but for simplicity I’ll just name the video file i want to load in my app. You’ll want the downloadable (as opposed to playable) distribution file. If like me, you plan to run the video intelligence API, the higher the quality, the more accurate the labelling will be.

Dependencies

There are not many – you’ll need to install them with npm or yarn.

Local dependencies

In my case, I’ll need to set the  mode I’m running in (would normally do this through environment variables, but for this demo I’m setting a global) and pick up my configuration secrets.

Setting up streaming from vimeo

The idea here is that we do an http request to the vimeo download link and immediately pipe the response to another stream which is connected to a cloud storage file.

Setting up streaming to Cloud Storage

By assigning a blob to a cloud storage file and associating a writestream with that blob, the data is magically piped from the input http stream.

Kicking it off

Just give it a file name on cloud storage, and the input url and that’s all there is to it.