Video and Audio Processing API

Overview

Currently, depending on the device in use, video and audio files can be saved in different formats. Our video and audio transcoding features help with converting the files to the type and format you would like to use in your application.

Video and Audio Transcoding are included on the Grow and higher plans.

How to get started

In order to start working with our Video and Audio transcoding feature you need to implement Webhooks. As it is described below, the video conversion process is an asynchronous task, and a response is not sent immediately. In order to receive the transformation results, there is a requirement to set up the endpoint that will be connected to our services. This can be implemented by logging into the Developer Portal -> Configurations -> Webhook tab.

Polling the URL will eventually yield a response with populated results.

Video transcoding process

Most of our transformations are synchronous tasks for which the results are sent immediately. Video and audio transcoding is an asynchronous process that returns job metadata and is divided into the three steps listed below:

I. request with video_convert task

II. an initial JSON response

III. final JSON response

First step - Transcoding request

Both video and audio require video_convert task in order to be transcoded, the basic example request looks like this:

https://cdn.filestackcontent.com/video_convert/HANDLE

You can add several parameters/options listed below, depending on your needs:

https://cdn.filestackcontent.com/video_convert=<params1>,<params2>/HANDLE

Second step - Initial JSON response

Initial JSON response should be returned immediately after the request is sent. It contains the UUID - the unique conversion identifier using which you can match the request with the final response (from the third step):

{
    "data":{},
    "metadata": {
        "result":{},
        "source":{}
    },
    "status":"started",
    "timestamp":"1522783969",
    "uuid":"5c0f33cc9c464e16592b7a01f6f8f5c1"
}

JSON response for video file:

{
    "data":{
        "thumb":"https://cdn.filestackcontent.com/HANDLE",
        "thumb100x100":"https://process.filestackapi.com/APIKEY/resize=w:100,h:100,f:crop/output=f:jpg,q:66/https://cdn.filestackcontent.com/HANDLE",
        "thumb200x200":"https://process.filestackapi.com/APIKEY/resize=w:200,h:200,f:crop/output=f:jpg,q:66/https://cdn.filestackcontent.com/HANDLE",
        "thumb300x300":"https://process.filestackapi.com/APIKEY/resize=w:300,h:300,f:crop/output=f:jpg,q:66/https://cdn.filestackcontent.com/HANDLE",
        "url":"https://cdn.filestackcontent.com/HANDLE"
    },
    "metadata": {
        "result": {
            "audio_bitrate":128,
            "audio_channels":1,
            "audio_codec":"aac",
            "audio_sample_rate":44100,
            "created_at":"2018/04/03 20:15:50 +0000",
            "duration":5615,
            "encoding_progress":100,
            "encoding_time":2,
            "extname":".mp4",
            "file_size":150791,
            "fps":30,
            "height":260,
            "mime_type":"video/mp4",
            "started_encoding_at":"2018/04/03 20:15:51 +0000",
            "updated_at":"2018/04/03 20:15:55 +0000",
            "video_bitrate":81,
            "video_codec":"h264",
            "width":350
        },
        "source": {
            "audio_bitrate":83,
            "audio_channels":1,
            "audio_codec":"aac",
            "audio_sample_rate":48000,
            "created_at":"2018/04/03 20:15:50 +0000",
            "duration":5568,
            "extname":".mp4",
            "file_size":383631,
            "fps":30,
            "height":320,
            "mime_type":"video/mp4",
            "updated_at":"2018/04/03 20:15:55 +0000",
            "video_bitrate":465,
            "video_codec":"h264",
            "width":560
        }
    },
    "status":"completed",
    "timestamp":"1522786635",
    "uuid":"UUID"
}

JSON response for audio file

{
   "conversion_type":"audio",
   "data":{
      "url":"https://cdn.filestackcontent.com/0jSokSPdTIq0vxRknqtY"
   },
   "metadata":{
      "result":{
         "audio_bitrate":192,
         "audio_channels":2,
         "audio_codec":"mp3",
         "audio_sample_rate":44100,
         "created_at":"2021/02/26 10:10:06 +0000",
         "duration":57182,
         "encoding_progress":100,
         "encoding_time":5,
         "extname":".mp3",
         "file_size":1373040,
         "mime_type":"audio/mpeg",
         "started_encoding_at":"2021/02/26 10:10:09 +0000",
         "updated_at":"2021/02/26 10:10:16 +0000"
      },
      "source":{
         "audio_bitrate":128,
         "audio_channels":2,
         "audio_codec":"aac",
         "audio_sample_rate":44100,
         "created_at":"2021/02/26 10:10:06 +0000",
         "duration":57144,
         "extname":".mp4",
         "file_size":7461775,
         "fps":23.976,
         "height":720,
         "mime_type":"video/mp4",
         "updated_at":"2021/02/26 10:10:16 +0000",
         "video_bitrate":913,
         "video_codec":"h264",
         "width":1280
      }
   },
   "status":"completed",
   "timestamp":"1614334206",
   "uuid":"5688a02e2f2ebf2d45e26a3ca92400dc"
}

Re-run video transcoding

video_convert=force:true

If you would like to re-run the failed conversion, you can use force:true option. Please make sure to add it only once to your URLs, otherwise, the new conversion will be generated every time you will send the request

Get video/audio file metadata

video_info

If you would like to receive the metadata information of the video/audio files you can use video_info task.

Example response:

{
   "audio":{
      "bit_rate":128045,
      "channels":2,
      "codec_id":"mp4a-40-2",
      "format":"AAC",
      "sampling_rate":44100
   },
   "codec_id":"mp42",
   "duration":57.144,
   "file_size":7461775,
   "format":"MPEG-4",
   "video":{
      "bit_rate":913519,
      "codec_id":"avc1",
      "format":"AVC",
      "frame_rate":23.976,
      "height":720,
      "width":1280
   }
}

Supported formats

Video transcoding options

Presets and codes

video_convert=preset:<preset>/handle

Depending on the device in use, different video file types can be uploaded. If you would like to make sure that all videos that you are using within your application have the same format, you can transcode them to one of the supported formats/codes.

Video properties

We have several parameters that allow you to add modifications to the output video dimensions and ratios.

Width

video_convert=width:<width>

Example:

 https://cdn.filestackcontent.com/video_convert=width:300/handle

Output:

Height

video_convert=height:<height>

Example

 https://cdn.filestackcontent.com/video_convert=height:300/handle

Aspect mode

video_convert=aspect_mode:<aspect_mode>

If you would like to specify the aspect mode of the outputted video you can use `aspect_mode` task and one of the following options:

Examples

video_convert=aspect_mode:preserve

https://cdn.filestackcontent.com/video_convert=aspect_mode:preserve/handle

Output:

video_convert=aspect_mode:pad

https://cdn.filestackcontent.com/video_convert=aspect_mode:pad,width:300,height:100,access:private/handle

Output:

Video file modifications

When it is required to change the file extension, video length or title, you can modify the video file by using one of the tasks below.

Extname

video_convert=extname:.<extension>

Note: In order to change the file extension, extname name needs to be included and the extension string needs to begin with “.” .

Example

 https://cdn.filestackcontent.com/video_convert=extname:.mp4/xBfYjbEJT0Wpkh2s0MK0

Title

video_convert=title:'My title'

Allows changing the title of output video/audio.

Example

 https://cdn.filestackcontent.com/video_convert=title:'My title"/xBfYjbEJT0Wpkh2s0MK0

Length of the video

video_convert=clip_length:<sec>

Allows changing the length of the output video. Need to be set as timestring(hh:mm:ss)

Example

 https://cdn.filestackcontent.com/video_convert=clip_length:00:00:01/xBfYjbEJT0Wpkh2s0MK0

Output:

Set the offset for your clip

video_convert=clip_offset

Clip starts at a specific offset.

Examples:

Output video starts on the 5th second.

https://cdn.filestackcontent.com/video_convert=clip_offset:00:00:05/handle

Output:

Output video start beginng on the 5th second and take 5 seconds:

https://cdn.filestackcontent.com/video_convert=clip_offset:00:00:05,clip_length:00:00:05/handle

Output:

Video Watermark

video_convert=watermark_*

In order to add watermark to the output video you can add one of the watermark tasks:

Example:

https://cdn.filestackcontent.com/video_convert=watermark_url:"https://cdn.filestackcontent.com/resize=w:200/watermark_handle"/handle

Output:

Video quality

Increasing the quality of the video is possible with Filestack video transcoding tasks.

Upscale

video_convert=upscale:true

Upscale the profile resolution to match profile (settings). Default set to true.

Video bitrate

video_convert=video_bitrate

Bitrate stands for the number of bits per second. It is a value of the quality of the video and audio files. It correlates positively with the file size, the higher the bitrate, the better the quality, and the larger the file size.

Example:

https://cdn.filestackcontent.com/video_convert=video_bitrate:8000/handle

Generating thumbnails for video files

In order to generate the thumbnail of the video file you can use one of the options listed below:

1. Run video transformation and wait for the webhook with the video transcoding result - you should find 3 thumbnails for the video in the response, for example:

{
   "thumb":"https://cdn.filestackcontent.com/tKMN5uTdTXqQHjMdmHgU",
   "thumb100x100":"https://process.filestackapi.com/AGGzy5zr9RNKuVEmf55MDz/resize=w:100,h:100,f:crop/output=f:jpg,q:66/tKMN5uTdTXqQHjMdmHgU",
   "thumb200x200":"https://process.filestackapi.com/AGGzy5zr9RNKuVEmf55MDz/resize=w:200,h:200,f:crop/output=f:jpg,q:66/tKMN5uTdTXqQHjMdmHgU",
   "thumb300x300":"https://process.filestackapi.com/AGGzy5zr9RNKuVEmf55MDz/resize=w:300,h:300,f:crop/output=f:jpg,q:66/tKMN5uTdTXqQHjMdmHgU",
   "url":"https://cdn.filestackcontent.com/sDebThSHWuzY4TnPZeUg"
}

2. Run video conversion with `preset:thumbnail`:

https://cdn.filestackcontent.com/video_convert=preset:thumbnail/handle

Output:

You can also add `thumbnail_offset` task to your request with one of the options below, in order to decide which clip should be chosen as the thumbnail:

Example:

https://cdn.filestackcontent.com/video_convert=preset:thumbnail,thumbnail_offset:5/handle

Output:

Muting audio track

video_convert=mute_audio_tracks

Two pass encoding

video_convert=two_pass

Key frame

video_convert=keyframe_interval

Video effects

We currently offer an easy way to improve your videos by adding video effects.

Request and response

Below you can find the guide of how to run video effects and receive the responses.

https://cdn.filestackcontent.com/<APIKEY>/<video_effect_task>/handle

https://cdn.filestackcontent.com/<APIKEY>/<first_effect>/<second_effect>/handle

https://cdn.filestackcontent.com/<APIKEY>/video_status=uuid:<UUID>/handle

Available effects

https://cdn.filestackcontent.com/video_boomerang/handle

https://cdn.filestackcontent.com/video_blur=amount:90/handle

https://cdn.filestackcontent.com/video_reverse/handle

https://cdn.filestackcontent.com/video_speed=tempo:0.5/handle

https://cdn.filestackcontent.com/video_noise=l:100/handle

https://cdn.filestackcontent.com/video_contrast=value:100/handle

https://cdn.filestackcontent.com/video_fadein=start:1,duration:5/handle

https://cdn.filestackcontent.com/video_brightness=value:100/handle

https://cdn.filestackcontent.com/video_saturation=value:100/handle

https://cdn.filestackcontent.com/video_vignette=angle:30/handle

https://cdn.filestackcontent.com/video_grayscale/handle

https://cdn.filestackcontent.com/video_noaudio//handle

https://cdn.filestackcontent.com/video_watermark=x:0,y:0,file:qOuSRuruTYCpWGXp9Yu3/handle

https://cdn.filestackcontent.com/video_watermark=x:0,y:0,text:Thanks for choosing Filestack!,color:red,shadow_color:grey/handle

https://cdn.filestackcontent.com/video_delogo=width:700,h:100,x:1,y:1,show:true/handle

https://cdn.filestackcontent.com/video_crop=width:800,height:500,x:100,y:0/handle

https://cdn.filestackcontent.com/video_progress_bar=height:100/handle

https://cdn.filestackcontent.com/video_progress_bar=height:50,placement:around/handle

https://cdn.filestackcontent.com/video_ambient_fill=width:500,height:500/handle

https://cdn.filestackcontent.com/video_vintage/handle

https://cdn.filestackcontent.com/video_auto_crop/handle

Video streaming options

Currently we offer two presets that allows creating the video_playlist: hls and dash presets.

When using the HLS/DASH preset, many files will be generated in your storage container and a special processing engine link will be returned to the playlist file:

https://cdn.filestackcontent.com/video_playlist/HANDLE

The video_playlist task generates the playlist dynamically so that all the segments in the playlist can have the same security policy and signature applied in the request if security is enabled for your API key.

Streaming tasks

Example

https://cdn.filestackcontent.com/<APIKEY>/video_convert=preset:dash.variant.playlist,quality:[360p,720p,4000x4000@60]/handle

Audio transcoding options

Both video and audio files can be transcoded using our services. We support variety of audio formats as an input. In this case also the video_convert task is required with one of the options below.

Audio codes

video_covert=audio_codec:<audio_codec>

Audio sample rate

video_convert=audio_sample_rate

Audio bitrate

video_convert=audio_bitrate

Audio channels

video_convert=audio_channels

Frame range

video_convert=ftp

Video and audio store options

All the video results are stored in our default S3 or in your custom bucket. Using the store option listed below, it is possilbe to change the storage location.

Video and audio cost calculation

Each second of audio or video transcoding counts againts transformations. We are calculating video based on the output file. we use the following ratios to convert the length and quality of video/audio transcoding requests from your account to conversions:

1. standard def (SD) transcoding: 2 transformations per second
2. high def (HD) transcoding: 3 transformations per second
3. ultra high def (UHD) transcoding: 6 transformations per second
4. audio transcoding: 3 transformations per second.

Please take your plan’s conversion limits into account before processing any audio, as you can quickly exceed your plan’s limits and incur overages.

Our system is recognizing video output file based on the video height: