Video Tagging

Detect general features of your videos.

Overview

Video tagging is one of intelligence tasks in Filestack platform. It allows you to detect general features in your videos.

This task is only available at Filestack Workflows. Read more about Workflows here.
By default, Video tags and Video SFW tasks support videos that are up to 10 minutes long. Please contact our Sales Team if you are interested in increasing this limit.

Workflows Task Configuration

Video Tagging task is available in the Intelligence tasks category in Filestack Workflows. To learn more about how to use the Workflows UI for configuring tasks (and the logic between them), visit Creating Workflows Tutorial.

Workflows Parameters

Task Name string Unique name of the task. It will be included in the webhook response.
Details boolean false Indicates if a detailed response should be returned.
As file boolean false Indicates if response should be returned as a file.

Webhook

Here is an example of a webhook that you will receive once your video is processed. details key is only present when Details parameter set to true:

{
    "id": 88085450,
    "action": "fs.workflow",
    "timestamp": 1570438682,
    "text": {
        "workflow": "1401680c-1234-81b4-bae3-04166d1a96ee",
        "jobid": "2c22748c-ce65-4979-8071-530b467e4d8b",
        "createdAt": "2019-10-07T08:50:15.001357914Z",
        "updatedAt": "2019-10-07T08:57:42.180333126Z",
        "sources": [
            "Yq31nO8kbj6ijRJGvTE"
        ],
        "results": {
            "video_tags_task": {
                "data": {
                    "tags": {
                        "user": null,
                        "auto": {
                            "electrical device": 94,
                            "face": 88,
                            "head": 51,
                            "human": 99,
                            "lecture": 51,
                            "man": 58,
                            "microphone": 94
                        }
                    },
                    "details": {
                        "tags": [
                            {
                                "confidence": 81,
                                "start_offset": 0,
                                "end_offset": 4571,
                                "name": "face"
                            },
                            {
                                "confidence": 92,
                                "start_offset": 1220,
                                "end_offset": 5234,
                                "name": "face"
                            }
                        ]
                    }
                }
            }
        },
        "status": "Finished",
        "ttl": 172800
    }
}

Because the response can get large for longer videos, you can receive video processing results as a file by setting As file parameter to true. In this case, webhook will look like this:

{
    "id": 88085450,
    "action": "fs.workflow",
    "timestamp": 1570438682,
    "text": {
        "workflow": "1401680c-1234-81b4-bae3-04166d1a96ee",
        "jobid": "2c22748c-ce65-4979-8071-530b467e4d8b",
        "createdAt": "2019-10-07T08:50:15.001357914Z",
        "updatedAt": "2019-10-07T08:57:42.180333126Z",
        "sources": [
            "Yq31nO8kbj6ijRJGvTE"
        ],
        "results": {
            "video_tags_task": {
                "url": "https://cdn.filestackcontent.com/YOUR_APIKEY/wf://1401680c-1234-81b4-bae3-04166d1a96ee/2c22748c-ce65-4979-8071-530b467e4d8b/acf95aff157ab3c950f1f71a682b68f5",
                "mimetype": "application/json",
                "size": 8978
            }
        },
        "status": "Finished",
        "ttl": 172800
    }
}

Please visit the webhooks documentation page to learn more.

Logic

Video Tagging can be used in Workflow logic only when it was configured with As File parameter set to false
Video Tagging task returns following response:

{
    "data": {
        "tags": {
            "auto": {
                "dog": 98,
                "sky": 82
            },
            "user": null
        },
        "details": {}
    }
}

Logic Parameters

data dictionary Indicates the response from this service.
tags dictionary Contains "auto" and "user" tags.
auto dictionary Includes all of detected general features from the image with their corresponding confidence numbers.
details dictionary This element is not useful in workflows logic.

Based on the task’s response you can build logic that tells the workflow how dependent tasks should be executed. For example, if you would like to run another task if a specific key was detected you can use the following rule:

tags.auto kex "sky"

In Workflows UI this command would look like in the example below:

Visit Creating Workflows Tutorial to learn how you can use Workflows UI to configure your tasks and logic between them.

Video Tagging Result

Below you can see how the response is structured. tags.auto key contains each tag that was detected in the video with its highest confidence.
More detailed information can be found under details.tags key. Each item contains a name, confidence and a start/end offset (time, in milliseconds from the start of the video, that the tag was detected).

{
    "tags": {
        "user": null,
        "auto": {
            "electrical device": 94,
            "face": 88,
            "head": 51,
            "human": 99,
            "lecture": 51,
            "man": 58,
            "microphone": 94
        },
    },
    "details": {
        "tags": [
            {
                "name": "human",
                "confidence": 93,
                "start_offset": 12379,
                "end_offset": 15382,
            },
            {
                "name": "microphone",
                "confidence": 89,
                "start_offset": 22134,
                "end_offset": 26233,
            }
        ]
    }
}