Timeline items

[Joery-M]

This document explains what 'timeline items' represent in the application.

Timeline items are the representation of elements that are rendered at a specific time. Timeline items dictate when an element is rendered, what effects are applied to the rendering step, and how the item can be interacted with (See transitions in effects (#291)).

Timeline items don't inherently define what media is played and when, that job is delegated to effects (#291).

Timeline items are not specific to video or audio, they are generic. This is also represented in the user interface, there is no split between audio and video, opting instead for using the sources defined on a timeline item.

Effects

A timeline item has a list of effects that are executed to compose an image or to modify an audio stream.

When rendering an image:

1. The source effect is executed to receive data about width and height of the media.
2. The item iterates over the remaining effects and executes the ones that are designated to modify images.
   • Each effect can modify a transformation matrix to alter the position, scale, rotation or skew of the end result. Transformations should always be relative to the last value.
3. The item takes the completed image and applies the transformation and any other effects required for composition, like blend mode.

When processing audio:

1. The source effect is executed to receive base audio data.
2. The item iterates over the remaining effects and executes the ones that are designated to modify audio.
   • Note: Experiment with how gain is handled to avoid clipping (if that even happens)
3. The item caches the processed audio chunk.

Reference

export interface TimelineItem {
    /**
     * Name provided by source effect
     *
     * @default "Blank item"
     */
    name: string;


    /**
     * What type this timelineItem should represent.
     *
     * In order:
     * @returns `Video` when a source contains video.
     * @returns `Audio` when a source contains audio.
     * @returns `Image` when a source is an image.
     * @returns `EffectLayer` when this timeline item contains no source effects,
     *          which will apply the other effects to subsequent layers.
     * @returns `Base` when no other option applies.
     */
    type: string;
    effects: Effect[];
    start: number;
    end: number;
    layer: number;
}

Storage

A timeline item has the following properties as a base:

• ID: string
• Name: string
• start: number
• end: number
• layer: number
• effects: A list of effects defined in effects (Effects #291)