This post isn't going to be a comprehensive introduction to MLT, the media lovers' toolkit. Instead, it's rather kind of a layman's memo as an aid to roughly understanding some basic of MLT.
MLT Concepts
At it's core, MLT has these four types of what it terms services:
- producers: these are the sources of video and audio frames, and they come as either basic producers or more complex ones, as we'll see below.
- consumers: these are sinks for video and audio frames. Examples are a screen display, or an audio/video encoder that writes different types of audio/video container files.
- filters: these are what Kdenlive terms effects instead; MLT filters sit somewhere in between producers and consumers, so they can modify frames as they are passing through them.
- transitions: these are the same as in Kdenlive, albeit the term transitions confuses a lot of Kdenlive users – to say the least. Simply spoken, transitions combine two input frames into a single output frame. (Yes, I know that technically, transitions still output two frames, with the so-called B frame coming out unmodified, but that doesn't help us here.)
Producers
The producers can be even further differentiated:
- basic producers: these produce frames directly from an audio/video source, which may be a file, device, or some even color clips, title clips, et cetera.
- playlists: are sequential containers for producers and blank spaces.
- tractors: these are convenience elements for working with tracks, filters, and transitions; as their name suggests, tractors pull frames from producers and then runs them through filters and transitions, with the result then pushed into the consumers.
- multitracks: are what MLT terms parallel containers for multiple producers that act as a single producer, yet produce multiple frame streams. With Kdenlive projects, multitracks are only implicit in that they don't appear in Kdenlive project files, but are created automatically when Kdenlive reads a project.
Basic Producers
Some notable producers in the context of Kdenlive are these here:
- avformat: this producer reads and audio/video file using the famous ffmpeg and then produces frames when asked, such as by a tractor. See also the MLT avformat producer documentation.
- avformat-novalidate: this producer is used to speed up loading playlists with known good media files. (See also this MLT commit message about avformat-novalidate.)
- timewarp: this is a rather new producer that allows to speed up (max. 20x) or slow down (down to 0.01x), and even to reverse, an encapsulated producer. Timewarp does not only handle video, but also audio. See also the MLT timewarp producer documentation. Kdenlive uses this producer internally to emulate the speed effect (which isn't, technically spoken, an MLT filter, but an MLT producer).
- pixbuf: produces frames from a wide range of bitmap and vector graphics formats. Under the hood, it uses the GDK pixbuf, hence its name (see also the MLT pixbuf producer documentation).
- This producer supports image sequences. For this, the resource property (that is, path and filename) needs to contain the wildcard "/.all.". The producer then loads all image files matching the following file extensions from the path.
- Some supported image and file formats are: PNG (.png), JPEG (.jpg, jpeg), SVG (.svg), TIFF, XPM, ...
- kdenlivetitle: given some XML it produces those beautiful title frames. The XML is directly included with this service using an "xmldata" property.
- color and colour: returns frames of a single color. For the internal black track, Kdenlive uses UK black from the colour producer. Any bin color clips use the stars-and-stripes color producer. The color (and also transparency) is encoded in the "resource" property (see also MLT color producer documentation):
- in form of a 32bit hex value in the form of RRGGBBAA. Here, RR is 8bit red, GG is 8bit green, BB is 8bit blue, and AA is 8bit alpha, where FF is opaque and 00 means transparent.
- or a few well-known color names, such as black and white.
Filters
...
Playlists
As their name already suggests: playlists are lists of producers to play out in sequence. You may compare playlists to individual timeline tracks. But that's only one way to see and deploy them. Internally, Kdenlive uses playlists also for several behind-the-scenes tasks, such as:
- muted video clips,
- sped-up or slow motion clips,
- for working around MLT causing audio distortion when mixing the same clip with itself on different tracks.
Tractors
Tractors play a crucial role in that they allow to use multiple producers at the same time, as opposed to playlists, which can deploy multiple producers only sequentially (one after another).
Transitions
Transitions are combining two frames from different tracks. They combine an A and a B frame into a new A frame (as well as an unmodified B frame). A frames come from the so-called A track, B frames from the B track. Usually, the A track is the lower track, whereas the B track is the upper track. Here, lower and upper are to be understood as in Kdenlive's timeline.
In MLT, transitions can only exist inside tractors. So, in contrast to what Kdenlive's graphical depiction of the timeline may suggest, transitions don't belong to tracks or clips.
Now, MLT tractors basically wire up the individual (timeline) tracks into a timeline. By the way, each track is simply a playlist, producing frames. The transitions then specify how to manipulate certain frames from these tracks. Often, transitions only apply to a limited timespan within the timeline only.
One important aspect is missing: how do MLT transitions identify the tracks supplying the A and B frames? MLT uses (track) indices. These indices for the A and B tracks (which supply A and B frames respectively) are 0-based. The 0th track is the first track listed with a tractor. From the perspective of Kdenlive's timeline, the black track with index 0 is the lowest track, but never shown. Kdenlive only depicts tracks from index 1 on.
Please note that the A track index usually is lower than the B track index. Otherwise, MLT simply reverses the A/B semantics, so that the B track is always the upper track, and track A always the lower one. Kdenlive however doesn't like reversed B/A indices and removes such transactions from projects.
