-*- text -*-

Synchronization in Flumotion relies on one thing only, which is ensuring
that data that is captured is timestamped relative to the same clock. If
the data has correct timestamps, then a muxer will be able to combine
streams in the proper order, ensuring proper muxing, and when the stream
is played back it the timestamps will be correct as well.

To understand how to produce timestamps, one must know how GStreamer
timestamps its data:

  timestamp = clock time - base time

To ensure that the timestamp of buffers captured on different machines
are synchronized, then we must have the same "base time" and the same
clock time.

In practice, this means that components that need synchronization need
to have the same clock. (Note that not all components need
synchronization -- e.g. the muxer does not need a clock to do its job.)
One clock should be chosen as the master, then Flumotion tells the other
components to slave their clocks to the master using the
NetTimeProvider/NetClientClock GStreamer interface.

The question becomes, how to determine which components need
syncronization, and how to choose the master clock? Whether or not the
component needs syncronization is a property of the component type --
all soundcard producers, for example, need to have their clocks
synchronized with other capture streams. Thus this property belongs in
the registry. We choose to represent this as

  <synchronization required="yes" />

in the component XML node, so that when the full registry is written out
there can be required="no" lines for components that do not need
synchronization.

The question of which master to choose is a more difficult issue. Even
in the simple case of synchronizing two sound cards, which is the same
component type, the user might have a preference that one card's clock
be preferred over the other. The user should be allowed that choice,
which implies that the choice of the master clock should be able to be
specified in the configuration XML (rather than the registry XML).

In the configuration XML, it might look like this:

  <component name="audio-source" type="audiotest-producer" worker="localhost">
    <clock-master>yes</clock-master>
    ...
  </component>

However, if the user does not specify a clock master, Flumotion needs to
choose a clock master on behalf of the user. It does this by finding the
components that need syncronization, and selecting the component with
the highest clock-priority as the master. The default priority of a
component is 100. Components can override this in the registry e.g. as:

  <syncronization required="yes" clock-priority="150"/>

This will allow audio clocks to be chosen over video clocks in the
normal case.

(This discussion is halfway implementation and halfway useful -- would
be nice to take out the part about how timestamps are calculated and
just leave the practical details, leaving the implementation information
to another file -- we don't talk about how it is implemented anyway...)

In each component, we check for discontinuities in the buffers it receives
on all its eaters, in time and in offset. This of course has no effect on
producers as there are no eaters there. A discontinuity in time means that
the buffer timestamp is not equal to the previous buffer's timestamp +
the previous buffer's duration.  A discontinuity in offset means that the
buffer offset is not equal to the previous buffer's offset end.

For eaters that receive raw video or raw audio, we can check for
discontinuities in both time and offset. Raw video and raw audio buffers
have offsets measured in frames and samples.
For video, a buffer typically has an offset end equal to offset + 1.

For eaters that receive encoded video or audio, we can check for
discontinuities in time only.

For eaters that receive muxed data, we can check for discontinuities in
offset only.  Muxed data buffers have offsets measured in bytes or packets.
The timestamps however can for example alternate between the time of the
audio or video packet, and thus the sum of all durations would end up being
twice the size of the total stream.

Timestamps generated by producers
---------------------------------

dvb-producer has 3 feeders: video, audio and mpegts. It takes input from a
DVB card and decodes it into raw audio and video. The new segment event
that is set on the gdp's stream headers for the video and audio feeders has
a start value as the timestamp in the MPEG transport stream. This can be
a value between 0 and approximately 26 hours. The first buffer on the video
and audio feeders starts close to that time.

firewire-producer has 2 feeders: video and audio. It takes input from a DV
camera and decodes it into raw audio and video. The new segment event that is
set on the gdp's stream headers for the video and audio feeders has a start
value of 0. The first buffer on the video and audio feeders start close to 0.
Firewire/DV is meant to guarantee that the audio can only drift slightly
relative to the video frame rate, so it will not drift out of sync over time.
However we have had instances of firewire converters that take analogue and
give a DV out produce desynchronized DV.  So it is only as good as the
hardware.

