Skip to content

Feature request: SegmentTimeline support for tiled thumbnails

Created by: dimitry-unified-streaming

Currently, the tiled thumbnails implementation in dash.js supports the following XML elements for specifying the segments (i.e. tiled thumbnail pictures) to be retrieved:

  • SegmentBase, where it reads the necessary information from an MP4 initialization segment. (This was added by @tecteun in pull request #2784, some time ago.)

  • SegmentTemplate, with a media attribute having $Number$ identifiers, like in the bbb_with_tiled_thumbnails example:

    <AdaptationSet id="3" mimeType="image/jpeg" contentType="image">
  <SegmentTemplate media="$RepresentationID$/tile_$Number$.jpg" duration="100" startNumber="1"/>
  <Representation bandwidth="12288" id="thumbnails_320x180" width="3200" height="180">
    <EssentialProperty schemeIdUri="http://dashif.org/thumbnail_tile" value="10x1"/>
  </Representation>
</AdaptationSet>

However, it does not yet support SegmentTemplate with a SegmentTimeline sub-element, and a media attribute having $Time$ identifiers. For example, something like this:

    <AdaptationSet
      id="2"
      group="4"
      contentType="image"
      width="672"
      height="400"
      sar="1:1"
      mimeType="video/mp4"
      codecs="jpeg">
      <Role schemeIdUri="urn:mpeg:dash:role:2011" value="main" />
      <SegmentTemplate
        timescale="12288"
        initialization="tears-of-steel-tiled-thumbnails-$RepresentationID$.dash"
        media="tears-of-steel-tiled-thumbnails-$RepresentationID$-$Time$.dash">
        <SegmentTimeline>
          <S t="0" d="737280" r="11" />
          <S d="172032" />
        </SegmentTimeline>
      </SegmentTemplate>
      <Representation
        id="img=4000"
        bandwidth="4000">
        <EssentialProperty
          schemeIdUri="http://dashif.org/guidelines/thumbnail_tile"
          value="3x4">
        </EssentialProperty>
      </Representation>
    </AdaptationSet>

Since dash.js already supports SegmentTimeline for playing regular DASH content, how much effort would be needed to add this to the tiled thumbnails feature?