Skip to content

Thumbnails support

Vinay Rosenberg requested to merge github/fork/epiclabsDASH/thumbnails into development

Created by: epiclabsDASH

This PR adds support for scrub bar thumbnails following DASH IF IOP spec: http://dashif.org/wp-content/uploads/2017/03/CR-image-thumbnails-v0.9.pdf (issue #1834 (closed)).

Some notes:

  • This PR reuses as much as possible already defined classes and architecture to manage other type of representations (manifest parsing, url template generation, baseURL, timing, etc). However, it is not using StreamProcessor/DashHandler type of "architecture" because they was designed to work with representations that are "linear" (one consecutive segment after the other). In the case of image/thumbnail type of representation, the way users consume it is not like a stream. Image "segments" are retrieved on demand given a specific time position, and typically users jump from one time position to other very quickly and without the need of going to consecutive times. In multiperiod streams, for example, it could happen that a user asks for thumbnails of a period that is not the one loaded/playing.

  • It supports single period and multiperiod vod streams.

  • I didn't test this implementation with live streams although given it is reusing previously defined timing and stream management methods I don't see any reason for not supporting them.

  • It supports multiple image renditions although ABR is not used. Quality changes are done manually. You can manage image qualities in the same way you do for video and audio representations: getBitrateInfoListFor('image'), getQualityFor('image'), setQualityFor('image', 0).

  • It leverages the use of browser cache. New define getThumbnail method of MediaPlayer returns a Thumbnail object that includes an url parameter that points to the image/sprite. In this version it is up to the browser caching the thumbnail image requests. I discarded implementing a thumbnails prefetch/cache layer (dash.js retrieving and caching all thumbnails when playback starts) at dash.js level because possible impact on video/audio ABR algorithm. Anyway, that's something that I will explore (and measure benefit/impact) once we have basic thumbnail support in.

  • Sample url (also added to "Other samples" sections of Dash reference player): http://dash.edgesuite.net/akamai/bbb_30fps/bbb_with_tiled_thumbnails.mpd

Merge request reports