Skip to content

Adds multiple audio/video adaptation support

Vinay Rosenberg requested to merge github/fork/MSOpenTech/multiple-audio-video-rc into development

Created by: KozhinM

PR summary

New API has been added to MediaPlayer:

  • getTracksFor(mediaType)
  • getCurrentTrackFor(mediaType)
  • setCurrentTrack(track)
  • setInitialMediaSettingsFor(mediaType)
  • getInitialMediaSettingsFor(mediaType)
  • getTrackSwitchModeFor(mediaType)
  • setTrackSwitchModeFor(mediaType, mode)

Adaptation handling

When an mpd is loaded dash.js searches for all the available AdaptationSets and adds them to the track list in MediaController providing these adaptations are supported by the player (check for codec and encrypted media support). MediaController is a new singleton class that keeps tracks for all media types and provides methods to get/set/add/change tracks. If several AdaptationSets are available for a media type dash.js selects the one, which media settings match the initial media settings (lang, viewpoint, role). If there are no initial settings set or there is no match, the first AdaptationSet is selected.

Settings storage

The latest used media settings are stored in localStorage. When playback starts, dash.js checks the initial media settings first. If they are not set, the settings are extracted from the storage and used as initial settings. This option can be switched on/off by using MediaPlayer.enableLastMediaSettingsCaching() method.

Buffer handling

Added logic to track the exact start and end positions of the appended segments. This has been achieved by tracking the changes in SourceBuffer.buffered ranges after appending each segment. Also added functionality to clear the SourceBuffer from data of the previous track. Clearing depends on the track switch mode. There are two modes supported - "always replace" and "never replace". The first one makes dash.js remove all the data from the buffer prior to the currentTime while the second one makes the player keep that data so that when users seek back they see/hear the old track. Besides, dash.js automatically clears the data buffered ahead of currentTime, but this happens only when there is enough data of the new track in the buffer (segmentDuratoin *2).

Quality switch

When tracks are switched RepresentationController now checks the average throughput, and select the most appropriate quality for it. If average throughput is not available the initial bitrate value is used instead.

Renaming partially renamed "track" into "representation" to avoid confusing with audio and video track naming convention. I am going to send out a separate email with clarifications of these changes.

Example page

A new multitrack.html demo page has been added to "samples\dash-if-reference-player" directory. It allows testing of all the new added features. The page duplicates original index.html except the following changes: • Added "Cache media settings" switch (right under the ABR switch) - allows to enable/disable caching of mediaSettings. • Added "Tracks" dropdown at the bottom of video and audio metric sections. Note: the list fulfills only after "Load" button is clicked, player is initialized and track list becomes available. • Added "Track switch mode" dropdown. Allows to choose one of two available modes. • Added text input that allows to set initial role for video track or lang for audio. Note: both audio and video support role, lang and viewpoint settings, so role and lang were chosen only as an example.

Merge request reports