To misquote Charles Dickens’s A Tale of Two Cities, ​“It was the 147^th of times, it was the 45^th of times.”

In mid-July 2024, in Sapporo, Japan, MPEG — an alliance of working groups established by ISO (the International Organization for Standardization) and IEC (International Electrotechnical Commission), that sets standards for media coding — met for the 147^th time.

And SC 29 (aka ISO/IEC JTC 1/SC 29, a standardization subcommittee of the Joint Technical Committee ISO/IEC JTC 1 of the ISO and the IEC) … well, they met for the 45^th time.

No one goes to an MPEG conference, especially all the way to Sapporo, unless the person’s knowledgeable about — and highly useful and attuned to — the field of media standardization.

Such a person is Mohamad Raad.

As standardization representative for Unified Streaming, Raad researches and shapes video streaming standards that are vital for the company and for the world of video streaming specifications.

After his trip, Raad presented his experience at MPEG to the rest of Unified Streaming. First, he listed a few facts about the conference.

Just the facts

1. MPEG ​“owns” the DASH standard, the ISOBMFF (ISO Base Media File Format) standard, and the video and audio standards that Unified Streaming products support. Any changes to these standards means changes to Unified’s own interoperability, customer expectations, and product features.

2. SC 29 is responsible for overseeing the work of MPEG and JPEG. The MPEG working groups are WG2, WG3, WG4, WG5, WG6, WG7, and WG8. JPEG is WG1.

3. Leading companies in the multimedia streaming industry are significantly involved in both MPEG working groups and the JPEG working group. The companies include Comcast, Apple, Google, Qualcomm, Dolby, Huawei, Nokia, Fraunhofer, MediaTek, Tencent, ByteDance, Bitmovin, Broadpeak, and others.

Achievements

Raad also explained Unified’s main achievements at the meeting.

1. Unified helped change the way that ISOBMFF recordings can be made, thereby saving resources. There’s a problem with the current method for very long recordings, and Unified’s contribution fixes that problem.

2. Technical changes to REaP (Redundant Encoding and Packaging) revolved around which HTTP methods to use.

• Raad presented Unified’s implementation of a REaP workflow and its results. Others now want the implementation to be the reference software starting point. (Adopters of the standard find it helpful to have a reference software implementation. Unified expects changes to be made to its implementation. That’s why this is a starting point.)
  

• Unified Streaming’s R&D identified a mistake in the REaP text, which allowed two HTTP methods to be used: either POST or PUT. So Unified determined that the only method that can be used is POST.
  
  The reason for nixing PUT is that it allows for existing segments to be overwritten, whereas POST will not. (So if the resource exists, then it will not be overwritten.) Allowing segment overwrite will cause errors on the egress, since the resource is either locked, or has been deleted, to be overwritten.
  

• Comcast and AWS are (or were) working on their own ReAP implementations.

Other groups, other developments

Naturally, progress was made in other working groups.

1. DASH 6^th edition (an update of the MPEG-DASH specification, owned by WG3) focused on CMCD (Common Media Client Data) in the MPD (Media Presentation Description, aka an XML-based manifest file for DASH). Also highlighted: low latency using sub-segments. In other words, segments broken up into smaller components to help speed up content delivery, especially at start-up.

2. In terms of content authenticity, JPEG, in collaboration with C2PA (Coalition for Content Provenance and Authenticity), is out in front. Every other WG wants to develop a standard in this domain.

• ITU‑T (the Geneva-based International Telecommunication Union Telecommunication Standardization Sector) also has a study group on multimedia authentication. The study group is named H.MMAUTH.

Bits and bytes

The multimedia space is teeming with various innovations.

• Lenslet video coding (3D representation without glasses)

• Neural network-based video coding (AI)

• Immersive formats (video and audio)

• Gaussian splatting (for point cloud compression)

• Radiance coding (instead of lightfield)

• Audio Neural Radiance Field coding (generates realistic audio scenes using what was originally a 3D image representation method)

Thoughts on the value and challenges of SC 29 and its WGs

1. Projects impact the current streaming market and establish conditions for future multimedia markets.

2. Projects move slowly. Average standardization period is 3 years after the call for proposals.

• Advancing to a call for proposals may take more than a year.

3. The process can seem archaic, but it helps create a level playing field. Small market entities can have a significant impact.

4. The working groups’ decision-making process is based on the scientific method.

5. The conference functions as a meeting point for actual stakeholders representing multiple interest groups.

Q&A with Mohamad Raad

Q. What’s your favorite part of attending MPEG?

Raad: Meeting the experts and discussing new ideas. MPEG attendees tend to be very well informed and are very sharp. Discussions with people at MPEG are always worthwhile.

Q. Which standards are you most excited about?

Raad: At the moment, DASH, REaP, and media authenticity. DASH still has a lot to offer in terms of multimedia delivery. And media authenticity is a very important topic in a world about to be dominated by generative AI.

Q. How do other researchers and developers agree on how to move forward?

Raad: The good thing about the SC 29 WGs is that decision making tends to be scientific, meaning that experts first agree on a set of metrics to use in the decision-making process, and then tend to adopt only technologies that contribute to an improvement of a standard based on those metrics. I don’t want to give the impression that this is an exact science, but we try to be as scientific as possible.

Q. What role does Unified play at these gatherings? Innovator? Mediator? Common ground finder? All of the above?

Raad: Unified has mostly played the role of innovator at MPEG, as representatives from Unified have focused on making technical contributions over the years. We have also occasionally been a ​“common ground finder.” Take, for example, the REaP standard, which is currently under development. Unified helped articulate the use cases and requirements for this activity by collecting input from multiple stakeholders.

Q. What are you looking forward to at MPEG 148, in Turkey?

Raad: On the technical front, I’m looking forward to progressing REaP to the final stage of standardization, as well as progressing the next edition of DASH, and discovering what MPEG is going to do about media authenticity. On a personal note, I’m looking forward to visiting Kemer again. The location (on Turkey’s southern Mediterranean coast) is very nice, and the organizing team is highly professional. Our last meeting there was great.