Handling multiple audio channels

[miclaus28]

Hi!
I managed to create my first OTIO timeline. Everything went smooth until I tried to add on the timeline a video that has multiple channels. I am generating both a .otio file and an .xml file using the fcpxml-adapter. I just want to know how would the logic of adding a video with multiple audio channel looks like. An example would be perfect! Thanks!

[reinecke]

Hi @miclaus28, thanks for reaching out. At the moment, the exact way audio channels are selected, mixed, and mapped to outputs is a bit loose. We have a proposal for full-featured audio channel mapping, but that lives in our "future feature" category.

Depending on what application your targeting to consume the timeline, you may be able to use the metadata dictionaries in OTIO objects to drive some of that, but that can also be a bit challenging.

Can you share a bit more context on what kinds of multi-channel things you're specifically trying to do and which application(s) you're targeting to consume these timelines?

[miclaus28]

Hi @reinecke, thank you for your answer!

I’m working with OTIO to generate timelines for Premiere Pro, DaVinci Resolve, and Final Cut Pro, and I’m struggling specifically with how to represent complex audio layouts.

Some of my source video files have things like:
• multiple stereo pairs,
• 7.1 surround tracks, or
• several separate mono channels.

I can detect and extract all this channel information correctly using ffprobe, but I don’t know how to translate that into an OTIO timeline so that, when I export (e.g. to XML) and import into the NLE, all audio tracks and channels appear correctly and in the right configuration.

Right now, I’ve hacked together a prototype where I:
1. Generate the XML using OTIO, then
2. Manually modify the XML with ElementTree to inject the correct audio track/channel structure.

This does work, but it’s fragile, hard to maintain, and quite buggy.

In short:
I am looking for a proper OTIO-based way to build timelines that preserve the original multi-channel audio structure (stereo pairs, mono channels, 5.1/7.1, etc.) so that Premiere, Resolve, and FCP correctly recognize and display those audio tracks on import, without having to post-process the XML manually.

[reinecke]

This context is super helpful.
While we haven't yet implemented that linked proposal, the FCP 7 XML adapter does use the fcp_xml key in metadata dictionaries to preserve unsupported fields from the XML. This means that structures you put in the fcp_xml key should show up in the output XML. You should be able to see examples of this data in .xml files you read into OTIO with the adapters.

I'd be interested to learn more about the exact post-processing you do - it would be pretty cool to implement the OTIO schema for this and then use what you're doing as a guide to implement support for the native channel mapping in the adapter. That would let us validate the new schema out in the world to see how it's working.

One other thing - since there's native OTIO import/export support in Resolve, have you looked into what they're doing for this? They include a pretty rich set of data in the metadata dictionaries - you may be able to make OTIO with those properly populated to have resolve directly import OTIO.

[miclaus28]

Please excuse my late answer.

I haven’t implemented the linked proposal yet, but my current workflow does use ffprobe to extract audio data, which I then parse into the .xml generated by OTIO using ElementTree. For Premiere Pro, this means I’m manually populating the XML structure rather than leveraging the fcp_xml key in the metadata dictionary.

For Resolve, I’ve successfully imported videos using a similar approach—extracting metadata via ffprobe and populating the dictionary—but I haven’t yet explored their native OTIO import/export in depth. That’s a great suggestion, and I’ll look into how they handle metadata to see if I can align with that.

Regarding post-processing, I’d be happy to share more details. My goal is to ensure accurate channel mapping and audio structure, so implementing an OTIO schema for this sounds like a great idea. It would definitely help validate the approach in real-world workflows.