The News Production use case uses the NLP technologies that are developed in the project to support the creation of media content. The focus lies on the creation of text (summaries for newsletters and video subtitles) and synthesized speech (video voice-over and audio podcasts).
In all cases, the Media Monitoring component can provide input material. This is especially true for the newsletter scenario, where relevant news material needs to be identified and summarized first.
However, it is also possible to upload input material directly into the system for further processing. This is especially true for the video subtitling and video voice-over scenarios, where it is sometimes necessary to process hand-selected videos. It also applies to the Podcast translator scenario, where selected podcasts are translated into other languages using speech synthesis.
We will experiment with different user interfaces to collect user corrections of the corresponding SELMA components. We will also experiment with the UI to show the automated post-processing suggestions (in WP3) and to accept or reject user suggestions, including hints on how the correction should be used for learning (e.g. topic-dependent, onetime only, global), We will explore how to benefit from user suggestions at multiple levels – (1) on the document level, the UI should show corrections based on user correction of terms in the same file and (2) on the user level, for other documents edited by the same user and (3) globally to improve the overall suggestions to all users.
News Podcast Creation
This section proposes a use case application where audio news bulletins are produced with the support of SELMA.
The goal is to increase the workflow’s efficiency by supporting the journalist in the production of daily audio news bulletins through SELMA.
The Reuters Digital New Report for 2020 highlights the rise of popularity of news podcasts over the last two years. One of the most well-known daily news podcasts is The Daily by the New York Times which attracts 2 million daily listeners. Based on this trend, DW’s Brazilian department started its own daily news podcast in August 2020. While the monthly usage rises consistently with more than 50.000 impressions in December 2020, it becomes equally clear how resource-intensive its production is.
Consequently, it is highly desirable to be able to produce audio news content in various languages with a short ramp-up time and with minimal personnel effort.
Video Subtitling
This section describes a DW use case application where SELMA output is further processed as subtitles.
The system creates subtitles based on the audio track, potentially translates them and makes them available for manual editing. Subsequently, it applies them to the video and thus enhances the accessibility of the audiovisual media item.
Video Voice-Over
This section describes a use case application where SELMA output is further processed as voice-over.
The extracted and edited subtitles are converted into an audio track via text-to-speech. The audio track is subsequently added to the video.
We use the same procedure as in the subtitling use case application, and follow the steps outlined there. In addition, we use a synthetic voice to produce a voice-over.
The objective is to make the speech output in the target language as expressive and natural as possible, resembling the expressiveness of the original version.
A possible extension of this use case application is to apply the speech-to-speech translation module that is part of experimental research.
