This site uses cookies.
Some of these cookies are essential to the operation of the site,
while others help to improve your experience by providing insights into how the site is being used.
For more information, please see the ProZ.com privacy policy.
Multi-modal AI is revolutionising language translation, enabling more accurate and nuanced communication across sectors like business, healthcare, and diplomacy.
In today’s interconnected world, language barriers are becoming increasingly significant as businesses and individuals seek to collaborate globally. The natural way to communicate isn’t through reading or writing; it’s through seeing, listening, and talking. Multi-Modal AI, which integrates text, audio, and visuals, is revolutionising real-time translation and interpretation. This technology can empower society by making knowledge and resources accessible to all, regardless of education or literacy. This article explores how multi-modal AI is revolutionising real-time translation, its impact on overcoming traditional language barriers, and the challenges it faces.
Introduction to Multi-Modal AI
Multi-modal AI combines diverse types of data inputs like text, images, and sounds to generate responses or translations. Unlike traditional AI models that rely solely on one form of input, multi-modal systems leverage multiple data types, allowing for more nuanced and accurate translations. Multi-modal AI not only helps with interpreting spoken languages, but also with contextualising the non-verbal cues such as body language or environmental factors. The convergence of these different data types makes multi-modal AI significantly more effective in fields like language translation, medical diagnosis, autonomous driving, and even creative arts.
According to a report by MarketsandMarkets, the global AI market is expected to grow from $150 billion in 2023 to $1.59 trillion by 2030, and multi-modal AI will account for a significant portion of this growth due to its diverse applications.
Orange wants to bring manga to as many readers as possible—but some fans are not happy.
A Japanese publishing startup is using Anthropic’s flagship large language model Claude to help translate manga into English, allowing the company to churn out a new title for a Western audience in just a few days rather than the two to three months it would take a team of humans.
Orange was founded by Shoko Ugaki, a manga superfan who (according to VP of product Rei Kuroda) has some 10,000 titles in his house. The company now wants more people outside Japan to have access to them. “I hope we can do a great job for our readers,” says Kuroda.
Orange’s Japanese-to-English translation of Neko Oji: Salaryman reincarnated as a kitten! IMAGES COURTESY ORANGE / YAJIMA
But not everyone is happy. The firm has angered a number of manga fans who see the use of AI to translate a celebrated and traditional art form as one more front in the ongoing battle between tech companies and artists. “However well-intentioned this company might be, I find the idea of using AI to translate manga distasteful and insulting,” says Casey Brienza, a sociologist and author of the book Manga in America: Transnational Book Publishing and the Domestication of Japanese Comics.
The Connexion speaks to Adriana Hunter as she prepares to tackle the poetic bestseller Son odeur après la pluie
Adriana Hunter has translated over one hundred books and won multiple awards
Adriana Hunter is one of the most experienced British translators of French books with more than a hundred under her belt. She is the recipient of multiple literary prizes and awards.
She has been the official translator for the English version of the Asterix series of graphic novels since 2018, when she replaced Anthea Bell, and has worked on the last four of them.
She loves ‘juggling with words’ and has been honing her craft to bridge the gap between both languages.
This can mean anything from unravelling the intimacy of Amélie Nothomb’s bestselling novels or conveying the play-on-words and puns in Asterix.
The Connexion interviewed Cédric Sapin-Defour for Son odeur après la pluie, an unexpected bestseller telling the intimate relationship between him and Ubac, his Bernese mountain dog, who died in 2017.
He was asked for his opinion about whoever would be chosen for the English translation.
He was not sure how that person would accomplish what he considered an almost impossible task.
Vasco Pedro, CEO of the Lisbon-based startup Unbabel, delivered a provocative forecast at the Web Summit in Lisbon when he said human translators may no longer be needed within three years. The statement came in parallel with the launch of Widn.AI, Unbabel’s new AI-powered translation service built on its proprietary large language model, Tower. Capable of handling translations in 32 languages, Widn.AI represents a significant shift from the company’s earlier hybrid model, which paired AI technology with human editors.
“The advantage humans have in translation is razor-thin,” Pedro said, asserting that AI has reached a stage where it can handle all but the most complex translation tasks. This advancement aligns with a broader trend of generative AI boosting enterprise innovation, as companies increasingly leverage AI for tasks once deemed exclusively human.
Implications for Jobs and Industry
Unbabel’s innovation comes as AI’s potential to replace jobs is sparking heated debates. While Unbabel foresees growth fueled by a surge in translated content, Pedro admitted that the revenue per word is likely to drop. This mirrors broader predictions about AI’s disruptive potential, such as Vinod Khosla’s claim that AI could perform 80 percent of tasks across 80 percent of jobs.
The Little Prince, Antoine de Saint Exupéry’s timeless masterpiece, has reached a historic milestone by becoming the world’s most translated book of fiction, with 600 translations to date!
On June 25, 2024, as part of the project “The Little Prince at the Bedside of the World’s Languages”, the 600th translation of this masterpiece was presented to the National Library of Panama, in Dulegaya, the language of the Indigenous Guna people of Northeastern Panama and Colombia. This event reinforces the role of the Little Prince as a universal work that unites peoples and contributes to the preservation of endangered languages.
Since its first publication in 1943 in New York, this philosophical tale, illustrated by Antoine de Saint Exupéry, has touched readers of all generations and cultures, offering a universal message of love, kindness, and humanity.
Today, with many languages on the brink of extinction, The Little Prince uniquely preserves languages and transmits cultures. Thanks to the passion of translators, this work has been translated into rare languages and endangered dialects, thus contributing to the preservation of the world’s linguistic heritage.
The 600th translation confirms the cultural and social impact of the Little Prince, which transcends borders and becomes a link between peoples. This story is more than a story: it is a celebration of cultural diversity. 1,500 copies of this Dulegaya edition, entitled Sagla Massi Bibbi, were printed in the spring of 2024 by the Panamanian publishing house El Hombre de la Mancha. They will be distributed in schools and libraries in the Guna Yala region as of 2025.
NEW YORK, NY, November 18, 2024 (EZ Newswire) — Pronto Translations, a leading translation service in New York, has been integrating generative AI technology such as ChatGPT into its workflows for the past 18 months to support its translation processes.
Following the initial report issued last April, which detailed common AI errors, continuous enhancements in deploying AI have necessitated an update due to the emergence of further drawbacks that impact AI translation processes. Despite ongoing improvements to AI engines, experiences at Pronto Translations confirm that while AI technologies like ChatGPT can assist with translation efforts, they are not capable of replacing human translators. Effective translation requires a collaborative approach between AI tools and skilled linguists. Below are the 10 most critical reasons identified by Pronto Translations:
Mistranslation Risks: ChatGPT generally excels more than many other machine translation tools in identifying the correct contexts for meanings. However, significant errors have been observed, such as confusing “nuts” meant for vehicle assembly with edible nuts, or misidentifying a washer as a laundry appliance in a car maintenance manual. These errors underscore the risks involved in relying solely on AI for translation.
Fabrication of Information: ChatGPT can occasionally generate inaccurate content, especially when dealing with less familiar or obscure terms and concepts. While it handles well-known information from its training data competently, it struggles in areas where the data is scant or the terms are not widely recognized. In such instances, ChatGPT may make educated guesses, leading to translations that are not only imprecise but also potentially misleading. This is particularly problematic in technical or specialized texts where each term has specific and significant implications.
In the wake of Han Kang’s Nobel Prize in Literature, the Literature Translation Institute of Korea has decided to actively pursue upgrading its translation academy to a graduate school. It will also strive to strengthen the global literary network by increasing exchanges with overseas writers, translators, and publishers.
“The Nobel Prize in Literature is not the end, but the beginning,” said Jeon Soo-yong at a press conference on her 100th day at the institute on Monday. “For Korean literature to become world literature, forming international discourse and building a foundation for critique must be strengthened,” she said, emphasizing the need to establish a graduate school of translation.
Currently, the Literature Translation Institute of Korea operates the Translation Academy, a non-degree program for students in seven languages. The plan is to upgrade it to a full-time master’s degree program, aiming to improve the quality of translations and create opportunities for local translators to take on roles in schools and other institutions.
It’s the latest offering from the German tech unicorn
German tech darling DeepL has (finally) launched a voice-to-text service. It’s called DeepL Voice, and it turns audio from live or video conversations into translated text.
DeepL users can now listen to people speaking a language they don’t understand and automatically translate it to one they do — in real-time. The new feature currently supports English, German, Japanese, Korean, Swedish, Dutch, French, Turkish, Polish, Portuguese, Russian, Spanish, and Italian.
What makes the launch of DeepL Voice exciting is that it runs on the same neural networks as the company’s text-to-text offering, which it claims is the “world’s best” AI translator.
As someone who’s just moved to a foreign country, I’m keen to try a voice-to-text translator that actually might work. All the ones I’ve tried so far aren’t real-time — there’s a lag that renders them pretty useless — and the translation quality is pretty poor.
For face-to-face conversations, you can launch DeepL Voice on your mobile and place it between you and the other speaker. It then displays your conversation so each person can follow translations easily on one device.
You can also integrate DeepL Voice into Microsoft Teams and video-conference across language barriers. The translated text appears on a sidebar as captions. It remains to be seen whether DeepL Voice will be available on platforms like Zoom or Google Meet anytime soon.
Award-winning writer, poet and translator Professor Makhosazana Xaba used the second annual AC Jordan commemoration lecture to take a stand for women translators, spotlighting the lack of value afforded to their intellectual labours, especially in respect of African languages.
The annual lecture, instituted in 2023 by the University of Cape Town’s (UCT) School of Languages and Literatures and the AC Jordan Chair in Africa Studies, provides a platform for critical reflections and engaging dialogues, highlighting African intellectual histories, scholars and scholarship across the continent, as part of efforts to advance decolonisation efforts at UCT.
The AC Jordan Chair was established at UCT in 1993, named for Professor Archibald Campbell Jordan, an academic pioneer of African scholarship, literature and linguistics, and renowned for his novel Ingqumbo yeminyanya (The Wrath of the Ancestors).
Taking to the stage to deliver her lecture at the end of October, Professor Xaba quickly dispensed with the original title of her speech, “On Translating The Wretched of the Earth into isiZulu: From Challenges and Pleasures to Epiphanies” – reflections on her recent translation into isiZulu of Frantz Fanon’s landmark text, Izimpabanga Zomhlaba (The Wretched of the Earth).
When it comes to solving climate change, every word counts. From the pitfalls of metaphors to the multiple meanings of the word “energy” – this is how translators at global climate negotiations navigate the language of global warming.
“I remember one morning we returned to our hotel at around 4:00am and slept for two hours. Then we were told that the final document was adopted so we had to rush back to the conference to translate the outcome documents,” says Jianjun Chen, a Chinese language translator at the United Nations, based in Geneva.
He is recounting the frantic hours before negotiators reached a deal at the 24th Convention of Parties to the United Nations Framework Convention on Climate Change (UNFCCC) – or COP24 – in Katowice, Poland, in 2018. Chen, who has worked at the UN for 14 years and translated multiple documents from the UN climate talks into Mandarin, isn’t fazed by the long hours or lack of sleep.
This year’s UN climate talks in Baku, Azerbaijan, will be his 13th. As world leaders gather for COP29 in Baku, Chen and 25 other translators are preparing for a slew of new climate vocabulary to enter the discourse – words that will dictate the ways countries and campaigners approach climate action.
The final text is the result of negotiations, sometimes very intense negotiations. So you have to be very careful about the wording – Jianjun Chen
Chen also translated important documents when the landmark Paris Agreement was signed at the UN climate talks in 2015 (COP21), pledging to try to prevent global warming to well below 2C, with a stretch target of a 1.5C limit. (Read more about why 1.5C is a critical threshold in this story by Martha Henriques). “I was called to start working in the middle of the night at 2 or 3am. Since there was always a tight deadline, we didn’t have the luxury to fall asleep,” he recalls.
Translators have raised concerns over Simon & Schuster-owned publisher Veen Bosch & Keuning’s (VBK) “disastrous decision” to use artificial intelligence (AI) to translate some titles into English.
TheBookseller revealed last week that the Utrecht-headquartered publisher is “working on a limited experiment with some Dutch authors for their books to be translated into English language using AI”. It was later reported in the Guardianthat the “project contains less than 10 titles—all commercial fiction”, and does not include any literary books or titles to which English rights would likely be sold at any point. The story was also followed up on Radio 4’s “Today” programme.
The publisher, which was acquired by Simon & Schuster (S&S) earlier this year, explained that this would include “one editing phase, and [that] authors have been asked to give permission for this”.
Although the Dutch publisher is owned by S&S, The Bookseller understands that the two publishers’ editorial decisions remain separate.
Industry figures have voiced their concerns about the potential “reputational damage” of this move, and the inefficiency that AI could introduce into the translation process.
Louise Rogers Lalaurie, who has translated 15 novels from French and is the author of Matisse: The Books (Thames & Hudson/University of Chicago Press), explained that the “end result” of an AI-generated translation can cost “more than a good human translation first time round” due to the time-consuming “post-editing” process.
Veen Bosch & Keuning, the largest publisher in the Netherlands, has confirmed plans to trial the use of artificial intelligence to assist in translation of commercial fiction
A major Dutch publisher plans to trial translating books into English using artificial intelligence.
Veen Bosch & Keuning (VBK) – the largest publisher in the Netherlands, acquired by Simon & Schuster earlier this year – is “using AI to assist in the translation of a limited number of books”, Vanessa van Hofwegen, commercial director at VBK said.
“This project contains less than 10 titles – all commercial fiction. No literary titles will nor shall be used. This is on an experimental basis, and we’re only including books where English rights have not been sold, and we don’t foresee the opportunity to sell English rights of these books in the future,” she added.
“There will be one editing phase, and authors have been asked to give permission for this,” a VBK spokesperson told the Bookseller. “We are not creating books with AI, it all starts and ends with human action.”
The fact that the publisher is planning to use AI translation only for commercial fiction, rather than literary titles, “assumes those books are purely formulaic and don’t contain many creative elements, which is rather insulting to the authors and readers involved”, said Michele Hutchison, who won the 2020 International Booker prize for her translation of Lucas Rijneveld’s The Discomfort of Evening.
“There’s only so far you can get” with machine translation post-editing – the process by which a human translator reviews an AI-generated translation. “The text might be superficially smooth but it is also likely to be very bland,” she added.
In the age of artificial intelligence (AI), foreign language learning can seem like it’s becoming obsolete. Why invest the time and effort to learn another language when technology can do it for you?
There are now translation tools to understand song lyrics, translate websites and to enable automated captions when watching foreign videos and movies. Our phones can instantly translate spoken words.
At the same time, foreign language programs are closing at New Zealand and Australian universities.
But while technology can translate messages, it misses an important component of human communication—the cultural nuances behind the words.
So, while AI translation might bridge language barriers and promote communication because of its accessibility, it’s important to be clear about the benefits and challenges it presents. Merely relying on technology to translate between languages will ultimately lead to misunderstandings and a less rich human experience.
Han Kang is lucky to have a translator as gifted and prominent as Deborah Smith, who enabled Han’s works to cross the borders of her country and meet foreign readers.
SEOUL – The news that novelist Han Kang has won the 2024 Nobel Prize in Literature came as a wonderful surprise to the Korean people. We feel it’s been a long time coming: Whereas Japan already has three Nobel laureates in literature and China has two, Korea had none until last week. At last, Korea has become a country with a Nobel Prize winner in literature.
These days, Korea is well known to the world, thanks to the immense popularity of Hallyu. In a sign of this recognition, McDonald’s sold BTS Meals for some time, and Coca-Cola is now selling a K-wave zero sugar product. In 2020, the Korean movie “Parasite” received six Academy Awards. In addition to the fame of its pop culture, Korea has now impressed the world with its literary works, too, which was acknowledged by the Nobel Prize, thereby accomplishing for Hallyu a “rondure complete,” as Walt Whitman might have put it.
To become universal, literature requires translation, because unlike music, dance or art, it can only be accessed through written language. Therefore, an excellent translation is a crucial prerequisite for a literary work to be read and praised around the world. Indeed, the famous Italian writer Italo Calvino said, “Without translation, I would be limited to the borders of my own country. The translator is my most important ally. He introduces me to the world.”
Han Kang is lucky to have a translator as gifted and prominent as Deborah Smith, who enabled Han’s works to cross the borders of her country and meet foreign readers. People say that had it not been for the superb translator Edward Seidensticker, Japanese novelist Yasunary Kawabata could not have received the Nobel Prize in Literature. The same thing may apply to Deborah Smith and Han Kang, because no matter how great a writer is, he or she cannot be known overseas without translation.
Chinese e-commerce company Alibaba has invested heavily in its fast-growing international business as growth slows for its China-focused Taobao and Tmall business.
BEIJING — Chinese e-commerce giant Alibaba’s international arm on Wednesday launched an updated version of its artificial intelligence-powered translation tool that, it says, is better than products offered by Google, DeepL and ChatGPT.
Alibaba’s fast-growing international unit released the AI translation product as an update to one unveiled about a year ago, which it says already has 500,000 merchant users. Sellers based in one country can use the translation tool to create product pages in the language of the target market.
The new version is based only on large language models, allowing it to draw on contextual clues such as culture or industry-specific terms, Kaifu Zhang, vice president of Alibaba International Digital Commerce Group and head of the business’ artificial intelligence initiative, told CNBC in an interview Tuesday.
“The idea is that we want this AI tool to help the bottom line of the merchants, because if the merchants are doing well, the platform will be doing well,” he said.
Large language models power artificial intelligence applications such as OpenAI’s ChatGPT, which can also translate text. The models, trained on massive amounts of data, can generate humanlike responses to user prompts.
Han Kang’s Nobel Prize in Literature has not only highlighted her works but also shone a spotlight on the importance of translation. Many say that translating from Korean to English can be challenging due to cultural differences and linguistic complexities. Unique cultural elements like honorifics, for example, may be difficult for English-speaking readers to understand.
Deborah Smith, 37, the British translator who translated Han’s “The Vegetarian” (2007), said she always keeps in mind how a book could be comprehended by native English speakers. In an interview with the Daesan Foundation after winning the International Booker Prize in 2016 with Han, she said that English readers may be unlikely to understand Korea’s social order based on the Confucian hierarchy.
To capture the oppressive atmosphere felt by the protagonist of “The Vegetarian,” who is in conflict with her family over her decision to become a vegetarian,Smith altered relational titles like “sister-in-law’s husband” to “Ji-woo’s mother” in her translation.
Smith translated “The Vegetarian” in 2013, just three years after beginning to learn Korean. She has since translated Han’s “The White Book” (2016), “Human Acts,” (2014) and “Greek Lessons” (2011), with the English version of “The White Book” being shortlisted for the International Booker Prize in 2018.
For a few years now, AI has been reshaping how audiovisual content is localized and used.
From AI-automated subtitling and captioning to AI dubbing and advancements in sign language, AI-generated audio description, and many other services, audiovisual translation (AVT) presents both exciting opportunities and challenges for language service providers (LSPs).
Data from the 2024 Slator Pro Guide: Audiovisual Translation shows that most established providers of AVT services already use AI tools in their workflow or are considering their implementation in the short to medium term.
In fact, over a third of LSPs surveyed by Slator now offer AI voiceover and subtitling services, with a quarter providing AI dubbing. This trend is driven partly by end clients actively asking for more efficient and scalable solutions.
As Ofir Krakowski, Co-Founder and CEO of Deepdub, aptly pointed out during a SlatorPod episode, we live in an age dominated by audiovisual content.
As it happens, today (October 10), there’s similar shortlist anticipation from another prominent book award, the United Kingdom’s Baillie Gifford Prize for Nonfiction, where executive director Toby Mundy, intends to research and report—on his own choice of timing and criteria after his awards—the impact of a Baillie Gifford Prize win.
In the aggregate, efforts of these kinds—to discern and evaluate the marketplce effects of a prize win—can establish some parameters of expecatation and actual influence, leading, presumably, to more clear-eyed enthusiasm—an honest appraisal of the impact of a “golden sticker” on a book cover, in terms of how much awards actually count.
And with ALTA’s focus on translation, particularly in the United States, it could be instructive to know if and how much an award might influence marketplace perceptions of translated work. The value of translation in literature has never been self-evident to broad swaths of many populations, and having actual numbers on how much an award might impact print-runs, unit sales, and so on, could be invaluable.
The role that interpreters play in health care is complex, difficult and largely unacknowledged. We couldn’t do our job without them.
The first patient arrives, flustered and tightly wound. She’s a refugee six months post-arrival in Australia. The initial flush of finally getting here has started to fade, the winter is getting colder and her children have to take three buses to school. She has increasing requests from her employment services provider to complete English classes and think about looking for work. In the country she left, the humanitarian situation continues to deteriorate and the messages from family left behind are less frequent but more distressing. She’s ostensibly here to follow up on some outstanding blood results from her new arrival refugee health assessment, but it’s clear from the outset there’s going to be a lot more to manage here.
Like almost all of our consults, this appointment is made possible through the Translating and Interpreting Services (TIS National). We dial in, provide some details and — almost miraculously, within minutes — are connected to someone who speaks English and the language of the patient in front of us.
Communication between the patient and doctor is perhaps the foundation of medicine. International Translation Day falls on 30 September, formalised by the United Nations in 2017 to acknowledge the work of all language professionals and their role in world peace and security. The date was chosen in honour of Saint Jerome, a priest from Northern Italy known primarily for translating the Bible into Latin from Greek and Hebrew and considered the patron saint of translators. We want to celebrate the daily contribution of a largely unacknowledged workforce in the clinical space, the Translating and Interpreting Service (TIS National).
The nature of the refugee health service where we work means we use mostly on-call telephone interpreters rather than pre-booked or face-to-face services. Although the interpreters are rostered on to take calls, they are going about their day — sometimes with washing dishes, barking dogs or pedestrian crossings in the background — when we’re connected by the operator, they step out of their worlds and immediately into the intimacy of a consult room.
Language watchdog insists French labelling requirements apply to all products, including musical instruments.
Xavier Twigg, owner of Twigg Musique Montréal, says instrument companies generally only create one product label in one language, English.(Steve Rukavina/CBC)
The president of a nationwide chain of musical instrument stores says a whole new twist on D-Day is coming to Quebec next June.
That’s when the province has promised to crack down on product labelling, ensuring everything has French wording emblazoned, inscribed or stickered on. The new law, known as Bill 96, requires any label or writing on musical instruments, products or packaging to have a French translation.
“When we get to D-Day, unless there’s some changes, it’s going to be very traumatic for our stores,” said Steve Long, president of Long & McQuade Musical Instruments, which has 100 locations from British Columbia to Newfoundland.
“We won’t be able to operate in Quebec.”
Long & McQuade has 18 stores in the province.
Long is among those raising concerns about the new law’s Article 51. It says every inscription on a product, its container, its wrapping or on any document or object supplied with it — including the directions for use and the warranty certificates — must be drafted in French.
Most of these products are imported, and English is the industry-standard language.
The translation news daily digest is my daily 'signal' to stop work and find out what's going on in the world of translation before heading back into the world at large! It provides a great overview that I could never get on my own.
susan rose (X)
ארצות הברית
I read the daily digest of ProZ.com translation news to get the essential part of what happens out there!
Share an article »
ProZ.com blog »
