As a high-stakes election takes place in the dizzying heat of India, a bewildering stream of deepfakes is appearing across India. The variety seems endless: AI impersonations, ventriloquism and deceptive editing effects. Some are vulgar, some are tongue-in-cheek, others are so obviously fake that they are impossible to believe are real.
The overall effect is confusing, adding to a social media already rife with misinformation. The volume of online rumors is too great for any electoral commission to keep track of, let alone debunk.
To fill the gap, a variety of fact-checking organizations have emerged. Although the law has been slow and uneven to enforce, the work of tracking deepfakes has been undertaken by hundreds of government workers and private fact-checking organizations in India.
“We have to be prepared,” said Surya Sen, a forestry official in Karnataka state who was transferred during the election to oversee a 70-member team charged with hunting down deceptive AI-generated content. “Social media has been a battleground this year.” When Sen’s team finds content they deem illegal, they tell social media platforms to remove it, expose the deception, or even call for criminal charges.
Celebrities have become common fodder for political maneuvers, including Hindi film star Ranveer Singh.
In a videotaped interview with an Indian news agency by the Ganges River in Varanasi, Singh praised strong Prime Minister Narendra Modi for praising “our rich cultural heritage.” But viewers heard something different in a modified version of the video circulating on social media, in which the voice sounded like Singh, lip-syncing almost perfectly.
“We call these lip-sync deepfakes,” said Pamposh Raina, head of the Deepfakes Analysis Unit at Indian media agency Nikki Hashim. The unit has set up a reporting hotline on WhatsApp where people can send suspicious videos and audio for review. She said Mr Singh’s video was a classic example of real footage edited with a voice cloned by artificial intelligence. The actor filed a complaint with the cyber crime unit of the Mumbai police.
In this election, no single party has a monopoly on deceptive content. Another manipulated clip begins with real footage of Modi’s most prominent opponent, Rahul Gandhi, participating in the mundane ceremony of swearing in a candidate. It is then superimposed with an AI-generated soundtrack.
Mr Gandhi did not actually quit his party. The video also contained a personal dig at Mr Gandhi, making it appear as if he was saying he “can’t pretend to be a Hindu any longer”. The ruling Bharatiya Janata Party presents itself as the defender of the Hindu faith and its opponents as traitors or imposters.
Sometimes, political deepfakes take a turn toward the paranormal. Deceased politicians can be resurrected in uncanny AI-generated likenesses to endorse their descendants’ real-life campaigns.
In a video that emerged days before voting began in April, H. Vasanthakumar, who died of Covid in 2020, came back to life, spoke indirectly about his own death and blessed his son Vijay, who is contesting for his father’s former parliamentary seat in the southern state of Tamil Nadu. The apparition follows the example of two other late Tamil political giants, Muthuvel Karunanidhi and Jayalalithaa Jayaram.
Modi’s government has been working on laws aimed at protecting Indians from deepfakes and other misleading content. The 2021 “IT Rules” bill makes online platforms liable, unlike in the United States, for all objectionable content, including parodies designed to cause insult. The Internet Freedom Foundation, an Indian digital rights group, considers the powers too broad and is tracking 17 legal challenges to the law.
But the prime minister himself appears to be open to certain types of AI-generated content. Two videos created using AI tools show two of India’s biggest politicians, Modi and one of his staunchest opponents, Mamata Banerjee, imitating the viral Youtube Video American rapper Lil Yachty is making his “toughest exit ever”.
Mr. Modi shared He watched the video on X and called the idea “delightful”. Mr. Sen, an election official in Karnataka, called it political satire: “Modi Rockstar is fine, there are no violations. People know it is fake.”
Police in West Bengal, where Banerjee is chief minister, issued notices to some people for posting “offensive, malicious and inflammatory” content.
In their hunt for deepfakes, Sen said his team in Karnataka, working for the opposition-controlled state government, vigilantly scanned social media platforms such as Instagram and X, searching for keywords and repeatedly refreshing the accounts of popular influencers.
The Deepfakes Analysis Unit has 12 fact-checking partners in the media sector, including several with close ties to Modi’s national government. Raina said her unit also works with external forensic labs, including one at the University of California, Berkeley. They use artificial intelligence detection software, such as True Mediawhich scans media files and determines whether they are trustworthy.
Some tech-savvy engineers are refining AI forensics software to identify which part of a video has been manipulated, down to a single pixel.
Pratik Sinha, founder of Alt News, India’s most respected independent fact-checking site, said the possibilities of deepfakes have not been fully exploited. One day, he said, videos could show politicians not just saying things they didn’t say but doing things they didn’t do.
Dr. Hany Farid, who has taught digital forensics at Berkeley for 25 years and has worked with the deepfakes analysis unit on some cases, said that while “we are catching bad deepfakes,” if more sophisticated forgeries enter the field, they may go unnoticed.
In India, as elsewhere, an arms race is underway between deep fakers and fact checkers — with all sides battling it out. Dr. Farid called this “the first year where I think we’re really starting to see the impact of AI in interesting and more nefarious ways.”
