AI-Generated Disinformation Overwhelms X During Iran Conflict
Key Points
- X's AI chatbot Grok misidentified Iran war footage and posted AI‑generated images.
- AI‑generated videos and images depicting false missile attacks, building fires, and antisemitic themes have garnered millions of views.
- Iranian officials and state media are using AI tools to create exaggerated war content.
- X temporarily demonetized blue‑check accounts that share unlabeled AI combat videos.
- Traditional disinformation continues, such as misattributing missile attacks on a school.
- Experts call for stronger regulation of AI‑generated misinformation.
Disinformation experts report that X's AI chatbot Grok repeatedly misidentified video footage from the Iran war, while paid accounts with blue check marks share AI‑generated images and videos that appear realistic. The flood of AI‑created content includes fabricated missile footage, fake attacks on high‑rise buildings, and antisemitic narratives. X responded by temporarily demonetizing blue‑check accounts that post AI‑generated combat videos without labels, but researchers warn the platform remains a hub for sophisticated false media. Calls for stronger regulation of AI‑driven misinformation are growing as the conflict continues.
AI Tools Fuel a Surge of False Content
Disinformation specialists observed that X's AI‑powered chatbot Grok failed to verify a post about Iranian missiles, repeatedly misidentifying the location and date of the original video. After the incorrect verification, Grok posted an AI‑generated image, illustrating how the system can produce misleading visuals. The broader platform has seen a dramatic increase in AI‑generated images and videos, including fabricated footage of a high‑rise building in Bahrain on fire, a US B‑2 bomber allegedly shot down, and Delta Force members purportedly captured. Some of these pieces have amassed millions of views before being removed.
Iranian officials and state media have also used AI tools to create videos that depict missile manufacturing inside caves and other exaggerated scenes of damage. In addition, pro‑regime networks on X have circulated antisemitic AI‑generated posts that depict Orthodox Jews leading American soldiers to war and celebrating American casualties. One viral video showing a line of young girls walking past former President Donald Trump in underwear reached millions of views before removal.
X's Partial Countermeasures
In response to the surge of AI‑generated combat media, X announced a temporary demonetization policy for blue‑check accounts that post such videos without a label. The platform has not disclosed how many accounts have been affected. Premium services that grant blue checks have been purchased by some Iranian officials, providing boosted engagement and potential earnings from posts.
Continued Non‑AI Disinformation
Traditional false narratives persist alongside AI fakes. For example, footage from elsewhere in the conflict has been repurposed to claim that Iran fired the missile that struck a primary school in Minab, despite verification that a US Tomahawk cruise missile hit a nearby naval base. This illustrates the blend of AI‑generated and conventional misinformation circulating on X.
Calls for Stronger Regulation
Experts warn that the ease of creating AI‑generated content with little consequence threatens factual discourse. The Institute of Strategic Dialogue highlighted the use of AI to push overtly antisemitic narratives, while analysts noted that detection tools are inconsistent. Meta’s Oversight Board criticized the company’s labeling approach as insufficient for the speed and scale of AI‑driven misinformation, especially during crises. Researchers argue that without robust regulation, the spread of AI‑based fake news could erode a fact‑based world.