The Mechanics of Deception: Inside the Disruption of Russia’s 'Bad Grammar' AI Influence Campaign

The landscape of information warfare is undergoing a fundamental shift. For years, state-sponsored influence operations (IO) relied on “troll farms”—warehouses filled with human operators manually crafting posts, managing hundreds of fake personas, and struggling to maintain linguistic fluency in foreign languages. Today, that model is being disrupted by generative AI.

In a recent disclosure, OpenAI detailed the disruption of a Russia-linked covert influence operation internally codenamed ‘Bad Grammar.’ This campaign represents a significant evolution in how adversarial actors utilize Large Language Models (LLMs) to manufacture credibility and scale their reach. Unlike previous iterations of Russian propaganda that were often plagued by obvious syntax errors and cultural disconnects, ‘Bad Grammar’ utilized ChatGPT to bridge the linguistic gap, translate complex narratives, and automate the creation of a “manufactured infrastructure” designed to deceive even sophisticated Western audiences.

The disruption of ‘Bad Grammar’ is a landmark case for several reasons. First, it highlights a proactive shift in the role of AI providers; OpenAI is no longer just a toolmaker but a frontline defender, actively hunting for state-sponsored abuse within its ecosystem. Second, it reveals the technical limitations of current AI-powered propaganda. Despite the efficiency gains, these campaigns often leave behind “digital fingerprints”—linguistic artifacts and behavioral patterns that allow defenders to trace the content back to its source. As we analyze the mechanics of this operation, we see a preview of a future where the battle for narrative control is fought through account clustering, behavioral analysis, and the constant cat-and-mouse game of AI safety guardrails.

Architecture of Deception: The International Burke Institute (IBI)

At the heart of the ‘Bad Grammar’ operation was a sophisticated attempt to “Live Off the Legitimate.” Instead of simply blasting slogans into the void of social media, the actors behind the campaign sought to anchor their narratives in a seemingly prestigious academic setting. They created the International Burke Institute (IBI), a fake think tank purportedly based in Israel.

The IBI was registered in February 2024, appearing at a time of heightened geopolitical tension. Its goal was to project an aura of objective, scholarly analysis while subtly weaving in pro-Russia and anti-Ukraine narratives. This strategy of LLM poisoning and fake think tanks is becoming a hallmark of modern influence operations. By mimicking the structure of a legitimate NGO, the actors aimed to bypass the immediate skepticism that greets anonymous social media accounts.

Technical Analysis of the IBI Website

A technical audit of the IBI’s web properties revealed a staggering level of automation. Out of 36 sampled articles published on the site, 34 were identified as plagiarized. However, this wasn’t simple “copy-paste” plagiarism. The actors used LLMs to:

1. Translate: Sourcing academic papers or news reports from Russian or other non-English sources.
2. Summarize: Condensing long-form content into digestible articles.
3. Sanitize: Adjusting the tone to sound like “neutral” academic prose while injecting specific talking points.

This process allows an attacker to generate a massive library of “expert” content in a fraction of the time it would take a human researcher. By repurposing legitimate academic work, they inherit the structural logic and vocabulary of real experts, making the deception significantly harder to detect through automated keyword filters.

Note: This “manufactured infrastructure” serves as a landing page for social media campaigns. When a bot on X (formerly Twitter) shares a link, it doesn’t point to a suspicious Russian domain; it points to the “International Burke Institute,” providing a veneer of credibility that can trick both users and platform algorithms.

The ‘Bad Grammar’ Toolkit: VPNs, LLMs, and Multi-Platform Distribution

The technical execution of ‘Bad Grammar’ relied on a blend of commodity tools and advanced AI. Because OpenAI restricts access to its services in Russia, the operators had to employ a variety of techniques to bypass geographic blocks.

Bypassing Geographic Restrictions

The primary tool for access was the use of Virtual Private Networks (VPNs). By routing their traffic through servers in Europe or North America, the operators masked their Russian origin. However, modern threat detection goes beyond simple IP geolocation. OpenAI’s security teams utilize behavioral signals—such as the timing of requests, the specific sequences of prompts, and the reuse of certain payment methods or API keys—to identify clusters of activity that originate from the same actor, even when hidden behind different VPN exit nodes.

Automated Content Generation

The ‘Bad Grammar’ toolkit allowed a single operator to simulate the output of an entire research department. The workflow typically followed this pattern:

This multi-platform approach is critical. By spreading content across Substack (for long-form “authority”), LinkedIn (for professional networking), and Telegram (for rapid dissemination), the actors created an echo chamber where their manufactured narratives appeared to be coming from multiple independent sources. This is a significant step up from the AI-powered cybercrime surges we’ve seen in other regions, as it focuses on psychological influence rather than direct financial theft.

Linguistic Forensics: Identifying the ‘Svetofor’ Artifacts

Despite the use of advanced LLMs, the ‘Bad Grammar’ campaign lived up to its name by leaving behind distinctive linguistic artifacts. These errors are the “smoking guns” of AI-powered propaganda, revealing the underlying Russian influence through failures in machine translation and cultural context.

The ‘Svetofor’ Case Study

One of the most telling examples identified by researchers was the misuse of the word “svetofor” (светофор), the Russian word for “traffic light.” In Russian political discourse, “svetofor” is sometimes used metaphorically to describe a specific type of political coalition or a signaling system.

When the LLM was tasked with translating these concepts into English, it often failed to find the culturally appropriate idiom. Instead of using terms like “green-lighting” or “political coalition,” the generated text would literally refer to a “traffic light” in contexts where it made no sense to an English speaker.

Identifying LLM Signatures

Beyond specific idioms, researchers use statistical methods to detect AI-modified content. These include:

• Perplexity Analysis: AI-generated text often has a lower “perplexity” (it is more predictable) than human writing.
• Burstiness: Human writing tends to have varied sentence lengths and structures, whereas LLMs often produce a more uniform rhythm.
• Instructional Leakage: Occasionally, the bots would accidentally post the AI’s internal instructions, such as “As an AI language model, I cannot generate content that…” or “Here is a summary of the article in a pro-Russia tone.”

These artifacts allow cybersecurity analysts to build “fingerprints” for specific campaigns. Once a signature like the “Svetofor” error is identified, it can be used to scan platforms for other accounts participating in the same operation.

Detection and Mitigation: The LLM Provider’s Perspective

For OpenAI and other LLM providers, the ‘Bad Grammar’ disruption represents a shift toward proactive threat hunting. They are no longer waiting for reports from external researchers; they are analyzing how their models are being used in real-time.

Account Clustering and Behavioral Analysis

The primary defense mechanism is account clustering. Security teams look for patterns that link seemingly unrelated accounts. For example:

• Accounts that were created within the same 10-minute window.
• Accounts that use the same credit card or phone number for verification.
• Accounts that consistently use the same “jailbreaking” prompts to try and bypass safety filters.

In the case of ‘Bad Grammar,’ OpenAI was able to identify a cluster of accounts that were all being used to generate content for the IBI website and its associated social media personas. By banning the entire cluster, they effectively decapitated the operation’s content engine.

The Privacy vs. Security Balance

This proactive stance comes with challenges. LLM providers must balance the need to monitor for abuse with the privacy of their legitimate users. This is why behavioral analysis (how the tool is used) is often more effective and less intrusive than content analysis (reading every prompt). By focusing on the mechanics of the abuse—VPN usage, rapid-fire API calls, and account linking—providers can disrupt state-sponsored actors without compromising the privacy of the broader user base.

This defensive posture is essential as we see Russian APTs weaponizing OAuth and other protocols to gain deeper access to Western digital infrastructure. The AI layer is simply the newest front in this ongoing conflict.

Impact Assessment: The Brookings Breakout Scale

Despite the technical sophistication of using LLMs to create a fake think tank, the ‘Bad Grammar’ campaign ultimately failed to achieve significant impact. To quantify this failure, researchers use the Brookings Breakout Scale, which measures how far an influence operation “breaks out” of its initial bubble.

Defining the Scale

• Category One: The operation stays within its own network of bots and fake accounts.
• Category Two: The operation is noticed by a small number of real users but fails to gain traction.
• Category Three: The operation is picked up by mainstream media or influential real-world figures.
• Category Four: The operation prompts real-world action or policy changes.

The ‘Bad Grammar’ campaign was assessed at Category One or Two. While it produced a high volume of content, it failed to engage authentic audiences. The posts on X and LinkedIn received very few likes, shares, or comments from real people.

The ‘AI Deflationary Spiral’ in Propaganda

This lack of engagement illustrates a phenomenon known as the AI deflationary spiral. As the cost of generating content drops to near zero, the sheer volume of “noise” on social media increases. However, as the volume increases, the value and impact of any single piece of content decrease.

Users are becoming increasingly skeptical of “perfectly” written but generic content. When an operation like ‘Bad Grammar’ floods the zone with AI-generated articles, it often creates a “dead internet” effect where bots are simply talking to other bots, while human users tune out the noise entirely.

Future Outlook: The Convergence of Infrastructure and Intelligence

The disruption of ‘Bad Grammar’ is not the end of AI-powered influence operations; it is merely the end of the beginning. As state actors learn from these failures, we can expect several trends to emerge:

1. Hyper-Personalization: Moving away from broad “think tank” narratives toward highly targeted content designed to exploit specific local grievances or micro-communities.
2. Deepfake Integration: Combining LLM-generated text with AI-generated video and audio to create even more convincing fake personas. This “multi-modal” approach is already being seen in the AI-edge vision and drone revolutions on the physical battlefield, and it will inevitably migrate to the information space.
3. Decentralized Infrastructure: Using blockchain or decentralized hosting to make it harder for providers like OpenAI or AWS to shut down fake think tanks and news sites.

The battle against ‘Bad Grammar’ and its successors requires a collaborative approach. LLM providers, social media platforms, and cybersecurity researchers must share intelligence in real-time to identify and neutralize these campaigns before they can reach Category Three or Four on the Breakout Scale.

As we move forward, the focus will shift from simply detecting “bad grammar” to identifying the subtle, systemic patterns of manufactured influence. The goal is not just to ban accounts, but to build an information ecosystem that is resilient to the automated deceptions of the AI age. The ‘Bad Grammar’ operation was a warning shot; the real test of our digital defenses is yet to come.