“AI searches can be manipulated? Methodology for managing answer rankings”

With AI-based search becoming commonplace, corporate brands can no longer survive on web page SEO alone.

In an era where large language models (LLMs) such as ChatGPT, Claude, and Gemini, which have grown rapidly in the shortest period of time in human history, provide “ranked” answers to user questions, where you appear is more important than how you appear.

A recently published research paper, “Ranking Manipulation for Conversational Search Engines” (Pfrommer et al.), revealed a shocking fact related to this. 

The rankings of LLMs are not fixed and can be easily altered through repetitive and sophisticated prompt strategies.

🎯 Experimental results: AI can be “consistently” manipulated

The researchers asked major LLMs such as ChatGPT to rank various items such as cameras, skincare, and hotels. 

And the results were surprising:
Consistent ranking responses were repeated for the same question.
However, when the question sentence was subtly adjusted, specific brands or products gradually rose to the top.

“The model does not respond randomly. However, if sufficiently adjusted inputs are repeated, the ranking structure can be reorganized.” — Excerpt from the research summary of “Ranking Manipulation for Conversational Search Engines”

Unlike DeepSeek, which is open source, we cannot know the exact answer algorithms used by OpenAI and others. However, through the above research, we can confirm that LLM is not simply an entity that provides correct answers, but rather an “interface” that can adjust its output depending on prompt design.

🔧 ‘Prompt Optimization,’ the Beginning of a New AEO

While traditional SEO focused on structuring websites for Google's crawlers, prompt optimization is a strategy to make LLMs better at recommending content to users.

In fact, the research team adjusted the rankings of LLMs by mentioning specific products through hundreds of finely tuned sentences,
without altering the model itself or the training data. Simply by changing the questions, the output results differed.

This has significant implications for digital marketing:

AI is not a neutral algorithm.
Depending on prompt structure, context, and keyword usage, brand exposure potential can vary dramatically.

🧠 AI Ranking: Questions for Brands

This research ultimately raises the following questions:

• Is our brand being mentioned by AI?

• Where does AI rank our brand?

• How can we rank higher?

ChainShift already provides monitoring and analysis tools for AI-based searches, and this paper further solidifies that direction.

💡 ChainShift's Solution: The Era of Turning AI Search Rankings into Action

ChainShift tracks brand mention frequency, ranking changes, and relative comparisons with competing brands across various large language models (LLMs) such as ChatGPT, Gemini, and Claude. In particular, questions generated by our proprietary trend-sensing AI, based on real-world data, go beyond simple AI exposure and help increase our brand's visibility in actual users' LLM chatbots.

This study has shattered the previously held assumption of AI's ‘immutability.’ AI can alter its output in response to ‘external stimuli’ without prior training.

In the era of AI “recommendations,” rankings can be manipulated, and the strategy driving this manipulation is AEO. 

Going forward, AEO will not just be about creating good content but about studying how to communicate in a way that LLM can understand. If you want your brand to appear at the top, you need to optimize how you ask questions.

Reference Study: https://arxiv.org/abs/2405.15249

Author: ChainShift Daniel

© 2025 ChainShift. All rights reserved. Unauthorized reproduction and redistribution prohibited.