The useful part
Large language models (LLMs) sometimes fabricate information, including web domains, when answering users' questions. Attackers are registering these hallucinated web domains to host phishing pages. Security researcher Seth Michael Larson, who dubbed this technique "slop squatting," told IEEE Spectrum that these attacks exploit the trust users place in AI tools.
How it works
- "For lots of users asking for a particular resource, the model will answer with the same hallucination some percentage of the time.
- They haven't." Larson advises users to treat AI results with vigilance, just as they would with unknown websites that appeared in search engine results.
- Have that in your mind when you're using these tools." Larson added, "We're not giving enough warnings to users that the output is not something you should just blindly accept.
- Human + AI KnowBe4 empowers the modern workforce to make smarter security decisions every day.
- Trusted by more than 70,000 organizations worldwide, KnowBe4 is the pioneer of digital workforce security, securing both AI agents and humans.
What to take from it
Platform provides attack simulation and training, email and collaboration security, and agent security powered by AIDA (Artificial Intelligence Defense Agents) and a proprietary Risk Score. The platform leverages 15 years of behavioral data to combat advanced threats including social engineering, prompt injection, and shadow AI. ABOUT KNOWBE4 TEAM The KnowBe4 Team delivers timely, expert-driven insights on cybersecurity trends, emerging threat intelligence, human risk and agent security best practices, compliance strategies and industry research to help organizations strengthen their digital defense layer and stay informed, resilient, and secure.
Example or evidence
- Users implicitly will take what an AI tells them to be authoritative.
- The trust boundary between this information and your computer is you." IEEE has the story: https://spectrum.ieee.org/ai-cyberattacks-llm-slop-squatting Topics:
- By securing humans and agents, KnowBe4 leads the industry in workforce trust and defense.
Details worth keeping
They'll assume some security trust decisions have happened. You should not be trusting this," Larson says. "As in any engineering field, you can't just go off of vibes.
Related coverage
- Cybersecuritynews: Fake storefronts appeared days after the launch of Jev, an artificial intelligence model that returns decisions rather than written answers.
- Bleepingcomputer: Threat actors are abusing trusted AI platforms to host malicious content, poison search results, and trick users into installing malware.
- Cofense: Threat actors are impersonating OpenAI and ChatGPT in phishing emails, using fake subscription payment alerts to create urgency and trick users into updating their billing information.
- Knowbe4: Scammers are using AI to generate phishing pages that impersonate antivirus products, according to Malwarebytes.