Perplexity has long been accused of ignoring internet rules and scraping content without respect. Now, a new report shows these practices may be getting worse. This puts Apple, which is considering acquiring the company, in an awkward position: will it compromise its principles to catch up in the AI race?
Article contents:
- Ongoing accusations of ignoring web rules
- Cloudflare report: new evidence of deliberate disregard
- Perplexity response: "PR stunt"
- Apple’s dilemma: does it really need this headache?
- The bigger question: will Apple compromise its principles?
- Conclusion: a deal that may cost Apple more than just money
- Frequently asked questions
Ongoing accusations of ignoring web rules
Perplexity has long been accused of intentionally bypassing anti-scraping measures to retrieve web content. While the company has historically dismissed these accusations as unfair or the result of misunderstandings, a new report shows that this practice is still happening and may even be worsening.
In June 2024, media outlets accused the company of ignoring the Robots Exclusion Protocol, a widely accepted standard that tells automated crawlers which parts of a website they should not access.
Cloudflare report: new evidence of deliberate disregard
Cloudflare published a report today claiming that even when a server explicitly denies all automated access and includes specific rules blocking crawling from Perplexity’s general web crawlers, Perplexity does it anyway. According to Cloudflare, the company is “going out of its way to get data that it is explicitly asked not to crawl.” This means the company is not only ignoring established norms but actively bypassing technical barriers set by site owners.
Perplexity response: "PR stunt"
In a statement to The Verge, Perplexity called the blog post a “PR stunt” and said that “there is a lot of misunderstanding in the blog post.” The company continues to deny the allegations, often arguing over the definition of “crawling” and “scraping” in specific use cases.
Apple’s dilemma: does it really need this headache?
These accusations put Apple in a tough spot as it reportedly weighs an acquisition of Perplexity. Nothing prevents Apple from closing the deal, and many believe it will. But given Apple’s strong stance on privacy and doing what is right, should it really acquire a company with such a controversial history?
It is entirely possible that Apple believes under its culture, leadership, and ethical practices, it can cleanse the incoming technology of its alleged “past sins.” But this will not erase the fact that Perplexity only reached where it is because it allegedly did what it did.
The bigger question: will Apple compromise its principles?
If Apple decides to acquire Perplexity, it could mean (hopefully) that the company did its due diligence and found nothing legally compromising. But it could also mean that Apple feels enough pressure to compromise, even slightly, on its core principles to catch up. If this turns out to be the case, it will be even more disappointing than its current lag in artificial intelligence.
Conclusion: a deal that may cost Apple more than just money
The potential acquisition of Perplexity represents a true test of Apple’s values. Will it prioritize speed and catching up to competitors at the expense of its reputation as an ethical company that plays by the rules? The decision it makes will not only determine its future in AI-powered search, but it will also tell us a lot about the kind of company Apple has become in the fiercely competitive AI era.
Frequently asked questions
What is web scraping?
It is the automated extraction of data from websites. While it has legitimate uses, it becomes controversial when used to pull massive amounts of content without permission or against the website owner’s wishes.
What is the Robots Exclusion Protocol (robots.txt)?
It is a text file placed by site owners on their servers to guide automated crawlers (such as those used by search engines) on which pages they should or should not access. It is not a law, but a voluntary and widely followed standard.
Why would Apple want to buy Perplexity?
Because Perplexity has developed a powerful AI-powered answer engine, which Apple currently lacks. Acquiring it would significantly accelerate Apple’s entry into this field and give it a ready-made competitor to Google.
