Apple has updated its App Store guidelines to tackle AI-generated nonconsensual intimate imagery. The new App Store AI deepfakes rules require developers to proactively prevent and remove this content, with the threat of removal from the App Store if they fail to act. Here’s a breakdown of what’s changing and why it matters.
Apple Expands App Store AI Deepfakes Rules
Apple has expanded its App Store moderation rules to cover AI-generated nonconsensual intimate imagery. Sexually explicit content was already banned, but under the updated guidelines, developers must now proactively prevent and remove AI-generated deepfakes from their platforms. Apple warns that apps with offending content could be removed from the App Store if developers fail to act.
The move represents one of the clearest signals yet that Apple intends to hold app developers accountable for how their AI tools are used. The App Store AI deepfakes policy shift places responsibility squarely on developers rather than relying solely on user reports or post-publication review.
Updated Rule 1.2 Targets AI Content Creation Apps
The revised 1.2 rule explicitly names AI content creation apps and could affect apps like Grok, among others. Users have complained about xAI’s Grok chatbot being used for nude and sexualized deepfakes. Earlier this year, Apple told xAI it would remove Grok from the App Store if it continued to permit nude deepfakes. xAI subsequently put safeguards in place, and Apple approved an updated version of the app.
The App Store AI deepfakes rule change signals that Apple is willing to enforce consequences when AI tools are misused. Developers of AI image generators, chatbots, and editing tools now face explicit obligations to monitor and moderate outputs.
New Guideline 2.5.19 Cracks Down on Hidden Features
Apple also added guideline 2.5.19 to clarify that developers who add features or functionality not present in the version of an app submitted for review risk removal from the App Store and Apple Developer Program. Apple links this rule to “nudification” apps that create sexually explicit images of real individuals without their consent.
The company suggests some apps have concealed their true purpose during review and added the nudify feature after approval. Apps containing hidden or undocumented features are also prohibited on the App Store. This includes apps that misrepresent their purpose during review and only reveal harmful capabilities—such as “nudification”—after approval, a tactic commonly known as bait-and-switch.
Section 5 Extended to Alternative App Distribution
Along with changes focused on AI-generated content, Apple updated section 5 to extend its ban on apps that facilitate human trafficking or child exploitation to alternative app distribution. This expansion ensures that the same prohibitions apply regardless of how apps reach users.
“We do not hesitate to remove apps, and terminate the developers behind them, when they violate these guidelines,” reads Apple’s developer news post on the rule changes.
Why the App Store AI Deepfakes Crackdown Matters
The updated App Store AI deepfakes rules signal a tougher stance on apps that enable the creation of nonconsensual intimate imagery. Developers of AI content creation tools must now take active steps to prevent misuse or face removal from the platform. For users, the changes aim to reduce the spread of harmful deepfakes through apps distributed on the App Store.
The new guidelines also close a loophole where apps could sneak harmful features past the review process by hiding them until after approval. By explicitly prohibiting bait-and-switch tactics, Apple is tightening enforcement against apps that misrepresent their purpose.
Apple’s move comes amid growing concern over AI-generated deepfakes and their potential for abuse. While the company acknowledges it cannot stop all harmful content on the web, it maintains that it should not contribute to the problem by allowing such apps on its platform.
What Developers Need to Know
Developers of AI content creation apps should review their moderation systems and ensure they can detect and remove nonconsensual intimate imagery. Apple’s guidelines make clear that passive moderation is no longer sufficient. Apps that allow users to generate realistic images of real people must implement safeguards before those features reach the App Store.
The App Store AI deepfakes rules also apply to apps that add new features after approval. Any functionality not present in the version submitted for review could trigger removal, especially if that functionality enables harmful content generation.
Apple’s updated guidelines reflect a broader industry shift toward accountability for AI-generated content. By targeting both the content itself and the deceptive practices used to distribute it, Apple is addressing two distinct problems: the harm caused by deepfakes and the dishonesty of developers who hide their true intentions during review.
The App Store AI deepfakes policy is not a complete solution. Harmful content can still spread through web browsers and other distribution channels. But Apple’s stance is clear: apps distributed through its platform must meet specific standards, and developers who violate those standards face consequences.
For now, the updated guidelines are in effect. Developers can review the full updated guidelines on Apple’s website.
