The App Store Is a Consent Surface

The app store used to sell itself as a velvet rope. Safer than the open web. Cleaner than a random download. Reviewed, ranked, sandboxed, paid through the house.

That story gets uglier when the thing being sold is a machine for sexual violation.

On July 17, 2026, San Francisco City Attorney David Chiu sent legal notices to Apple and Google demanding action against 13 apps that could create AI-generated nonconsensual intimate images, according to WIRED and TechCrunch. The notices did not only ask the companies to remove listed apps. They pushed at the commercial machinery around them: app review, developer accounts, search, payments, and the platform fee.

That is the important part.

The lazy framing is content moderation. A bad app slipped through. Someone flagged it. The company removed it. Roll credits.

But nonconsensual intimate-image tools are not just files on a shelf. They are businesses that need distribution, payment, renewal, recommendation, ratings, and the soft legitimacy that comes from sitting inside a trusted storefront. A phone does not say “this app is morally clean.” It says something more useful to the user and more profitable to the platform: this app is available here, and you can pay us without leaving the room.

That is a consent surface.

The Gate Is Part Of The Product

Apple and Google both told reporters they prohibit this kind of content. Google said it had suspended the five Android apps flagged by Chiu’s office and removed hundreds of violating apps. Apple said it had removed three of the named apps and was moving against some developer accounts.

Good. Also insufficient.

Reactive cleanup treats abuse as a terms-of-service problem. It waits for watchdogs, journalists, victims, schools, parents, city attorneys, and search screenshots to do the dangerous work first. Then the platform appears with a mop and a quote about safety.

The harder question is whether the store should profit from dual-use intimacy tools until someone proves the harm loudly enough.

The Tech Transparency Project reported in January that it found dozens of AI apps across the Apple and Google stores that could create sexualized images of women. In April, it reported that store search and ads could still steer users toward these tools, despite platform rules against explicit sexual abuse.

The scale matters because it kills the “edge case” defense. WIRED reported that apps identified by the watchdog were estimated at roughly 480 million downloads and about $120 million in combined revenue. Those numbers can move. The shape does not. Abuse at app-store scale is not a basement problem. It is a supply-chain problem.

The Disguise Is The Strategy

The ugliest trick is that many of these tools do not need to announce themselves as abuse products.

A recent safety audit from Alaa Daffalla, Sarah Chao, and Eric Zeng found that face-swap apps can present as ordinary entertainment tools while still supporting synthetic nonconsensual intimate image abuse. The researchers identified 420 face-swap apps, manually tested 155 eligible apps, and found that 70 percent had no technical safeguards against creating nude face swaps. The paper is a preprint, so treat it as evidence to weigh, not gospel. But its finding matches the market pattern: the abuse hides inside plausible utility.

That creates a moderation problem with teeth.

If the store only blocks apps that advertise the harm in neon, it will miss products that keep the dirty feature one tap, one template, or one off-platform prompt away. If the store only checks screenshots and copy, it will miss capabilities. If it only responds to exact banned words, it teaches developers better camouflage.

For AI image tools, safety review has to test behavior, not just branding. Can the app generate sexualized images of a real person without consent? Does it reject that path? Does it detect the target as a real person? Does it block nude-source swaps? Does it make abuse harder by design, or does it leave a velvet curtain over the obvious door?

The platform cannot claim innocence if its gatekeeping stops at vibes.

Consent Is Not A Checkbox After The Fact

Synthetic intimate abuse is not fake harm because the image is fake. The target is real. The social blast radius is real. The humiliation is real. The coercive power is real. The school, workplace, relationship, and family consequences can be real before any lawyer uses a precise term.

This is where the consent language matters.

The phrase “nonconsensual intimate image” can sound sterile. It should not. It describes a system that takes an ordinary photo and turns someone’s body into a weapon against them. It also describes a market that treats the victim’s lack of consent as a missing input, not a hard boundary.

California law now gives that boundary more force. California Civil Code Section 1708.86 creates civil claims around digitized sexually explicit material and includes liability for people or entities that knowingly facilitate or recklessly aid prohibited conduct. The statute also lets public prosecutors seek penalties. I am not giving legal advice. The point is cultural before it is procedural: the law is starting to look past the image generator and toward the services that keep it alive.

That is why this story belongs in the same room as sex-tech cloud consent and desire surveillance. Intimacy tools do not become ethical because they are wrapped in UI chrome. A marketplace can be part of the intimate system, even if no one in Cupertino or Mountain View wants to think of it that way.

What A Real Fix Looks Like

The fix is not one grand ban button. It is boring and structural.

App stores should run adversarial safety tests on AI image, face-swap, avatar, and “beauty” tools before approval and after major updates. They should treat explicit nonconsensual-image capability as a high-risk behavior, not as a marketing-category issue. They should block payment flows for services that fail those tests. They should remove repeat developers, not only repeat apps. They should stop search, autocomplete, and ad systems from laundering demand into discovery.

They should also publish enough enforcement data to make the trust claim auditable. How many apps were rejected for synthetic intimate abuse capability? How many were removed after approval? How many used in-app purchases? How fast did the platform respond after notice? How many developer accounts came back under new names?

Safety theater hates logs. Real safety leaves receipts.

There is a risk here, too. Platform control can become overbroad. Age-verification schemes can become surveillance traps. Sexual expression and adult speech can get swept into crude safety filters. A serious consent regime cannot become a prudish dragnet.

But this is not a hard case for adult freedom. Nonconsensual sexual deepfakes do not expand erotic expression. They convert another person’s face, body, reputation, and fear into product inventory.

The clean line is consent.

Bottom Line

San Francisco’s move matters because it names the marketplace as part of the abuse stack. Not just the model. Not just the user. Not just the developer hiding behind “face swap” copy. The store, the search box, the payment processor, and the platform fee all sit close enough to the harm to deserve scrutiny.

The app store is not a neutral hallway. It is a machine that decides what reaches bodies.

If Apple and Google want the trust premium, they have to earn it where trust is most expensive. Not in the keynote. Not in the privacy billboard. Here, at the point where a stranger’s photo can become a threat.

Consent should be enforced before the checkout screen.

Sources

• WIRED, “San Francisco Demands Apple and Google Delete AI ‘Nudify’ Apps From App Stores,” July 17, 2026.
• TechCrunch, “Apple and Google ordered to purge ‘nudify’ apps from App Stores,” July 17, 2026.
• Tech Transparency Project, “Nudify Apps Widely Available in Apple and Google App Stores,” January 27, 2026.
• Tech Transparency Project, “Apple and Google Are Steering Users to Nudify Apps,” April 2026.
• Daffalla, Chao, and Zeng, “Dual-Use AI Face Swap Apps Are Mostly Unsafe: A Systematic Safety Audit,” arXiv, May 23, 2026.
• California Civil Code Section 1708.86, “Digitized sexually explicit material,” amended effective January 1, 2026.