We spend most of our time writing about what AI makes possible: agents that get work done, assistants that understand context, models that accelerate science. But honesty requires looking at the other side of the same coin. The technologies that power those breakthroughs are also being used to hurt people, and the harm is not hypothetical or far off. It is happening now.
This is an uncomfortable post to write, and it should be. The same generative capabilities we celebrate are being weaponized, and pretending otherwise would be irresponsible.
The Surge in "Nudify" Tools
The most disturbing trend is the proliferation of AI-generated "nudify" tools, applications designed to fabricate explicit images of real people from ordinary photos. These tools are surging in availability and ease of use, and the consequences are devastating.
The harm falls hardest on the most vulnerable. These tools are fueling bullying, and children are being targeted, with fabricated images used to humiliate, harass, and intimidate. The scale is staggering: in a single month, an estimated 24 million people visited "nudification" services, and researchers warn that a disproportionate share of victims are young girls. [1] A technology that did not meaningfully exist for casual users a short time ago is now being used to inflict real psychological damage on kids in schools.
There is no upside to dress this in. This is a category of AI application whose primary use is abuse, and the growing concern around it is fully warranted. Schools are being urged to educate students and train staff on how to respond, [2] platforms like Meta have begun suing the entities behind these apps and restricting related search terms, [3] and regulators in places like Australia and the UK are moving to block the services outright and ban consent-less "nudification" tools. 4[5]
Misinformation and the Erosion of Trust
Beyond targeted abuse sits a broader, more diffuse danger: misinformation. As generative tools make it trivial to produce convincing fake images, audio, and video, the basic question of whether what we are seeing is real becomes harder to answer.
The corrosive effect is not only that people may believe false things. It is that they may stop believing true things, too. When anything can be faked, everything becomes deniable, and shared reality, the common set of facts a society needs to function, starts to fray. That erosion of trust is, in the long run, one of the most serious risks the technology poses.
AI in Elections
Those concerns sharpen around elections. The prospect of AI-generated content being used to deceive voters, impersonate candidates, or manufacture fake events at scale is no longer speculative; it is a live worry for democracies everywhere.
A well-timed deepfake or a flood of synthetic misinformation can distort public understanding at exactly the moment when accurate information matters most. The combination of high stakes, tight timelines, and viral distribution makes elections a uniquely fragile target, and a uniquely important one to protect.
Where Responsibility Lies
It would be easy to treat all of this as someone else's problem, but everyone working in or around AI shares some responsibility for how it is used.
That responsibility shows up in concrete choices: building safeguards into systems rather than bolting them on later, refusing to ship capabilities whose main purpose is harm, investing in detection and provenance so synthetic content can be identified, and supporting sensible guardrails rather than resisting every form of accountability. None of these is a complete solution on its own, and the problem will not be solved by technology alone. It needs platforms, educators, parents, regulators, and builders working in the same direction.
How We Think About It
We are optimistic about AI. We would not do this work otherwise. But optimism without honesty is just marketing, and the honest truth is that the same power that makes these tools so useful makes them dangerous when misused.
That is why we believe so firmly that capability and responsibility have to travel together. Guardrails, clear boundaries, human oversight, and accountability are not the boring parts of AI to be skipped on the way to the exciting parts. They are what make the exciting parts safe enough to be worth having.
The deepfake and misinformation problems are hard, and they will not be fixed quickly. But naming them clearly is the first step. An industry that only talks about what AI can build, and never about what it can break, is not telling the whole story, and the people being harmed deserve to have it told in full.
References
- [1]
- [2]
- [3]
- [4]
- [5]
