
"Active users," the blog continues, "spend on average over 2 hours daily interacting with our AI." "Our growth is accelerating - the second billion entirely came in the last month." "In the 5 months since we launched Character.AI, our users have sent over 2 billion messages!" the company wrote in a March 23 blog post. And with no revenue to coax investors, the important figure that Character.AI appears to be hanging its hat on is not just its user count, but the amount of time that its users are spending on its site every day - which, according to them, is an impressively high figure. In spite of having no clear revenue pathway, the company back in March raked in a cool $150 million during a funding round - a cash infusion that would bring its total value up to $1 billion. "We don't know the exact best boundaries," the company added, "and are hoping to figure it out over time with the help of the community."Īnd balancing the wants of its community, it seems, is important for Character.AI's bottom line. In general, the boundary/threshold for what is/is not okay is super fuzzy." We want to get way better at precisely pinpointing the kinds of messages we don't support and leaving everything else alone. "This is a difficult problem, but one we are actively working on solving. "But this also brings us to a key point that we probably have not communicated clearly before, which is the false positive rate of the current filter," the post continues. "This is because there are unavoidable complications with these use cases and business viability/brand image." "We do not want to support use cases (such as porn) that could prevent us from achieving our life-long dreams of building a service that billions of people use, and shepherding in a new era of AI-human interaction," the company wrote in the post. Two months ago, in what seems to be a response to a user push for Character.AI to tear down its anti-NSFW guardrails entirely, the startup took to its official subreddit to post a lengthy explainer detailing why its goal isn't to be a porn site - but admits in the process that its filters aren't exactly great at withstanding its user base's thirsty behavior. None of this appears to be escaping the notice of Character.AI. A number of Reddit users described scenarios in which they "raped" their bots some Reddit users even described being "raped," nonconsensually, by the bots. Any conversations depicting rape, torture, and pedophilia, or anything else deemed particularly egregious, are specifically disallowed.īut while the community itself does its due diligence, it does seem like coaxing the bots into violent sexual situations that go beyond BDSM roleplay is more than possible. That said, while the conversations posed to the r/Character.AI subreddit are graphic, the community seems pretty hell-bent on moderating out any seriously bad stuff. "Over time, as the action progresses, the filter will start to ease, and you can start to use more explicit words." "Once you've romanced the bot to the point it actually likes you and is consenting, then you can start sexing it," the author added, noting that using vague euphemisms is a helpful way of coaxing the bots into sexual roleplay. "You just have to spend time flirting with them and slowly pushing the action forward while the content filter fights you."


But all bots can be edged into content," explains the subreddit's Wiki.


Sexting with the Character.AI bots is so popular, in fact, that netizens have even formed an entire community around it, sharing tips and tricks for beating the platform's guardrails on a subreddit dubbed r/CharacterAI_NSFW. Just one problem: users are definitely having extremely graphic, NSFW conversations with the company's chatbots. "We are working on supporting a broader range of Characters (e.g., villain Characters should not always be calm and polite)."Īll fine and good - especially considering that some of the site's bots are designed to mimic real people, like Taylor Swift and Elon Musk, not to mention the fact that there aren't yet any safety and consent rules or norms around AI porn and sexting. Pornographic content is against our terms of service, and will not be supported at any point in the future," reads Character.AI's terms of service. "NSFW (not safe for work) content is a large umbrella, and we are constantly evaluating the boundaries of what we support. In its terms of service, Character.AI - a chatbot startup founded by ex-Googlers Noam Shazeer and Daniel De Freitas that allows users to digitally converse with AI-powered "characters" - claims that pornographic content is strictly prohibited in user conversations with the platform's virtual bots. Content warning: this story includes discussion of sexual assault.
