Safety Is a Game Design Problem

EXAMPLE · Among Us style chat constraints; Gated guild or rank communication

Moderation starts before the report button

When people hear "trust and safety," they picture the cleanup crew: a player says something harmful, it gets reported, a moderator reviews it, action is taken. All of that matters — but it's the last line, not the first. By the time a report is filed, the game has already let the interaction happen, and often made it easy, visible, or hard to avoid.

On a Player Driven panel, Christina Camilleri (Netflix Games) broke moderation into three layers: a proactive side (tooling and detection that stop bad content appearing), a reactive side (removing it after), and the one most teams underuse — game design itself.

Communication rules are design choices

Her example: games that drop you into open text chat the second you load in. "Like Among Us, for example, you go in, it's competitive and there's open chat. It's going to be a bad experience." The fix isn't always to lock chat down — it's that who can say what, and when is a design decision that shapes player culture before a moderator sees a single report.

From the same conversation:

• Earn your voice. Kim Wilke described a game where players have to join a guild — itself gated behind a rank — before they can communicate at all. You play solo for a while first, which gives you time to learn the norms before you get a microphone.
• Templated speech. Club Penguin let kids talk in preset phrases. A smart start — but over-censor until "every second word you tried to hit enter on was censored," and you've taught players the game is broken, not safe.

Easy, hard, what happens when it goes wrong

The lesson isn't "every game should restrict chat." It's that every social feature is a safety decision, whether you treat it like one or not. Open voice chat makes fast coordination easy — and makes harassment immediate and personal. That's not just a communication feature. It's a design choice about what your game makes easy, what it makes hard, and what a player can do when it goes wrong.

When you look at any social surface, run it through three questions:

1. What behavior does this make easy? 2. What behavior does this make hard? 3. What happens when it goes wrong — and who has to absorb it?

Your move

Take the riskiest social surface in your game — open chat, DMs, lobbies, names. Ask: what does it make easy, what does it make hard, what happens when it goes wrong? Then mark one design change that prevents harm the report queue is currently mopping up.

EXAMPLE · Minecraft Good Game; CyberSafe lessons

Teach players the norms you want

Rules tell players what not to do. Norms teach players what kind of community they've just walked into. The difference matters more than it looks. If the first time a player learns a rule is the moment they get warned or banned, you've already missed the teaching moment — and you've cast yourself as the cop instead of the host.

A code of conduct buried on a website isn't a norm. The player-facing version sounds like answers to real questions: How do we treat each other here? What kind of competition is welcome? What do I do when I see something harmful? Players can't follow a social contract they've never been shown.

Let players help write the rules

At a Player Driven session, Carlos Figueiredo, Director of Player Safety at Minecraft, described building Good Game — a CyberSafe title where the core mechanic is players authoring their own norms instead of receiving them.

That reframes safety from control to participation. The game doesn't hand kids a policy — it drops them into dilemmas and lets them practice. A teammate is cheating with a hack pack but it's helping you win: do you report it? Someone has an inappropriate username: do you flag it or ignore it? The chat is being spammed with pings until it's unbearable: what's the proportionate response? Each one is a social learning moment, not a clause in a document. And Figueiredo is careful that this adds to the studio's systems rather than replacing them: "This is on top of existing systems — chat filtering, player abuse reporting. As a player, what can you do to also play your part?"

Turn a rule into a norm

The move is small but it changes everything: rewrite the prohibition as a description of the community you want.

• Rule: "Do not harass other players." → Norm: "We compete hard, but we don't make anyone feel targeted."
• Rule: "Do not spam chat." → Norm: "We use chat to help the group play, learn, and connect."
• Rule: "Report bad behavior." → Norm: "If something feels unsafe, we use the tools and help keep the space playable for everyone."

Rules are still necessary — enforcement needs a clear line. But a norm teaches the behavior before anything goes wrong, and it treats players as people you trust to keep the place good, not just risks to be managed.

Your move

Take one safety rule from your game or community and rewrite it as a player-facing norm. Start the sentence with: "In this community, we…" — for example, "In this community, we compete without targeting people personally." Then put it where players actually see it, before they ever hit the report button.

EXAMPLE · Fortnite voice chat opt-outs; Destiny 2 privacy settings; Rainbow Six Siege muting and reporting

Give players agency over their experience

There's a business case for safety that too many teams miss, and Christina Camilleri (Netflix Games) put it in five words on a Player Driven panel. Trust and safety often gets treated like security — a cost center, something you do for legal reasons, never something that drives the success of the game. She thinks that's exactly backwards.

She backs it with research: studies she cited showed a majority of players would quit a game over harmful interactions, and many who stayed cut their spending after running into bad behavior. The point isn't that safety is only about retention. It's that players are far more likely to stay when they feel they have some control — and if a player has to endure harm until a moderator can review a report, the system is asking them to absorb too much.

Agency means choices in the moment

Reporting tells the studio what happened. It does nothing for the player right now. Real agency is a set of immediate, low-conflict controls a player can reach for the instant an interaction starts to feel bad — mute, block, recent-player controls, voice opt-outs, friend-only communication, privacy and visibility settings. These don't replace moderation. They give players a way to protect their own experience while the larger system does its slower work.

Crucially, good controls fit the context. Camilleri pointed to how the best games tailor this:

A competitive match carries different risks than a cozy co-op space. A child account has different needs than an adult one. Voice creates more emotional intensity than text. One-size-fits-all controls fail somebody. The ADL, which Camilleri referenced, makes the same case: open interaction systems without clear player controls exacerbate harm, especially for marginalized players — and being able to tailor your own experience is as important as having moderation at all.

Three questions for any control

When you evaluate the agency your game gives a player, ask:

1. Can the player act immediately, without waiting on review? 2. Can the player act without escalating the conflict? 3. Can the player control future exposure, not just this one moment?

A report button answers none of these well — it says "tell us what happened." A mute says "you can stop hearing this now." A block says "you can reduce future contact." A privacy setting says "you decide who gets access to you in the first place." They work as a stack, not substitutes.

Your move

Choose one player interaction in your game that can go wrong — open voice in a lobby, a trade request, a DM. Name one immediate control the player should have in that exact moment. Start with player control, not punishment.