About the Field

Any online platform that brings people together must engage in due diligence to keep users safe. Dating apps like Tinder and Hinge, Social media apps like Facebook and X, “gig economy” services like Uber, Care.com and Angie’s list, and online gaming platforms like Roblox and Epic Games all bear liability for interactions on and off their platform. These services employ robust Trust and Safety departments to uphold community guidelines and ensure that dangerous individuals are not grooming children, luring victims into dangerous situations, stealing data, spreading misinformation or using the platform to perpetuate fraud schemes. If you enjoy these platforms, your expertise and experience with them may make you uniquely qualified for trust and safety roles in some of the largest tech employers in the world.

Job Titles to Know

Safety Analyst

This job title could take several forms including “Child Safety Analyst” , “User Safety Analyst”, “Platform Safety Analyst”, etc. These positions could be proactive in nature, meaning that it will be your job to identify emerging risk indicators to prioritize investigations. They could also be reactive in nature, meaning that you’re more of an investigator who will respond to content that gets flagged by automatic monitoring algorithms (e.g. High-risk words or phrases are used in direct messages) or when other users file complaints.

Escalation Analyst

The word “Escalation” is telling in the Trust and Safety Space. An “escalation” occurs after an initial flag has brought certain content to the Trust and Safety Team. First-line analysts or investigators then review the content and, if its determined to be severe enough, they will escalate the content to the Escalation Team who can take more severe action. That action may involve reporting to law enforcement, banning users from the platform, or even taking civil action.

Threat Detection

The phrase “Threat Detection” almost always implies the use of machine-learning or AI to identify bad actors on a platform. Since bad actors are always finding new ways to execute schemes, prediction models have to constantly be tuned to adapt to emerging threats. In some companies, these algorithms can direct the workflows for hundreds or thousands of analysts and investigators.

Get Qualified

Trust and Safety roles vary greatly and there’s not a certification program or hard-skill that is universally in demand. Analysts with investigation support, OSINT or data analysis skills may want to highlight those when applying.

Job Search Keywords

User Safety, Moderation, Community Safety, Escalation, Intelligence Analyst

Example employers

Uber, Lyft, Hinge, Tinder, Bumble, Epic Games, Roblox, Microsoft, Sony, Care.com, Linkedin

Learn More

Jeff Dunn is the VP Of Trust and Safety for Hinge. His personal site is a wealth of knowledge for anyone looking to learn more about the Trust and safety industry. You can follow Jeff HERE.

Kyra Wakin is a former Roblox Trust and Safety Manager who now runs her own consulting firm. Her site is a rich resource for anyone looking to explore the Trust and Safety Space. Her site includes job boards, resume tips and key words for job searches. You can utilize her resources HERE.