What is Koko?

Koko is a nonprofit dedicated to ending the youth mental health crisis by reaching young people where they are—online. We provide free, digital mental health support directly on the social platforms young people use every day, so help is just one click away.  

How can I support Koko’s mission?

You can make a donation directly to support our mission here. If you have other ideas on how you might help us expand our impact, feel free to reach out to us here.   

Is Koko a nonprofit?

Yes, we are a US-based 501(c)3 nonprofit organization. 

I have questions or feedback. Who should I contact?

You can reach out to us directly here. 

I’d love to get involved. Are you looking for volunteers or partnerships?

Yes! We are always eager to collaborate and hear from those inspired by this work. You can reach out to us directly here to share more about your ideas.

How does Koko keep young users safe?

Koko exists to keep young people online safe and thriving. To ensure our work is protecting its users, we have a lot of safeguards in place. 

• To access our services, users must be at least 13 years old. 
• Koko maintains comprehensive ethical oversight through an independent external ethics review board that evaluates research and product development. 
• Additional protections include: no collection of personally identifiable information, automated content moderation, complete user anonymity with no direct messaging capabilities, and clear crisis response protocols. All research and platform changes undergo thorough ethical review before implementation.

How does Koko protect young users from sharing personal information with other users?

Koko protects its young users by not allowing its users to chat privately with each other, which significantly limits the likelihood of sharing personal information. To ensure the safety and anonymity of our users, we use large language models to identify ‘off-platform’ attempts by users to share personal information i.e., user names, phone numbers. If a user violates our community policies, we remove them from the platform or send a warning, depending on the nature of the violation.

How does Koko protect its users’ privacy?

Please find our comprehensive privacy policy here.

What is Koko’s consent policy for its users?

Users must agree to Koko's terms of service and privacy policy to use the service. Users can choose whether to participate in research studies conducted by universities partnering with Koko. These research studies have consent policies that are reviewed and approved by each university's Institutional Review Board.

How is AI used in Koko's tools?

Koko uses AI for content moderation and to help guide users to services that best meet their needs. You can view all of our ethical commitments here including our detailed protocol for ethical use of AI.

How do you ensure that your support features are safe for young people? What if they get harmful advice from the community?

Koko prioritizes safety in our peer support service, which we've refined over ten years. Our safety systems and algorithms have been utilized by major platforms including Pinterest, Twitch, OpenAI, and Tumblr.

We maintain safety through multiple layers of protection:

• Users cannot directly message each other
• Content is monitored through AI and human review to enforce strict safety standards
• Our systems identify users who may need additional help and connect them with appropriate crisis resources - we partner with over 300 helplines worldwide
• No images or videos are allowed on the platform
• Automated systems remove personal identifying information
• Our approach to connecting high-risk users with help has been validated through randomized trials conducted with research partners, including Harvard University

How does a user access Koko's services?

Koko is integrated into many online platforms - some through in-platform integrations and others through advertising and direct targeting. Users can also access some of our services directly at getsupport.org.

How is Koko's model different from traditional forums or peer chat options?

Koko takes a unique approach to peer support, distinct from traditional forums and chat services. Users share brief, anonymous messages of hope for a user who is struggling. The platform teaches users how to support each other constructively, whilst exposing users to key elements of cognitive therapy. Our approach is backed by research - a randomized controlled trial with MIT and Northwestern showed that users experienced reduced depression symptoms and improved their ability to manage stress through cognitive reframing techniques.

How does automated content moderation take place? How do you continually improve the tool and what industry standards do you peg your practices to?

Koko employs a multi-layered approach to content moderation, developed through our deep expertise in AI safety. Our moderation system combines:

• AI-powered automated screening of all content
• User reporting capabilities for flagging concerning content
• Human review by trained team members for flagged content
• Continuous model improvement based on moderator feedback
• Our content policies have been developed and refined over many years in consultation with clinical research psychologists. We maintain extremely strict safety standards, aiming for fewer than 1 in 10,000 pieces of content to violate our highly conservative policies.

How does Koko continue to ensure that its platform is improving and keeping up with the latest best practices with AI as an ever-evolving space?

Our approach balances innovation with responsibility, ensuring that any new AI capabilities are thoroughly validated before being integrated into our platform. This process is strengthened by the guidance of our ethics advisory board, who bring deep expertise in responsible AI deployment.

To regularly maintain platform excellence and keep up with the latest AI best practices, our Koko team ensures:

• Rigorous evaluation of new technologies before implementation, requiring clinical validation through evidence-based methods
• Continuous refinement of our AI moderation systems based on real-world feedback and outcomes
• Close collaboration with leading AI companies and research institutions
• Regular review and updates to our content policies in consultation with clinical research psychologists
• Commitment to transparency about AI use and its limitations
• Oversight from our external ethics advisory board, comprising world experts in AI ethics and responsible AI use
• Adaptation of our practices as AI technology evolves, prioritizing ethical implementation over rapid adoption

How has Koko responded to the criticism it received in 2023 about its testing practices for youth users? What specific changes have been made to Koko's research protocols and consent practices?

At Koko, the success of our services depends on the trust of our users. Following recent questions surrounding our research processes, our team has been dedicated to ensuring transparency across our operations, with an emphasis on our research mechanisms. Koko has formalized its research processes and developed an external advisory committee that reviews research and product initiatives. You can also review our ethical research commitments here.

How does Koko obtain informed consent for its testing practices, particularly for minor participants?

To use our service, users must consent to participate in research. For university-led research, users may need to provide additional consent, depending on IRB recommendations.

Can you explain Koko's current Institutional Review Board (IRB) and research oversight mechanisms?

Koko regularly collaborates with academic institutions in conducting research and leverages university IRBs to ensure oversight throughout this process.

Does Koko receive funding or have partnerships with social media companies?

Yes, Koko receives funding and ad credits to help enhance the safety of these platforms. Koko never sells user data.

Has Koko's work been studied by third-party, clinical professionals and credited experts?

Yes. Koko’s platform has been analyzed in over eight peer-reviewed publications, including three randomized controlled trials. You can view this work here.

Does Koko work with mental health professionals to conduct research and evolve its technology?

Yes, Koko regularly collaborates with clinical researchers and practitioners.

What processes has Koko implemented to ensure the ethical development and use of its technology?

Koko has established ethical commitments, as well as clearly defined internal and external mechanisms for ensuring safety and independent third-party review.

How does Koko account for cultural sensitivities in the development of its technology?

At Koko, we hope to continue expanding the benefits of our services to young people everywhere. Our goal is to ensure that our services can continue to meet the needs of our users no matter where they live, therefore we are constantly adapting our services to ensure quality localization and address cultural sensitivities. To date, Koko has adapted its services for regions such as Taiwan and India, with support and guidance from local influencers and users.

What is the role of Koko's Ethics Advisory Board?

The Ethics Advisory Board is a team of professional ethicists and scholars specializing in technology, research, and health ethics. The Ethics Advisory Board has two primary roles: first, providing ethical advice and counsel to Koko on potentially difficult operational decisions and initiatives, research pursuits, and delivery; and, second, providing external, third-party ethical oversight and accountability to ensure that Koko is staying true to its stated ethical commitments and principles.

How does the Advisory Board contribute to Koko's ethical decision-making?

The Ethics Advisory Board acts as a sounding platform and potential ethical check on Koko’s initiatives and maintains a level of independence from Koko that lets it speak honestly and directly to the ethical issues Koko encounters. Whenever Koko leadership reaches a key ethical decision point, they reach out to the Ethics Advisory Board. The Ethics Advisory Board explains what values are at stake, what stakeholders should be considered, what ethical principles may be in tension, and how best to proceed to serve the interests and wellbeing of Koko’s community. The Ethics Advisory Board is always on-hand for a quick review or consultation.

How does Koko ensure diverse perspectives are represented in its ethical oversight?

Koko’ team is diverse and incorporates a number of different perspectives and Koko consults with varied viewpoints when making crucial decisions about the platform. Importantly, both Koko’s moderation community and the Ethics Advisory Board include people from varied walks of life and with diverse points of view that inform the development of our services. The Ethics Advisory Board in particular ensures representation across different genders and racial and ethnic groups, and people with varied and professional backgrounds and expertise. The Ethics Advisory Board strives to ensure that all voices are heard and consensus is reached when addressing difficult ethical issues.

How does Koko balance technological innovation with ethical considerations?

Koko understands that there is often some tension between innovation and ethics. New technologies can offer effective solutions to difficult problems, but they might also bring about new or increased risks. Our team always aims to assess the possible effects of a new innovation before implementation. This includes a survey of likely consequences and an assessment of stakeholder rights and responsibilities. The team also believes that innovating well is itself an ethical necessity—in a rapidly evolving online mental health ecosystem, it’s important that those who care the most about users’ rights and wellbeing are also those who reach the most people, many of whom are in need. We always strive to innovate ethically.

What mechanisms are in place to continuously evaluate the ethical implications of Koko's technologies?

Whenever Koko makes a potentially significant change to its technology or its platforms, the team follows a set of protocols to first assess whether the proposal coheres with Koko’s ethical commitments and to evaluate ethically sensitive impacts: things like privacy, wellbeing, equitable access, and fairness. Then, before implementing a change, the Koko leadership team consults with core community constituents, including community moderators and participants and the Koko Ethics Officer, as well as the Ethics Advisory Board, to ensure Koko’s values are represented by and expressed in our technology.