AI Chatbot Evaluator
Position: AI Chatbot Evaluator (Health & Wellness Study) Ideal candidates enjoy using and conversing with AI chatbots, are very comfortable following step-by-step instructions independently, open to feedback and comfortable collaborating on an iterative workflow to provide thoughtful, detailed feedback on their AI interactions. Project Type: Contractor Assignment Location: Remote (U.S.) Compensation: $180 flat fee-minimum of 4 tasks, $45 for any additional tasks completed Time Commitment: ~ 2-3 hours total (including about 30 minutes for instructions/setup and 30 minutes for task completion). Project Summary Productive Playhouse is supporting a new upcoming project to evaluate the performance of leading AI chatbots on health and wellness topics. The objective of this study is to determine whether personalization improves response quality when users interact with these AI models. Participants will use their existing personal accounts and conversation history to engage with their preferred AI chatbot, such as Gemini, ChatGPT, or Claude. The goal is to evaluate how effectively the chatbot uses user context to deliver high-quality, meaningful, and personalized insights. What You'll Do: Interact with your assigned AI model on designated health and wellness topics. Use your personal AI chatbot account and existing conversation history to evaluate personalized response quality. Assess how well the system adapts to your inputs and provides accurate, relevant responses. Address feedback and implement revisions to support high quality submissions. Complete a structured evaluation form based on your experience. Requirements: Must be 18 years of age or older Currently residing in the United States. Must be fully proficient in reading and writing instructions in English. Must have one of the following existing, active personal accounts: A personal Google account to use Gemini, not newly created or a test account; An active, paid ChatGPT subscription; or An active, paid Claude Pro subscription. **The selected AI chatbot account must have at least 1 month of active usage history. As this project continues to evolve, we want to ensure we set clear expectations from the start. Given the iterative nature of this test, there will be times when we need to adapt and make adjustments along the way. We ask that you remain flexible and open to changes as they arise Application Process Apply now and check your email immediately to complete the mandatory Intake Form (where you will specify which AI platform you currently use). Once submitted, our project leads will review your application and contact qualified participants with setup instructions as soon as the project goes live. About Us: As a global data company, Productive Playhouse “PPH”, is pioneering our approach to language and data services, while incorporating their roots as a production company. Originally creating content to support children's language acquisition, our commitment to excellence, forward-thinking strategies, and worldwide cultural experience has proven key for delivering exceptional service. Originally founded as an educational production company, Productive Playhouse made a mark with our award-winning children’s series, which taught fundamental subjects through engaging and effective programming. This early success paved the way for our evolution into a comprehensive data services provider. Since 2011, Productive Playhouse has expanded rapidly to offer an extensive suite of data services. Our current offerings include transcription, translation, linguistic analysis, rating, systems testing, localization, field and studio recording, language skill verification, and specialized data handling with a focus on sensitivity and diversity. Our commitment to innovation means we continually enhance our service portfolio to meet the evolving needs of our clients. At Productive Playhouse, we are proud of our reputation for addressing complex challenges with agility and delivering premium, secure data solutions across diverse environments. Our dynamic team is dedicated to maintaining the highest standards and ensuring exceptional service every time. Disclaimer The job description provided is designed to convey information essential to understanding the scope of the position and the general nature of the work performed. It is not intended to be an exhaustive list of duties, responsibilities, or qualifications associated with the job. Productive Playhouse reserves the right to modify or revise the job description as necessary Productive Playhouse is an equal opportunity employer. We celebrate diversity and are committed to creating an inclusive work environment for all employees. Employment with Productive Playhouse is at-will, meaning that either the employee or the employer can terminate the employment relationship at any time, with or without cause or notice. All offers of employment at Productive Playhouse are contingent upon the candidate’s ability to provide valid documentation of identity and eligibility to work in the United States or relevant hiring location. Productive Playhouse participates in E-Verify. Productive Playhouse provides reasonable accommodation for qualified individuals with disabilities. If you need assistance or accommodation due to a disability, please contact the Human Resources Department. Apply To This Job