Annotation Guidelines
1. Your Role
As a Chinho annotator, you compare pairs of AI-generated Bangla responses and choose the one that better answers the prompt. Your judgments train AI systems to produce more accurate, natural, and helpful responses in Bangla.
2. How to Judge Responses
Consider these criteria in order of importance:
- Factual accuracy — Does the response contain correct information? Factual errors are the most serious problem.
- Completeness — Does the response fully address the question? Missing key details is a significant issue.
- Naturalness — Does the response sound natural in Bangla? Awkward phrasing or transliteration problems reduce quality.
- Helpfulness — Is the response practical and actionable for a Bangla-speaking user?
3. Writing Justifications
Every choice requires a brief justification in Bangla. Explain why the chosen response is better — not just that it is better. Reference specific differences between the two responses. One or two sentences is enough.
4. Voice Recording Tasks
For voice recording tasks, speak naturally in your own regional dialect. Do not use formal/standard Bangla unless that is your natural way of speaking. Record in a quiet environment, keep your microphone close, and stay within the 2-minute time limit.
5. Quality Standards
Your work is periodically checked against expert judgments (gold items). Maintaining a quality score above 70% is required to continue annotating. If your score drops, you will receive feedback on how to improve.
6. Payment
You earn a fixed amount per completed task, displayed before you begin each batch. Payouts are processed via bKash or Nagad within 24 hours of your request. The minimum payout threshold is ৳100.
7. Code of Conduct
- Do not rush through tasks — quality matters more than speed.
- Do not share task content or screenshots externally.
- Do not use automated tools or AI to complete your tasks.
- Report any issues or unclear prompts using the skip button with an appropriate reason.