Remote Data Labeling Specialist (New York)

<strong>Remote Data Labeling Jobs in New York (US-Remote)<br><br></strong>Rex.zone connects candidates to full-time remote data labeling and evaluation projects that create high-quality training data for AI systems across NLP and computer vision. You will label text, images, and conversations and contribute directly to model performance improvement through consistent taxonomy decisions, QA evaluation, and guideline-driven annotation.<br><br><strong>What You Will Do<br><br></strong><ul><li>Perform data labeling and data annotation for NLP and computer vision tasks using provided tools and schemas</li><li>Execute QA evaluation workflows (spot checks, inter-annotator agreement reviews, review queues, systematic error tagging)</li><li>Complete RLHF evaluation including preference ranking, pairwise comparisons, and rubric-based scoring for LLM responses</li><li>Conduct prompt evaluation to grade instruction-following, reasoning quality, and safety behaviors</li><li>Maintain annotation guidelines compliance by escalating ambiguous cases and documenting edge-case rationales</li><li>Contribute to training data quality via feedback loops, consistency audits, and taxonomy refinement<br><br></li></ul><strong>Core Workstreams (project-dependent)<br><br></strong><ul><li>Named entity recognition (NER) and entity linking</li><li>Computer vision annotation (bounding boxes, polygons/segmentation, keypoints, attribute tagging)</li><li>Content safety labeling (harassment, self-harm, hate, violence, adult content)</li><li>LLM evaluation (helpfulness, factuality, refusal quality, conversational coherence)</li><li>Data curation (deduplication flags, language identification, domain tagging)<br><br></li></ul><strong>Requirements<br><br></strong><ul><li>Professional experience in data labeling, annotation operations, or evaluation workflows</li><li>Strong attention to detail and consistent application of rubrics and taxonomies</li><li>Familiarity with QA evaluation concepts (sampling, agreement checks, review processes)</li><li>Comfort working with ambiguity and writing clear decision rationales</li><li>Ability to meet throughput and quality targets while following versioned guidelines<br><br></li></ul><strong>Compensation<br><br></strong>Base pay range: <strong>$30–$50/hour</strong> (USD, HOURLY), depending on project scope and demonstrated quality metrics.<br><br>This role is remote within the US and optimized for New York-based job seekers.