Machine Learning Engineer job at Raising The Village

Job Title: Machine Learning Engineer
Department/Group: VENN
Reporting To: Senior Data Scientist
Years of Experience: 3+ years
Location: Mbarara
Travel Required: Up to 30%

Background

At Raising The Village (RTV), we are dedicated to eradicating ultra-poverty in Sub-Saharan Africa. As a dynamic, rapidly growing international development organization, we’ve assembled a team of over 250 passionate individuals in Uganda, alongside an additional 17 professionals in North America and 15 in Rwanda. Together, we are committed to elevating communities out of ultra-poverty by implementing innovative solutions and leveraging advanced data analytics to drive impact.To date, our holistic approach has positively impacted over 1 million lives since 2012, and we’re poised to achieve even greater milestones, aiming to assist 1 million individuals annually by 2027. Our growth and success are fueled by the invaluable support of global partners who share our vision of sustainable change. Learn more about our impactful programs at www.raisingthevillage.org

The VENN department is the data and technology backbone of our organization, connecting advanced analytics, and custom software tools with field implementation to ensure data-informed decision-making at every level.

Job Description

The Machine Learning Engineer is responsible for building, deploying, and continuously improving RTV's production LLM applications, which are currently live across multiple platforms and actively used by field teams and program staff across Uganda, Rwanda, and the Democratic Republic of Congo. The role sits within the Predictive Analytics / VENN department and focuses on advancing agentic LLM architectures, RAG systems, and evaluation infrastructure as RTV scales its AI capabilities to new countries and deepens integration with mobile field tools and the data warehouse. A core area of responsibility is the SBCC (Social and Behavior Change Communication) system, which generates personalized, practice-specific behavior change messaging for field officers across agriculture, health, livestock, and community domains, and is currently being integrated into RTV's mobile check-in application. The engineer will work closely with the Data Engineer, Data Scientists, the Software Engineering team, and field program teams to deliver reliable, context-aware LLM applications that integrate with RTV's data warehouse, mobile implementation apps, and the broader WorkMate AI ecosystem. This role also contributes to RTV's strategic partnership with The Agency Fund (TAF) AI Accelerator, supporting shared technical challenges in knowledge base architecture, multi-country scaling, and LLM evaluation governance.

Key Responsibilities

• Design and implement agentic LLM architectures including multi-step reasoning pipelines, tool use, memory management, and autonomous workflow orchestration using LangChain and related frameworks, applied across both conversational and generative AI use cases.
• Build, maintain, and optimize Retrieval-Augmented Generation (RAG) pipelines for context-grounded LLM responses, including embedding strategy design, chunking approaches, and retrieval optimization tailored to diverse content types such as program documentation, household data, and behavioral practice guidelines.
• Manage and evolve RTV's vector database infrastructure (Chroma or Qdrant) including index management, namespace organization, and multi-domain retrieval tuning to support distinct organizational use cases.
• Design, build, and maintain end-to-end ML pipelines covering data ingestion, feature engineering, model training, evaluation, and deployment, ensuring reproducibility and version control across all pipeline stages.
• Apply knowledge of core ML algorithms — including supervised learning, classification, regression, clustering, and neural network architectures — to select appropriate modeling approaches for diverse problem types across RTV's AI workstreams.
• Develop and manage the full LLM application lifecycle — from prompt engineering and chain construction through deployment, versioning, and production monitoring — using LangChain and LangSmith as the primary development and observability stack.
• Design and implement LLM evaluation frameworks using LLM-as-a-judge approaches, automated metrics, and human evaluation protocols to assess response quality, factual grounding, cultural appropriateness, and content safety across generative outputs.
• Instrument production LLM applications with LangSmith tracing, logging, and feedback collection pipelines to enable continuous performance monitoring, failure analysis, and iterative improvement cycles.
• Build and deploy RESTful API endpoints for LLM-powered services, ensuring stable integration with WorkMate and the RTV mobile implementation app used by field officers during household visits.
• Develop and maintain personalized content generation pipelines that leverage household segmentation, behavioral data, and program-specific context from the data warehouse to produce targeted, practice-specific outputs at scale.
• Implement offline and low-connectivity strategies including message caching and fallback mechanisms to ensure AI-powered tools remain accessible to field officers in remote locations.
• Collaborate with the Applied Learning team to incorporate validated program content into knowledge bases and generation templates, ensuring evidence-based alignment and content quality across all LLM outputs.
• Write clear technical documentation for agent architectures, RAG pipeline designs, evaluation frameworks, and API specifications to support team collaboration and organizational knowledge continuity.

Technical Requirements

• Bachelor's or Master's degree in Computer Science, Artificial Intelligence, Data Science, Statistics (Computing Major) or a related quantitative field.
• 3+ years of hands-on experience building and deploying production LLM applications, with a demonstrable portfolio.
• Proficiency in:
  • LangChain for agentic pipeline construction, tool use, memory integration, and RAG implementation.
  • LangSmith for LLM application tracing, evaluation, dataset management, and production monitoring.
  • Vector databases (Chroma and/or Qdrant) including embedding management, indexing, and retrieval optimization.
  • Agentic design patterns including ReAct, plan-and-execute, multi-agent orchestration, and tool-augmented reasoning.
  • LLM evaluation methodologies including LLM-as-a-judge frameworks, reference-based and reference-free metrics, and human-in-the-loop evaluation workflows.
  • Python for LLM application development, API construction (FastAPI or equivalent), and pipeline automation.
  • OpenAI API and prompt engineering best practices including few-shot prompting, structured output generation, and system prompt design.
  • Cloud deployment on AWS, including containerized application hosting, environment management, and API infrastructure.
• Experience integrating LLM applications with structured data sources (SQL databases, data warehouses) for analytics-augmented generative AI capabilities.
• Solid understanding of core ML algorithms including supervised and unsupervised learning, classification, regression, ensemble methods, and neural network architectures, with the ability to select and apply appropriate approaches for varied problem types.
• Hands-on experience building and managing ML pipelines including data preprocessing, feature engineering, model training, evaluation, experiment tracking (Weights & Biases or equivalent), and production deployment using CI/CD practices.
• Familiarity with mobile application integration and offline-first design patterns for low-connectivity deployment environments is an asset.

Personal Attributes

• Genuine commitment to using AI for social impact and poverty alleviation.
• Strong engineering discipline with attention to reliability, safety, and cultural sensitivity in AI-generated content.
• Ability to translate complex LLM system outputs into accessible insights for non-technical field staff and program managers.
• Collaborative and communicative team player who can work across analytics, software development, and field program teams.
• High degree of ownership, intellectual curiosity, and drive to stay current with the fast-moving LLM engineering landscape.

• Design and implement agentic LLM architectures including multi-step reasoning pipelines, tool use, memory management, and autonomous workflow orchestration using LangChain and related frameworks, applied across both conversational and generative AI use cases.
• Build, maintain, and optimize Retrieval-Augmented Generation (RAG) pipelines for context-grounded LLM responses, including embedding strategy design, chunking approaches, and retrieval optimization tailored to diverse content types such as program documentation, household data, and behavioral practice guidelines.
• Manage and evolve RTV's vector database infrastructure (Chroma or Qdrant) including index management, namespace organization, and multi-domain retrieval tuning to support distinct organizational use cases.
• Design, build, and maintain end-to-end ML pipelines covering data ingestion, feature engineering, model training, evaluation, and deployment, ensuring reproducibility and version control across all pipeline stages.
• Apply knowledge of core ML algorithms — including supervised learning, classification, regression, clustering, and neural network architectures — to select appropriate modeling approaches for diverse problem types across RTV's AI workstreams.
• Develop and manage the full LLM application lifecycle — from prompt engineering and chain construction through deployment, versioning, and production monitoring — using LangChain and LangSmith as the primary development and observability stack.
• Design and implement LLM evaluation frameworks using LLM-as-a-judge approaches, automated metrics, and human evaluation protocols to assess response quality, factual grounding, cultural appropriateness, and content safety across generative outputs.
• Instrument production LLM applications with LangSmith tracing, logging, and feedback collection pipelines to enable continuous performance monitoring, failure analysis, and iterative improvement cycles.
• Build and deploy RESTful API endpoints for LLM-powered services, ensuring stable integration with WorkMate and the RTV mobile implementation app used by field officers during household visits.
• Develop and maintain personalized content generation pipelines that leverage household segmentation, behavioral data, and program-specific context from the data warehouse to produce targeted, practice-specific outputs at scale.
• Implement offline and low-connectivity strategies including message caching and fallback mechanisms to ensure AI-powered tools remain accessible to field officers in remote locations.
• Collaborate with the Applied Learning team to incorporate validated program content into knowledge bases and generation templates, ensuring evidence-based alignment and content quality across all LLM outputs.
• Write clear technical documentation for agent architectures, RAG pipeline designs, evaluation frameworks, and API specifications to support team collaboration and organizational knowledge continuity.

• LangChain
• LangSmith
• Vector databases (Chroma and/or Qdrant)
• Agentic design patterns
• LLM evaluation methodologies
• Python
• OpenAI API
• AWS cloud deployment
• ML algorithms
• ML pipelines
• Mobile application integration
• Offline-first design patterns

• Bachelor's or Master's degree in Computer Science, Artificial Intelligence, Data Science, Statistics (Computing Major) or a related quantitative field.
• 3+ years of hands-on experience building and deploying production LLM applications, with a demonstrable portfolio.
• Experience integrating LLM applications with structured data sources (SQL databases, data warehouses) for analytics-augmented generative AI capabilities.
• Solid understanding of core ML algorithms including supervised and unsupervised learning, classification, regression, ensemble methods, and neural network architectures, with the ability to select and apply appropriate approaches for varied problem types.
• Hands-on experience building and managing ML pipelines including data preprocessing, feature engineering, model training, evaluation, experiment tracking (Weights & Biases or equivalent), and production deployment using CI/CD practices.