AI Engineer

About the Role

Join Us at PlanSource: Your Exciting Career Awaits!

At PlanSource, a leading cloud-based provider of benefits administration technology and services, we are on a mission to simplify how people choose, use, and manage benefits through cutting-edge, AI-powered experiences. This is an incredible time to launch your career with us!

PlanSource is seeking an AI Engineer to join our Data Science & Engineering team. In this role, you’ll help design, build, deploy, and support AI-powered products and shared platform capabilities that improve experiences for employees, administrators, clients, and internal teams.

This is a hands-on engineering role focused on delivering production-grade AI solutions in a secure, regulated SaaS environment. You’ll work closely with product managers, platform engineers, architects, security teams, and AI specialists across both the United States and India to deliver scalable, reliable, and compliant AI capabilities.

You’ll receive mentorship and technical guidance from experienced AI and platform engineering leaders while being expected to independently own features, troubleshoot production issues, and contribute to the ongoing evolution of our AI platform. This role is best suited for someone who has practical experience taking software or AI-powered solutions from development through production deployment and ongoing operational support.

What You’ll Work On

AI-Powered Applications

• Develop and maintain backend services powering AI-driven user experiences.
• Build and optimize agentic workflows that support complex business processes.
• Implement prompt engineering, content filtering, and response quality improvements using large language models.
• Integrate secure handling of sensitive information through masking, tokenization, and other privacy-preserving techniques.
• Support conversational experiences, workflow automation, human escalation scenarios, and streaming responses.
• Implement feedback mechanisms, session management, and audit capabilities.

Retrieval-Augmented Generation Services

• Build and enhance document ingestion pipelines for structured and unstructured content.
• Design and optimize chunking, embedding, retrieval, and ranking strategies.
• Improve semantic search quality through evaluation and experimentation.
• Implement secure multi-tenant access controls and data isolation.
• Support service-to-service integrations and authentication across AI components.

AI Quality, Observability & Operations

• Deploy and maintain observability tooling for AI applications, including tracing, latency analysis, cost monitoring, and prompt debugging.
• Build evaluation frameworks to measure retrieval quality, response quality, and task completion success rates.
• Investigate production issues involving model behavior, retrieval accuracy, latency, and integrations.
• Develop monitoring, alerting, operational dashboards, and support runbooks.
• Contribute to incident response, problem management, and continuous improvement activities.

Platform & Infrastructure

• Containerize and deploy AI services on cloud-native infrastructure.
• Manage CI/CD pipelines, container registries, and deployment automation.
• Support cloud infrastructure, storage services, and security controls.
• Address security vulnerabilities and technical debt in AI-related services.
• Participate in architecture reviews and operational readiness reviews.

Collaboration

• Partner closely with product managers to translate business requirements into technical solutions.
• Collaborate regularly with AI engineers, architects, and engineering teams across the United States and India.
• Work effectively within a blended delivery model where architecture, design, implementation, and support responsibilities may be shared across global teams.
• Participate in engineering design discussions, technical reviews, and solution delivery planning.
• Contribute to engineering standards, documentation, and operational excellence practices.
• Participate in agile planning, sprint execution, and cross-functional collaboration.

Required Qualifications

• Demonstrated experience taking software or AI-powered solutions from development through production deployment and ongoing operational support.
• Practical experience building, delivering, and supporting production software or AI-powered solutions in a professional environment.
• Strong Python development skills, including APIs, backend services, and asynchronous programming.
• Experience building applications using large language models such as OpenAI or equivalent platforms.
• Experience with Retrieval-Augmented Generation, embeddings, vector databases, and document processing pipelines.
• Experience with agent orchestration frameworks such as LangGraph, LangChain, or similar technologies.
• Experience deploying and supporting cloud-native applications in production environments.
• Working knowledge of Kubernetes, Docker, CI/CD pipelines, and infrastructure automation.
• Experience working with relational databases and SQL.
• Strong troubleshooting and problem-solving skills across distributed systems.
• Strong collaboration and communication skills, including experience working across geographically distributed teams.

Preferred Qualifications

• Experience with observability platforms such as Langfuse, LangSmith, Arize Phoenix, or similar tools.
• Experience implementing AI evaluation, testing, or quality measurement frameworks.
• Familiarity with PII handling, masking, tokenization, and privacy-preserving architectures.
• Experience building multi-tenant SaaS applications.
• Understanding of compliance frameworks such as HIPAA, SOC 2, GDPR, or CCPA.
• Experience with OAuth 2.0 and modern authentication patterns.
• Familiarity with AWS services including EKS, S3, IAM, CloudWatch, and container-based deployments.
• Exposure to data engineering, ETL pipelines, or analytics platforms.

Technology Stack

Backend

• Python
• FastAPI
• LangGraph
• LangChain

AI Services

• OpenAI APIs
• Embedding models
• Transcription and speech services

Data & Retrieval

• PostgreSQL
• pgvector
• Document processing pipelines

Infrastructure

• AWS
• EKS
• Docker
• Kubernetes
• S3
• ECR

Observability

• Langfuse
• CloudWatch

Security

• PII masking services
• Vulnerability scanning
• OAuth 2.0
• Secrets management

DevOps

• Git
• Automated CI/CD pipelines

What Success Looks Like in the First 6 Months

• Deliver production-ready features for AI-powered applications and shared platform services.
• Become a trusted engineering partner for Product, Platform Engineering, and AI leadership.
• Contribute meaningful improvements to AI quality, reliability, and observability.
• Independently troubleshoot and resolve production issues affecting AI services.
• Improve retrieval quality and application performance through data-driven experimentation.
• Establish strong working relationships with both U.S. and offshore engineering teams.
• Demonstrate ownership, accountability, and operational excellence for the systems you support.

About the Team

You’ll join a small, growing Data Science & Engineering team responsible for building and operating AI-powered capabilities across PlanSource. The team works closely with Product Engineering, Platform Engineering, Security, Infrastructure, and Product Management to deliver secure, scalable, and supportable AI solutions.

We believe AI products must be reliable, measurable, supportable, and secure. Our team emphasizes collaboration, continuous learning, operational excellence, and responsible use of AI technologies to solve real business problems.

Why Join PlanSource

PlanSource is investing in AI as a core part of how we improve our products, support our clients, and scale our operations. This role offers the opportunity to work on practical, production AI systems in a complex SaaS environment where reliability, security, and business impact matter.

You’ll have the opportunity to contribute to AI-powered products that solve real problems, work with a globally distributed engineering team, and help shape the foundation for future AI capabilities across the company.

What we Offer:

Due to character limit, please see full section on PlanSource Careers role.

Top 5 Reasons to join PlanSource

#5: Be part of what’s next in benefits technology

#4: Make an impact with technology our customers trust

#3: Be recognized, supported, and set up to thrive

#2: Grow your career with intention

NOTE: We will consider a remote employee for approved cities PlanSource can accommodate.

If you’re selected, you’ll receive your offer and be asked to complete a background check. This will include a drug screen, criminal record search and verification of previous employment and/or education.

Recruiting Scams: Unfortunately, scams targeting job seekers are common. To protect our candidates, we want to remind you that authorized representatives of PlanSource will only contact you from an email address ending in @plansource.com. PlanSource will never ask for personally identifiable information such as Date of Birth (DOB), Social Security Number (SSN), banking/direct/tax details, etc. via email or any other non-secure system, nor will we instruct you to make any purchases related to your employment. If you believe you’ve encountered a recruiting scam, report it to the Federal Trade Commission and your state’s Attorney General.

REF 4843