About SEOPulse
SEOPulse is an SEO and market intelligence platform that provides data-driven insights into the search landscape. As search engines shift toward AI-driven answers and dynamic layouts, our data collection infrastructure needs to scale accordingly. We are expanding our team to ensure our data pipeline remains accurate, efficient, and reliable.
Role Overview
We are looking for a Data & Backend Engineer to take ownership of our data collection systems and contribute to our core platform backend. In this role, you will design and maintain an asynchronous pipeline built to handle a high volume of web requests, initially launching at hundreds of thousands of daily multi-layered web crawls and rapidly scaling upward as our platform grows.
This is a combined system architecture and backend development role. You will be responsible for managing high-concurrency workloads, optimizing proxy configurations, preventing database bottlenecks, and integrating LLM APIs efficiently. Additionally, during periods of pipeline stability, you will work alongside our engineering team to build core backend services and user-facing features for our main web application.
Key Responsibilities
1. Data Ingestion & Pipeline Architecture
- Implement a decoupled Message Queue architecture to ingest high-volume webhooks from external providers without overloading production databases.
- Design a hot/cold storage strategy, routing raw HTML data to object storage while indexing structured metadata into the relational database.
- Integrate LLM APIs for structured text and entity extraction, implementing token-bucket rate limiting and prompt caching to keep API costs predictable as data volume increases.
2. Advanced Web Page Scraping & Systems Resilience
- Maintain and optimize Python and Node.js web crawlers designed to parse complex, dynamic search engine results and secondary target webpages.
- Build automated failover mechanisms: orchestrate local browser automation instances via residential proxies, with automatic fallback routes to premium web scraping APIs when encountering advanced anti-bot barriers.
- Monitor and optimize proxy networks (residential, datacenter, mobile) to maintain high success rates while managing bandwidth costs at scale.
3. Database Performance & Optimization
- Resolve and prevent database connection drops by configuring serverless connection pooling patterns (e.g., database connection pooling or HTTP-based data routing).
- Optimize relational database schemas, indexes, and queries to handle massive, rapidly growing datasets smoothly.
4. Core Platform Backend Development
- Transition to core platform development when pipeline infrastructure tasks are complete.
- Build, test, and deploy RESTful APIs and backend services using Node.js and Python.
- Collaborate with the product team to transform raw scraped metrics into functional dashboard features for our users.
Required Skills & Qualifications
- Experience: 5+ years of professional experience in Backend Engineering, Data Engineering, or Systems Architecture.
- Core Languages: Strong proficiency in both Python (Asyncio, Celery, FastAPI) and Node.js (TypeScript, Express/NestJS).
- Pipeline Orchestration: Practical experience managing distributed task queues or workflow orchestrators handling asynchronous, high-concurrency workloads.
- Database Management: Solid understanding of relational databases including MySQL, PostgreSQL, including connection management, relational data modeling, and performance tuning at scale.
- Browser Automation & Scraping: Hands-on experience with scraper library and browser automation tools including request interception, browser fingerprint management, and proxy rotation.
- Infrastructure: Experience with Docker, cloud container services (AWS ECS Fargate or GCP Cloud Run), and basic cloud cost management.
- Location: Candidate MUST BE located in Hong Kong or Taiwan.
Nice-to-Haves
- Experience with vector databases and word embeddings.
- Background working in the SEO industry or building scrapers for AI-generated search environments.
- Familiarity with source-level browser fingerprint modification.
- Experience with open-source AI scraping frameworks or multi-agent architectures.
What We Offer
- Competitive salary and performance bonuses.
- Dedicated operational budget for premium proxy networks, solving services, and cloud infrastructure.
- A direct impact on the technical direction of our core data collection platform.
- Flexible, remote-first working arrangements (aligned with Hong Kong time zone collaboration).