Job alert: Open-source LiteLLM raises $1.6M, now hiring founding engineer

LiteLLM, an open-source LLM Gateway with 27,000+ GitHub stars, is seeking a founding backend engineer to help scale their platform that allows companies to call 100+ LLM APIs in OpenAI format. The Y Combinator-backed startup has raised $1.6M in seed funding and serves enterprise clients including NASA, Adobe, and Rocket Money who need unified access to multiple AI models.

What you should know: LiteLLM provides both a Python SDK and FastAPI server that standardizes calls across major AI providers like OpenAI, Anthropic, Bedrock, Azure, and VertexAI.

• The role offers $160K-$220K salary plus 0.50%-3.00% equity for candidates with 1+ years of backend experience.
• The position requires working closely with the CEO and CTO to maintain API transformations that convert OpenAI-format requests to provider-specific formats.
• Companies upgrade to LiteLLM Enterprise when they need production monitoring through Prometheus metrics and secure access management via SSO or JWT.

Key technical challenges: The founding engineer will tackle critical infrastructure improvements to handle enterprise-scale deployments.

• Migrate key systems from httpx to aiohttp to achieve 10x higher throughput.
• Add support for new features like Anthropic’s thinking parameter on Bedrock.
• Handle provider-specific limitations, such as OpenAI o-1’s lack of streaming support.
• Ensure the platform can compute aggregate spending with 1M+ database logs.
• Implement cost tracking and logging for Anthropic’s /v1/messages API.

The tech stack: The platform runs on Python and FastAPI for backend services, with JavaScript/TypeScript handling frontend components.

• Infrastructure includes Redis, Postgres, S3, Google Cloud Storage, Datadog, and Slack API integrations.
• The team prioritizes open-source development and direct user engagement to understand enterprise requirements.

Interview process: Candidates face a compressed one-day virtual interview spanning 3.5 hours total.

• Two 60-minute coding sessions work through real LiteLLM GitHub issues, evaluating problem-solving and code quality.
• A 45-minute architecture interview covers AI infrastructure and scalability scenarios.
• A 30-minute founder chat discusses working style and company direction.

Company background: Founded in 2023 by Krrish Dholakia and Ishaan Jaffer, LiteLLM emerged from Y Combinator’s W23 batch as a two-person team.

• The startup addresses the complexity enterprises face when integrating multiple AI providers into their workflows.
• Their open-source approach has attracted significant developer adoption, with enterprise clients upgrading for production-grade features.