Design and implement scalable data pipelines and workflows for Behavioral AI products. Collaborate with cross-functional teams to define data schemas and standards. Optimize data processes for cost efficiency and reliability.
Key Highlights
Technical Skills Required
Benefits & Perks
Job Description
About The Company
Yobi is a rapidly growing Behavioral AI company dedicated to ethically democratizing the benefits of data and artificial intelligence. Since its inception in 2019, Yobi has established itself as a leader in building large-scale, consented behavioral datasets across the United States, extending beyond the confines of traditional Big Tech platforms. Unlike conventional language model companies, Yobi specializes in creating foundation models of human behavior rooted in real-world actions such as purchases and store visits. Our approach emphasizes privacy-by-design, enabling state-of-the-art personalization and decision-making for prominent brands and agencies while safeguarding user privacy, safety, and ethics. Our current focus is on delivering the performance of closed-web user acquisition to the open web and connected TV, providing brands with effective results without the barriers of walled gardens. At its core, Yobi is constructing the behavioral intelligence layer for any system that makes personalization decisions, aiming to revolutionize how data-driven insights are leveraged responsibly and ethically.
About The Role
As the Lead Data Engineer at Yobi, you will play a pivotal role in shaping how data flows, scales, and empowers our Behavioral AI products. This is a hands-on engineering position that involves working across various virtual teams and projects, focusing on schema modeling, orchestration, abstractions, workload efficiency, reliability, and observability. You will be responsible for designing and implementing robust data workflows that ensure high performance, modularity, and clarity. Your expertise will influence the architecture of our data systems, helping align best practices and standards across teams. The role requires a strategic thinker with strong technical skills who can navigate an evolving environment, balancing technical excellence with pragmatic delivery. You will lead efforts to optimize data pipelines, ensure data quality, and implement scalable solutions that support our mission to ethically democratize AI-driven personalization. Your work will directly impact the efficiency and reliability of our platform, enabling Yobi to continue delivering innovative behavioral insights at scale.
Qualifications
- Proven experience in data engineering, with a strong understanding of data workflows and architecture
- Hands-on experience with Delta Lake, Scala Spark, Python Spark, and Airflow
- Deep knowledge of schema evolution, data contracts, dependency graphs, and metadata management
- Ability to design scalable, reliable, and efficient data pipelines
- Strong problem-solving skills and attention to detail
- Experience leading or influencing cross-functional engineering teams
- Familiarity with various data pipeline patterns including push-based, pull-based, batch, streaming, and eventual consistency
- Excellent communication skills and ability to work collaboratively in a remote or hybrid environment
- A proactive attitude with a focus on delivering high-quality solutions
Responsibilities
- Design, develop, and maintain scalable data pipelines and workflows to support Behavioral AI products
- Collaborate with cross-functional teams to define data schemas, models, and standards
- Implement data orchestration, workload management, and observability solutions to ensure reliability and performance
- Optimize data processes for cost efficiency and minimal wall-clock time
- Ensure data quality through schema validation, metadata management, and data contracts
- Lead efforts to influence best practices and standards across engineering teams
- Troubleshoot and resolve data pipeline issues promptly to minimize downtime
- Contribute to the evolution of our data infrastructure, balancing technical excellence with pragmatic delivery
- Stay updated on emerging data engineering trends and incorporate innovative solutions as appropriate
Benefits
- Competitive base salary aligned with industry standards
- Meaningful equity and financial upside, offering a share of the company's success
- Annual bonus target based on personal and company performance
- Comprehensive health, dental, and vision insurance plans with minimal or no out-of-pocket costs
- Unlimited paid time off to support work-life balance and impact-driven work
- 401(k) plan with company matching contributions
- Fully remote or hybrid work options from several hubs including San Francisco Bay Area, Seattle, and New York City
- Opportunity to work with a world-class team of machine learning and data engineering experts
- Participation in innovative projects that are shaping the future of Behavioral AI and personalization
Equal Opportunity
Yobi is an equal opportunity employer committed to fostering an inclusive environment. We celebrate diversity and are dedicated to providing equal employment opportunities regardless of race, gender, age, disability, sexual orientation, gender identity, or any other protected characteristic. We believe that a diverse team enhances our ability to innovate and serve our clients effectively.