Senior Software Development Engineer in Test (SDET) - AWS Lakehouse, Python, PySpark, Kafka Streaming
Remote
AI Summary
We are seeking a Senior Software Development Engineer in Test (SDET) to join our team. As an SDET, you will be responsible for designing and implementing automated testing frameworks for our data engineering workflows. You will work closely with our data engineering team to ensure the quality and reliability of our data pipelines.
Key Highlights
Design and implement automated testing frameworks for data engineering workflows
Collaborate with data engineering team to ensure quality and reliability of data pipelines
Develop and maintain testing strategies for batch and streaming architectures
Technical Skills Required
Benefits & Perks
100% Remote
12+ Months Contract
Job Description
Sr. SDET - (AWS Lakehouse, Python, PySpark, Kafka Streaming)
100% Remote
12+ Months Contract
Job Description:
Skills & Technologies:
- Python (Pandas, PySpark), Apache Spark
- Kafka / Kafka Streaming
- Delta Lake
- Advanced SQL (PostgreSQL, Window Functions, CASE)
- Shell scripting, Unix/Linux
- Multi-threading & Performance Testing
AWS & Cloud:
- AWS Lakehouse Architecture
- AWS Glue, S3, Data Lakes
- Glue Migration
- AWS ETL & Data Pipeline Testing
- AWS Cloud Practitioner–level knowledge
Testing & Automation:
- SDET (Data/Lakehouse)
- ETL / ELT Data Validation & Automation
- ICEDQ / QuerySurge
- Data Testing Strategy (Batch & Streaming)
- TDD for Spark & Kafka
- Test Automation Frameworks
DevOps / CI-CD / IaC:
- GitHub, GitHub Actions, Jenkins
- Terraform
- CI/CD Pipeline Testing
Platform & Domain:
- Data Engineering workflows
- Streaming & Batch architectures
- Healthcare Data & Compliance (HIPAA)
Nice to Have:
- Healthcare / Health Insurance experience
- AWS Certifications (Cloud Practitioner, Data Analytics, Solutions Architect)