Senior Data Engineer

Centraprise • United State
Visa Sponsorship
Apply
AI Summary

We are seeking a Senior Data Engineer to develop and maintain scalable data pipelines using Python and PySpark. The ideal candidate will have experience with Big Data Technologies, Databases, and Cloud Platforms. This role requires strong technical skills and collaboration with teams on cloud deployments, CI/CD, and production support.

Key Highlights
Develop and maintain scalable data pipelines
Work with Databricks and Delta Lake for data processing and optimization
Support real-time streaming solutions using Kafka or similar frameworks
Key Responsibilities
Develop and maintain scalable data pipelines using Python and PySpark
Work with Databricks and Delta Lake for data processing and optimization
Support real-time streaming solutions using Kafka or similar frameworks
Optimize data processing performance and partitioning strategies
Collaborate with teams on cloud deployments, CI/CD, and production support
Integrate REST APIs and support application-level data requirements
Contribute to GenAI and LLM-based workflow initiatives where applicable
Technical Skills Required
Data Engineering Data Pipelines Data Processing ETL Large-Scale Data Processing Python PySpark Apache Spark Databricks Delta Lake PostgreSQL MongoDB Redis SQL NoSQL Kafka Kafka Streaming Real-Time Data Processing Azure AWS Cloud Data Services Performance Optimization Data Partitioning Performance Tuning In-Memory Optimization DevOps & Deployment CI/CD Pipelines Production Deployment Linux Unix REST APIs Application Development System Design Software Architecture Multi-User Data Platforms Access Management GenAI & AI Concepts LLM Workflows Prompt Versioning Evaluation Frameworks
Benefits & Perks
$80-85K+ Benefits
H1b Transfers are Workable

Job Description


Data Engineer ($80-85K+ Benefits) + H1b Transfers are Workable

Dallas, TX

Fulltime (Permanent)


5 Positions


Job Description:


Experience: 5 Years


Core Skills:

  • Data Engineering: Data Pipelines, Data Processing, ETL, Large-Scale Data Processing
  • Big Data Technologies: Python, PySpark, Apache Spark, Databricks, Delta Lake
  • Databases: PostgreSQL, MongoDB, Redis, SQL, NoSQL
  • Streaming & Real-Time Processing: Kafka, Kafka Streaming, Real-Time Data Processing
  • Cloud Platforms: Azure, AWS, Cloud Data Services
  • Performance Optimization: Data Partitioning, Performance Tuning, In-Memory Optimization
  • DevOps & Deployment: CI/CD Pipelines, Production Deployment, Linux, Unix
  • APIs & Architecture: REST APIs, Application Development, System Design, Software Architecture
  • Platform & Security: Multi-User Data Platforms, Access Management
  • GenAI & AI Concepts: LLM Workflows, Prompt Versioning, Evaluation Frameworks


Key Responsibilities:

  • Develop and maintain scalable data pipelines using Python and PySpark.
  • Work with Databricks and Delta Lake for data processing and optimization.
  • Design and manage SQL/NoSQL database integrations and workflows.
  • Support real-time streaming solutions using Kafka or similar frameworks.
  • Optimize data processing performance and partitioning strategies.
  • Collaborate with teams on cloud deployments, CI/CD, and production support.
  • Integrate REST APIs and support application-level data requirements.
  • Contribute to GenAI and LLM-based workflow initiatives where applicable.

Similar Jobs

Explore other opportunities that match your interests

Senior Principal AI/ML Engineer

Data Science
•
10h ago

Premium Job

Sign up is free! Login or Sign up to view full details.

•••••• •••••• ••••••
Job Type ••••••
Experience Level ••••••

palo alto networks unit 42

United State

Senior AI Data Engineer

Data Science
•
10h ago

Premium Job

Sign up is free! Login or Sign up to view full details.

•••••• •••••• ••••••
Job Type ••••••
Experience Level ••••••

Deloitte

United State

VIE Analyst, People Analytics

Data Science
•
1d ago

Premium Job

Sign up is free! Login or Sign up to view full details.

•••••• •••••• ••••••
Job Type ••••••
Experience Level ••••••

Pernod Ricard

United State

Subscribe our newsletter

New Things Will Always Update Regularly