Senior Data Engineer (PySpark)
Job Title: Senior Data Engineer
Location: Remote
Experience: 5-10 Years
About the Opportunity
We are looking for a highly skilled Senior Data Engineer (PySpark) to join a global data transformation initiative within the pharmaceutical domain. The role involves designing, developing, and optimizing large-scale data pipelines and ETL/ELT processes on modern Data Lake platforms.
This is a fully remote opportunity with exposure to international teams, large-scale data ecosystems, and modern cloud-based data engineering practices.
Key Responsibilities
- Design, develop, and optimize ETL/ELT pipelines using PySpark and Python.
- Build scalable data flows from source systems to Data Lake environments.
- Develop and support data integration and transformation processes.
- Ensure data quality, performance, and reliability of data pipelines.
- Document technical solutions and maintain development standards.
- Collaborate with cross-functional teams in Agile Scrum environments.
- Participate in sprint planning, estimation, and delivery activities.
- Work with Azure DevOps, Jira, and Confluence for project execution.
Required Skills
- Strong experience in PySpark and Python.
- Solid background in Data Engineering.
- Hands-on experience with ETL/ELT design and development.
- Experience working with Data Lakes and large-scale data processing.
- Strong SQL and data modeling skills.
- Experience in Agile/Scrum methodologies.
- Familiarity with Azure DevOps and/or Jira.
- Experience using Confluence for documentation.
- Excellent verbal and written communication skills.
Preferred Skills
- Experience with Palantir Foundry.
- Exposure to Databricks, Snowflake, Azure Data Factory, or similar platforms.
- Experience working with global or distributed teams.
Education
Bachelor's degree in Computer Science, Information Technology, Engineering, or a related field.
