Job Details
Job Description ? Azure Data Engineer with Pypark & ADF skills
Mandatory Skills / Top Skills
Must have over all 3-4 years of experience, which should include:
- Good Knowledge on Data warehousing concepts(CDC,SCDs)
- Exposure to Azure Data Factory ( MUST )
- exposure to Azure Databricks
- exposure to Python ( PySpark) (MUST ) & Spark sql
- Good in complex SQL queries
- worked on at least 2 - EDW / Data Migration / Data Integration projects
Preferably 3-4 Years experience of PySpark coding
Must have low level development skills. Should be able to develop a solution for given use cases
Must have experience in building ETL/ELT through PySpark
Exposure to Python, Databricks notebooks development
Must have experience in designing job orchestration (ADF), sequence, metadata design, Audit trail, dynamic parameter passing and error/exception handling
Agile delivery - Person must be able to show/review design and code on daily basis
Desirable :
Preferred to have understanding/hands-on experience in Azure components like Azure DB, Azure Synapse, Logic App, Azure Key vault, Azure functions and other components
Datawarehousing projects? experience
Past experience in Migrating applications from on premise to Azure Cloud