Databricks Python Developer

Must Have Technical/Functional Skills • 10+ years of experience in Data Engineering, Data Warehousing, or Big Data development. • 5+ years of hands-on experience with Databricks, Python, and PySpark. • Strong hands-on experience in Databricks, Python, and PySpark development. • Experience building and supporting large-scale ETL/ELT data pipelines. • Strong knowledge of Spark SQL, Delta Lake, and Data Lakehouse architecture. • Experience working with structured and unstructured data from multiple sources. • Strong SQL skills and experience with relational databases. • Experience with data modeling, data transformation, and data quality processes. • Knowledge of orchestration tools such as Airflow, ADF, or similar. • Experience with Git, CI/CD, and Agile development methodologies. • Strong analytical, debugging, and performance tuning skills. • Excellent verbal and written communication skills. Roles Responsibilities • Design, develop, and maintain data pipelines using Databricks, Python, and PySpark. • Build scalable data ingestion, transformation, and integration solutions. • Collaborate with Data Architects, Analysts, and business teams to deliver data solutions. • Develop and optimize batch and real-time data processing workflows. • Ensure data quality, integrity, security, and governance standards are met. • Monitor, troubleshoot, and resolve production data pipeline issues. • Optimize Spark jobs, SQL queries, and Databricks workloads for performance. • Participate in data modeling, code reviews, and platform enhancement activities. • Create technical documentation and support production deployments. • Participate in Agile ceremonies and contribute to continuous improvement initiatives.