Data Platform Engineer
Project Role Description : Assists with the data platform blueprint and design, encompassing the relevant data platform components. Collaborates with the Integration Architects and Data Architects to ensure cohesive integration between systems and data models.
Must have skills : Databricks Unified Data Analytics Platform
Good to have skills : NA
Minimum 12 year(s) of experience is required
Educational Qualification : 15 years full time education
Summary: As a Data Platform Engineer, you will assist with the data platform blueprint and design, collaborating with Integration Architects and Data Architects to ensure cohesive integration between systems and data models. You will play a crucial role in shaping the data platform components. Roles and Responsibilities: • Lead the design, development, and optimization of complex data pipelines, ensuring high performance and scalability using Py Spark, Spark, and Big Data technologies. • Architect data engineering solutions across major cloud platforms like AWS, Azure, or GCP, enabling smooth and secure data ingestion, transformation, and storage. • Lead efforts to design and build modular, reusable ETL/ELT pipelines to integrate data from various sources into cloud-based data lakes and data warehouses. • Collaborate closely with cross-functional teams, including data scientists, analysts, and other engineering teams to develop and maintain large-scale data processing systems. • Guide and mentor junior engineers in best practices for data engineering, cloud technologies, and big data solutions. • Optimize and tune Spark and big data processing jobs to handle high-volume data efficiently and at scale. • Ensure data security, privacy, and governance are maintained by enforcing best practices and policies across cloud platforms. • Implement automated data testing and monitoring solutions to ensure data quality and pipeline reliability. • Drive initiatives for real-time data processing, batch jobs, and analytics pipelines to support both operational and analytical needs. • Advocate for continuous improvement, researching new tools, and technologies to drive data engineering excellence. Required Skills & Qualifications: • Proven experience with AWS, Azure, or GCP in designing data engineering workflows and solutions. • Expert-level knowledge of Py Spark, Spark, and Big Data technologies. • Extensive experience in data pipeline architecture, design, and optimization. • Strong hands-on experience with large-scale ETL/ELT processes and integrating data into cloud-based platforms like S3, Blob Storage, Big Query, and Redshift. • Proficiency in Python, SQL, and scripting for data pipeline development. • Ability to work with real-time data streaming tools like Kafka and batch processing tools. • Experience leading cross-functional teams in the implementation of cloud-based data platforms and workflows. • Solid understanding of data security, compliance, and governance in a cloud environment. Nice to Have: • The candidate should have a minimum of 12 years of experience in Databricks Unified Data Analytics Platform. • Familiarity with Dbt (Data Build Tool) for managing data transformation and data quality. • Exposure to CI/CD pipelines, DevOps practices, and containerization technologies (Docker, Kubernetes). • Certifications or experience with data tools like Apache Airflow, Five Tran, Informatica, or Talend. • Experience with analytics and BI tools like Power BI, Tableau, or Looker for visualization over cloud-based data platforms. Educational Qualification: - 15 years full time education is required.
Bengaluru
Equal Employment Opportunity Statement
All employment decisions shall be made without regard to age, race, creed, color, religion, sex, national origin, ancestry, disability status, veteran status, sexual orientation, gender identity or expression, genetic information, marital status, citizenship status or any other basis as protected by federal, state, or local law.
Please read Accenture’s Recruiting and Hiring Statement for more information on how we process your data during the Recruiting and Hiring process.
We work with one shared purpose: to deliver on the promise of technology and human ingenuity. Every day, more than 775,000 of us help our stakeholders continuously reinvent. Together, we drive positive change and deliver value to our clients, partners, shareholders, communities, and each other.
We believe that delivering value requires innovation, and innovation thrives in an inclusive and diverse environment. We actively foster a workplace free from bias, where everyone feels a sense of belonging and is respected and empowered to do their best work.
At Accenture, we see well-being holistically, supporting our people’s physical, mental, and financial health. We also provide opportunities to keep skills relevant through certifications, learning, and diverse work experiences. We’re proud to be consistently recognized as one of the World’s Best Workplaces™.
Join Accenture to work at the heart of change. Visit us at www.accenture.com.
We have been alerted to the existence of fraudulent messages asking job seekers to set up payment to cover various costs associated with establishing employment at Accenture. No one is ever required to pay for employment at Accenture. If you are contacted by someone asking for payment, please do not respond, and contact us at india.fc.check@accenture.com immediately.