Skip to main content Skip to footer

Lead Data Engineer

Packaged Application Development Team Lead/Consultant | Full time | Experience: 5-10 years
Job No. R00335398 | Kochi
Apply for this job

Job Title - Lead Data Engineer - Level 9 - ACS Song

Management Level: Level 9 – Lead/Specialist

Location: Kochi

Must have skills: Azure and Databricks

Good to have skills:

Experience: 5-8 years of experience is required

Educational Qualification: Graduation (Accurate educational details should capture)

Job Summary

We are seeking a Senior Data Engineer specializing in Azure and Databricks to design, build, and optimize scalable data engineering solutions across cloud-based and distributed data platforms. The role will focus on developing robust ETL/ELT pipelines, workflow orchestration, data processing frameworks, and backend automation using Python. The ideal candidate will bring strong hands-on experience with Databricks, Azure Functions, Apache Airflow, Kedro, Kafka, PostgreSQL, Docker, and modern software engineering practices (all must haves). This position requires close collaboration with architects, UI developers, and data engineering teams to deliver reliable, production-ready, and high-performing data solutions. 

Roles and Responsibilities

  • Design and deliver scalable, maintainable, and production-grade data pipelines and data processing workflows using Azure, Databricks, Python, and related technologies. 
  • Enable efficient data ingestion, transformation, orchestration, validation, and automation across batch and streaming data environments. 
  • Support the development of modular, reusable, and well-documented data engineering components using frameworks such as Kedro and Apache Airflow. 
  • Ensure high standards of code quality, performance, reliability, troubleshooting, and collaboration within cross-functional engineering teams. 
  • Develop, maintain, and enhance scalable data pipelines, ETL/ELT workflows, and data processing solutions on Azure and Databricks. 
  • Work extensively with Kedro for pipeline development, modularization, feature implementation, code changes, and bug fixes. 
  • Build, maintain, and optimize workflow orchestration using Apache Airflow, including DAG creation, monitoring, troubleshooting, and issue resolution. 
  • Develop and maintain Azure Functions using Python for backend processing, integrations, automation, and event-driven workflows. 
  • Write clean, scalable, maintainable, and production-ready Python code, including use of Pydantic models, data validation, modular design, and API/backend integrations. 
  • Design, optimize, and troubleshoot complex SQL queries and transformations, with hands-on work in PostgreSQL for database development and data management. 
  • Work with Kafka and event-driven or streaming architectures for data ingestion, processing, and integration workflows. 
  • Use Docker for containerization and support local development, testing, and deployment workflows. 
  • Use GitHub for version control, pull requests, code reviews, branching strategies, and collaborative software development. 
  • Collaborate with architects, UI developers, data engineers, and other stakeholders while participating in sprint planning, technical discussions, code reviews, documentation, testing, CI/CD, monitoring, and production support activities 

Professional and Technical Skills:

  • Minimum 6 years of total professional experience in data engineering, cloud data platforms, backend data processing, or related software engineering roles. 
  • Strong experience designing, developing, and supporting production-grade data pipelines and ETL/ELT workflows. 
  • Hands-on experience working in enterprise or production environments with cloud-based data engineering solutions. 
  • Proven ability to debug, troubleshoot, and resolve application issues, pipeline failures, DAG issues, and production bugs. 
  • Strong programming experience in Python for data engineering, backend processing, pipeline development, and automation. 
  • Good understanding of Python programming concepts, including modular code development, Pydantic models, data validation, backend integrations, and API-based workflows. 
  • Hands-on production experience with Kedro for pipeline development, modularization, and maintainable data engineering design. 
  • Strong hands-on experience with Apache Airflow, including DAG creation, workflow orchestration, scheduling, monitoring, and troubleshooting. 
  • Experience with Databricks and distributed data processing environments. 
  • Hands-on experience developing and maintaining Azure Functions using Python. 
  • Strong working knowledge of the Microsoft Azure ecosystem and cloud-based data engineering services. 
  • Experience with Kafka and event-driven or data streaming architectures. 
  • Strong SQL skills with hands-on experience in PostgreSQL, including query optimization, data validation, and database development. 
  • Hands-on experience with Docker and containerized development or deployment workflows. 
  • Experience using GitHub for collaborative development, pull requests, code reviews, and version control.

Additional Information      

Behavioral and Collaboration Skills 

  • Strong analytical, troubleshooting, and problem-solving skills. 
  • Ability to work effectively in cross-functional engineering teams. 
  • Strong communication skills (English) with the ability to participate in technical discussions and explain implementation approaches clearly. 
  • Comfortable collaborating with architects, UI developers, data engineers, and other stakeholders. 
  • Proactive mindset with ownership of assigned tasks, production issues, and continuous improvement initiatives. 

About Our Company | Accenture (do not remove the hyperlink)

  • Minimum 6 years of total professional experience in data engineering, cloud data platforms, backend data processing, or related software engineering roles. 
  • Strong experience designing, developing, and supporting production-grade data pipelines and ETL/ELT workflows. 
  • Hands-on experience working in enterprise or production environments with cloud-based data engineering solutions. 
  • Proven ability to debug, troubleshoot, and resolve application issues, pipeline failures, DAG issues, and production bugs. 
  • Strong programming experience in Python for data engineering, backend processing, pipeline development, and automation. 
  • Good understanding of Python programming concepts, including modular code development, Pydantic models, data validation, backend integrations, and API-based workflows. 
  • Hands-on production experience with Kedro for pipeline development, modularization, and maintainable data engineering design. 
  • Strong hands-on experience with Apache Airflow, including DAG creation, workflow orchestration, scheduling, monitoring, and troubleshooting. 
  • Experience with Databricks and distributed data processing environments. 
  • Hands-on experience developing and maintaining Azure Functions using Python. 
  • Strong working knowledge of the Microsoft Azure ecosystem and cloud-based data engineering services. 
  • Experience with Kafka and event-driven or data streaming architectures. 
  • Strong SQL skills with hands-on experience in PostgreSQL, including query optimization, data validation, and database development. 
  • Hands-on experience with Docker and containerized development or deployment workflows. 
  • Experience using GitHub for collaborative development, pull requests, code reviews, and version control.

Kochi

Requesting an Accommodation

Accenture is committed to providing equal employment opportunities for persons with disabilities or religious observances, including reasonable accommodation when needed. If you are hired by Accenture and require accommodation to perform the essential functions of your role, you will be asked to participate in our reasonable accommodation process. Accommodations made to facilitate the recruiting process are not a guarantee of future or continued accommodations once hired.

If you would like to be considered for employment opportunities with Accenture and have accommodation needs such as for a disability or religious observance, please call us toll free at 1 (877) 889-9009 or send us an email or speak with your recruiter.

 

Equal Employment Opportunity Statement

We believe that no one should be discriminated against because of their differences. All employment decisions shall be made without regard to age, race, creed, color, religion, sex, national origin, ancestry, disability status, veteran status, sexual orientation, gender identity or expression, genetic information, marital status, citizenship status or any other basis as protected by federal, state, or local law. Our rich diversity makes us more innovative, more competitive, and more creative, which helps us better serve our clients and our communities.

For details, view a copy of the Accenture Equal Opportunity Statement

Accenture is an EEO and Affirmative Action Employer of Veterans/Individuals with Disabilities.

Accenture is committed to providing veteran employment opportunities to our service men and women.

 

Other Employment Statements

Applicants for employment in the US must have work authorization that does not now or in the future require sponsorship of a visa for employment authorization in the United States.

Candidates who are currently employed by a client of Accenture or an affiliated Accenture business may not be eligible for consideration.

Job candidates will not be obligated to disclose sealed or expunged records of conviction or arrest as part of the hiring process. Further, at Accenture a criminal conviction history is not an absolute bar to employment. 

The Company will not discharge or in any other manner discriminate against employees or applicants because they have inquired about, discussed, or disclosed their own pay or the pay of another employee or applicant. Additionally, employees who have access to the compensation information of other employees or applicants as a part of their essential job functions cannot disclose the pay of other employees or applicants to individuals who do not otherwise have access to compensation information, unless the disclosure is (a) in response to a formal complaint or charge, (b) in furtherance of an investigation, proceeding, hearing, or action, including an investigation conducted by the employer, or (c) consistent with the Company's legal duty to furnish information.

California requires additional notifications for applicants and employees. If you are a California resident, live in or plan to work from Los Angeles County upon being hired for this position, please click here for additional important information.

Please read Accenture’s Recruiting and Hiring Statement for more information on how we process your data during the Recruiting and Hiring process.

We work with one shared purpose: to deliver on the promise of technology and human ingenuity. Every day, more than 775,000 of us help our stakeholders continuously reinvent. Together, we drive positive change and deliver value to our clients, partners, shareholders, communities, and each other.

We believe that delivering value requires innovation, and innovation thrives in an inclusive and diverse environment. We actively foster a workplace free from bias, where everyone feels a sense of belonging and is respected and empowered to do their best work.

At Accenture, we see well-being holistically, supporting our people’s physical, mental, and financial health. We also provide opportunities to keep skills relevant through certifications, learning, and diverse work experiences. We’re proud to be consistently recognized as one of the World’s Best Workplaces™.

Join Accenture to work at the heart of change. Visit us at www.accenture.com.

Discover where this job fits at Accenture

Software engineer jobs: Imagine it, build it, scale it

Create software that will power change and empower people.

Learn more

Technology jobs: Be the catalyst

Get hands-on with the technologies that our clients need to reinvent, work in new ways and change the world for the better.

Learn more

Jobs in technology platforms: Create foundations for the future

You’ll build the tech that transforms how business works for the better.

Learn more