Data Engineer
Project Role Description : Design, develop and maintain data solutions for data generation, collection, and processing. Create data pipelines, ensure data quality, and implement ETL (extract, transform and load) processes to migrate and deploy data across systems.
Must have skills : Data Engineering, Data Architecture Principles, Databricks Unified Data Analytics Platform, Microsoft Azure Data Services
Good to have skills : NA
Minimum 3 year(s) of experience is required
Educational Qualification : 15 years full time education
Summary:
As a Data Engineer, a typical day involves designing, developing, and maintaining comprehensive data solutions that support the generation, collection, and processing of data. This role requires creating efficient data pipelines and ensuring the integrity and quality of data throughout its lifecycle. The position also involves implementing processes to extract, transform, and load data, facilitating seamless migration and deployment across various systems. Collaboration with different teams to align data strategies and optimize workflows is an integral part of daily activities, ensuring that data infrastructure supports organizational needs effectively and reliably. Utilizes software engineering principles to deploy and maintain fully automated data transformation pipelines that combine a large variety of storage and computation technologies to handle a distribution of data types and volumes in support of data architecture design. A Data Engineer designs data products and data pipelines that are resilient to change, modular, flexible, scalable, reusable, and cost effective.
Roles & Responsibilities:
- Design, develop, and maintain data pipelines and ETL processes using Microsoft Azure services (e.g., Azure Data Factory, Azure Synapse, Azure Databricks, Azure Fabric)
- Utilize Azure data storage accounts for organizing and maintaining data pipeline outputs. (e.g., Azure Data Lake Storage Gen 2 & Azure Blob storage)
- Collaborate with data scientists, data analysts, data architects and other stakeholders to understand data requirements and deliver high-quality data solutions
- Optimize data pipelines in the Azure environment for performance, scalability, and reliability
- Ensure data quality and integrity through data validation techniques and frameworks
- Develop and maintain documentation for data processes, configurations, and best practices
- Monitor and troubleshoot data pipeline issues to ensure timely resolution
- Stay current with industry trends and emerging technologies to ensure our data solutions remain cutting-edge
- Manage the CI/CD process for deploying and maintaining data solutions.
- Expected to be an SME, collaborate and manage the team to perform.
- Responsible for team decisions.
- Engage with multiple teams and contribute on key decisions.
- Provide solutions to problems for their immediate team and across multiple teams.
- Lead the design and implementation of scalable data architectures to support business requirements.
- Oversee the continuous improvement of data processing workflows to enhance efficiency and reliability.
- Mentor junior team members to foster skill development and knowledge sharing within the team.
Professional & Technical Skills:
- Must To Have Skills: Proficiency in Data Engineering, Data Architecture Principles, Databricks Unified Data Analytics Platform, Microsoft Azure Data Services.
- Strong experience in building and optimizing data pipelines and ETL frameworks.
- In-depth knowledge of cloud-based data storage and processing solutions.
- Ability to work with large datasets and ensure data quality and consistency.
- Familiarity with scripting and programming languages commonly used in data engineering.
- Experience in monitoring and troubleshooting data workflows to maintain system performance.
- Learning agility
- Technical Leadership
- Consulting and managing business needs
- Strong experience in Python is preferred but experience in other languages such as Scala, Java, C#, etc is accepted
- Experience building spark applications utilizing PySpark.
Additional Information:
- The candidate should have minimum 5 years of experience in Data Engineering.
- This position is based at our Pune office.
- A 15 years full time education is required.
Pune
Requesting an Accommodation
Accenture is committed to providing equal employment opportunities for persons with disabilities or religious observances, including reasonable accommodation when needed. If you are hired by Accenture and require accommodation to perform the essential functions of your role, you will be asked to participate in our reasonable accommodation process. Accommodations made to facilitate the recruiting process are not a guarantee of future or continued accommodations once hired.
If you would like to be considered for employment opportunities with Accenture and have accommodation needs such as for a disability or religious observance, please call us toll free at 1 (877) 889-9009 or send us an email or speak with your recruiter.
Equal Employment Opportunity Statement
We believe that no one should be discriminated against because of their differences. All employment decisions shall be made without regard to age, race, creed, color, religion, sex, national origin, ancestry, disability status, veteran status, sexual orientation, gender identity or expression, genetic information, marital status, citizenship status or any other basis as protected by federal, state, or local law. Our rich diversity makes us more innovative, more competitive, and more creative, which helps us better serve our clients and our communities.
For details, view a copy of the Accenture Equal Opportunity Statement
Accenture is an EEO and Affirmative Action Employer of Veterans/Individuals with Disabilities.
Accenture is committed to providing veteran employment opportunities to our service men and women.
Other Employment Statements
Applicants for employment in the US must have work authorization that does not now or in the future require sponsorship of a visa for employment authorization in the United States.
Candidates who are currently employed by a client of Accenture or an affiliated Accenture business may not be eligible for consideration.
Job candidates will not be obligated to disclose sealed or expunged records of conviction or arrest as part of the hiring process. Further, at Accenture a criminal conviction history is not an absolute bar to employment.
The Company will not discharge or in any other manner discriminate against employees or applicants because they have inquired about, discussed, or disclosed their own pay or the pay of another employee or applicant. Additionally, employees who have access to the compensation information of other employees or applicants as a part of their essential job functions cannot disclose the pay of other employees or applicants to individuals who do not otherwise have access to compensation information, unless the disclosure is (a) in response to a formal complaint or charge, (b) in furtherance of an investigation, proceeding, hearing, or action, including an investigation conducted by the employer, or (c) consistent with the Company's legal duty to furnish information.
California requires additional notifications for applicants and employees. If you are a California resident, live in or plan to work from Los Angeles County upon being hired for this position, please click here for additional important information.
Please read Accenture’s Recruiting and Hiring Statement for more information on how we process your data during the Recruiting and Hiring process.
We work with one shared purpose: to deliver on the promise of technology and human ingenuity. Every day, more than 775,000 of us help our stakeholders continuously reinvent. Together, we drive positive change and deliver value to our clients, partners, shareholders, communities, and each other.
We believe that delivering value requires innovation, and innovation thrives in an inclusive and diverse environment. We actively foster a workplace free from bias, where everyone feels a sense of belonging and is respected and empowered to do their best work.
At Accenture, we see well-being holistically, supporting our people’s physical, mental, and financial health. We also provide opportunities to keep skills relevant through certifications, learning, and diverse work experiences. We’re proud to be consistently recognized as one of the World’s Best Workplaces™.
Join Accenture to work at the heart of change. Visit us at www.accenture.com.