Data Platform Engineer
Project Role Description : Assists with the data platform blueprint and design, encompassing the relevant data platform components. Collaborates with the Integration Architects and Data Architects to ensure cohesive integration between systems and data models.
Must have skills : Microsoft Azure Databricks
Good to have skills : NA
Minimum 5 year(s) of experience is required
Educational Qualification : 15 years full time education
Summary:
As a Microsoft Azure Databricks Architect, you will be responsible for defining, designing, and governing enterprise-scale data processing and analytics solutions using Azure Databricks. You will lead architectural decisions across multiple initiatives, ensuring alignment with business objectives, cloud strategy, performance, security, and cost optimization standards. Your role involves architecting end-to-end Databricks-based solutions, guiding teams on implementation best practices, reviewing technical designs, and driving standardization. You will collaborate closely with stakeholders, platform teams, and delivery leads while mentoring technical leaders and improving overall data platform maturity.
Roles & Responsibilities: -
Act as the Azure Databricks Architecture Lead and SME across projects and programs.
Analyze business, data, and non-functional requirements and translate them into end-to-end Databricks architecture designs.
Design scalable data ingestion, transformation, and analytics architectures using Azure Databricks, Azure Data Lake Storage Gen2, and Azure Data Factory.
Define and govern cluster architecture, job design, and workload orchestration strategies.
Establish and enforce data processing patterns including batch, incremental, and near–real-time pipelines.
Drive adoption of medallion architecture standards for raw, curated, and serving layers.
Ensure performance optimization, scalability, and cost efficiency through cluster tuning, job optimization, and resource management.
Define data security and governance practices including access control, secrets management, and compliance standards.
Review solution designs, notebooks, and deployment approaches for architectural compliance.
Collaborate with enterprise architects, security teams, cloud infrastructure teams, and stakeholders on key decisions.
Mentor team leads and senior engineers conduct design and architecture reviews.
Provide architectural guidance during production incidents, RCA, and platform optimization initiatives.
Professional & Technical Skills: - Must-Have Skills with Microsoft Azure Databricks as listed below
Strong hands-on and architectural experience with Azure Databricks, including:
o Cluster configuration and management
o Job scheduling and orchestration
o Notebook and workflow optimization
Advanced expertise in Spark architecture, PySpark, and Spark SQL performance tuning.
Strong proficiency in SQL for data transformations and analytics workloads.
Deep understanding of data engineering and data warehousing concepts, including medallion architecture.
Experience implementing secure, scalable, and governed Databricks platforms.
Strong knowledge of cloud security, access control, and governance in Azure environments.
Experience defining CI/CD and Databricks deployment strategies using Git-based workflows.
Ability to evaluate architectural trade-offs across performance, cost, scalability, and maintainability.
Strong experience with Azure Data Lake Storage Gen2 and Delta Lake.
Additional Information: -
The candidate should have 9–12 years of experience in data engineering, analytics, or big data platforms with architecture ownership.
This position is based at our Bengaluru, Chennai, Hyderabad office.
Minimum 15 years of full-time education (or equivalent).
Drives architecture governance and technical standards across multiple teams.
Responsible for platform scalability, stability, and optimization.
Accountable for delivering robust, secure, and high-performance Databricks solutions.
Bengaluru
Equal Employment Opportunity Statement for Australia and New Zealand
At Accenture, we recognise that our people are multi-dimensional, and we create a work environment where all people feel like they can bring their authentic selves to work, every day.
Our unwavering commitment to inclusion and diversity unleashes innovation and creates a culture where everyone feels they have equal opportunity. Our range of progressive policies support flexibility in ‘where’, ‘when’ and ‘how’ our people work to ensure that Accenture is an organisation where you can strive for more, achieve great things and maintain the balance and wellbeing you need.
We encourage applications from all people, and we are committed to removing barriers to the recruitment process and employee lifecycle. All employment decisions shall be made without regard to age, disability status, ethnicity, gender, gender identity or expression, religion or sexual orientation and we do not tolerate discrimination. If you require any accommodations or adjustments for interviews and/or at work, please reach out to exectalent@accenture.com or contact us at +61 2 9005 5000 (Australia) or +64 44666056 (New Zealand).
To ensure our workplace is inclusive and diverse we are setting bold goals and taking comprehensive action. To achieve these goals, we collect information that allows us to track the effectiveness of our Inclusion and Diversity programs. Learn how Accenture protects your personal data and know your rights in relation to your personal data. Read more about our Privacy Statement.
We work with one shared purpose: to deliver on the promise of technology and human ingenuity. Every day, more than 775,000 of us help our stakeholders continuously reinvent. Together, we drive positive change and deliver value to our clients, partners, shareholders, communities, and each other.
We believe that delivering value requires innovation, and innovation thrives in an inclusive and diverse environment. We actively foster a workplace free from bias, where everyone feels a sense of belonging and is respected and empowered to do their best work.
At Accenture, we see well-being holistically, supporting our people’s physical, mental, and financial health. We also provide opportunities to keep skills relevant through certifications, learning, and diverse work experiences. We’re proud to be consistently recognized as one of the World’s Best Workplaces™.
Join Accenture to work at the heart of change. Visit us at www.accenture.com.