Skill required: Data Scientist - Data Science
Designation: I&F Decision Sci Practitioner Analyst
Qualifications:Diploma In CS and Engineering
Years of Experience:3 to 5 years
About Accenture
Accenture is a global professional services company with leading capabilities in digital, cloud and security.Combining unmatched experience and specialized skills across more than 40 industries, we offer Strategy and Consulting, Technology and Operations services, and Accenture Song— all powered by the world’s largest network of Advanced Technology and Intelligent Operations centers. Our 699,000 people deliver on the promise of technology and human ingenuity every day, serving clients in more than 120 countries. We embrace the power of change to create value and shared success for our clients, people, shareholders, partners and communities.Visit us at www.accenture.com
What would you do? Data & AI In this role, you will be working on the interdisciplinary field about scientific methods, processes and systems to extract knowledge or insights from data in various forms, either structured or unstructured.
What are we looking for? • Construct, design, and put into practice scalable data pipelines with PySpark, and other pertinent technologies • Build and maintain data lakes and data warehouses, ensuring data quality, integrity, and availability • Process and transform large datasets using PySpark to support analytics and business intelligence initiatives • Develop and deploy Docker containers to package and manage data processing applications • Work together to comprehend data requirements and provide solid data solutions with data scientists, analysts, and other stakeholders • Monitor and optimize data pipelines for performance, scalability, and reliability • Implement data governance and security practices to protect sensitive information • Experience with Linux/Unix Systems and Scripting • Experience with Version Control Systems, such as Git • Knowledge on scheduling job using informatica ETL, and Autosys, as required • Create and implement containerization using Kubernetes and orchestration using Docker containers for data processing application packaging and management • Agility for quick learning • Commitment to quality • Written and verbal communication • Adaptable and flexible • Ability to work well in a team
Roles and Responsibilities: • Design and develop new data pipelines using Databricks notebooks and ADLS Gen2.0 to ingest and process raw data efficiently and ensuring reliability and scalability of the pipeline. Combines data gathering to model creation, use Databricks Notebooks to unify the process and instantly deploy to production. • Integrates with a wide variety of data stores and services such as Azure SQL Data Warehouse, Azure Cosmos DB, Azure Data Lake Store, Azure Blob storage, Azure Event Hubs, Azure IoT Hub, and Azure Data Factory. • Utilize Databricks and Delta/parquet tables to optimize the performance of both new and existing data processing pipeline to reduce job run time and improve efficiency. • Maintain the data platform focusing on process monitoring, troubleshooting, and data readiness, ensuring high-quality data for regular reporting and system optimization. • Work with other data engineers to design and implement enhancements to the overall data platform, improving functionality and performance. • Work independently on end-to-end implementation of data processing pipelines, from development-testing-deployment using Databricks workflow. Should be proficient in PySpark operations to extract-transform-load data from/to Azure Delta Lake, creating reports to support business request to meet multiple priorities in given time.