Skip to main content Skip to Footer

BLOG


April 14, 2016
DC/OS at Accenture Labs
By: Carl Dukatz and Miguel Bernandin

What’s DC/OS?
DC/OS (a.k.a. Datacenter Operating System) is a sleek technology that lowers the barrier of entry to the Apache Mesos frameworks. It operates as a resource manager and leverages frameworks to manage and maintain a cluster of computers. It’s exceptional at addressing application provisioning and scalability requirements. It also makes use of exhibitor which makes your Mesos environment a bit more resilient as it manages your zookeeper quorum and performs periodic backups.

English please? - It lets you run applications on an entire data center with the same experience you have installing and running applications on your smartphone (e.g. grab it from an online store, hit the run button, not worry about which “core” or “computer” it’s on, just start using it!)

And, there are also a good number of applications that it supports today which are easy to deploy via containers. Check out a few of them on github; you have a few good tools to choose from the DC/OS Universe. These tools are available to use out of the box.

Also, if you have any doubts about the underlying technology, Apache Mesos, don’t. It’s been battle hardened and proven infrastructure tool that runs in many large infrastructures such as Twitter and AirBnB to name a few.

Why are we using DC/OS at Accenture Labs?
At any given point in time there are numerous research initiatives underway simultaneously in the Accenture Labs. Mesos provides a simple and consist platform for these teams to share the on-site resources available. All the teams have to do is input the amount of compute and memory they need for their application and it’s allocated appropriately. This is great because they teams don’t have to be concerned with the messy details of where in our data center the application is actually running.

For example, our Artificial Intelligence team needs to be able to work collaboratively on algorithms as well as execute the algorithms as large deep learning workloads. They use Apache Zeppelin which is deployed as a docker container on DC/OS. This web-based notebook enables interactive data analytics that persist using HDFS (using an NFS gateway) on top of the entire cluster. Since Zeppelin is using Marathon if they happen to crash the application with some work in progress test algorithm Marathon will relaunch the application on a different machine with seconds and since the notebooks are saved in HDFS, the user does not see any data loss. Secondly, the user is interacting with Zeppelin using Mesos-DNS rather than a specific IP so they never have to change the URL even when Zeppelin may be running on different machines at various times.

Another team in the Accenture Labs focuses on Digital Experiences. The Digital Experiences focuses on interactions between users and technology. They are frequent creating demos to showcase features and functionality of prototype wearables (including Virtual and Augmented reality). The team needs to rapidly test their front end applications and DC/OS enables them to create the required environments easily from a tailored json template file. When a change needs to be made to the application, Marathon gives us the ability to perform a rolling upgrade by allowing users to specify whether all old instances can be killed, before the new version is deployed or if all instances of the new version is deployed side by side, before the old version is destroyed. With this functionality the demo gods are leaning towards our favor. Also, running applications in DC/OS gives users the ability to run in a microservices style fashion where they can independently scale each tier such as front end, middleware or database tier like Cassandra.

Our Systems and Platforms team is constantly exploring innovations on the cluster itself, such as validating different resource configurations and changing the management tools. With Mesos as the underlying resource manager, when configurations are modified the other team's work is not affected! They would really have our heads… Additionally, when we suffer from hardware failures such as devices, services, hard drives, memory, etc, or need to run upgrades/patches the applications and services which are affected heal themselves automatically, meaning that we don’t have to be on 24/7 support for the environment. We’ve also taken the liberty to script up many of our common recovery functions using Ansible and they integrate extremely well with Mesos.

These are just a few examples of how we’re running DC/OS right now. As we continue forward we’ll probably take it a step further by leveraging Marathon Health Checks, Artifacts Store Backend and Event Bus when the need arises. These are all great ways to improve how applications are deployed at scale.

Why on-prem DC/OS?
You can setup DC/OS in the cloud or on your own hardware. We set it up on our own hardware because we like the way that it gives us a cloud like experience without having to pay per use. When installing it on-premise, there is a web automated installer that walks you through the setup. The installer will ask for the IP addresses of the master and slave nodes. It will also ask where your master and slave nodes reside via IP and will perform preflight and postflight tests to ensure your environment is fully functional. There are some prerequisites for with DC/OS; it supports CentOS and RHEL currently. For those of you running Ubuntu, it will be supported in the 15.04 release as it will be leveraging systemd which DC/OS currently supports.

Want some starter tips on DC/OS?
DC/OS comes with a UI and a CLI. The UI provides a sleek dashboard that gives a high level overview of the cluster state, tasks, health metrics, cpu / memory utilizations and more. The CLI gives administrators control of many frameworks deployed on cluster. If a user wanted to deploy an application against Marathon, they can issue this command:

They are able to see the status of the application by performing this command:

dcos marathon app add .json

They are able to see the status of the application by performing this command:

dcos marathon app list

If you’re like us and tend towards using NoSQL for operational data then you might find it interesting that you can run Cassandra on top of DC/OS. And DC/OS just makes it simple and easy! Simply run:

dcos package install cassandra

Users who want to build their own tools and packages can simply add them by using this command:

dcos package repo add ‹myrepo› ‹http://my_repo.com/repo›

You can also add from your own internal gitlab or a private github repository.

Industry & topics highlighted

Technology

Popular Tags

    More blogs on this topic

      Archive