Discovering and Exploiting Multiple Vulnerabilities in Avaya Aura

This blog is authored by Gerardo Iglesias, member of Accenture’s FusionX team, taking part in advanced adversary simulation engagements.

On a recent client engagement, we found ourselves in the unenviable position of needing to breach the network of a client with a very robust anti-phishing posture. In an attempt to avoid phishing entirely, we decided to perform a detailed review of the exposed external network. What follows is the story of how we found and chained multiple zero-day vulnerabilities in Avaya Aura® Utilities Server, starting with a pre-auth, low privileged remote code execution (RCE) and ultimately resulted in full remote code execution as root.

We were initially attracted to an exposed Avaya Aura Utilities Server because the application appeared to be using a dated technology stack, and it was one of the few complex services that this particular client had exposed without multi-factor single sign-on. Initial testing did not reveal much beyond an authentication page. With few other options, we acquired a similar version of the software for our lab environment. This allowed us to explore the filesystem of the application for potential vulnerabilities.

The Avaya Aura Utilities Server provides a variety of functionality to support Avaya IP Telephony. Among these is the ability to serve firmware and configurations via TFTP to Avaya phones as well as an API test suite. The service is built on top of RedHat Linux and uses Apache to serve a PHP application with some elevated actions occurring via a SUID-as-root CGI binary or at least that is how the versions we observed are built and work.

After understanding the basic technology behind the application, we began searching for dangerous PHP methods. We then attempted to back-trace these calls to publicly accessible functionality. While multiple potentially dangerous methods were discovered, none of them initially looked as if they were reachable from our unauthenticated vantage point. After an extensive review, a series of oversights were identified and chained together to gain access. We found that a web service test page (“/sms/sms_test.php”) was accessible pre-authentication via direct browsing.

<<< Start >>>

<<< End >>>

The purpose of this and the similar “/smsxml/smsxml_test.php” pages appear to be to assist administrator in debugging SOAP or XML API requests which can be used to interact with other Avaya products. From what we could ascertain these SOAP and XML services do not exist on this particular server, but the server will helpfully spin up a dummy SOAP or XML server dynamically to aide in testing. We do not believe that these debug pages should have been exposed pre-authentication as they directly allow an attacker to launch server-side request forgery (SSRF) attacks against internal services via partially controlled SOAP and XML requests. Unfortunately, SSRF attacks were not the most critical attacks that could be conducted with this functionality.

Through a complicated chain of method calls, and a carefully crafted POST request, our Adversary Team was able to poison a parameter that was eventually relayed to the dynamically created SOAP server, where the poisoned parameter was then concatenated into a string that was executed at the command line, thus resulting in low privilege code execution. This entire chain of over 10 steps and the nuances around this exploit are detailed in the disclosure report that was provided to Avaya. This report is linked at the end of this blog post. The end result of this exploit chain was that we could execute code as the “apache” user by sending a request like the following:

<<< Start >>>

<<< End >>>

The “/var/www/avayadir/avaydirini/” folder in the previous screenshot is one of the few locations served by Apache where the “apache” user has write permissions. With the above code execution, we were able to further explore the server and perform limited network proxying via PHP. With this access we were able to determine that this machine had access to the client’s internal network and could ping their domain controllers. To be effective, we needed a more robust way of tunneling traffic through the server. There are multiple, public solutions which allow proxying SOCKS traffic over PHP, however, these solutions are noisy, and prone to breakage. We were also limited by a rather restrictive NGINX reverse proxy that was between us and the vulnerable server, which prevented us from extending our access by adding our own SOCKS proxy on a high port or taking advantage of websockets.

Early in our attempts to exploit this service, we had tested the SSRF capabilities of the previously mentioned vulnerable pages, however this outbound HTTP traffic was apparently detected by the client, and outbound traffic was quickly blocked. Because of this, we elected against uploading a typical C2 agent that worked over HTTP(S). We decided that a reverse SSH tunnel might be our best bet, but the “apache” user account we were running under was not configured to allow interactive login, so we could not simply drop an SSH key for this user and then SSH back in over a reverse SSH tunnel. Before resorting to installing any malicious software on the server, we proceeded to look for ways to escalate privileges to “root”.

Normal privilege escalation techniques were quickly ruled out. In our older lab version of the application, we noted that several privilege escalation opportunities existed if we could become the “admin” user due to weak sudo permissions. These vulnerabilities are laid out in detail in the disclosure report linked at the end of this blog, but we will only talk about the one we used later on in this attack chain. We also investigated common publicly released privilege escalation vulnerabilities like “Dirty COW”. Though this machine was very old and missing many security patches, it was too old for some of the newer known privilege escalation vulnerabilities, and it was patched for the older ones that we attempted. With these common, reliable, and relatively safe privilege escalation paths ruled out, we went back to looking for application-specific paths.

We noticed that the application had a CGI binary (“/opt/avaya/smi/cgi-bin/cgi_main”) with SUID-as-root privileges that was called to perform elevated activities.

By further reverse engineering and testing, we discovered that authentication to the application was performed against the local “/etc/shadow” file and users. When a user was successfully authenticated, the “cgi_main” binary would drop privileges to the account of the authenticated user, except for specific privileged operations. As previously mentioned, we also noted that some users like the “admin” user potentially had privilege escalation capabilities through their sudo access. For instance, in our older lab version of the application, the “admin” user had the ability to run the vi command on certain paths with sudo. This would allow trivially escalation via dropping to shell or “bang” execution within vi.

Below is a screenshot of us demonstrating root access via “!whoami” from within vi in the lab environment:

We would later learn that this oversight had been corrected in the version of the application that the client was running, but other opportunities still existed and are laid out and described in our disclosure report.

Though we spent considerable time trying to find a direct privilege escalation to “root” from the “apache” user account, we were not successful. Eventually we turned our focus toward figuring out if we could become the “admin” user. At this point we had found several escalation paths if we had this account’s permissions. We started looking into how PHP session management worked for this application. In particular, we realized that some sort of session hand-off existed that allowed PHP to pass the session to the “cgi_main” binary when needed. Since PHP ran as the “apache” user under the Apache webserver, it seemed likely that there might be a weak point in this hand-off process, especially since we currently had command execution as the same “apache” user.

By testing authentication in our lab environment, we discovered that PHP sessions are stored as PHP serialized objects in files under “/var/lib/php/session”. When a new session is created, a file is created in this folder with the name “sess_<session id>”. Initially this file is owned by the “apache” user account. Inside the serialized object is the user’s UID, username, IP address, and other information, though the user information is initially blank. Once a user is authenticated, the file is then chown’d by the “cgi_main” binary to have the permissions of the logged-in user. Below is a screenshot showing this behavior:

We discovered that there was a gap in this session file hand-off process which allowed us to become any user allowed to login through the web application. By correctly forging a session file with accurate user information, then requesting the post-authentication landing page with the corresponding session cookie, the application would accept our forged session information, and chown the session file with the appropriate account permissions and grant the session the privileges of the user described in the session file. Again, it is important to note that there are nuances to this process, and sessions are tied to IP addresses. These details are explained in more detail in the report.

This session forgery primitive was extremely powerful. Not only could we become the most powerful “admin” user that a normal Avaya customer can become, but we could also become various accounts that Avaya has created for providing customer support like the following accounts: “csadmin”, “inads”, “rasaccess”, and “craft”. We did not fully explore if this other access would have granted us an easier path to root, as we had already identified multiple potential paths to “root” as the “admin” user. We could not use this attack to directly become “root” because the “root” user is not allowed to log into the web-application.

As mentioned, we had discovered several paths to “root” from the “admin” user and all of them are laid out in detail in the full disclosure report. The least disruptive method since the sudo “vi” path was not possible in the client’s instance appeared to be via abusing the backup and restore feature. This feature allowed the “admin” user to backup system configurations into a downloadable tar file. Similarly, these files could be restored. There were attempts to prevent malicious use of this feature by careful use of the tar “-C” option to prevent path traversal and only allowing restoration of approved files. However, the functionality allowed the backup and restore of “/etc/cron.d” folder. At this point we could construct a malicious cron job which was configured to run as “root” every minute, and have it execute a bash file of our choosing. In fact, we tested this successfully in our lab environment. By simply creating the appropriate path structure with our single malicious cron job file, building the expected nested tar archives and handing the backup to the web application we gained code execution as “root” in our lab environment.

As is often the case with real-world exploitation, we ran into problems when we attempted this attack on the client’s server. The first issue was that none of the administrative functionality that we had been relying on was exposed on the client’s server. We are uncertain if the client had manually removed these administrative sections from being served in their Apache configuration, or if there was some sort of configuration option in their version of the application where Avaya makes it possible to not expose these administrative sections. In either case, we realized that though we had a privilege escalation path, we had no way of reaching the vulnerable code via HTTP(S). We did not have permission as the “apache” user to alter the Apache configuration to re-expose this access.

We were almost ready to declare defeat when it occurred to us that we should be able to access this functionality by making direct CGI calls ourselves. With this new insight we set about writing a Python script to automate the process. After much trial-and-error, we eventually figured out how to make both GET and POST requests via CGI command line calls. Eventually we had a successful proof of concept for our attack chain, however we then ran into the next problem.

A valid Utilities Server backup archive contains two or three internal tar files. One contains PostgreSQL database backups, one contains firmware backups for various phones (but this is optional), and the third contains the system configuration files. We only care about the last archive. In our older version of the application, a backup file that was missing the database backup files would cause the restore script to throw errors, but it would still complete the process of restoring system files. Unfortunately, in the client’s newer version, a backup file missing the database backups would cause the process to exit before “restoring” our malicious backup cron job. Since we did not have the required file permissions to view this restore script on the client’s server, we figured out this discrepancy via a painful process of trial and error.

To get around the backup restoration error checking while not damaging the client’s server, we constructed calls to first create a backup of the existing server. This backup file gets stored in “/tmp” which is web servable. We then untar this file into a folder and delete everything but the database backup. We cannot extract the existing file backup and simply add our malicious script because the file permissions and ownership of these files would not survive which could result in a badly broken system on restoration. Fortunately, the file restore process does not check for the existence of any particular file, it merely ensures that any files that are present are within an approved list. So all that remained was to construct the correct sub-path of “/etc/cron.d/” and drop our malicious cron job before tar’ing everything back up and rebuilding the final backup tar.gz file… Or was it? We attempted this in the client’s environment and initially everything seemed to have worked. We could see our malicious cron job sitting in “/etc/cron.d”, but it never ran.

Once again, we were stymied by differences between our lab environment and the client’s newer version. The lab environment was old enough to have lax permission requirements around cron jobs. The client’s version was new enough that it performed safety checks around cron jobs and would not run them if they are not owned by the appropriate user account, and with the correct file permissions to prevent other accounts from modifying them. By modifying our exploit to set chmod permissions on the malicious cron job to “644” and then using the “--owner” and “--user” tar arguments to force the owner and user to UID/GID of 0 (“root”) on the file, we were finally able to craft a backup tar that would drop a cron job with appropriate permissions. With our cron job in place and executing every minute, we now had a reliable way of executing any command or script as “root”.

We would go on to establish a successful pivot into the client’s internal network and achieve significant internal compromise.

Remediation Recommendations:

Avaya has released the corresponding advisories and updates to fix some of these issues on the affected products that are still within support lifecycle. More information can be found at:
https://downloads.avaya.com/css/P8/documents/101076479https://downloads.avaya.com/css/P8/documents/101076523

We have created a simple vulnerability checking script to help validate whether a server is vulnerable to the initial code execution vulnerability as well as a full proof-of-concept for the full chain. These scripts can be found in the link below.

To mitigate these vulnerabilities, we recommend not exposing these servers to the intranet if possible. In addition, please block all access to the “/sms/” and “/smsxml/” URL paths. This will at least prevent the initial remote code execution vulnerability that Accenture discovered. We also recommend blocking all access to the “/tmp/” URL path as we discovered multiple sensitive files can be leaked into this folder.

Finally, we recommend blocking inbound and outbound traffic on all ports that are unneeded. For instance, many customers might be surprised to know that any account that they log into the web application with also has SSH access. Please see the following link for the detailed vulnerability disclosure as well as proof of concept code for the exploit chain: https://github.com/Accenture/AARO-Bugs/tree/master/Avaya-Aura-2021

Timeline:

01/04/2021 - Initial disclosure email to Avaya
01/07/2021 - Attempted to contact Avaya via phone
01/07/2021 - Contact attempt via Partner Onboarding
01/07/2021 - Confirmation received from Avaya about issues being investigated
02/04/2021 - Follow-up email sent requesting update information
02/18/2021 - Reached out to MITRE requesting either CVE assignment or assistance in contacting Avaya
02/25/2021 - MITRE reached out to Avaya
03/01/2021 - Message to Avaya requesting for updates
03/01/2021 - Avaya confirmed they are close to have a report with the assessment of the reported issues
03/18/2021 - Avaya provided a report with details on the assessment, confirming vulnerabilities and affected products. Release dates for the fixes are set for June
06/25/2021 - Avaya releases advisory ASA-2021-087 (https://downloads.avaya.com/css/P8/documents/101076479)
06/25/2021 - Avaya releases advisory ASA-2012-088 (https://downloads.avaya.com/css/P8/documents/101076523)

We would also like to thank Shelby Spencer for his help in drafting this post.

Accenture Security is a leading provider of end-to-end cybersecurity services, including advanced cyber defense, applied cybersecurity solutions and managed security operations. We bring security innovation, coupled with global scale and a worldwide delivery capability through our network of Advanced Technology and Intelligent Operations centers. Helped by our team of highly skilled professionals, we enable clients to innovate safely, build cyber resilience and grow with confidence. Follow us @AccentureSecure on Twitter or visit us at www.accenture.com/security.

Accenture, the Accenture logo, and other trademarks, service marks, and designs are registered or unregistered trademarks of Accenture and its subsidiaries in the United States and in foreign countries. All trademarks are properties of their respective owners. All materials are intended for the original recipient only. The reproduction and distribution of this material is forbidden without express written permission from Accenture. The opinions, statements, and assessments in this report are solely those of the individual author(s) and do not constitute legal advice, nor do they necessarily reflect the views of Accenture, its subsidiaries, or affiliates. Given the inherent nature of threat intelligence, the content contained in this article is based on information gathered and understood at the time of its creation. It is subject to change. Accenture provides the information on an “as-is” basis without representation or warranty and accepts no liability for any action or failure to act taken in response to the information contained or referenced in this report.

This document makes reference to marks owned by third parties. All such third-party marks are the property of their respective owners. No sponsorship, endorsement or approval of this content by the owners of such marks is intended, expressed or implied.

Copyright © 2021 Accenture. All rights reserved.