Detecting Memcached DDoS Attacks Targeting GitHub

Contributors to this blog include Nathaniel “Q” Quist and Dan Kaiser.

On February 28 and March 5, 2018, Memcached DDoS attacks targeted GitHub. LogRhythm Labs performed an investigation into the cause, effect, and outcome of these attacks. The following will help give you an understanding of the background that allowed the DDoS attack to be possible, the vulnerability that was exposed, mitigation techniques, and AI Engine rules that can detect Memcached attacks targeting or originating from a client environment.

Understanding the Memcached DDoS Attacks Against GitHub

The recent Distributed Denial of Service (DDoS) attacks raised the bar for attackers even higher with two highly publicized attacks against GitHub—an online repository for open source computer code. Both attacks were made possible by vulnerable Memcached servers. Memcached stands for memory cache, and these servers allowed threat actors to perform heavily amplified reflection attacks against GitHub by exploiting a recently released attack.

The first attack took place on Wednesday, February 28, 2018, and set a record high bandwidth usage of 1.35 TBPS—shattering the previously held record for the Imperva Incapsula network attack. This latest record was short-lived, however, and just 5 days later, on March 5, 2018, the record was broken again when GitHub was targeted a second time with a DDoS that reached 1.7 TBPS of traffic!

Memcached Servers and Their Vulnerability

Fortunately, architecture changes in the method of deployment of Memcached servers can easily prevent this type of attack. However it is important to understand how the cybersecurity industry got to the point where a system designed to make services more flexible and more accessible, could be used to bring any network to its knees. Memcached is a storage service providing large quantities of RAM for data storage and retrieval. Memcached servers typically support database-driven websites by storing frequently requested data and objects in the local memory. Since memory is much faster to query than most other resources, this data is more readily available once cached on the Memcached server. If the Memcached server does not hold the desired data or information, the webserver will query the slower database and then store the result in Memcached.

The vulnerability affecting Memcached was discovered the week of February 26, 2018. The vulnerability targets the Memcached listener hosted by default on UDP Port 11211. At the time of this blog, there are two unique, open-source Proof of Concept (PoC) exploits published: one exploit targets a static list of 17,000 unsecured Memcached servers, and the other allows the user to select a target. According to researchers from Cloudflare who have performed analytics on the Memcached PoCs, as well as on the live attacks, the reflection attack has an extremely large amplification vector at more than 51,000 times the original request!

“15 bytes of request triggered 134KB of response,” Cloudflare states. “This is amplification factor of 10,000x! In practice, we’ve seen a 15-byte request result in a 750kB response (that’s a 51,200x amplification).”

DDoS Attack Mitigation

At this point it is unclear how many Memcached systems are currently available for participation within a MemCrashed exploit. It’s likely, however, that a DDoS using more than 6,000 Memcached servers is possible. In order to mitigate the usage of Memcached servers being exploited in these attacks, it is recommended that the following actions be taken:

• Remove Memcached servers from publically accessible networks
• If servers must be on publically accessible networks, do not use the UDP listener (disabled by default on the latest version)
• Implement source address validation, Best Practice Guide (BPG) 38 and 84
• Implement authentication controls on any server responding to public requests
• Block unauthorized ingress access to UDP Port 11211

LogRhythm AI Engine Rules

The LogRhythm Labs team has created the following rules to assist in the identification of Memcached attack traffic either targeting or originating from the client network. These rules are not designed to mitigate any malicious Memcached attacks, but are designed rather to alert security operations center (SOC) analysts should any Memcached attack attempts be made. These rules must have Knowledge Base (KB) 7.1.430.1 or higher before being deployed within the environment.

CAT : Attack : Allowed Potential Memcached Reflection Attack

Figure 1: First AI Engine Rule Designed to Detect Memcached Attacks—CAT : Attack : Allowed Potential Memcached Reflection Attack

CAT : Attack : Denied Potential Memcached Reflection Attack

Figure 2: Second AI Engine Rule Designed to Detect Memcached Attacks— CAT : Attack : Denied Potential Memcached Reflection Attack

CAT : Reconnaissance : Allowed Potential Memcached Traffic

Figure 3: Third AI Engine Rule Designed to Detect Memcached Attacks—CAT : Reconnaissance : Allowed Potential Memcached Traffic

CAT : Reconnaissance : Denied Potential Memcached Traffic

Figure 4: Fourth AI Engine Rule Designed to Detect Memcached Attacks— CAT : Reconnaissance : Denied Potential Memcached Traffic

How to Implement AI Engine Rules

In order to implement these AI Engine Rules, please perform the following steps:

1. Download all four of the AI Engine Rules
2. Open the LogRhythm Console
3. Navigate to the following window:
   • LogRhythm Console > Deployment Manager > AI Engine
4. Click on Actions > Import
5. Select the four rules, Click “OK”
6. Within the AI Engine Rule Name selector field, type in the following: “CAT :”
7. After tuning, select the four new rules and select, Right Click > Actions > Enable
8. Restart the AI Engine Service
9. Tuning:
   • Each of the rules should use the appropriate network traffic Log Sources for your environment:
     • These rules were specifically designed to use Firewall-specific log sources.
   • False positives:
     • The rules are created as Threshold rules, which allow the client to adjust how many logs of a specific rule are required to trigger
     • If a rule is triggered too often, increasing the threshold number will require more logs to be matched, resulting in a lower number of times the rule will trigger

The Memcached DDoS attacks targeting GitHub attacks set record high bandwidth usage, indicating a high potential for impact on the business and its users. Hackers are evolving both their scope and method of attack. To keep up, your organization needs to be able to rapidly detect threats such as the Memcached DDoS attacks. LogRhythm AI Engine rules can help automate threat detection to ensure your business stays one step ahead of cyberattacks.