Security software that uses 'code hooking' opens the door to hackers

Researchers from data exfiltration prevention company enSilo found six common security issues affecting over 15 products when they studied how software vendors use 'hooking' to inject code into a process in order to intercept, monitor or modify the potentially sensitive system API (application programming interface) calls made by that process.

Most of the flaws enSilo found allow attackers to easily bypass the anti-exploit mitigations available in Windows or third-party applications, allowing attackers to exploit vulnerabilities that they couldn't otherwise or whose exploitation would have been difficult. Other flaws allow attackers to remain undetected on victims' computers or to inject malicious code into any process running on them, the enSilo researchers said in a report sent via email that's scheduled to be published Tuesday.

The hooking method is used extensively in the antivirus world to monitor for potentially malicious behavior, but is also used by anti-exploitation, virtualization, performance monitoring and sandboxing applications. Some malware programs also hook browser processes to launch so-called Man-in-the-Browser attacks.

Antivirus programs accounted for most of the affected products the security company identified, but one vulnerability also exists in a commercial hooking engine developed by Microsoft and used by over 100 other software vendors.

EnSilo identified affected products from AVG, Kaspersky Lab, McAfee/Intel Security, Symantec, Trend Micro, Bitdefender, Citrix, Webroot, Avast, Emsisoft and Vera Security.

Some of these vendors have released fixes for the flaws, but patching is not easy because it often requires recompiling each affected product individually. Microsoft plans to release patches for its Microsoft Detours hooking engine in August, enSilo said. 

Bitdefender said it fixed the problem on Jan. 19, and has pushed the fix out to affected customers. Microsoft and Symantec did not immediately respond to a request for comment.

The researchers plan to release technical details of the vulnerabilities during the upcoming Black Hat security conference in Las Vegas in early August.

This is not the first time this type of security weakness has been introduced by antivirus products. Last year, the enSilo researchers found that products from Intel Security, Kaspersky Lab and AVG allocated a memory page with read, write and execute permissions to user-mode processes. This allowed attackers to bypass anti-exploitation protections such as address space layout randomization (ASLR) and data execution prevention (DEP) for third-party applications.

In fact, it was that discovery that fuelled this larger research into hooking implementations used by different products.

It's hard to estimate the number of vulnerable systems, but since the affected software includes the Microsoft Detours engine, the enSilo researchers believe that hundreds of thousands of users might be affected. According to them, the Detours vulnerability has existed for at least eight years.

"Companies using affected software should get patches from the vendors, if available," the enSilo researchers said. "Customers using software from the affected vendors should contact their vendors and demand that the software be patched."