Anti-Spyware Gets HIP

The growing spyware problem may drive proactive, behavioral-based intrusion prevention onto enterprise desktops.

By Andrew Conry-Murray

09/01/2005, 12:00 AM ET

Promise: Anti-spyware vendors are developing behavior-based detection technologies that prevent spyware from reaching enterprise desktops without the use of signatures. The relentless pace of spyware development--and the irrepressible urge for users to install spyware-laden programs--may drive security architects toward proactive solutions.

Players: Much of the innovation in anti-spyware technology comes from a host of small companies, including Tenebril, Webroot, Aluria, Finjan, and Eset.

Prospects: Security architects and IT staff struggle daily with spyware and adware infestations and may thus be willing to experiment with proactive anti-spyware technology. Over time, they may grow more comfortable extending the approach to a broader class of malware.

Anti-spyware start-ups are rolling out proactive solutions that can stop new and unknown programs from invading PCs. Over time, anti-spyware software will likely evolve from threat-specific technologies into Host-based Intrusion Prevention Systems (HIPS) designed to protect desktops and laptops from a broad class of malware.

Current HIPS software hasn't enjoyed widespread desktop deployment because of concerns over false positives and the complexities of policy creation for a diverse population of enterprise users. Proactive anti-spyware software may be a logical avenue for security architects to introduce HIPS-like technology to the enterprise.

It may also be a logical avenue for small, innovative vendors to gain a foothold in corporate desktops. One reason is that spyware and adware have an immediate and persistent impact on PC performance, and IT staff are desperate for relief. Another is that end users are often responsible for infestations. Despite years of warnings against clicking links and downloading software, users are the number-one reason why unwanted programs get installed on enterprise machines. Proactive blocking technology could save users from themselves.


Annti-Spyware Options Proliferate
Click to Enlarge

Spyware may be the entrance point of HIPS into the enterprise because it--and its sinister cousin, adware--cause IT administrators daily pain. Traditional HIPS solutions such as McAfee's Entercept, Cisco Systems' Cisco Security Agent, and Sana Security's Primary Response are sold primarily as protection against widespread malware outbreaks from zero-day worms exploiting OS or application vulnerabilities. These outbreaks are devastating when they occur, but also relatively infrequent. In addition, though traditional HIPS products are evolving to block spyware installations, at this point they can't remove spyware from infested PCs.

That's a significant drawback, considering that IT departments are currently beleaguered by spyware and adware. These programs suck up processing power and can render PCs essentially inoperable. Besides affecting user productivity, spyware and adware drain time and money from the IT department. Jeff Pelot, CTO at Denver Health Hospital, knows firsthand. He says 25 percent of all his support calls were spyware- or adware-related, and that simply generating the help desk tickets to deal with infested PCs cost him $6,600 a month.

Part of the problem is self-inflicted. Some spyware and adware gets installed through drive-by downloads in which the user is blameless. But the great majority comes bundled with other software, including games, screen savers, file-sharing software, utilities, and add-ons such as weather trackers and emoticon generators. And it's the users who bring all this junk software onto the PCs.


Most anti-spyware software includes some prevention features, such as the ability to stop Browser Helper Objects (which are companion applications for Internet Explorer and a popular vehicle for adware) from being installed, or create white lists of programs that are acceptable on the PC while blocking all others. They can also warn users if a program attempts to install itself or perform other behaviors, such as changing registry entries.

Anti-spyware software is also digging deeper into the OS. Aluria Software's corporate anti-spyware solution, Paladin, includes a kernel driver to prevent spyware programs from installing on PCs.

However, these prevention measures rely on signatures and will only stop programs that already have a definition in the threat database.

At the same time, the need for proactive prevention is rising because spyware is getting more difficult to remove once it infects a PC. In fact, cutting-edge spyware is beginning to mimic rootkits. "We've seen some spyware that will hook the disk access API. So if you're scanning the hard disk, the spyware tells Windows API not to tell you it's there," says Mike Green, director of product management at anti-spyware maker Webroot Software. "If Windows won't let you see it, how can you delete it?"

Other spyware programs can monitor their own registry keys so that if a portion of the code gets removed, they can call home and get those portions reinstalled.

"The really good developers are reading Microsoft Systems Journal and looking at the same boards as virus creators to learn the darkest Windows API secrets to delve deeper and deeper into the OS," says Fred Felman, senior vice president of marketing at anti-spyware company Tenebril.

To address this problem, vendors are moving beyond signature detection by analyzing the behavior of unknown programs. This September, Tenebril will introduce SpyCatcher 4.0. In addition to using a database of known spyware definitions, the 4.0 version will add a kernel-based software agent that monitors system and API calls to look for potentially malicious behavior from programs that aren't listed in the spyware database.

SpyCatcher's approach is identical to that of established HIPS products from McAfee, Cisco, and Sana. But unlike those products, SpyCatcher focuses exclusively on spyware and adware. It won't look for buffer overflows, which are commonly used by new worms and Trojans to gain entry into target machines.

"Eventually it may have applications in other areas, but right now we're just pursuing spyware with it," says Felman.

Webroot's Spy Sweeper Enterprise 2.5, which was released late this summer, uses heuristic analysis to prevent malicious behavior, such as changing a user's home page or resetting the host file to point to an unwanted Web site or spoofed versions of e-commerce or online banking sites. Webroot's Green says future versions of Spy Sweeper will also include kernel-based components to monitor system calls and provide more proactive protection against unknown attacks.

Other small vendors are pushing proactive solutions at the desktop. Eset, which makes the NOD32 desktop software, uses advanced heuristics to catch unknown spyware, viruses, Trojans, and other malware without signatures. An emulation engine on the desktop analyzes incoming programs from the Web and e-mail.

"You analyze the file to see what it's trying to do, see what it's calling--for instance, trying to hook a registry key at startup, or trying to install a Bowser Helper Object," says Andrew Lee, Eset's CTO. Lee says this emulation process adds about 6 percent of overhead to the desktop processing load. NOD32 backs up its heuristic analysis with a signature database.

Panda Software's TruPrevent offers a full-blown HIPS solution that includes the ability to block new spyware and other malware. The software includes a signature-based IPS, a heuristic-based engine to analyze the intent of incoming programs, and a kernel-based rules engine to monitor file systems, registries, and active processes. Panda also claims to have an artificial intelligence engine that can correlate events from all the security components to help evaluate whether a piece of code is malicious.

Some products eschew behavioral analysis altogether when dealing with unknown programs. Host-based software from start-up GreenBorder creates a virtual environment that allows any untrusted executable to run without hooking into essential files and registries. At the end of the user session, these untrusted programs are simply flushed from the computer. (For more on GreenBorder, see "IPS Odyssey" July 2005)

Of course, the most significant drawback of behavioral-based prevention is the risk of false positives. Security architects must weigh the benefits of proactive security against potential disruptions in employee productivity and irate calls to the help desk.

Anti-spyware vendors also insist that signature-based detection will continue to be an essential weapon in their arsenals. Not only are signatures essential for removing known spyware from infected machines, but they also reduce the likelihood of false positives and help administrators track and report on remediation efforts.