Our fellow security professionals at Damballa have written a pretty good explanation of IP reputation and the benefits of applying it. Since our business at ThreatSTOP is to provide IP reputation perhaps we should ask them to write more copy... However, while the article, as a whole is good, there are a few places where I think it could be improved.
For example, it's always nice to have a concrete example or two. As I blogged about before, some IP addresses are recidivists that are found doing one bad thing after another and in fact that particular IP address - 98.124.198.1 - and its neighbor 98.124.199.1 recently showed up in the logs of yet another one of our customers. And they have now had even more badness added as they are currently also featured on the Autoshun Block List and the Malware Domain List as well. So far the only obvious list these addresses have missed is the Spamhaus DROP one - and I have a feeling that they'll soon be on that as well.
In fact the AS they belong to - AS21740, eNom Inc. - is a great example of an address space you probably don't want your computers talking to. I ran a check on our database for historical activity and found the following:
AS21740
================================
Summary
Querying 8.5.0.0-8.5.1.255: 34 hits found
Querying 63.229.62.0-63.229.62.255: 0 hits found
Querying 63.251.174.0-63.251.174.255: 0 hits found
Querying 64.74.223.0-64.74.223.255: 28 hits found
Querying 69.64.144.0-69.64.147.255: 51 hits found
Querying 69.64.150.0-69.64.159.255: 77 hits found
Querying 70.42.37.0-70.42.37.255: 1 hits found
Querying 74.202.67.0-74.202.67.255: 0 hits found
Querying 98.124.192.0-98.124.193.255: 2 hits found
Querying 98.124.196.0-98.124.199.255: 18 hits found
Querying 98.124.216.0-98.124.225.255: 0 hits found
Querying 98.124.244.0-98.124.244.255: 0 hits found
Querying 98.124.248.0-98.124.255.255: 3 hits found
Querying 207.8.85.0-207.8.85.255: 0 hits found
Querying 207.238.43.0-207.238.43.255: 0 hits found
Total 214 hits from 12288 addresses
The number of hits found for this comparatively small number of addresses is astonishingly high - 1&1 only has 1386 total and its AS consists of 352,256 addresses (almost 30 times as many by my calculations) - and there are ASes with similar numbers of addresses with no more than one or two, typically ancient.
Secondly the question the article asks - and sort of answers - is fine.
So, getting back to the question – can dynamic reputation systems replace IDS systems? If the purpose is to identify what just happened and why a computer was just compromised, then the answer is No. If the purpose of the IDS is to stop the attack from actually happening (which would actually make it an IPS), then mostly Yes – a suitable dynamic reputation system could thwart most of the attacks the protection functions of a typical IPS deployment is designed to stop; the caveat being upon the class of threat being protected against.
But the next paragrpah kind of misses the point:
A dynamic reputation system focuses upon the sources of the threat rather than the payload of the attack. For example, the location of the drive-by download sites rather than a particular exploit embedded within the JavaScript of the malicious website.
The point is, as the example IP address I showed, that once an IP address is bad you really don't want to have anything to do with it. Just because the address was used as (say) a phishing site once doesn't mean it will remain one. In fact quite often it will cycle through being a phisher, a malware dropper and a botnet C&C host. So the fact that the IP reputation system doesn't say why an address is bad is not a drawback at all. This is, if you like, profiling but the evidence suggests that on the Internet profiling works.
Thirdly, while the metaphor of IP reputation compared to the reputation of streets or neighborhoods is fine as far as it goes the problem is that you have to combine it with something else. No you don't want to visit the bad neighborhood but also you don't want people from the bad neighborhood visiting you. And that is something that we don't, as a general rule, enforce in the physical world because we simply have no way to do so. We do to some extent whitelist people (clearance, security badges etc.) but, except in extremely totalitarian regimes, we don't insist that every visitor to anywhere show their id papers.
IP reputation is really the only way to stop zero-day exploits. And it works precisely because the system doesn't care what 'crime' the IP address has committed. It just stops all communication to and from it. Thus it can stop new bot "call home" methods, phishes, malware droppers, server exploits and so on because the bad guys are forced to reuse their compromised hosts. Of course it will never be a perfect defense and no you should never rely solely on IP reputation because criminals will always compromise new addresses, but IP reputation allows the good guys to share information so that each bad IP address only gets a couple of opportunities to attack before it is identified and blocked by all the good guys.