This weekend we have put our new log-parsing and reporting code into production. The new code significantly increases our speed of log parsing (by about two orders of magnitude) and it provides a lot more help to help our users research what particular blocked threats were caused by. As product manager I am very pleased to say that it is a massive improvement over the previous stuff but, for our existing users, there are a couple of niggles.
The first is very simple: please clear your browser cache before visiting the reporting section of the ThreatSTOP website because otherwise it will look very strange as there are clashes in the javascript/css code. The second is that one of the new features - filter report by feed - is not working consistently on the production server/database, although needless to say it worked and still works flawlessly on our test servers. We are, of course, working on identifying why it does not always work, but in the mean time be aware that if you do filter by feed and get 0 responses this may not be accurate (if you get a non-zero number then this will be correct, the issue is that sometimes we get 0 results back when we should have had some) Update: The search by feed only works with logs parsed by the new log parser system and thus is unlikely to work for logs uploaded before last Friday.
Ok beyond the niggles what will our customers notice? the log parsing code is effectively invisible, but the recent delays between log submission and being able to view reports should now be gone. On the reporting front the changes are rather more obvious:
When looking at the detailed reporting pages users will immediately notice that IP addresses are all underlined with a dotted line. Hovering a mouse over a particular IP address brings up a menu of drill down options.
The top link on the drilldown list (Research) is a report on what we at ThreatSTOP know about the address, that is to say it is essentially the same as the what we used to report in a separate popup window. The astute will note that the new popup window has options to allow the user to add this ip address to either a custom block list or a custom white (allow) list.
Clicking on the Add to White list button leads to a dialog like this (The block list button has an equivalent one):
The next item down (Domains) is a popup that displays a report from the SIE passive DNS database about what DNS queries have resolved to that address. This is generally more useful for outbound analysis where the list can identify a phishing site or a known botnet C&C host domain name. Although the example above shows just three names, frequently these ip addresses will have been used by thousands of different domains (note that we do not display more than 10,000 domains).
The remainder of the links open external windows to other (malware) research sites that we find useful in identifying information about a particular ip address. If anyone has other sites they find useful we will be glad to add them too.
All these reporting changes are also added, where relevant, to other sections of the website such as the check logs feature.
Finally - if you are still reading this far - here is a worked example of how to use the new features to confirm a bot C&C host and thus identify infected machines on your network. This is a real customer but the internal infected addresses have been obfuscated to preserve some anonymity. First run a report for recent outbound traffic filtering on the "BOTNET" feed and then click on "outbound connections" list.
Next click on the "Research" tab for the Destination IP to learn that this address is a known C&C host:Then confirm by checking on the Domains link to see what sorts of names have been resolving to this address.
Finally click on say the McAfee link to confirm that this is not just a figment of ThreatSTOP's imagination. Now that it has been confirmed that the IP address is bad the source IP addresses (192.168.x.51 and 192.168.x.52 in this case) should be noted down and the computers using them identified for immediate remediation.
