OSSIM hands-on 3: Creating a custom OSSIM plugin

This one is the third post regarding the series of practical exercises for OSSIM users. Now we are going to create a custom plugin to process Exchange Web Server logs through the SIEM engine.

Exchange Web SMTP server logs

The log file used for this practical hands-on exercise can be downloaded here: exchangews.log

Once downloaded, open the file to see the logs we are going to parse. Here are some sample lines:

 2011-10-09 05:00:19 1.1.1.1 36A42160 SMTPSVC1 MEE-PDC 192.168.1.2 0 QUIT - 36A42160 240 6219 68 4 0 SMTP - - - -  
 1.1.1.10 - 1.1.1.9 [11/Oct/2011:13:16:40 -0600] "HELO -?+1.1.1.9 SMTP" 250 46  

Creating the plugin configuration file - exchangews.cfg

1.- Global plugin configuration settings

• Copy ssh.cfg into another one named exchangews.cfg
• Change the plugin_id field (use 9001 as it is part of the user range that goes up to 10000)
• Change location to point to the log file /var/log/exchangews.log
• Delete startup and shutdown fields since those are not going to be used (there is no application associated to this plugin)
• Create new translation table:
• HELO=1
• MAIL=2
• RCPT=3
• DATA=4
• QUIT=5
• xxxx=6
• DEFAULT_=9999

2.- Creating new rules, filling up the fields below

We will create two regular expressions to parse the data, since we have two different formats in the log file.

 [exchangews - Generic rule]
 #2011-10-09 05:00:15 1.1.1.1 36A42160 SMTPSVC1 MEE-PDC 192.168.1.2 0 HELO - +36A42160 250 0 48 13 0 SMTP - - - -  
 #2011-10-09 05:00:16 1.1.1.1 36A42160 SMTPSVC1 MEE-PDC 192.168.1.2 0 MAIL - +FROM:+<test@sample1.com> 250 0 57 45 0 SMTP - - - -  
 event_type=event  
 regexp="(?P<date>\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2})\s(?P<src_ip>\IPV4)\s(?P<userdata2>\S+)\s(?P<hostname>\S+)\s(?P<userdata3>\S+)\s(?P<dst_IP>\IPV4)\s\d\s(?P<type>\w+)"  
 date={normalize_date($date)}  
 plugin_sid={translate($type)}  
 dst_ip={resolv($dst_ip)}  
 src_ip={resolv($src_ip)}  
 hostname={$hostname}  
 userdata2={$userdata2}  
 userdata3={$userdata3}  

 [exchangews = Generic rule 2 NCSA Format]  
 #1.1.1.10 - 1.1.1.9 [11/Oct/2011:13:16:40 -0600] "HELO -?+1.1.1.9 SMTP" 250 46  
 #1.1.1.10 - 1.1.1.9 [11/Oct/2011:13:16:41 -0600] "MAIL -?+FROM:+<Keith@testdomain.com> SMTP" 250 46  
 event_type=event  
 regexp="(?P<src_ip>\IPV4)\s-\s(?P<dst_ip>\S+)\s\[(?P<date>\d\d\/\w{3}\/\d{4}:\d\d:\d\d:\d\d)\s-\d{4}\]\s\"(?P<type>\w+)"  
 date={normalize_date($date)}  
 plugin_sid={translate($type)}  
 dst_ip={resolv($dst_ip)}  
 src_ip={resolv($src_ip)}  

3.- test exchangews.cfg file against the log file (/var/log/exchangews.log)

/usr/share/ossim/scripts/regexp.py /var/log/exchangews.log /etc/ossim/agent/plugins/exchangews.cfg q

Creating database file - exchangews.sql

1.- Create the file using the following examples

INSERT INTO plugin (id, type, name, description) VALUES (9001, 1, 'exchangews', 'Exchange E-mail Web server');

INSERT INTO plugin_sid (plugin_id, sid, category_id, class_id, name, priority, reliability) VALUES (9001, 1, NULL, NULL, 'exchangews: HELO' ,3, 2);

…

INSERT INTO plugin_sid (plugin_id, sid, category_id, class_id, name, priority, reliability) VALUES (9001, 9999, NULL, NULL, 'exchangews: Generic exchange event' ,3, 2);

2.- Insert file values into the database at the server box (10.0.0.30 in case you are following the other hands-on exercises)

cat exchangews.sql | ossim-db

3.- Apply changes in the SIEM

/etc/init.d/ossim-server restart

Enable the plugin on the sensor

• Edit /etc/ossim/agent/config.cfg.orig and add a new line with exchangews plugin.
• Enable the plugin using ossim-setup command.
• Check that ossim-agent is reading the log file with lsof +d command.
• Insert new logs into the file (using cat >>).
• Check in the web interface that those events have been processed.

Results and troubleshooting

For troubleshooting check the log files at:

• /var/log/ossim/agent.log
• /var/log/ossim/server.log

This exercise resulting files can be downloaded here:

• exchangews.cfg
• exchangews.sql
• exchangews.log

References

http://alienvault.com
http://communities.alienvault.com