Tuesday, July 1, 2014

How to create a Debian package

I have actually found really useful documentation in the Internet (see references section below) that explains the package creation process in great detail. Nevertheless I thought it could make sense for me to put together a simplified tutorial, using a simple "hello world" program as an example.

As well, as a continuation of this post, I will write another article explaining how to use Pbuilder to compile the binary package for the different Debian distributions, currently wheezy, jessie and sid, using chroot environments.

For this purpose I've used a 64 bits server with Debian wheezy, current stable, version 7.4 (you can check yours at /etc/debian_version). This way I've been able to create packages for both i386 and amd64 architectures.

1.- The package creation process

There are several tools involved in this process, used to build, check and sign the package. Debuild is a wrapper that will call them appropriately, so we don't need to do it manually. Here is a brief description of the tools it invokes:
  • dpkg-buildpackage: It creates a temporary directory with the package files, building later the .deb package with its content. To work properly, it heavily relies on the files in the special debian subdirectory: control, rules, changelog, etc.
  • lintian: Dissects Debian packages trying to find bugs or policy violations.
  • debsign: Signs packages (.dsc and .changes files) using GPG or PGP.

2.- Installing necessary software to build our packages

 apt-get install dh-make build-essential
 apt-get install devscripts fakeroot debootstrap pbuilder

3.- Setting up environment variables

 DEBEMAIL="your_email_address@domain.com"  
 DEBFULLNAME="Your Name"  
 export DEBEMAIL DEBFULLNAME  

4.- Uncompressing our source code (format of the tar.gz file is software-version.tar.gz)

In my case, for the purpose of this little how-to, I will build the Debian package for a simple "hello world" program written in C.
 root@debian-package:/opt# tar -xvzf hello-0.1.tar.gz   
 hello-0.1/  
 hello-0.1/Makefile  
 hello-0.1/hello_world.c  
hello_world.c:
 #include <stdio.h>  
 main ()  
 {  
      printf("Hello World");  
 }  
Makefile:
 DESTDIR=/  
 INSTALL_LOCATION=$(DESTDIR)/usr/  
 CFLAGS:=$(shell dpkg-buildflags --get CFLAGS)  
 LDFLAGS:=$(shell dpkg-buildflags --get LDFLAGS)  
 all: hello_world  
 hello_world: hello_world.o  
      cc $(CFLAGS) $(LDFLAGS) -o $@ hello_world.o  
 install: hello_world_install  
 hello_world_install:  
      mkdir -p $(INSTALL_LOCATION)/bin  
      cp hello_world $(INSTALL_LOCATION)/bin  
      chmod 755 $(INSTALL_LOCATION)/bin/hello_world  
 clean:  
      rm -f *.o hello_world   
There are a two interesting things that we can see in our Makefile:
  • It uses DESTDIR variable to support the DESTDIR convention
  • Dpkg-buildflags is used to get C compiler options (CFLAGS) as well as linker options (LDFLAGS). This complies with the hardening requirements described in Debian documentation.
On the other hand, if your software has any external dependencies, you would need to install those, so you can compile it successfully. Typically you would be able to install them using apt-get.

5.- Building the Debian files skeleton

 root@debian-package:/opt# cd hello-0.1  
 root@debian-package:/opt/hello-0.1# dh_make -f ../hello-0.1.tar.gz   
 Type of package: single binary, indep binary, multiple binary, library, kernel module, kernel patch?  
  [s/i/m/l/k/n] s  
 Maintainer name : Your name
 Email-Address  : your_email_address@domain.com   
 Date       : Tue, 24 Jun 2014 21:50:02 +0000  
 Package Name   : hello  
 Version     : 0.1  
 License     : blank  
 Type of Package : Single  
 Hit <enter> to confirm:   
 Done. Please edit the files in the debian/ subdirectory now. You should also  
 check that the hello Makefiles install into $DESTDIR and not in / .  
As we only want to build a single binary package, I chose that option. Multiple binary package option would in fact, build multiple .deb packages.

Now we have a new directory, called "debian", with all the necessary Debian files that we need to build our package, including examples. This includes important files like:
  •  control: includes meta data about the package
  •  rules: specifies how the package is going to be built
  •  changelog: history of the debian package
  •  copyright: copyright information
As well, some other example files are created by dh_make, that we won't use at this point and can be deleted safely.
 root@debian-package:/opt/hello-0.1/debian# rm -f *.ex *.EX README.*  
 root@debian-package:/opt/hello-0.1/debian# ls  
 changelog compat control copyright docs rules source  

6.- Control file

The control file has two sections, the first part refers to the source package and the second to the binary one. More information about the different fields can be found in deb-control manual page.
 Source: hello  
 Maintainer: Your Name <your_email_address@domain.com>   
 Build-Depends: debhelper (>= 8.0.0)   
 Standards-Version: 3.9.3   
 Section: utils
  
 Package: hello  
 Priority: extra  
 Architecture: any   
 Depends: ${shlibs:Depends}, ${misc:Depends}  
 Description: Test package for hello world  
  This software literally prints "hello world".   
The variable ${shlibs:Depends} will be substituted by the shared library dependencies needed to build our binary package. Those are calculated automatically by dh_shlibdeps, one of the tools of the debhelper suite.

7.- Changelog file

 root@debian-package:/opt/hello-0.1# cat debian/changelog   
 hello (0.1-1) unstable; urgency=low  
  * Initial release (Closes: #100) # there was no previous ITP  
  -- Your Name <your_email_address@domain.com> Wed, 25 Jun 2014 19:50:08 +0000  
ITP stands for Intend to Package and, for our package to be included in a Debian distribution, the changelog file should close an existing bug. For our example we closed bug #100, this way we won't see lintian warnings requiring for this number later.

Note: We can use "dch -i" command to edit our changelog file.

8.- Copyright file

 root@debian-package:/opt/hello-0.1# cat debian/copyright   
 Format: http://www.debian.org/doc/packaging-manuals/copyright-format/1.0/  
 Upstream-Name: hello  
 Files: *  
 Copyright: 2014 Your Name <your_email_address@domain.com>  
 License: GPL-2  
  This package is free software; you can redistribute it and/or modify  
  it under the terms of the GNU General Public License as published by  
  the Free Software Foundation; either version 2 of the License, or  
  (at your option) any later version.  
  .  
  This package is distributed in the hope that it will be useful,  
  but WITHOUT ANY WARRANTY; without even the implied warranty of  
  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the  
  GNU General Public License for more details.  
  .  
  You should have received a copy of the GNU General Public License  
  along with this program. If not, see <http://www.gnu.org/licenses/>  
  .  
  On Debian systems, the complete text of the GNU General  
  Public License version 2 can be found in "/usr/share/common-licenses/GPL-2".  

9.- Rules file

The rules file invokes the original software Makefile script, as well as the debhelper suite of tools (with the prefix "dh_). These tools handle different tasks, including the creation of the .deb file (dh_builddeb).
 #!/usr/bin/make -f  
 # -*- makefile -*-  
 # Uncomment this to turn on verbose mode.  
 #export DH_VERBOSE=1  
 %:  
     dh $@ 
The rules file can be run with different targets: clean (invokes make clean), build (invokes make) and binary (invokes make install). Usage of fakeroot command is recommended so you don't need to build your packages as root.
 root@debian-package:/opt/hello-0.1# fakeroot debian/rules clean  
 dh clean   
   dh_testdir  
   dh_auto_clean  
 make[1]: Entering directory `/opt/hello-0.1'  
 rm -f *.o hello_world   
 make[1]: Leaving directory `/opt/hello-0.1'  
   dh_clean  
 root@debian-package:/opt/hello-0.1# fakeroot debian/rules build  
 dh build   
   dh_testdir  
   dh_auto_configure  
   dh_auto_build  
 make[1]: Entering directory `/opt/hello-0.1'  
 cc -g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security  -c -o hello_world.o hello_world.c  
 cc -g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -Wl,-z,relro -o hello_world hello_world.o  
 make[1]: Leaving directory `/opt/hello-0.1'  
   dh_auto_test  
 root@debian-package:/opt/hello-0.1# fakeroot debian/rules binary  
 dh binary   
   dh_testroot  
   dh_prep  
   dh_installdirs  
   dh_auto_install  
 make[1]: Entering directory `/opt/hello-0.1'  
 mkdir -p /opt/hello-0.1/debian/hello/usr//bin  
 cp hello_world /opt/hello-0.1/debian/hello/usr//bin  
 chmod 755 /opt/hello-0.1/debian/hello/usr//bin/hello_world  
 make[1]: Leaving directory `/opt/hello-0.1'  
   dh_install  
   dh_installdocs  
   dh_installchangelogs  
   dh_installexamples  
   dh_installman  
   dh_installcatalogs  
   dh_installcron  
   dh_installdebconf  
   dh_installemacsen  
   dh_installifupdown  
   dh_installinfo  
   dh_pysupport  
 dh_pysupport: This program is deprecated, you should use dh_python2 instead. Migration guide: http://deb.li/dhs2p  
   dh_installinit  
   dh_installmenu  
   dh_installmime  
   dh_installmodules  
   dh_installlogcheck  
   dh_installlogrotate  
   dh_installpam  
   dh_installppp  
   dh_installudev  
   dh_installwm  
   dh_installxfonts  
   dh_installgsettings  
   dh_bugfiles  
   dh_ucf  
   dh_lintian  
   dh_gconf  
   dh_icons  
   dh_perl  
   dh_usrlocal  
   dh_link  
   dh_compress  
   dh_fixperms  
   dh_strip  
   dh_makeshlibs  
   dh_shlibdeps  
   dh_installdeb  
   dh_gencontrol  
 dpkg-gencontrol: warning: File::FcntlLock not available; using flock which is not NFS-safe  
   dh_md5sums  
   dh_builddeb  
 dpkg-deb: building package `hello' in `../hello_0.1-1_amd64.deb'.  
At this point we have already our .deb file created!. We can see the shared libraries needed to build our program (in this case libc6, because of the stdio.h include) listed in our debian/hello.substvars file:
 root@debian-package:/opt/hello-1.0# cat debian/hello.substvars   
 shlibs:Depends=libc6 (>= 2.2.5)  
 misc:Depends=  

10.- Inspecting package contents

 root@debian-package:/opt/hello-0.1# find debian/hello  
 debian/hello  
 debian/hello/DEBIAN  
 debian/hello/DEBIAN/control  
 debian/hello/DEBIAN/md5sums  
 debian/hello/usr  
 debian/hello/usr/bin  
 debian/hello/usr/bin/hello_world  
 debian/hello/usr/share  
 debian/hello/usr/share/doc  
 debian/hello/usr/share/doc/hello  
 debian/hello/usr/share/doc/hello/copyright  
 debian/hello/usr/share/doc/hello/changelog.Debian.gz  
 root@debian-package:/opt/hello-0.1# dpkg --contents ../hello_0.1-1_amd64.deb   
 drwxr-xr-x root/root     0 2014-07-02 02:07 ./  
 drwxr-xr-x root/root     0 2014-07-02 02:07 ./usr/  
 drwxr-xr-x root/root     0 2014-07-02 02:07 ./usr/bin/  
 -rwxr-xr-x root/root   6160 2014-07-02 02:07 ./usr/bin/hello_world  
 drwxr-xr-x root/root     0 2014-07-02 02:07 ./usr/share/  
 drwxr-xr-x root/root     0 2014-07-02 02:07 ./usr/share/doc/  
 drwxr-xr-x root/root     0 2014-07-02 02:07 ./usr/share/doc/hello/  
 -rw-r--r-- root/root    940 2014-07-02 01:12 ./usr/share/doc/hello/copyright  
 -rw-r--r-- root/root    174 2014-06-30 23:51 ./usr/share/doc/hello/changelog.Debian.gz  

11.- Package maintenance scripts

It is possible to supply scripts that will run when the package is installed, upgraded or removed. These scripts are the control information files: preinst, postinst, prerm, postrm. And in some cases, may prompt the user if necessary, typically through a program such as debconf. More information on how to create this scripts can be found at the Debian Policy Manual.

12.- Debuild

As mentioned before, we can use debuild to build the Debian binary and source packages, check it with lintian, and sign it with debsign. We can use "debuild -us -uc" to build the packages without signing the .changes file. More information can be found with "man debuild".
 root@debian-package:/opt/hello-0.1# debuild -us -uc  
  dpkg-buildpackage -rfakeroot -D -us -uc  
 dpkg-buildpackage: warning: using a gain-root-command while being root  
 dpkg-buildpackage: source package hello  
 dpkg-buildpackage: source version 0.1-1  
 dpkg-buildpackage: source changed by Your Name <your_email_address@domain.com>  
  dpkg-source --before-build hello-0.1  
 dpkg-buildpackage: host architecture amd64  
  fakeroot debian/rules clean  
 dh clean   
   dh_testdir  
   dh_auto_clean  
 make[1]: Entering directory `/opt/hello-0.1'  
 rm -f *.o hello_world   
 make[1]: Leaving directory `/opt/hello-0.1'  
   dh_clean  
  dpkg-source -b hello-0.1  
 dpkg-source: info: using source format `3.0 (quilt)'  
 dpkg-source: info: building hello using existing ./hello_0.1.orig.tar.gz  
 dpkg-source: info: building hello in hello_0.1-1.debian.tar.gz  
 dpkg-source: info: building hello in hello_0.1-1.dsc  
  debian/rules build  
 dh build   
   dh_testdir  
   dh_auto_configure  
   dh_auto_build  
 make[1]: Entering directory `/opt/hello-0.1'  
 cc -g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security  -c -o hello_world.o hello_world.c  
 cc -g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -Wl,-z,relro -o hello_world hello_world.o  
 make[1]: Leaving directory `/opt/hello-0.1'  
   dh_auto_test  
  fakeroot debian/rules binary  
 dh binary   
   dh_testroot  
   dh_prep  
   dh_installdirs  
   dh_auto_install  
 make[1]: Entering directory `/opt/hello-0.1'  
 mkdir -p /opt/hello-0.1/debian/hello/usr//bin  
 cp hello_world /opt/hello-0.1/debian/hello/usr//bin  
 chmod 755 /opt/hello-0.1/debian/hello/usr//bin/hello_world  
 make[1]: Leaving directory `/opt/hello-0.1'  
   dh_install  
   dh_installdocs  
   dh_installchangelogs  
   dh_installexamples  
   dh_installman  
   dh_installcatalogs  
   dh_installcron  
   dh_installdebconf  
   dh_installemacsen  
   dh_installifupdown  
   dh_installinfo  
   dh_pysupport  
 dh_pysupport: This program is deprecated, you should use dh_python2 instead. Migration guide: http://deb.li/dhs2p  
   dh_installinit  
   dh_installmenu  
   dh_installmime  
   dh_installmodules  
   dh_installlogcheck  
   dh_installlogrotate  
   dh_installpam  
   dh_installppp  
   dh_installudev  
   dh_installwm  
   dh_installxfonts  
   dh_installgsettings  
   dh_bugfiles  
   dh_ucf  
   dh_lintian  
   dh_gconf  
   dh_icons  
   dh_perl  
   dh_usrlocal  
   dh_link  
   dh_compress  
   dh_fixperms  
   dh_strip  
   dh_makeshlibs  
   dh_shlibdeps  
   dh_installdeb  
   dh_gencontrol  
 dpkg-gencontrol: warning: File::FcntlLock not available; using flock which is not NFS-safe  
   dh_md5sums  
   dh_builddeb  
 dpkg-deb: building package `hello' in `../hello_0.1-1_amd64.deb'.  
  dpkg-genchanges >../hello_0.1-1_amd64.changes  
 dpkg-genchanges: warning: missing Priority for source files  
 dpkg-genchanges: including full source code in upload  
  dpkg-source --after-build hello-0.1  
 dpkg-buildpackage: full upload (original source is included)  
 Now running lintian...  
 warning: the authors of lintian do not recommend running it with root privileges!  
 W: hello: binary-without-manpage usr/bin/hello_world  
 Finished running lintian.  
Both source and binary packages have been built now. One of Lintian checks warned us about our package not including a manual page for hello_world binary. Although it is just a reminder, more information about this warning can be displayed using "lintian-info -t lintian_tag" command. In our case lintian_tag is "binary-without-manpage".
 root@debian-package:/opt/hello-0.1# ls -l ../  
 total 32  
 drwxr-xr-x 3 root root 4096 Jul 2 03:07 hello-0.1  
 -rw-r--r-- 1 root root 3020 Jul 2 03:09 hello_0.1-1_amd64.build  
 -rw-r--r-- 1 root root 1407 Jul 2 03:08 hello_0.1-1_amd64.changes  
 -rw-r--r-- 1 root root 3338 Jul 2 03:08 hello_0.1-1_amd64.deb  
 -rw-r--r-- 1 root root 1378 Jul 2 03:07 hello_0.1-1.debian.tar.gz  
 -rw-r--r-- 1 root root 735 Jul 2 03:07 hello_0.1-1.dsc  
 -rw-r--r-- 1 root root 441 Jun 30 23:43 hello_0.1.orig.tar.gz  
 -rw-r--r-- 1 root root 441 Jun 30 23:43 hello-0.1.tar.gz  

13.- Extracting sources

Finally we can use dpkg-source command to extract the sources from our package:
 root@debian-package:/opt# dpkg-source -x hello_0.1-1.dsc
 dpkg-source: warning: extracting unsigned source package (hello_0.1-1.dsc)  
 dpkg-source: info: extracting hello in hello-0.1  
 dpkg-source: info: unpacking hello_0.1.orig.tar.gz  
 dpkg-source: info: unpacking hello_0.1-1.debian.tar.gz  

References

https://www.debian.org/doc/debian-policy/
http://www.debian-administration.org/articles/336
https://www.debian.org/doc/manuals/packaging-tutorial/packaging-tutorial.en.pdf

Tuesday, December 10, 2013

IPMI for remote management

I've been recently playing around with IPMI (Intelligent Platform Management Interface) which was developed by Intel and implemented in Dell, HP, Supermicro and a variety of other Intel servers.

In my lab, I am using it to manage a Linux Server, and here is how I configured it.

1.- Loading IPMI kernel modules

I added ipmi_si and ipmi_devintf to my /etc/modules so those can load at startup, but also loaded those manually.
 root@donkey:~# modprobe ipmi_si  
 root@donkey:~# modprobe ipmi_devintf    

2.- Configuring IPMI network settings with ipmitool

 root@donkey:~# ipmitool lan set 1 ipsrc static  
 root@donkey:~# ipmitool lan set 1 ipaddr 192.168.0.201  
 Setting LAN IP Address to 192.168.0.201  
 root@donkey:~# ipmitool lan set 1 netmask 255.255.255.0  
 Setting LAN Subnet Mask to 255.255.255.0  
 root@donkey:~# ipmitool lan set 1 defgw ipaddr 192.168.0.1  
 Setting LAN Default Gateway IP to 192.168.0.1  
 root@donkey:~# ipmitool lan print  
 Set in Progress     : Set Complete  
 Auth Type Support    : NONE MD2 MD5 PASSWORD  
 Auth Type Enable    : Callback : MD2 MD5 PASSWORD  
             : User   : MD2 MD5 PASSWORD  
             : Operator : MD2 MD5 PASSWORD  
             : Admin  : MD2 MD5 PASSWORD  
             : OEM   : MD2 MD5 PASSWORD  
 IP Address Source    : Static Address  
 IP Address       : 192.168.0.201  
 Subnet Mask       : 255.255.255.0  
 MAC Address       : 00:25:90:XX:XX:XX  
 SNMP Community String  : public  
 IP Header        : TTL=0x00 Flags=0x00 Precedence=0x00 TOS=0x00  
 BMC ARP Control     : ARP Responses Enabled, Gratuitous ARP Disabled  
 Default Gateway IP   : 192.168.0.1  
 Default Gateway MAC   : 00:00:00:00:00:00  
 Backup Gateway IP    : 0.0.0.0  
 Backup Gateway MAC   : 00:00:00:00:00:00  
 802.1q VLAN ID     : Disabled  
 802.1q VLAN Priority  : 0  
 RMCP+ Cipher Suites   : 0,1,2,3,6,7,8,11,12  
 Cipher Suite Priv Max  : aaaaXXaaaXXaaXX  
             :   X=Cipher Suite Unused  
             :   c=CALLBACK  
             :   u=USER  
             :   o=OPERATOR  
             :   a=ADMIN  
             :   O=OEM  

3.- Configuring user credentials with ipmitool

Default admin user is 'ADMIN', here is how I set his password:
 root@donkey:~# ipmitool user list 1  
 ID Name       Callin Link Auth IPMI Msg  Channel Priv Limit  
 2  ADMIN      false  false   true    ADMINISTRATOR  
 root@donkey:~# ipmitool user set password 2 yourpassword  

4.- Accessing IPMI interface

We can now access the IPMI interface and log into the system by going to:

https://192.168.0.201 (this is the IP I assigned to the IPMI interface).

References

Sunday, January 27, 2013

Installing Cuckoo Sandbox on VirtualBox Ubuntu Server LTS

Quoting their website Cuckoo sandbox is an Open Source automated malware analysis system. To do so it uses custom components that monitor the behavior of the malicious processes while running in an isolated environment (typically a Windows operating system).

It can retrieve the following type of results:
  • Traces of win32 API calls performed by all processes spawned by the malware.
  • Files being created, deleted and downloaded by the malware during its execution.
  • Memory dumps of the malware processes.
  • Network traffic trace in PCAP format.
  • Screenshots of Windows desktop taking during the execution of the malware.
  • Full memory dumps of the machines.
In our case I decided to use a combination of Ubuntu LTS server and VirtualBox to setup the platform where we are going to run Cuckoo. For further details on how to install these systems you can see my previous posts:
Cuckoo (version 0.5) has been developed in Python and integrated with MongoDB, Yara, SSDEEP, Tcpdump for different purposes. That is why my recommendation is to install all these packages including Cuckoo Python dependencies. Here are the necessary steps to do it:

1.- Installing Python and dependencies

 $ apt-get install python # installed by default  
 $ apt-get install python-magic # for identifying file formats  
 $ apt-get install python-dpkt # for extracting info from pcaps  
 $ apt-get install python-mako # for rendering html reports and web gui  
 $ apt-get install python-sqlalchemy  
 $ apt-get install python-jinja2 # necessary for web.py utility  
 $ apt-get install python-bottle # necessary for web.py utility  

2.- Installing SSDEEP for calculating fuzzy hashes

 $ apt-get install ssdeep  
 $ apt-get install python-pyrex # required for pyssdeep installation  
 $ apt-get install subversion  
 $ apt-get install libfuzzy-dev   
 $ svn checkout http://pyssdeep.googlecode.com/svn/trunk/ pyssdeep  
 $ cd pyssdeep  
 $ python setup.py build  
 $ python setup.py install # run as root user  

3.- Installing MongoDB and Python support

 $ apt-get install python-pymongo # for mongodb support  
 $ apt-get install mongodb # includes server and clients  

4.- Installing Yara and Python support

 $ apt-get install g++  
 $ apt-get install libpcre3 libpcre3-dev  
 $ wget http://yara-project.googlecode.com/files/yara-1.6.tar.gz  
 $ tar -xvzf yara-1.6.tar.gz  
 $ cd yara-1.6  
 $ ./configure  
 $ make  
 $ make check  
 $ make install # finished yara installation  
 $ wget http://yara-project.googlecode.com/files/yara-python-1.6.tar.gz  
 $ tar -xvzf yara-python-1.6.tar.gz  
 $ cd yara-python-1.6  
 $ python setup.py build  
 $ python setup.py install # finished python support installation  

5.- Modifying Tcpdump running privileges

This is necessary so Cuckoo can run Tcpdump as non-root user.
 $ apt-get install libcap2-bin  
 $ setcap cap_net_raw,cap_net_admin=eip /usr/sbin/tcpdump  
 $ getcap /usr/sbin/tcpdump # to check changes have been applied  

6.- Installing Cuckoo Sandbox

 $ sudo useradd cuckoo  
 $ usermod -a -G vboxusers cuckoo # add cuckoo to vboxusers group  
 $ id cuckoo # checks cuckoo user details  
 $ apt-get install git   
 $ git clone git://github.com/cuckoobox/cuckoo.git   

7.- Configuring Windows Guest virtual machine

At this point we need to install Cuckoo python agent in the virtual machine that we want to use to run the malware. I am going to continue the work described in my previous post and use WindowsXPVM1 for this purpose.

First steps to prepare the Windows Guest system:
Next we copy the Python agent to our Windows shared folder:
 $ cp /home/santiago/cuckoo/cuckoo/agent/agent.py /home/santiago/cuckoo/shares/WindowsXPVM1/  
I also renamed it to agent.pyw to prevent the command prompt from showing. We can run it manually or configure it to run at Windows startup following these steps:
  • Copy to C:\Python27\agent.pyw
  • Add it to HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\Run: Name:'Agent' Type:'REG_SZ' Data:"C:\Python27\agent.pyw"
After executing the Python script on the virtual machine a new socket should be listening on 0.0.0.0:8000



Our virtual machine is now ready to run malware so it's time to save the system state creating a VirtualBox snapshot.
 $ vboxmanage snapshot "WindowsXPVM1" take "WindowsXPVM1Snap01" --pause  
And these are the commands we can use to restore the snapshot.
 $ vboxmanage controlvm "WindowsXPVM1" poweroff  
 $ vboxmanage snapshot "WindowsXPVM1" restorecurrent  
 $ vboxheadless --startvm "WindowsXPVM1"  

8.- Starting Cuckoo sandbox

Before starting Cuckoo for the first time, we need to configure Cuckoo VirtualBox settings to specify the virtual machine the system will use to analyze a malware sample. To do it we edit cuckoo/conf/virtualbox.conf file and set the following variables.
  [virtualbox]  
  # Specify which VirtualBox mode you want to run your machines on.  
  # Can be "gui", "sdl" or "headless". Refer to VirtualBox's official  
  # documentation to understand the differences.  
  mode = headless  
  # Path to the local installation of the VBoxManage utility.  
  path = /usr/bin/VBoxManage  
  # Specify a comma-separated list of available machines to be used. For each  
  # specified ID you have to define a dedicated section containing the details  
  # on the respective machine. (E.g. cuckoo1,cuckoo2,cuckoo3)  
  machines = WindowsXPVM1  
  [WindowsXPVM1]  
  # Specify the label name of the current machine as specified in your  
  # VirtualBox configuration.  
  label = WindowsXPVM1  
  # Specify the operating system platform used by current machine  
  # [windows/darwin/linux].  
  platform = windows  
  # Specify the IP address of the current machine. Make sure that the IP address  
  # is valid and that the host machine is able to reach it. If not, the analysis  
  # will fail.  
  ip = 192.168.56.101  
Finally we can start our freshly installed Cuckoo sandbox.
  root@donkey:/home/santiago/cuckoo/cuckoo# python cuckoo.py
 
                                 _|                            
     _|_|_|  _|    _|    _|_|_|  _|  _|      _|_|      _|_|    
   _|        _|    _|  _|        _|_|      _|    _|  _|    _|  
   _|        _|    _|  _|        _|  _|    _|    _|  _|    _|  
     _|_|_|    _|_|_|    _|_|_|  _|    _|    _|_|      _|_|

  Cuckoo Sandbox 0.5  
  www.cuckoosandbox.org  
  Copyright (c) 2010-2012  
  Checking for updates...  
  Good! You have the latest version available.  
 2013-01-26 23:25:33,216 [lib.cuckoo.core.scheduler] INFO: Using "virtualbox" machine manager  
 2013-01-26 23:25:33,290 [lib.cuckoo.core.scheduler] INFO: Loaded 1 machine/s  
 2013-01-26 23:25:33,290 [lib.cuckoo.core.scheduler] INFO: Waiting for analysis tasks...  

9.- Analyzing a malware sample

I decided to analyze the following malware sample: efeb717fdbb98d8043eb4c51254d9b74 You can find virustotal description here. We can use submit.py util for it.
 root@donkey:/home/santiago/cuckoo/cuckoo/utils# python submit.py /home/santiago/binaries/efeb717fdbb98d8043eb4c51254d9b74  
 Success: File "/home/santiago/binaries/efeb717fdbb98d8043eb4c51254d9b74" added as task with ID 4  
And these are Cuckoo logs while performing the malware analysis.
 2013-01-26 23:34:00,275 [lib.cuckoo.core.scheduler] INFO: Starting analysis of FILE "/home/santiago/binaries/efeb717fdbb98d8043eb4c51254d9b74" (task=4)  
 2013-01-26 23:34:00,286 [lib.cuckoo.core.scheduler] INFO: File already exists at "/home/santiago/cuckoo/cuckoo/storage/binaries/8dafb21e7d106a6c98f745f30c2577ee7b0984ec7ba2c4107f7ddcd0d127baf6"  
 2013-01-26 23:34:00,304 [lib.cuckoo.core.scheduler] INFO: Task #4: acquired machine WindowsXPVM1 (label=WindowsXPVM1)  
 2013-01-26 23:34:00,312 [lib.cuckoo.core.sniffer] INFO: Started sniffer (interface=vboxnet0, host=192.168.56.101, dump path=/home/santiago/cuckoo/cuckoo/storage/analyses/4/dump.pcap)  
 2013-01-26 23:34:02,063 [lib.cuckoo.core.scheduler] INFO: Task #4: analysis procedure completed  
Then we can web.py Cuckoo tool to view the output of the analysis.
 root@donkey:/home/santiago/cuckoo/cuckoo/utils# python web.py   
 Bottle server starting up (using WSGIRefServer())...  
 Listening on http://0.0.0.0:8080/  
 Hit Ctrl-C to quit.  
And at this point we can connect to our host through the web http://192.168.0.200:8080 and see our analysis report.


References

http://www.cuckoosandbox.org/
http://www.virtualbox.org/
http://www.virustotal.com/
http://blog.michaelboman.org/

Saturday, January 26, 2013

Setting up a Windows Guest on VirtualBox

I recently installed VirtualBox on Ubuntu LTS as described in my previous post. Now I am going to install a Windows XP Guest on it, so it can later be used as a platform to run malware for automatic analysis with Cuckoo sandbox.

In this case, instead of using Phpvirtualbox web interface, I will choose to use the command line so it will be easier in the future to automate the virtual machine creation process by using a bash script.

These are the specs I am going to use for the Windows XP:
  • 1GB RAM memory
  • 20GB of Hard Disk space
  • VDI format for the virtual disk
  • Dynamically allocated storage

1.- Creating the virtual machine

The command vboxmanage can be used to create the virtual machine, using settings above, and to attach a DVD drive with the ISO image of the Windows XP. In my case I decided to name it WindowsXPVM1.
 $ vboxmanage createvm --name "WindowsXPVM1" --ostype WindowsXP --register  
 $ vboxmanage modifyvm "WindowsXPVM1" --memory 1000 --acpi on --boot1 dvd --nic1 nat  
 $ vboxmanage createhd --filename "WinXP.vdi" --size 20000  
 $ vboxmanage storagectl "WindowsXPVM1" --name "IDE Controller" --add ide --controller PIIX4  
 $ vboxmanage storageattach "WindowsXPVM1" --storagectl "IDE Controller" --port 0 --device 0 --type hdd --medium "WinXP.vdi"  
 $ vboxmanage storageattach "WindowsXPVM1" --storagectl "IDE Controller" --port 0 --device 1 --type dvddrive --medium /pathtoyouriso/windowsxp.iso  
At this point we can start the virtual machine to start the Windows installation procedure.
 $ VBoxHeadless --startvm "WindowsXPVM1"  
In order to connect to the system we can both use Phpvirtualbox console or directly connect through Remote Desktop Protocol (RDP) to the host.

2.- Installing guest additions in our virtual machine

 $ wget http://dlc.sun.com.edgesuite.net/virtualbox/4.1.12/VBoxGuestAdditions_4.1.12.iso  
Once downloaded we need to mount the ISO file at the Windows XP and follow the installation wizard.

3.- Adding a shared folder and recording the network traffic

 $ vboxmanage controlvm "WindowsXPVM1" poweroff  
 $ mkdir -p /home/santiago/cuckoo/shares/WindowsXPVM1  
 $ vboxmanage sharedfolder add "WindowsXPVM1" --name "WindowsXPVM1" --hostpath /home/santiago/cuckoo/shares/WindowsXPVM1 --automount  
 $ vboxmanage sharedfolder add "WindowsXPVM1" --name setup --hostpath /home/santiago/cuckoo/shares/setup --automount --readonly  
 $ vboxmanage modifyvm "WindowsXPVM1" --nictrace1 on --nictracefile1 /home/santiago/cuckoo/shares/WindowsXPVM1/dump.pcap  
 $ vboxheadless --startvm "WindowsXPVM1"  

4.- Configuring virtual machine to use a host-only adapter

 $ lsmod | grep vboxnetadp # module needed to add a new host-only interface at the host  
 $ vboxmanage list hostonlyifs # checks host-only interfaces at the host  
 $ vboxmanage hostonlyif create # leaving default IP 192.168.56.1/24  
 $ vboxmanage list dhcpservers # checks dhcp servers  
 $ vboxmanage list vms # checks virtual machines  
 $ vboxmanage showvminfo "WindowsXPVM1" # checks NICs information  
 $ vboxmanage controlvm "WindowsXPVM1" poweroff   
 $ vboxmanage modifyvm "WindowsXPVM1" --nic1 hostonly  
 $ vboxmanage modifyvm "WindowsXPVM1" --hostonlyadapter1 vboxnet0  
 $ vboxheadless --startvm WindowsXPVM1  
The gateway (192.168.56.1) and DNS Server (in this case I will use Google's 8.8.8.8) need to be configured manually at the Guest using Windows settings.

5.- Configuring the Host IP forwarding and firewall filters

 $ iptables -A FORWARD -o eth0 -i vboxnet0 -s 192.168.56.0/24 -m conntrack --ctstate NEW -j ACCEPT  
 $ iptables -A FORWARD -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT  
 $ iptables -A POSTROUTING -t nat -j MASQUERADE  
 $ sysctl -w net.ipv4.ip_forward=1  
We can add these commands to our /etc/rc.local file if we want those to be executed every time the server wakes up or restarts.

6.- Starting and stopping the virtual machine

To start VirtualBox web service and the virtual machine we need to run the following commands:
 $ vboxwebsrv -b  
 $ vboxmanage list vms # Optional to list virtual machines  
 $ vboxheadless --startvm "WindowsXPVM1"  
And this is how we can stop it:
 $ vboxmanage controlvm "WindowsXPVM1" poweroff  
And we are done. We should now be able to use our fresh installation of our virtual Windows XP.

References

http://www.virtualbox.org/manual/
http://blog.michaelboman.org/

Installing VirtualBox on Ubuntu Server LTS

I decided to install VirtualBox on Ubuntu server so I can use it later with Cuckoo Sandbox for malware analysis.

The steps followed for this installation are:
  • Download and installation of Ubuntu Server LTS (current version 12.04.1)
  • VirtualBox and dependencies installation (current stable version 4.1.12)
  • Phpvirtualbox installation for headless servers (version 4.1-11)
  • VirtualBox extension pack installation for VRDP support
  • Starting VirtualBox and connecting to Phpvirtualbox web user interface

1.- Download and installation of Ubuntu Server LTS

I decided to use Ubuntu Server LTS as it is stable and does not require the installation of a Desktop environment, which I won't use for my purposes. The server used has a 64 bits CPU, 12GB RAM, and 514GB of hard disk space, what is more than enough to run several virtual machines in parallel.

A fresh Ubuntu Server image can be downloaded from: http://www.ubuntu.com/download/server

Then you can choose to run the ISO from a USB stick or CD-ROM drive. My recommendation is to install only the base system, so we keep the server clean from packages that we won't use. The only extra package I installed was the SSH server so I can access it remotely.

Once finished the installation processes lets also upgrade the Debian packages to the latest version by
running these commands:
 $ apt-get update  
 $ apt-get dist-upgrade  
As well I setup the hostname and network settings at /etc/hostname and /etc/network/interfaces.

2.- VirtualBox and dependencies installation

Installing Virtualbox with apt-get:
 $ apt-get install virtualbox  
Checking installed packages:
 $ dpkg -l | grep -i virtualbox  
 ii virtualbox             4.1.12-dfsg-2ubuntu0.2    x86 virtualization solution - base binaries  
 ii virtualbox-dkms          4.1.12-dfsg-2ubuntu0.2    x86 virtualization solution - kernel module sources for dkms  
 ii virtualbox-qt           4.1.12-dfsg-2ubuntu0.2    x86 virtualization solution - Qt based user interface  

3.- Installing Phpvirtualbox

First we need to install apache2 and php:
 $ apt-get install apache2  
 $ apt-get install php5  
Then we can install Phpvirtualbox, setting the permissions of the directory to your own username (mine is santiago):
 $ cd /var/www/  
 $ wget http://phpvirtualbox.googlecode.com/files/phpvirtualbox-4.1-11.zip  
 $ unzip phpvirtualbox-4.1-11.zip  
 $ chown -R santiago:santiago /var/www/phpvirtualbox/  
 $ cp /var/www/phpvirtualbox/config.php-example /var/www/phpvirtualbox/config.php  
Then edit /var/www/phpvirtualbox/config.php and set the username and password for the system user that runs VirtualBox:
 var $username = 'santiago';  
 var $password = 'yourpassword';  

4.- VirtualBox extension pack installation for VRDP support

Installing the extension pack will allow us to control the virtual machines desktop remotely.
 wget http://download.virtualbox.org/virtualbox/4.1.12/Oracle_VM_VirtualBox_Extension_Pack-4.1.12.vbox-extpack  
 vboxmanage extpack install Oracle_VM_VirtualBox_Extension_Pack-4.1.12.vbox-extpack  

5.- Starting VirtualBox and connecting to Phpvirtualbox user interface

The following command is used to start VirtualBox web services
 $ vboxwebsrv -b  
Then we can connect to the user interface from our browser at http://yourserverip/phpvirtualbox
user: admin
password: admin

We should be know able to use our fresh installation of VirtualBox.

References

http://www.virtualbox.org/manual/
http://codesupply.net/content/setup-ubuntu-1110-64bit-server-headless-virtualbox-host