Log Information

All of the daemons running on the frontend node of the cloud infrastructure provide log files of their activity. This information can be used to debug problems or to investigate security issues.

The virtual machine logs are probably the most useful for debugging problems and understanding the history of a particular machine. These log files are located in the directory /var/log/one. The logs of the virtual machines are numbered like <VM_ID>.log.

This log contains information about the entire lifecycle of the virtual machine as well as information to associate the virtual machine with its user and resources. For example,

Tue Feb 22 20:17:22 2011 [DiM][I]: New VM state is ACTIVE (248, 10, loomis)
Tue Feb 22 20:17:22 2011 [VMM][I]: acquired lease (IP=134.158.75.32, MAC=0a:0a:86:9e:49:20)
Tue Feb 22 20:17:22 2011 [LCM][I]: New VM state is PROLOG.
...
Tue Feb 22 20:17:25 2011 [LCM][I]: New VM state is BOOT
...
Tue Feb 22 20:17:26 2011 [LCM][I]: New VM state is RUNNING
...
Tue Feb 22 20:18:56 2011 [DiM][I]: New VM state is DONE
Tue Feb 22 20:18:56 2011 [VMM][I]: released lease (IP=134.158.75.32, MAC=0a:0a:86:9e:49:20)
...

It also contains output from the various hook scripts that were executed during the machine's lifecycle. For example, the “clone” output:

Tue Feb 22 20:17:25 2011 [TM][I]: tm_mkswap.sh: Creating directory /var/lib/one/vms/248/images
Tue Feb 22 20:17:25 2011 [TM][I]: tm_mkswap.sh: Executed "mkdir -p /var/lib/one/vms/248/images".
Tue Feb 22 20:17:25 2011 [TM][I]: tm_mkswap.sh: Executed "chmod ug+w,o-rwx /var/lib/one/vms/248/images".
Tue Feb 22 20:17:25 2011 [TM][I]: tm_mkswap.sh: Creating 1024Mb image in /var/lib/one/vms/248/images/disk.1
Tue Feb 22 20:17:25 2011 [TM][I]: tm_mkswap.sh: Executed "/bin/dd if=/dev/zero of=/var/lib/one/vms/248/images/disk.1 bs=1 count=1 seek=1024M".
Tue Feb 22 20:17:25 2011 [TM][I]: tm_mkswap.sh: Initializing swap space
Tue Feb 22 20:17:25 2011 [TM][I]: tm_mkswap.sh: Executed "/sbin/mkswap /var/lib/one/vms/248/images/disk.1".
Tue Feb 22 20:17:25 2011 [TM][I]: tm_mkswap.sh: Executed "chmod ug+w,o-rwx /var/lib/one/vms/248/images/disk.1".

To allow detailed analysis of a virtual machine after it has completed running, all of the machine files are quarantined for a period of time defined by the administrator in /etc/stratuslab/quarantine.cfg. The default is to quarantine the files for 48 hours.

The quarantined files for each virtual machine can be found in the directory $VM_DIR/quarantine/<VM_ID>/images. The deployment descriptor is in the file deployment.0 with the associated disk images in files disk.<ID>.

A log file for the cleanup cron can be found in /var/log/one/one-quarantine-cleanup.log.

The OpenNebula daemon and associated plugins produce a variety of different log files. The following table summarizes them and their contents; all of the files are located in the /var/log/one directory.

The log file for the Authentication Proxy is available in /opt/stratuslab/one-proxy/logs. The logs are named by date and rotated daily. This file contains information about the authentication of users and can be used to understand problems associated with running login modules.

Successful authentication will always emit a message about forwarding a request like:

Feb 22, 2011 8:16:52 PM eu.stratuslab.authn.OneProxyServlet$ProxyHandler prepareRequestParameters
INFO: forwarding request from loomis:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa

Note that the “aaa…” above is a dummy password which is passed to OpenNebula and not used by the server.

This log is particularly useful for debugging problems with the use of certificates, VOMS proxies, and LDAP servers. Often this log will contain the information needed to resolve those issues.