HP MicroServer G7 RAC and Linux

How to flash a new firmware and how to set the network remotely without rebooting.

Download the new firmware (version 1.4 as writing) from the HP website

The flashing utility SOCFLASH is available on the Aspeed website (direct link). The utility is available also for Linux both 32 and 64 bit.

Flash the new RAC firmware

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
$ ./socflash.sh all.1.4.bin old.1.3.bin
ASPEED SOC Flash Utility v.1.09.04
Find ASPEED Device 1a03:2000 on 3:0.0
Relacate IO Base: e800
MMIO Virtual Address: 3e18000
Static Memory Controller Information:
CS0 Flash Type is SPI
CS1 Flash Type is NOR
CS2 Flash Type is SPI
Boot CS is 2
Option Information:
CS: 2
Flash Type: SPI
[Warning] Don\'t AC OFF or Reboot System During BMC Firmware Update!!
[SOCFLASH] Flash ID : 180101
Find Flash Chip #1: SpansionS25FL128 SE64KB
Backup Flash Chip O.K.
Check Flash Chip #1 at: 440000

Setup IPMI in the OS

1
2
$ modprobe ipmi_si type=kcs ports=0xca2
$ echo "ipmi_si type=kcs ports=0xca2" > /etc/modprobe.d/ipmi.conf

Change the network settings

1
2
3
4
5
6
7
8
9
10
$ ipmitool shell
ipmitool> lan set 1 ipsrc static
ipmitool> lan set 1 ipaddr 172.16.12.238
Setting LAN IP Address to 172.16.12.238
ipmitool> lan set 1 netmask 255.255.255.0
Setting LAN Subnet Mask to 255.255.255.0
ipmitool> lan set 1 defgw ipaddr 172.16.12.10
Setting LAN Default Gateway IP to 172.16.12.10
ipmitool> mc reset warm
Sent warm reset command to MC

HP MicroServer + RAC on Fedora 18

After upgrading my HP MicroServer with Remote Access Card from Fedora 17 to Fedora 18 (kernel-3.8.3-203.fc18) the RAC KVM console (remote and VGA) stopped to work with ‘out-of-range’ signal after:

[ 4.072580] [drm] Initialized drm 1.1.0 20060810
[ 4.104362] [drm] AST 1100 detected
[ 4.104500] [drm] dram 816000000 1 16 04000000
[ 4.104785] [TTM] Zone kernel: Available graphics memory: 2024944 kiB
[ 4.104913] [TTM] Initializing pool allocator
[ 4.105035] [TTM] Initializing DMA pool allocator
[ 10.522147] [sched_delayed] sched: RT throttling activated

The MicroServer RAC has an AST1100 video controller that currently doesn’t work very well with the Linux KMS (Kernel Mode Setting).

HP RAC KVM out of sync

 

So, to fix this issue you need to disable the KMS for the ast module:

  • Edit /etc/sysconfig/grub and at the end of the GRUB_CMDLINE_LINUX line add “ast.modeset=0 nomodeset
    GRUB_CMDLINE_LINUX="rd.md=0 rd.lvm=0 rd.dm=0 SYSFONT=latarcyrheb-sun16 rd.luks=0 KEYTABLE=it LANG=en_US.UTF-8 ast.modeset=0 nomodeset quiet"
  • Update the grub2 configuration
    grub2-mkconfig -o /boot/grub2/grub.cfg

OPTIONAL

  • Create /etc/modprobe.d/ast.conf with
    options ast modeset=0
  • Update the initramfs with dracut
    dracut --force

The bug has been reported at RedHat Bugzilla bug #926064.

References:
modinfo ast:

filename:       /lib/modules/3.8.3-203.fc18.x86_64/kernel/drivers/gpu/drm/ast/ast.ko
license:        GPL and additional rights
description:    AST
author:         Dave Airlie
alias:          pci:v00001A03d00002010sv*sd*bc03sc*i*
alias:          pci:v00001A03d00002000sv*sd*bc03sc*i*
depends:        drm,drm_kms_helper,ttm,i2c-core,i2c-algo-bit
intree:         Y
vermagic:       3.8.3-203.fc18.x86_64 SMP mod_unload
parm:           modeset:Disable/Enable modesetting (int)

SNMP hell (part 1): proxy to multiple devices

Last week I started working on the @GEMwrld HPC cluster at the Eidgenössische Technische Hochschule Zürich (ETH). The first task was to implement a kind of system monitoring and resource profiling. We already have a centralized graphical web console, built upon Observium, that collect resources statistics from all of our servers; usually we use SNMP v2c and, just for a few cases, the munin-node daemon (more on this will be in the part 2).

On the ETH cluster we have full access to every node, but the only incoming connections allowed by the ETH firewall are the SSH sessions (on port 22), so there’s a first problem: how can we transport SNMP data from the agent to the poller?

The answer is easy: SSH tunnel! But: usually SNMP uses UDP protocol; making an UDP SSH tunnel is a bit painful. To workaround this issue we had simply used the TCP protocol for SNMP.

First of all you need an SSH tunnel. In this case the tunnel is made by the monitoring server:

ssh -N monitoring@serverip -L 16000:localhost:161

The 161 is the remote TCP port to be forwarded and 16000 is the local port.

To use SNMP on TCP you have to modify the net-snmp daemon (snmpd) command line parameters (and not the snmpd config file); just edit /etc/default/snmpd (on ubuntu/debian, for RHEL the path is /etc/sysconfig/snmpd) as following:

SNMPDOPTS='-LF 6 /var/log/snmpd.log -u snmp -g snmp -I -smux -p /var/run/snmpd.pid TCP:161'

The important part is ‘TCP:161’. This will bind snmpd on every interface and on the TCP port 161 (the default SNMP port).
On the poller side you will need to configure your monitoring system to poll localhost:16000 with tcp protocol. For Observium you can add the host with:

./addhost.php localhost  community v2c 16000 tcp

Now there’s another problem. The Zurich GEM cluster is made by eight nodes and I do not want to start an SSH tunnel on every node. Then my idea was to use a kind of proxy on the control node:

eth_snmp

net-snmp helped me with the snmpd proxy capabilities: http://www.net-snmp.org/wiki/index.php/Snmpd_proxy

Continue reading

WordPress caching with nginx

As a part of my friend @zenkay post on his blog here it is a simple but efficient nginx configuration for proxying and caching an Apache + WordPress installation.

Assuming that:

  • nginx and Apache are on different nodes: if nginx and Apache are on the same machine is highly advised to serve static files directly from nginx;
  • static files expire header is managed by Apache (through mod_expires); if static stuff is served directly you need to specify the expire through nginx;
  • you have installed the WordPress Nginx proxy cache integrator;
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
server {
    # Listen IPv6 and IPv4 socket
    listen       [::]:80; #on Linux this means both IPv4 and IPv6
    # Name-based virtualhosts
    server_name  *.mydomain.com mydomain.com;
 
    # Add Cache-Status debug header on replies
    add_header X-Cache-Status $upstream_cache_status;
 
    # Set the vhost access-log
    access_log  /var/log/nginx/access-mydomain.log  main;
 
    location / {
 
        # Skip^1 caching variable init
        set $nocache 0;
        # Bypass^2 caching variable init
        set $purgecache 0;
 
        # Bypass^2 cache on no-cache (et al.) browser request
        if ($http_cache_control ~ "max-age=0")
            { set $purgecache 1; }
        if ($http_cache_control ~ "no-cache")
            { set $purgecache 1; }
        # Bypass^2 cache with custom header set on request
        if ($http_x_cache_purge ~* "true")
            { set $purgecache 1; }
        # Skip^1 caching when WordPress cookies are set
        if ($http_cookie ~* "comment_author_|wordpress_(?!test_cookie)|wp-postpass_" )
            { set $nocache 1; }
 
        # Cache pool
        proxy_cache             proxy-one;
        # Bypass^2 cache when $purgecache is set to 1.
        # Bypass means that content is served fresh and the cache is updated
        proxy_cache_bypass      $purgecache;
        # Skip^1 caching when $nocache is set to 1
        # Do not cache when browsing frontend as logged user
        proxy_no_cache          $nocache;
        # Define the cache resource identifier. Be careful to add $nocache
        proxy_cache_key         "$scheme$http_host$request_uri$args$nocache";
        proxy_connect_timeout   10;
        proxy_read_timeout      10;
        # use stale cache on backend fault
        proxy_cache_use_stale   error timeout invalid_header updating http_500 http_502 http_503 http_504;
        proxy_cache_valid       200 302 15m;
        proxy_cache_valid       404 1m;
        proxy_set_header        Host             $host;
        proxy_set_header        X-Real-IP        $remote_addr;
        proxy_set_header        X-Forwarded-For  $proxy_add_x_forwarded_for;
        proxy_pass              http://apache_remote_ip;
    }
 
    location ~* \/blog\/wp\-.*\.php|\/blog\/wp\-admin {
        proxy_cache             off;
        proxy_pass              http://apache_remote_ip;
    }
 
    error_page   500 502 503 504  /50x.html;
    location = /50x.html {
        root   /usr/share/nginx/html;
    }
 
}

1^ see: http://wiki.nginx.org/HttpProxyModule#proxy_no_cache
2^ see: http://wiki.nginx.org/HttpProxyModule#proxy_cache_bypass

CentOS 5 on KVM: reduce host CPU load

To reduce host CPU usage with a CentOS 5 VM on KVM is important to add

divider=10

to grub.conf as kernel parameter

kernel /vmlinuz-2.6.18-348.1.1.el5 ro root=LABEL=/ console=ttyS0,115200 divider=10

This will reduce the internal kernel timer from 1000 Hz to 100 Hz.

Although additional parameters are not required, the divider=10 parameter can still be used. Guests with this parameter will produce less CPU load in the host, but will use more coarse-grained timer expiration. (http://s19n.net/articles/2011/kvm_clock.html)

On MicroServer the CPU load reduce is quite visible:

MicroServer CPU usage

MicroServer CPU usage (made with http://www.observium.org/)

For more info read http://s19n.net/articles/2011/kvm_clock.html.

 

Installing ObsPy on Fedora 18

obspy-logo

ObsPy is an open-source project dedicated to provide a Python framework for processing seismological data. It provides parsers for common file formats, clients to access data centers and seismological signal processing routines which allow the manipulation of seismological time series (see Beyreuther et al. 2010, Megies et al. 2011).

The goal of the ObsPy project is to facilitate rapid application development for seismology.

https://github.com/obspy/obspy/wiki

ObsPy packages are available only for Debian/Ubuntu, however is possible to install it on Fedora via Python Package Index (PyPI). Not all deps are explained on ObsPy wiki pages so here is the complete procedure to install the software on Fedora 18:

First set up all the dependencies via yum

[root@sam ~]# yum install -y python-devel python-setuptools numpy scipy python-matplotlib python-matplotlib python-matplotlib gcc-gfortran python-suds python-sqlalchemy python-lxml

Now you can install ObsPy as explained in theirs wiki (https://github.com/obspy/obspy/wiki/Installation-on-Linux-via-PyPI)

[root@sam ~]# easy_install -U distribute; easy_install -N obspy

To test if the installation was successful use obspy-runtests

[daniele@sam ~]$ obspy-runtests
................................................................................................................................................................................................s.....................................................
[...]
----------------------------------------------------------------------
Ran 839 tests in 65.348s
 
OK

That’s all.

References:

 

 

Re-enable “Log out” in GNOME 3.6

user_menu

 

In GNOME 3.6, with a single user, the option “Log out” in user menu is no more available by default.

To re-enable it open a terminal and type:

gsettings set org.gnome.shell always-show-log-out true

Now restart the Shell by pressing Alt+F2 and typing r<return>

References:

https://wiki.archlinux.org/index.php/GNOME#Always_show_the_.22Log_Out.22_entry_in_the_user_menu