Difference between revisions of "Snark"

From Netsoc Wiki
Jump to: navigation, search
Line 10: Line 10:
 
==Description==
 
==Description==
  
snark is a much-better-than-spoon-but-not-quite-cube 2U monster that was bought on adverts.ie in April 2012. Anything relating to it -  including what the hell we plan to do with it - should be maintained here.
+
snark is a much-better-than-spoon-but-not-quite-cube 2U monster that was bought on adverts.ie in April 2012.
 +
==Current role==
  
==Current role==
+
* Primary services server
 +
* Netsoc webserver
 +
* User webserver
 +
* Primary LDAP server
 +
* Authoritative DNS server
 +
* Secondary mail server
 +
* IRC server
 +
* IRC services
  
Snark is serving as a system on which the Xen virtualisation system is running. It is running Debian wheezy with kernel version 3.2.0-2-amd64 and xen version 4.1.3-rc2-pre. Currently only admins have access to both the dom0 (snarkroot) and the domUs (the various VMs that are running on it - see below for a list). It is connected to the College network via it's eth0 interface, which is running as a bridge for Xen to allow the domUs communicate. The configuration of the interface is given below:
+
Snark runs most of our essential services, partitioned into VMs under the Xen hypervisor. It is running Debian wheezy with kernel version 3.2.0-2-amd64 and xen version 4.1.3-rc2-pre. Currently only admins have access to both the dom0 (snarkroot) and the domUs (the various VMs that are running on it - see below for a list). It is connected to the College network via it's eth0 interface, which is running as a bridge for Xen to allow the domUs communicate. The configuration of the interface is given below:
 
<pre>
 
<pre>
 
auto xenbr0
 
auto xenbr0
Line 30: Line 38:
  
 
==Planned roles==
 
==Planned roles==
 
===Primary services server===
 
 
We're going to move the following off [[Cube]], and onto VMs on snark:
 
 
* Netsoc website/database
 
* Primary LDAP
 
* Primary DNS
 
* Secondary mail
 
* IRC
 
* IRC services
 
 
By doing this we will be able to greatly reduce the load on cube, and leave it as a big massive dev box for people to run all their crazy stuff on. It'll also make it easier to rebuild cube on Xen, and end the OpenVZ pain forever.
 
  
 
===Backup login server===
 
===Backup login server===

Revision as of 06:24, 7 January 2014

Specification

  • Dell Poweredge 2950
  • Dual quad-core 2.0GHz L5335 Low voltage Xeon processors with 8MB L2 Cache
  • 16GB DDR2F RAM
  • 2×146GB + 4×300GB 15,000 RPM SAS hard disks
  • Dual gigabit ethernet
  • Dual, redundant, hot-swappable power supplies

Description

snark is a much-better-than-spoon-but-not-quite-cube 2U monster that was bought on adverts.ie in April 2012.

Current role

  • Primary services server
  • Netsoc webserver
  • User webserver
  • Primary LDAP server
  • Authoritative DNS server
  • Secondary mail server
  • IRC server
  • IRC services

Snark runs most of our essential services, partitioned into VMs under the Xen hypervisor. It is running Debian wheezy with kernel version 3.2.0-2-amd64 and xen version 4.1.3-rc2-pre. Currently only admins have access to both the dom0 (snarkroot) and the domUs (the various VMs that are running on it - see below for a list). It is connected to the College network via it's eth0 interface, which is running as a bridge for Xen to allow the domUs communicate. The configuration of the interface is given below:

auto xenbr0
iface xenbr0 inet static
        bridge_ports eth0
        address 134.226.83.58
        gateway 134.226.83.1
        broadcast 134.226.255.255
        netmask 255.255.255.0
        dns-nameservers 134.226.83.27
        dns-search netsoc.tcd.ie

There are two domUs running on snark at the moment: Freedom and snark. Freedom is a freeBSD 9.0 HVM whilst snark is a paravirtualised domU running Debian Wheezy. snark's IP is 134.226.83.59. both snark and freedom are running SSH servers, but neither are interfaced with the LDAP system and so only local accounts will work (and only root accounts exist at this time).

Planned roles

Backup login server

We'll also include a small login VM called 'snark', so that people have another place to log into in the case of downtime. This will have roughly Spoon-level resources allocated to it, and a modestly-sized /home. We won't be installing any dev software (fancy compilers, obscure libs, etc.) here, to discourage people from ruining it with coursework.

Backup server

An important service which will run on snark is backups. We now have the right card and cable to drive the PowerVault disk array, and we have enough disks now. We'll back up everything on Spoon, Cube and snark to it, including all member data. Because the machine is hosted in TCHPC, this provides us with redundancy and means that if some day either the Maths department or TCHPC burns down, member data will be safe.


Name

snark is a weapon in the original Half Life game by "Valve":http://www.valvesoftware.com/. It is from the border world, Xen, which is also the name of the virtualisation software on snark.

Storage

The storage on snark is set up as a RAID1 of the 2x146GB drives whilst the 4x300GB are in a RAID5, both controlled by the onboard PERC 6/i card. LVM is set up on top of the RAID5 completely in a physical volume and volume group both called onboard-raid5 (/dev/sdb1). The 146GB RAID1 is split to a 200MB ext2 /boot partition, a 50GB ext4 /, a 24GB swap, and a 72GB LVM with its physical volume called onboard-raid1. The onboard-raid1 volume contains / for the Dom0, and space for miscellaneous storage.The onboard-raid5 volume contains the logical volumes for the various VMs the machine runs.

Snark goes hand-in-hand with the Snark disk array, which is connected via an infiniband cable to the PERC 6/E card. It contains four 1TB SATA disks, which are pooled as a RAID5, providing 3TB of storage. The disk array's storage appears in a volume group called diskarray-raid5.

Networking

Snark uses transparent (bridged, or level 2) firewalling for the VMs.

Networking in Xen

Xen has two ways of setting up networking for the VMs: routed, or bridged. In snark, we have used the bridged networking option. By default this means that all VMs and the Dom0 bind to a bridge called xenbr0. Inside each of the VMs, the interfaces are always called the usual eth0. In the Dom0, each of the interfaces that connect the bridge to the VMs is called vifX.X. For example, vif0.0 is the interface that connects the Dom0 to the bridge, but the Dom0 doesn't see it as vif0.0, but as eth0. This is because Xen masks the real name for the more conventional names. vif1.0 and vif1.1 may be two interfaces given to a VM.

Networking in Snark

However we have changed how this works, and all VMs should not use xenbr0, but instead a bridge called xenbrdmz. xenbrdmz is passed to the firewall VM snark-fw along with xenbr0. This VM does the transparent firewalling between the two bridges, which it sees as two normal ethernet interfaces (eth0 for xenbr0 and eth1 for xenbrdmz)

Networking in snark

Shorewall is used to make the firewall rules. Shorewall has had support for bridge firewalls for a while, but has recently gotten good at it. There are a number of documents on the Shorewall website about bridge firewalls, as well as using Xen with Shorewall (Mind: some of the stuffrelating to Xen is out of date, and the 3.0 documentation isn't necessarily compatible with Shorewall 4.5). There are two firewalls needed: one on the Dom0, snarkroot which does its own firewalling, as well as passing everything through the bridge to be filtered by snark-fw.

TODO: Talk about shorewall rules, policies, and the bport zones.

Virtualization

Xen is used as the hypervisor. The processors in snark support VT-x. Unlike on Cube, each VM is able to have its own kernel (as opposed to sharing a single kernel with the hardware node).

Adding new vms

It's easy to make a new VM, allocate its disk space, and install debian on it. First, you need to make a partition definition file in /etc/xen-tools/partitions.d. These files are used to tell Xen how to lay out the partitions on the machine you're creating. For example, /etc/xen-tools/partitions.d/snark is as follows:

[root]
size=2G
type=ext3
mountpoint=/
options=sync,errors=remount-ro

[swap]
size=2G
type=swap

[home]
size=200G
type=ext3
mountpoint=/home
options=nodev,nosuid

[opt]
size=2G
type=ext3
mountpoint=/opt
options=nodev

[tmp]
size=1G
type=ext3
mountpoint=/tmp
options=nodev,nosuid

[usr]
size=4G
type=ext3
mountpoint=/usr
options=nodev

[var]
size=4G
type=ext3
mountpoint=/var
options=nodev,nosuid

[var-tmp]
size=1G
type=ext3
mountpoint=/var/tmp
options=nodev,nosuid

To create this machine, you would run

xen-create-image --hostname NAMEHERE --partitions=/etc/xen-tools/partitions.d/snark --mem=4Gb --swap=1Gb --ip=134.226.83.foo --gateway=134.226.83.1 --netmask=255.255.0.0 --dist=wheezy

then xm create /etc/xen/NAMEHERE.cfg, then xm console NAMEHERE to get a shell, but an ssh server should come up by default.

REMEMBER TO UPDATE ips

snarkroot

In the spirit of Cube, the Dom0 is called snarkroot. Nothing will be set up on this machine except Xen, shorewall, and things which require access to hardware, like RAID and LVM and so on. It runs Debian stable

snark

Again in the spirit of Cube, the main machine that users log into will be called snark. It runs testing.

snark-irc-services

Exactly what it says on the tin.

Running squeeze, because wheezy has no hybserv package. No longer using hybserv, will upgrade.



snark-ns

This machine is relatively small, and runs an authoritative DNS server based on bind. For security, bind is chrooted, and a minimal installation of debian is used.

snark-www

This machine will eventually replace cubewww on cube. We're debating which of lighttpd and nginx to run on it.

snark-backups

This machine will host whatever backup software we decide to run (fwbackups, bacula, amanda...), and coordinate backups of all machines. It will only be accessible by admins (for now), and will run a stable release. If the disk array is to be used for backups only, we'll make this machine the only one with access to it.

Others

A few somewhat sillier VMs also exist on snark. They're not powered on all the time though.

b3ndial

I can't believe I'm writing this, but this is an instance of Windows Server 2012, set up by DUCSS admin k3ypad.

Freedom

A FreeBSD VM, for people to play with.


Other proposed VMs are;

  • Morpheus - Matrix clone with netris.real and amazing (bad?) tcsh prompts. OpenIndiana, most likely, as Solaris itself is verging on really-quite-non-free.
  • Shipwreck - A proposed VM for Titanic that ran legacy operating systems inside emulators (preferably ones that don't eat CPUs)