Difference between revisions of "Snark"

From Netsoc Wiki
Jump to: navigation, search
(Adding new vms)
(Updates)
Line 28: Line 28:
  
 
There are two domUs running on snark at the moment: [[Freedom]] and snark. Freedom is a freeBSD 9.0 HVM whilst snark is a paravirtualised domU running Debian Wheezy. snark's IP is 134.226.83.59. both snark and freedom are running SSH servers, but neither are interfaced with the [[LDAP]] system and so only local accounts will work (and only root accounts exist at this time).
 
There are two domUs running on snark at the moment: [[Freedom]] and snark. Freedom is a freeBSD 9.0 HVM whilst snark is a paravirtualised domU running Debian Wheezy. snark's IP is 134.226.83.59. both snark and freedom are running SSH servers, but neither are interfaced with the [[LDAP]] system and so only local accounts will work (and only root accounts exist at this time).
 
The storage on snark is set up as a RAID1 of the 2x146GB drives whilst the 4x300GB are in a RAID5, both controlled by the PERC 6/i card. LVM is set up on top of the RAID5 completely in a physical volume and volume group both called raid5-home (/dev/sdb1). The 146GB RAID1 is split to a 200MB ext2 /boot partition, a 50GB ext4 /, a 24GB swap, and a 72GB LVM with its physical volume called raid1-vms. It houses the logical volumes used for the various VMs (which are typically 15-20GB per VM).
 
  
 
==Planned roles==
 
==Planned roles==
  
===Primary development server===
+
===Primary services server===
 
 
The goal will be to minimize the number of essential services running on snark. If we can do this, we can keep it as a development server. That way, people can run all their crazy projects on it, instead of breaking stuff on cube when memory usage gets too high. In the spirit of it being a development server, we'll install a more unstable distro, so that people can get access to the latest versions of packages as quickly as possible. Current suggestions for this include
 
 
 
* 6-month-cycle Ubuntu
 
* Debian unstable (currently wheezy)
 
  
the SAS disks inside snark run at 15,000 RPM, giving very fast local storage for people to work on projects with.
+
We're going to move the following off [[Cube]], and onto VMs on snark:
  
Another happy consequence of not running essential services is that we can set up a virtualization environment, and fill it with lots of interesting machines for people to log into. Having learned lessons from [[Cube]], we won't be using OpenVZ for this purpose. Instead, we'll install Xen, which gives better control over resource limits. Also, unlike OpenVZ, virtualization in Xen isn't done on the OS-level, so we will be able to install all kinds of other operating systems for people to log into an experiment with. Current suggestions include:
+
* Netsoc website/database
 +
* Primary LDAP
 +
* Primary DNS
 +
* Secondary mail
 +
* IRC
 +
* IRC services
  
* One or more BSDs (OpenBSD, FreeBSD, NetBSD)
+
By doing this we will be able to greatly reduce the load on cube, and leave it as a big massive dev box for people to run all their crazy stuff on. It'll also make it easier to rebuild cube on Xen, and end the OpenVZ pain forever.
* SunOS/Solaris/OpenSolaris (including a possible clone of [[matrix]] from the backups in /srv/public on cube)
 
* plan9
 
  
===Secondary login server===
+
===Backup login server===
  
Hopefully people won't go overboard with resources too much, and others will still be able to use snark for day-to-day work. We'll be installing the usual login software like screen and tmux, but maybe not NX, since it combined with GNOME can be a bit of a hog.
+
We'll also include a small login VM called 'snark', so that people have another place to log into in the case of downtime. This will have roughly [[Spoon]]-level resources allocated to it, and a modestly-sized /home. We won't be installing any dev software (fancy compilers, obscure libs, etc.) here, to discourage people from ruining it with coursework.  
  
 
===Backup server===
 
===Backup server===
  
The only 'essential' (but not as essential) service which will run on snark is backups. snark comes with the kind of card we need to drive the PowerVault disk array, so when we've finally got it filled with disks we'll back up everything on [[Spoon]], [[Cube]] and snark to it, including all member data. We're going to try to get it hosted somewhere other than in the maths department, meaning that member data will be perfectly safe in the event that either server room burns down.
+
An important service which will run on snark is backups. We now have the right card and cable to drive the PowerVault disk array, and we have enough disks now. We'll back up everything on [[Spoon]], [[Cube]] and snark to it, including all member data. Because the machine is hosted in TCHPC, this provides us with redundancy and means that if some day either the Maths department or TCHPC burns down, member data will be safe.
 
 
===Package mirror===
 
  
We don't plan to offer massive disk quotas on snark (if people want big storage, they can use cube for that purpose). This will give us a lot of space on very fast disks that we can do cool things with. We hope to run one or more package mirrors on snark for various OSes, including Debian, Arch, and maybe Macports for OS X users.
 
  
 
==Name==
 
==Name==
 
A few names have been suggested.
 
 
===snark===
 
  
 
snark is a weapon in the original Half Life game by "Valve":http://www.valvesoftware.com/.
 
snark is a weapon in the original Half Life game by "Valve":http://www.valvesoftware.com/.
 
===Freedom===
 
 
This name refers to the fact that if we don't run essential services we are free to run whatever crazy stuff we want, as well as the fact that Xen will let us run non-free operating systems like BSD. It's also a reference to http://freedom.redbrick.dcu.ie/, the amazing website made by "bunbun":http://www.redbrick.dcu.ie/~bunbun/
 
  
 
==Storage==
 
==Storage==
  
Fast local storage will be provided via the 6 (six!) internal hard drive bays. /, /var, /tmp and such will go on the two 146GB disks, to be combined in RAID1. The four 300GB disks will be put in RAID5, providing 900GB of storage.
+
The storage on snark is set up as a RAID1 of the 2x146GB drives whilst the 4x300GB are in a RAID5, both controlled by the onboard PERC 6/i card. LVM is set up on top of the RAID5 completely in a physical volume and volume group both called onboard-raid5 (/dev/sdb1). The 146GB RAID1 is split to a 200MB ext2 /boot partition, a 50GB ext4 /, a 24GB swap, and a 72GB LVM with its physical volume called onboard-raid1. The onboard-raid1 volume contains / for the Dom0, and space for miscellaneous storage.The onboard-raid5 volume contains the logical volumes for the various VMs the machine runs.
  
In theory, not-as-fast but abundant storage (see [[Seth]]) for backups will be provided via the SAS card and the disk array. In practice, we don't have any disks for the array.
+
The disk array is connected via an infiniband cable to the PERC 6/E card. It contains four 1TB SATA disks, which are pooled as a RAID5, providing 3TB of storage. The disk array's storage is in a volume group called diskarray-raid5.
  
 
==Virtualization==
 
==Virtualization==
  
Xen will be used as the hypervisor. The processors in snark support VT-x. Unlike on [[Cube]], each VM will be able to have its own kernel (as opposed to sharing a single kernel with the hardware node).  
+
Xen is used as the hypervisor. The processors in snark support VT-x. Unlike on [[Cube]], each VM is able to have its own kernel (as opposed to sharing a single kernel with the hardware node).  
  
 
===Adding new vms===
 
===Adding new vms===
Line 91: Line 76:
 
====snarkroot====
 
====snarkroot====
  
In the spirit of [[Cube]], the hardware node will be called snarkroot (or <name>root, depending on what we decide to call it). Nothing will be set up on this machine except Xen, shorewall, and things which require access to hardware, like RAID and LVM and so on. It will probably run Debian stable, or $distro stable, where $distro has the best Xen support in its stable release.
+
In the spirit of [[Cube]], the Dom0 is called snarkroot. Nothing will be set up on this machine except Xen, shorewall, and things which require access to hardware, like RAID and LVM and so on. It runs Debian stable
  
 
====snark====
 
====snark====
  
Again in the spirit of [[Cube]], the main machine that users log into will be called [[snark]]. This will run unstable, as discussed above, and will be allocated the majority of snarkroot's resources. If ISS only gives us firewall control of one VM, this will be it.
+
Again in the spirit of [[Cube]], the main machine that users log into will be called [[snark]]. It runs testing.
  
 
====snark-irc-services====
 
====snark-irc-services====
Line 104: Line 89:
 
<br/><br/>
 
<br/><br/>
  
===snarkbackups===
+
====snark-ns====
 +
 
 +
This machine is relatively small, and is currently in the process of being set up as a DNS server with bind.
 +
 
 +
====snark-www====
 +
 
 +
This machine will eventually replace [[cubewww]] on [[cube]]. We're debating which of lighttpd and nginx to run on it.
 +
 
 +
====snark-backups====
  
 
This machine will host whatever backup software we decide to run (fwbackups, bacula, amanda...), and coordinate backups of all machines. It will only be accessible by admins (for now), and will run a stable release. If the disk array is to be used for backups only, we'll make this machine the only one with access to it.
 
This machine will host whatever backup software we decide to run (fwbackups, bacula, amanda...), and coordinate backups of all machines. It will only be accessible by admins (for now), and will run a stable release. If the disk array is to be used for backups only, we'll make this machine the only one with access to it.
  
===snarkmirror===
+
===Others===
  
If we decide to set up package mirrors, they'll sit here.  
+
A few somewhat sillier VMs also exist on snark. They're not powered on all the time though.
  
===Others===
+
====b3ndial====
 +
 
 +
I can't believe I'm writing this, but this is an instance of Windows Server 2012, set up by DUCSS admin [[k3ypad]].
  
Stream of consciousness time:
+
====Freedom====
  
<code>$ cat /dev/brain/netsoc/snark/VMs</code>
+
A FreeBSD VM, for people to play with.
  
The machines running the various other operating systems will be given hilarious names, based on puns on the operating systems they run. It has already been decided that the FreeBSD machine will be called rms. We'll probably have to firewall off most of these from the College network. Some hax with loopback filesystems might be possible, allowing users to have a single home directory on snark, accessible from all the different VMs.
 
  
 
Other proposed VMs are;
 
Other proposed VMs are;
 
* Morpheus - Matrix clone with netris.real and amazing (bad?) tcsh prompts. OpenIndiana, most likely, as Solaris itself is verging on really-quite-non-free.
 
* Morpheus - Matrix clone with netris.real and amazing (bad?) tcsh prompts. OpenIndiana, most likely, as Solaris itself is verging on really-quite-non-free.
* b3ndial - Windows Small Business server VM because lol. (Actually, if we do have a Windows server, it could be good for some games servers since the Source servers for Linux suck a bit)
 
 
* Shipwreck - A proposed VM for Titanic that ran legacy operating systems inside emulators (preferably ones that don't eat CPUs)
 
* Shipwreck - A proposed VM for Titanic that ran legacy operating systems inside emulators (preferably ones that don't eat CPUs)

Revision as of 13:02, 24 January 2013

Specification

  • Dell Poweredge 2950
  • Dual quad-core 2.0GHz L5335 Low voltage Xeon processors with 8MB L2 Cache
  • 16GB DDR2F RAM
  • 2×146GB + 4×300GB 15,000 RPM SAS hard disks
  • Dual gigabit ethernet
  • Dual, redundant, hot-swappable power supplies

Description

snark is a much-better-than-spoon-but-not-quite-cube 2U monster that was bought on adverts.ie in April 2012. Anything relating to it - including what the hell we plan to do with it - should be maintained here.

Current role

Snark is serving as a system on which the Xen virtualisation system is running. It is running Debian wheezy with kernel version 3.2.0-2-amd64 and xen version 4.1.3-rc2-pre. Currently only admins have access to both the dom0 (snarkroot) and the domUs (the various VMs that are running on it - see below for a list). It is connected to the College network via it's eth0 interface, which is running as a bridge for Xen to allow the domUs communicate. The configuration of the interface is given below:

auto xenbr0
iface xenbr0 inet static
        bridge_ports eth0
        address 134.226.83.58
        gateway 134.226.83.1
        broadcast 134.226.255.255
        netmask 255.255.255.0
        dns-nameservers 134.226.83.27
        dns-search netsoc.tcd.ie

There are two domUs running on snark at the moment: Freedom and snark. Freedom is a freeBSD 9.0 HVM whilst snark is a paravirtualised domU running Debian Wheezy. snark's IP is 134.226.83.59. both snark and freedom are running SSH servers, but neither are interfaced with the LDAP system and so only local accounts will work (and only root accounts exist at this time).

Planned roles

Primary services server

We're going to move the following off Cube, and onto VMs on snark:

  • Netsoc website/database
  • Primary LDAP
  • Primary DNS
  • Secondary mail
  • IRC
  • IRC services

By doing this we will be able to greatly reduce the load on cube, and leave it as a big massive dev box for people to run all their crazy stuff on. It'll also make it easier to rebuild cube on Xen, and end the OpenVZ pain forever.

Backup login server

We'll also include a small login VM called 'snark', so that people have another place to log into in the case of downtime. This will have roughly Spoon-level resources allocated to it, and a modestly-sized /home. We won't be installing any dev software (fancy compilers, obscure libs, etc.) here, to discourage people from ruining it with coursework.

Backup server

An important service which will run on snark is backups. We now have the right card and cable to drive the PowerVault disk array, and we have enough disks now. We'll back up everything on Spoon, Cube and snark to it, including all member data. Because the machine is hosted in TCHPC, this provides us with redundancy and means that if some day either the Maths department or TCHPC burns down, member data will be safe.


Name

snark is a weapon in the original Half Life game by "Valve":http://www.valvesoftware.com/.

Storage

The storage on snark is set up as a RAID1 of the 2x146GB drives whilst the 4x300GB are in a RAID5, both controlled by the onboard PERC 6/i card. LVM is set up on top of the RAID5 completely in a physical volume and volume group both called onboard-raid5 (/dev/sdb1). The 146GB RAID1 is split to a 200MB ext2 /boot partition, a 50GB ext4 /, a 24GB swap, and a 72GB LVM with its physical volume called onboard-raid1. The onboard-raid1 volume contains / for the Dom0, and space for miscellaneous storage.The onboard-raid5 volume contains the logical volumes for the various VMs the machine runs.

The disk array is connected via an infiniband cable to the PERC 6/E card. It contains four 1TB SATA disks, which are pooled as a RAID5, providing 3TB of storage. The disk array's storage is in a volume group called diskarray-raid5.

Virtualization

Xen is used as the hypervisor. The processors in snark support VT-x. Unlike on Cube, each VM is able to have its own kernel (as opposed to sharing a single kernel with the hardware node).

Adding new vms

xen-create-image --hostname NAMEHERE --size=20Gb --mem=4Gb --swap=1Gb --ip=134.226.83.NEWHERE --gateway=134.226.83.1 --netmask=255.255.0.0 --dist=wheezy
then xm create /etc/xen/NAMEHERE.cfg, then xm console NAMEHERE to get a shell, but an ssh server should come up by default.

REMEMBER TO UPDATE ips

Existing

snarkroot

In the spirit of Cube, the Dom0 is called snarkroot. Nothing will be set up on this machine except Xen, shorewall, and things which require access to hardware, like RAID and LVM and so on. It runs Debian stable

snark

Again in the spirit of Cube, the main machine that users log into will be called snark. It runs testing.

snark-irc-services

Exactly what it says on the tin.
Running squeeze, because wheezy has no hybserv package.



snark-ns

This machine is relatively small, and is currently in the process of being set up as a DNS server with bind.

snark-www

This machine will eventually replace cubewww on cube. We're debating which of lighttpd and nginx to run on it.

snark-backups

This machine will host whatever backup software we decide to run (fwbackups, bacula, amanda...), and coordinate backups of all machines. It will only be accessible by admins (for now), and will run a stable release. If the disk array is to be used for backups only, we'll make this machine the only one with access to it.

Others

A few somewhat sillier VMs also exist on snark. They're not powered on all the time though.

b3ndial

I can't believe I'm writing this, but this is an instance of Windows Server 2012, set up by DUCSS admin k3ypad.

Freedom

A FreeBSD VM, for people to play with.


Other proposed VMs are;

  • Morpheus - Matrix clone with netris.real and amazing (bad?) tcsh prompts. OpenIndiana, most likely, as Solaris itself is verging on really-quite-non-free.
  • Shipwreck - A proposed VM for Titanic that ran legacy operating systems inside emulators (preferably ones that don't eat CPUs)