Tuesday, July 5, 2011

Validate This!

So, for my first work related blog, I thought I would describe the insane world I am creating for myself.  I am a QA tester working on the Ubuntu armel platform.

I have one specific talent for this:  I find the bugs that make engineers bang their heads on the wall in frustration.  Some of my early work (early 90's)  is still listed in Microsoft DevNet (Access date formatting bug, database corruption issue, etc).

My office is (was) an unused bedroom in my house.  As part of my remodeling project, I have wired the entire house with Gigabit Ethernet and Wifi 802.11N (for when I am either outside enjoying the Oregon weather or in the bathroom running things remotely).  I have 4 Intel systems and 15 Arm systems currently online in my office, plus a rack cabinet in the basement housing my main server (Apache, MySQL, Postgresql, Quassel Core, Mediatomb, Jenkins, and a mirror of the Ubuntu binaries updated hourly), my firewall, and two other arm systems that I keep alive for SRU update testing.  Other systems are also running in the house, but I don't care about them as much (wife's Windows box for instance).

The last few years have been revolving around testing the desktop/netbook images for Ubuntu on Arm platforms (FreeScale iMX51, Marvell, Ti Omap3/4).


Most of this testing involves making sure the apps just work.  Not a lot of them were developed with ArmV7 in mind, let alone SMP on Arm (this opens up a whole new barrel of monkeys).  For example, one fine piece of code had this comment in their atomic memory handler for smp:
/* SWP on ARM is very similar to XCHG on x86.  Doesn't lock the
* bus because there are no SMP ARM machines. */
Erm, yea.  Sure. 
This cycle, we are starting to do server stacks on Arm (queue maniacal laughter).  To add to the "fun", the current hardware I have available is essentially equivalent to a cell phone (Droid cluster computing anyone?).  With the greatly expanded number of systems now cluttering my newly dedicated test table, I "can" test a large plethora of server stacks.

I say that figuratively, as most of the test automation tools I need to implement revolve around x86/amd64 (virtualization, CD ISO files, etc).  Not easy when the boot methods revolve around an SD card. Access to these test systems will be primarily serial console, with ssh only after install.

Don't get me wrong, I am in no way saying this hardware is bad.  Just that this particular system (for which I have an odd abundance of - 5 online and 2 on order) is really designed for cell phone and tablet/smart book use.  There is no SATA, 100mb Ethernet on the USB bus only, etc.  What this system excels at is its ability to control 2 1080p monitors with HD video playback.  It does have 1G memory, and built in wifi.  It will make a great nettop unit and I recently heard that it is the first platform to get HD certification from Netflix on Android.

But running a server stack can be a lot different.  Sure, it should be able to handle dhcp and dns loads without yawning, but when you get into NFS, SQL, or LAMP stacks, things may get a little more dicey.  At least in theory.

My main goals for this release (Oneiric Ocelot) are to ensure that the software we want to run can actually run on Arm SMP (with none of the code issues listed above).  Don't expect benchmarks from me, other than as a cursory "We ran it and it worked".  I'm not interested in making it fail any faster than it may already at this stage.  My goal is to make sure it runs.  I will be running benchmark suites, mainly as they are already written and are good at stress testing the entire stack.  Any benchmarks I generate will only be useful in ensuring that any changes between now (Alpha 2) and release in October don't get worse as features are added & fine tuned.

Stick with me on my journey into madness, as I add more systems to my rack cabinet in the basement to be used for various tests (iSCSI host, web client traffic generators, NFS root server, etc).  I may even throw a Windows Server system down there for Active Directory testing.

So, with that in mind, queue the maniacal laughter, get your freak on and join me on my journey into madness.