Various Ramblings and half-truths.

Linux Permissions - Time To Start Again....

2012-02-03T02:17:00.000+11:00

Lets take a trip back to my beginnings in unix land, which is around SunOS 3.x (i think, thats around 1992 so im not sure if thats correct or not anymore).

Permissions in unix land back then were simple:

root could do anything
users had a restricted set of permissions
user<->user interaction was based on a very simple permission setup

What i mean is simply that users could not (for example) bind ports below 1024. They couldnt mount/unmount file systems, they couldnt snoop network traffic - etc etc etc.

This has changed quite alot over the years, acls were added (unix and later linux), xattr's were added, selinux and more recently setcap (the thing that actually spawns this blog post). The problem with all these extensions is there implementation. Consider this:

testuser@boson:~/mystuff$ ls -al

total 9776

drwxr-xr-x 2 testuser testuser 4096 2012-02-03 00:31 .

drwxr-xr-x 3 testuser testuser 4096 2012-02-03 00:29 ..

-rwsr-xr-x 1 root root 917888 2012-02-03 00:31 bash

-rw------- 1 testuser testuser 1048576 2012-02-03 00:32 important_document.doc

-rwxr-xr-x 1 testuser testuser 8029400 2012-02-03 00:31 php

testuser@boson:~/mystuff$

Quite a simple directory listing in reality. Most unix users will know what all this means and possibly poop their pants at that "bash" binary.

For the untrained, that bash binary can run by any user and will switch to the root user upon exectution with no authentication:

testuser@boson:~/mystuff$ ./bash -p

bash-4.1# id

uid=1001(testuser) gid=1002(testuser) euid=0(root) groups=0(root),1002(testuser)

bash-4.1#

Scary eh? However, to most people this is sort-of nice (in terms of layout), its obvious whats going on in that directory cause the "ls" program has always made it easy to see the basic set of permissions a file (or set of files) has. Nautilus (the gnome file manager) wont (by default) should you the set-uid-ness of that bash file, but thats less of an issue.

This has never changed, while the upstream linux permission set has. Lets say I want to use ACL's (and by the way, alot of linux systems still build with a default of acl's off - good or bad, you decide). But if i turn it all on, and lets say i (testuser) want to allow a user named "rancid" to access my important document. Without ACL's this is tricky cause if we didnt have a group common to both of us (and only us) theres no real way unless we both keep copies of the file (annoying). Enter acl's. ACL's allow you to do this:

testuser@boson:~/mystuff$ setfacl -m user:rancid:rwx important_document.doc

testuser@boson:~/mystuff$ ls -la

total 9776

drwxr-xr-x 2 testuser testuser 4096 2012-02-03 00:31 .

drwxr-xr-x 3 testuser testuser 4096 2012-02-03 00:33 ..

-rwsr-xr-x 1 root root 917888 2012-02-03 00:31 bash

-rw-rwx---+ 1 testuser testuser 1048576 2012-02-03 00:32 important_document.doc

-rwxr-xr-x 1 testuser testuser 8029400 2012-02-03 00:31 php

Nice, but WTF? Why did the group permissions change like that? its an internal acl utility thing (and IMHO, quite idiotic). Now, in LS I can see a little + next to the permissions indicating this file has some extended set of permissions (and it wasn't always this way). You wont see jack (by default) anywhere other then LS and getfacl (yes, programs like nautilus wont give you any indication of extented acls, nor a way to manage them - by default) which is a shame because its a useful function of linux. Sadly, there are people out there I know who consider themselves unix admins that dont even know what that little "+" means - don't scoff, it happens too easily.

The + is sadly easy to miss if your an admin, and ultimately is kinda annoying cause it doesnt actually tell you much useful and when confronted with a directory of these things you just sigh and wish you were dead.

But thats where the problem is, ACL's were simply a hack tacked onto the base unix/linux permissions set with very little regard to anything else. To get the actual permission of that file, you do this:

testuser@boson:~/mystuff$ getfacl important_document.doc

# file: important_document.doc

# owner: testuser

# group: testuser

user::rw-

user:rancid:rwx

group::---

mask::rwx

other::---

And so you see the actual extended permission set. I like ACL's, its something unix needed, but it needs a better implementation. ACL's also dont extend far enough - can you acl up a permission that allows users to mount file systems? well yes and no, its not a direct function of acl's, but you could make a setuid mount binary that can do just that - so long as mount "user" permissions are set in fstab.

God this gets messy. When you sit down and look at how poorly permissions get dolled out in linux and by whom you start to wonder who's in control of what mess here. Take mount for example, if its a setuid binary (run as root) and the fstab says "users" can mount the file system, then users can then mount/umount any "user" labelled fs in /etc/fstab. It has its uses, but the whole process is really controlled exclusively by the mount binary itself. It doesn't pass any control/auth/etc back to where it should go (i.e. the kernel really should be making the decisions). Often the kernel doesn't really take many active roles in the permissions users end up with. Setuid binaries mean the kernel really only sees a root-spawned activity and not the original user who ran the program, meaning your dependant on the security provided by the setuid binaries that may exist on your system.

Sadly, there are way too many examples of that exact type of functionality within linux - some are justifiable (such as changing passwords, it would be hard without setuid, and the kernel doesnt actually have any direct control over user/group creation or even authentication). The linux kernel itself doesn't even really care much about users themselves, all it knows is that the root user (the user with the uid 0) has a "can do anything" flag and everything else (without the uid 0) cant. The kernel itself doesn't really care what uid 1024 is doing as long as it has the permissions to do it (i.e. the kernel will decide if uid 1024 can create a file in /home/something/someplace based on the permissions in the file system), but it wont care how a process owned by uid 1024 came into being in the first place. Alot of people miss that distinction. The kernel itself has not the vaguest idea as to whether a user id is actually a valid user or not. That happens entirely in the user-space binaries that your OS provides. The kernel will happily spawn a process with any user ID regardless of whether your OS knows of existence of that user id.

I do have a point to make here, but bear with me. Consider SElinux as an example. SELinux is another permission hack-job on top of the linux kernel. It gives you some amazing control over what can happen on your system. For eg, you could say people in a group of "x" are not allowed to do very specific kernel level operations. Again, keep in mind the kernel doesnt have a clue whether UID actually exists or not. But the operation of SELinux sets completely within the grasp of selinux. The kernel component of SELinux simply reads a bunch of SELinux rules and decides whether to allow or deny the system call for a specific operation. For example, you could make it such that even if you set the permissions of /etc/passwd such that anyone can edit it, SELinux can stop it from happen regardless of that permission set. SELinux also does understand what users exists and what users dont exist too, but it really doesn't make alot of use of it.

Ultimately, is user X capable of doing something now depends on three separate things:

The base linux permissions set
Hack 1 - ACLs
Hack 2 - SELinux

Determining the actuality of what user X is capable of becomes very tricky, just with that set of hacks because each is an entity entirely unto itself and your dependant on knowing how to "see" the resultant set of permissions based on your ability to use the user-land utility functions that come with each hack.

Then when you take into account the permissions hacks software-level things add (mount and the plethora of setuid binaries for example) you then also need to truly comprehend the kinds of permissions mount is capable of giving out - more complexity.

Let me just take a quick side step and show you what I mean when I say the kernel makes no distinction between users that exists and those that dont, consider this directory listing:

testuser@boson:~/mystuff$ ls -al

total 9776

drwxr-xr-x 2 testuser testuser 4096 2012-02-03 00:31 .

drwxr-xr-x 3 testuser testuser 4096 2012-02-03 00:47 ..

-rwsr-sr-x 1 10093023 324897 917888 2012-02-03 00:31 bash

Here i've created a file (the bash shell) with a completely random user/group id and i've set it setuid, which i can then execute and become this non-existant user:

testuser@boson:~/mystuff$ ./bash -p

bash-4.1$ id

uid=1001(testuser) gid=1002(testuser) euid=10093023 egid=324897 groups=1002(testuser)

bash-4.1$ ps -ef |grep bash

10093023 17561 17117 0 01:24 pts/6 00:00:00 ./bash -p

testuser 17566 17417 0 01:24 pts/1 00:00:00 grep --color=auto bash

Note, I now have a process with a uid of an unknown user running rampant on my system - theres not much you can do about that really. In truth there are reasons for it, and it does make the kernel quite efficient. But now onto the next hack.

Xattr's. Im going to glance over these quickly cause to explain them in a blog will make this post 5 times the length it is and as you can see its already quite lengthy. Surfice it to say xattr's get applied to files on a filesystem and can have fruity permission consequences to the file they are applied to. Yet another level of permission complexity that if you dont know exists can bite you, and give you hours of debugging fun!.

So onto setcap, the latest hack. Going back to the original kernel, it knew root which can do anything, then users which can do only a subset of those things. For example, root can "listen" on port 80, a user cannot. This is good and bad, it means a user cant start a web server on your linux machine, but it also means that any process that runs as a webserver must (at some point) be root - this has been the cause of many a security problem in unix over the years - in order to get a port listening, a binary needs to be root, someone breaks the binary via a remote execution and viola, they have a root shell.

Setcap allows for a break from that problem. It allows the linux kernel to recognize an attribute (attached to a file) that then allows that file to perform certain (previously root-only) operation's without requiring root permissions.

Heres where it gets nasty - anyone capable of running that binary, gets the permission. Consider our directory listing now:

testuser@boson:~/mystuff$ ls -la

total 9776

drwxr-xr-x 2 testuser testuser 4096 2012-02-03 01:42 .

drwxr-xr-x 3 testuser testuser 4096 2012-02-03 00:47 ..

-rwxr-xr-x 1 testuser testuser 917888 2012-02-03 00:31 bash

-rw------- 1 testuser testuser 1048576 2012-02-03 01:42 important_document.doc

-rwxr-xr-x 1 testuser testuser 8029400 2012-02-03 00:31 php

Looks pretty harmless right? You might (as an admin) wonder why testuser has a php binary sitting in his home directory, but generally speaking you probably wouldn't care.... unless you knew to do this:

root@boson:/home/testuser/mystuff# getcap *

php = cap_net_admin,cap_net_raw+eip

Then you would go "well f**k me". You would do this because cap_net_admin means "i can do nasty things to your network interfaces" and "cap_net_raw" means "i can capture all network traffic coming in and out of your system".

Heres an even worse one:

root@boson:/home/testuser/mystuff# getcap *

php =p

It means php is essentially now a setuid-root binary, it can perform all the capabilities of a root level process and you would never know (less you run getcap) simply because ls -al shows you nothing to indicate the php binary has been fiddled with in this way. Anything root can do (everything) can now be done via this scriptable php binary and its permissions allow anyone to use it. FREAKING SCARY! Even better, when it appears in the process list, its running as a normal user so you wouldn't even realise when it starts re-writing your password file for you that it was the process responsible for such a nightmare.

Essentially setcap lets you assign root-only "capabilities" (at a reasonably granular level) to binary files on your FS. Its a good idea in some ways. its also completely invisible, you wont know if a binary file has a root capability pinned to it unless you actually run "getcap" on it. Nasty. On the plus side it means alot of binaries and daemons no longer need to be run as root or setuid root. Consider ping:

root@boson:/home/testuser/mystuff# ls -al `which ping`

-rwsr-xr-x 1 root root 35648 2010-07-29 00:51 /bin/ping

Its setuid root because it needs to send out some fairly raw packets (i.e. it manipulates the ttl in the header of the outgoing packet, something only root can do). But if someone managed to make ping spawn a process thru some form of manipulation, then its also a hole into your system. Setcap fixes this problem - the hole gets plugged to a certain degree.

I cant help but think that this has been poorly implemented. It should be obvious that the binary has been setcap'd, and there should be some control over who gets to run it with the "setcap" in place. this might be something SELinux could control. I.e. "I really dont want random user x binding a privledged port unless he's running this set of programs" - but im not entirely sure about that.

Of course, you do have permissions you can set to control who can access the binaries (such as setting the permissions to 750 and using group-level permissions), but thats somewhat painful.

Personally I think its time someone sat down and went, "we need a more structured approach to this". Something that combines the entire base, ACL, xattr, setcap, etc into one "chunk" of crapola, makes it obvious what is in use and then gives the user a simple layer of control over it.

I cant help thinking that its time the kernel grouped users into three places, "root", "users i know about", "the rest" or implemented a more role-based security mechanism, or ... something. Someone really needs to consider the whole picture and produce a new set of layers that take it all into account at once - right now its BLOODY UGLY and getting worse. The sheer volume of ways you could hide something nasty in a linux box that no one may know was there (just simply because they're not a genius with knowledge of every permissions tree within linux - a large group of people there really) is rather a poor choice.

As always, this is my random brain-dump. I love linux, i really do, but sometimes the things that are getting added to it scare me a little from a "who's gunna exploit that" point of view.

GLCAS pt 2 - PXE Build Services

2012-01-27T00:52:00.004+11:00

GLCAS is a project I started some time ago, it stands for "Generic Linux Central Administration Services".

GLCAS's original purpose was to be a central location from which you could administrate a large set of linux boxes - be they RHEL based, debian based or others. It would have plugins so support for administrating just about any service would be possible.

This all started with my repo proxy - and IMHO its pretty good. It understands (currently) yum and apt repos. From a client perspective it looks like a real repo, but in truth it only downloads packages you use. Unlike a proxy server (such as squid) it also understands (intimately) the layout of repos which allows me to do things like snapshot a repo at any point and present different versions of the same repo to different machines. It will also be capable of repo management in a sophistocated way, for eg, I want the package "blah" from the external repo "myrepo", I have machines that are centos 5 and 6, so i tell GLCAS "install blah to these machines from repo myrepo". GLCAS is sophisticated enough to add the appropriate repo to both centos 5 and 6, then also install the package for you.

But i've taken a break for a little while cause I want to work on the next component of the equation - the build services. Build services were always going to be the next component of GLCAS I worked on once I'd gotten the repo components past an alpha state - this simply makes sense to me. However, for various reasons i've decided its time to switch and work on the build services for a bit. Some of it comes down to "i need a decent pxe control solution", and some of it comes down to the length of ideas I have for this bit of software has gotten me excited enough to want to do it "right now".

I have alot experience with netbooting - I wouldn't say im the PXE master, but i've had alot of experience and i've managed to get things working over PXE that were just never designed to work that way in the first place. I've also used alot of net build services and so I feel I have alot of experience with them - enough to say that I really don't like any of them to any large degree. Also, I believe I have some ideas which will make it alot more intelligent, OS agnostic and extendible.

First up, i've written my own TFTP daemon (from scratch) - its in PHP and controlled by the build services web site. Initially my plan was to migrate this to C, C++ (or at least some binary language that made sense). However, the performance of the thing so far has actually blown my socks off to a certain degree. Enough that leaving it in PHP comes with a number of advantages, such as OS agnosticism. The original purpose of the custom tftpd was to tie boot processes to control at the web front end.

For EG, the typical boot process of a machine (with pxelinux.0) goes something like this:

get dhcp address (gives boot server and boot filename parameter also)
Next, tftp to the boot server address and request the boot file.
Next, boot the boot file
pxelinux.0 then contacts the tftp server again requesting various menu files which start with a UUID, then its MAC address (i.e. 00-22-11-33-55-44 for eg), and eventually it shortens this up until it gets to "default"
Once it has a config file it then grabs a menu presenting file (such as menu.c32) and presents whatever the config file says
User then chooses an option from the list which tells it to grab a kernel and initrd (most likely) via tftp
Machine boots kernel + initrd and often from then on alot of traffic happens over a different protcol (such as http/ftp/nfs).

Here's why the php tftpd. My tftpd doesnt present a filesystem at all, it has a config (driven by the website) which says "when a machine asks for something, give them *Stuff*" and its the stuff thats interesting.

When a machine asks for pxelinux.0, it'll just service up pxelinux.0. However, at the same time the machine itself will appear on the website with a status of "network booting". Next, the machine asks for a boot file. If the next file the machine asks for matches a certain pattern (such as mac address, ip address or default) then it'll present the default menu (which you can specify), unless someone already said "when machine x boots, present this option". The default menu is cool cause what it does is then set a timeout of 5 with the default menu as the default boot option. So when a machine boots, it'll continuously sit at the same menu reloading it via tftp. How is this any use do you say?

Well, on the website we see a machine trying to boot and the website says "at default menu". At this point you can now choose (at the website) which profile to boot (build, rescue, live cd, you name it) and when you do, the tftpd will now know and the next time the machine loads the menu it gets a different menu, with a timeout of 0 and default boot of whatever was chosen.

This by itself is something i've wanted to do for a long time, but it doesn't end there. Consider one of the most popular (for linux) network boot bits of software - cobbler. I've used it on and off for a long long time and I always end up going back to a flat tftp file system that I remain in control of for the simple fact that cobbler is good for one thing and one thing only, controlling a tftp directory with the view of booting and installing redhat (and derivatives). I was most disappointed to see ubuntu's orchestrator boot services to simply be a hack job on top of cobbler. Cobblers well written dont mistake me, but i find its limitations are always its biggest downfall, and it simply misses some key things (such as booting ubuntu in a sensible way). Some of my bigger annoyances have simply been trying to get it to boot something like SystemRescueCD. But back to what im doing.

The next component of my boot services is OS independence. That doesn't mean it knows nothing about any OS, it simply means getting an OS or any network bootable software it doesn't know about shouldn't be a hack-job compared to the ones it does know about (at this point that is debian, ubuntu, RHEL, fedora, centos, SystemRescueCD, clonezilla live - but adding OS's is easy as, and most people who produce OS's shouldn't have a hard time of it).

The next component is cascading profiles. This extends to any type of profile, but for example I want all my machines to boot with a domain of x, a filesystem layout of y, a root password of z, add a webserver, add webmin, connect to this ldap server for accounts, log to this server for syslog, etc etc - consider all the things you'd like to be able to globally define. My system allows this, so when you create an ubuntu 10.10 server 64 bit profile it'll do a few amazing things:

it'll download the netboot kernel/initrd automatically from the ubuntu archive for you (it knows where to get netboots for centos, debian, fedora, centos by itself)
it knows the format of the preseed so it can then translate a global setting to an OS specific one
it'll give you a preseed editor that makes sense (rather then just a text file)
it'll know the difference between preseeds for various versions.

Right now, it can do that for RHEL, Centos, fedora, ubuntu, debian (though version knowledge at this point is sparse).

It also extends to rescue CD's. For eg, SystemRescueCD is capable of being told its root password, start a VNC server on port x, start ssh, etc etc on boot and so there are global params for these things for rescue CD's - so if another rescue cd is capable of doing that, and someone writes a plugin to handle the profile - it'll be a simple thing to do.

All up, i've spent about a week on it so far and in about another week it'll make its appearance on github. Quite looking forward to how the netbooting community will perceive it.

Star Wars: The Old Republic, My Review.

2012-01-22T03:54:00.002+11:00

I'm going to launch straight into my score and my thoughts for the future of the game, then i'll explain why. Keep in mind, this is an opinion piece - MY opinion. If you don't like my opinion, your free to voice your own, elsewhere.

I believe this game deserves 5.5/10 (its ok, lacks originality and fails to deliver in some really important ways).

My predictions for its future. This is a littler harder, but I personally think that SWTOR will likely go the same way as Lord of the Rings online, Star Trek online, etc. I think within the next 3-4 months, the number of subscribers will peak. In about 6 months after that, it'll be half of its peak. 3 months from that, they'll be letting people play the game for free, just to get them on board.

Right now, I believe quite a number of WoW players will have jumped on the SWTOR bandwagon and initially they'll probably be enjoying the game, but once its apparent that all they're really doing is playing a version of WoW that is 3-4 years behind what WoW has managed to create, they'll come back to WoW.

So now, onto the review and why I think its a pretty average game.

Let me summarise the whole thing with the following:

80% of the time, I feel like im playing WoW (world of warcraft)
Space combat is depressingly dull.
Lacks any other form of involvement in space (its STAR WARS for crying out loud)
Some of the things they have done to make the game unique, become tedious in short order

If I considered SWTOR solely on its own merits as a game apart from every other game (i.e. ignore the similarities) it'd get a 8-9/10. But this is one of those (rare) games where they have largely simply copied someone else and so to consider it (and only it) when scoring it is unfair to the people they have copied.

There are so many opportunities to make a new and much more compelling MMO in truth, and that EA/Bioware have forsaken that opportunity and created something that is very similar to WoW rather then creating something unique is a real shame. EA/Bioware have the people and money to achieve such a lofty goal rather then simply copying vast portions of the WoW space and I see SWTOR is a huge opportunity missed. There aren't many out there that could not only achieve such a goal, but support it with more resources then alot of other people are going to have at their disposal and hence be the "next big thing" in MMO's.

Starwars Galaxies, while it had many down sides, few of them had to do with the ideas and so forth that the universe was developed on. Most of the problems with SWG were simple either the fact that SOE created it or simple things they could have done to fix it. Ultimately SWG could have been a much more impressive MMO with a real following if it hadn't of fallen into a few holes along the way (such as letting SOE code it - i harp on this, but I really do think the fact SOE were allowed to develop this title was one of its main problems). Alot of the concepts in it were unique and worth pursuing - it really was an attempt at creating something new. Its a pity EA/Bioware didn't attempt to do the same.

A bit about me.

I've played (in truth) many MMO's. From this style of game with a game client and a heavily involving (and evolving) universe, to web based MMOs, phone based games, etc. I've played many. Even prior to MMO's I used to play MUD's (from which MMO's essentially evolved, of which EQ was probably one of the first notable ones), and they were fun too. But the short list of MMOs (in this style) that i've played:

Dark age of camelot
EVE Online
Vendetta online
The Matrix Online
Starwars Galaxies
Jumpgate
Pirates of the burning sea
City of heroes
Anarchy online
World of warcraft
Lord of the rings online

There are more, but i've forgotten what else.

ITs WoW with light sabre's.

First, let me talk about the WoW similarities. If you go back to around the time WoW originally came out, you'd find a few MMO's around at the time and one I played most was probably Dark Age Of Camelot. It was a fun MMO, had a very nice PvP system (one of the best at the time, IMHO). You could compare and find alot of similarities - but thats about it, similarities, and in reality you'll find alot of similarities between many MMO's. But thats all you'll find, you wont find "exactly the same". Sadly EA/Bioware have chosen to break with that ideal and simply copy the way WoW works in many instances. Some of the game components and mechanics are so similar in fact that often it feels like all your doing is playing a new expansion to WoW where they've jumped into the future a fair way disabled some of the useful features of their game and given people light sabre's to run around with.

That ain't good. There were some key things about WoW that everyone wants gone - most notably the boredom of the level grind. Level grinding and questing annoy me, the only thing that makes it fun is when something drops a rare (or better) item. Other then that its simply a case of find quest giver, get quest, do quest, hand in quest, next quest. I wont say this is one of those things that SWTOR has copied exactly - but its very close.

Level grinding and questing is always the same thing. Why oh why couldn't EA/Bioware, with the large sum of money and people at their disposal come up with a game that was more unique. Im not bothered by people writing games that take queues or learn from other games when it comes to game play - but to copy them exactly I find a little sad especially when its a big name like this one.

Ultimately, you will struggle to find a game (MMO) that copies WoW so closely (or in general one game that copies another game so closely) - some say this is a good thing - no it really is not.

If SWTOR and WoW were phones, blizzard would be suing the pants off EA/Bioware, much like Apple are suing Samsung, only Blizzard would have alot more to sue for. Thats not to say I agree with Apple, I personally believe Apples lawsuits are without merit and the fact that Apple can patent something like "slide to unlock" is indicative of a system that is truly flawed. I also believe Blizzard have no reason (and are unlikely to) sue EA/Bioware, however, I do believe EA/Bioware will be punished to some degree by their duplication of WoW.

At the end of my 30 days of free gameplay, I doubt i'll continue on much longer for the simple fact that there isn't enough in SWTOR to keep it interesting for me - im bored. It really does feel like an old version of WoW with lots of pieces of missing. Chances are that at the end of that 30 days im going to hit up star trek online, just to see why it failed so badly. Or perhaps diablo 3 will be out, who knows?

The Unique bits

Quest Dialogues

First, theres the questing system itself. While basically the same as WoW in structure, its the delivery of quests that has been changed. You can search on youtube for tonnes of examples of this, but when you start a conversation with an NPC it launches into a KOTOR style dialogue - complete with speech and, the characters mouths move in time, everything - you also have to make choices during the dialogue, some that effect your character (i.e. light vs dark choices). Its initially quite nice. By the time I got to level 35 I was so utterly sick of this and would just hit space till I got to a choice prompt, hit a choice, and continue - its like being delivered a story, one sentence at a time.

Even worse, I see this being an even bigger drag if I decide to start a new character. But by level 35, I was seriously bored, I've played WoW on and off since its first iteration and so these days I rarely start a new char in the game cause I just don't want to go through the level grind again (though, in WoW there are tools to make this much much easier - thank you UI customisations!). But since I feel like im just continuing to play WoW its gotten very boring very quickly.

Ultimately, seeing those quest dialogues made me realise where all their money/time went/was wasted.

Consider how many quests there are in a game such as WoW, this one has more cause it also has a very complete set of class-based quests as well, and thats a shit-load of dialogue (that EA/Bioware had to pay someone to act - not to mention the time it would take). At least - that was my original thought. Now that i've been playing it a while im beginning to notice that alot of the quest givers don't speak English and im sure i've heard the same (alien) dialogue more then once now. But, thats beside the point. SWTOR has tried to make questing more involving. Initially it is, but its repeatability is in the toilet.

Space

Space - the one thing something like star wars should have had going for it. Space is used in two ways, as a small thing you can see through the window of your ship as you fly between planets (and no, you do not "drive" your ship, you simply tell it which planet to go to and your there), and for space combat.

Both of these are a serious disappointment. Flying from one planet to another has all the fun of getting on a ship (sailing) in WoW to travel between continents. You simply get to your ship, click a button then exit your ship... how dull.

Then there's combat. Initially its quite fun until you realise that you can completely ignore the twists and turns of your ship, all you do is point your mouse pointer at a red thing, or a yellow thing and fire lasers or missiles at it (in reality you just keep your finger down on the left mouse button which is lasers, and occasionally hit your right to fire some missiles at something bigger). Its very much like being on a roller coaster with two guns strapped down in front of you. You have no control over where your ship flies. Its also entirely single player.

The next pain of space combat is the repetitiveness of it. Repetition in single player games is death, space combat in SWTOR is a single player game, make no mistake about it. Yet you get a handful of missions and daily quests to complete those missions. Sadly theres nowhere near enough of them and in some cases they even repeat the exact same game between two different missions. Any single mission has no randomness to it, to the point where once you've done it 4 or more times, you'll be able to complete the mission (perfectly) without even looking at the screen - the baddies appear on screen exactly the same way and place every time you do it. As the space combat happens entirely on the client, even LAG wont effect the gameplay of space combat.

Daily missions are nothing new - WoW has them in abundance for faction grinding. In WoW though, they can differ slightly each time, in that they happen within the WoW universe and such other people are doing them also and so mobs you might be after are hard to find or whatever. Sometimes they even may have 20 daily missions for a single faction but only offer 5 on any given day. My point being that WoW dailies often aren't simply a "get quest, load macro, finish quest, hand in quest", where space combat in WoW could be EXACTLY that. WoW dailies are boring, no mistake about it, but when it comes to boring, space combat in SWTOR has probably excelled to new heights unseen in any MMO or single player game to date.

All this makes space combat a very pathetic experience. Aside from the improved graphics, it has even less going for it then... well, lets just say Wing Commander (1990) is light years ahead of it in terms of "fun" and game mechanics. Seriously EA/Bioware? This is the best you could do? When it comes to space combat, I know coders who could have punched out code for what EA/Bioware have done here in maybe 2 weeks (single handed). That is a very very VERY sad state of affairs.

To try and compare the mechanics of SWTOR's "space combat" experience I found myself reaching back a very very long way - not many people have tried for such boring game mechanics for a long time and eventually I realised that what it most reminds me of is Atari's 1980 game, Missile Command. Yep, thats not a typo. In missile command, missiles would rain down on your city and you had to fire missiles up so they exploded at the right time to take out the incoming missiles. SWTOR is even easiler in that you dont have to use some form of prediction for where you need to fire in order for your shots to take out the baddies. You simply point your cursor at the point on the screen and shoot - you can even do it one handed.

In essence, you can completely ignore the roller coaster ride thats going on on your screen and simply point your mouse at the moving things that need to die on your screen, the twists and turns of your ship are 99% irrelevant.

As I said - in todays world, thats just nowhere near good enough.

Companions

The companion system is possibly the only "fun" and unique part of the game. It is really quite interesting. Companions are like pets, only they interact with you quite a fair bit - your ultimately in control, but occasionally a companion will have conversations with you and maybe quests and so forth. They also form a large part of the crafting system for SWTOR - more on that later.

You end up with a number of companions that you pick up as you journey through the SWTOR universe. Once you get your first one you'll always have one by your side (except perhaps in flashpoints where you probably/hopefully wont need them and PvP where you cant). You cant have more then one travelling with you at any time, but thats ok, the others are used for your crew skills (such as crafting).

The best part of the companion system is how it adds a method to your gameplay thats somewhat unique depending on the companion you have out. What I mean is that my first class was a Jedi Sage (healer/stealth type) and I went for the seer tree (healer). My first three companions were a lizard-man (quite an interesting character actually), a truly annoying robot (you get this when you get your ship) and a up-himself doctor (who has a female hologram that is perhaps a little weird). The lizard man is a melee combat tank, the doctor is a ranged DPS healer. I tended to use the lizard when they were my only two choices as i was a good healer, but whenever I pulled the doctor out I had to play a completely different way - that adds a level of flexibility to the gameplay that is both fun and enjoyable. You'll also find you'll get quite caught up in your companions (specially the first one) and their "stories".

Exactly as WoW does, SWTOR has a skill tree system (this was one of my bigger dislikes, not that it has one, but that it exactly copies WoW). However, you can't have two skill tree's (multi-spec) like WoW. BUT unlike WoW, its viable (thanks to companions) to have a skill tree thats not DPS. You can happily quest (level grind) as either a tank or a healer (technically you can do this in WoW, its just harder - in SWTOR its not).

EA/Bioware have even added something no other MMO I know of has with this, the concept of physical relationships. Even class has at least one companion you can romance. Im still not entirely sure whether I find this concept a little silly or intriguing. It has a days-of-our-lives drama quality to it that often I find to be more of a turn-off. Much like many soaps though, its hard not to get caught up in it however. Im not simply refering to the romance component of the soap opera that comprises your interactions with companions either. I find the lizard man to be quite interesting - EA/Bioware have done a good job when it comes to keeping that guy in character throughout the game.

The companion system is unlike any kind of pet system you'll have played with prior to this game and truly unique - its fun ultimately (and sadly the only part of the game i've found truly enjoyable)

Crafting

Here's another unique system SWTOR has come up with. The crafting system is indeed generally quite unique. To be fair, its not the end result thats unique. Much with any other game, you gets some materials and you create something. But this is where your companions are important, you get these things called crew skills, one of which is your crafting profession (you can only have one) and two others that are ultimately gathering skills.

You (and only you) can also pull things apart again for the chance to create a better version of it (much like wow, you get green items, blue items and purple items - in order of awesomeness).

The companions can either be at your side, on your ship, or performing a crew skill. While level grinding you'll find it hard to keep up with the crafting system because of the astronomical cost of your training each time you level. Companion gathering skills are similar, they cost you money (and its very hard to afford both them and your level training as you progress), but you can usually also gather materials as you venture around the universe (though theres at least one component from every item you wont be able to gather and have to buy).

Its a good system for the most part. But you (as in your character) aren't heavily involved in it - that is a good thing by the way. Where it falls apart a bit is the cost, its hard to keep up with the cost.

And thats where the "uniqueness" ends....

But one more thing I should touch on...

PvP

PvP in SWTOR is much the same as it is in WoW, you queue for a battleground (in SWTOR you cant choose which battleground you join - this is bad), you join a bunch of random players and you compete against each other for some pre-set goal.

The actual battlegrounds are different, but ultimately its the same stuff. They do have one thats somewhat different called "huttball" which is kind like football. It is kinda unique, but ultimately its not too different to capture the flag style play of some of WoW's battlegrounds.

Much like WoW, there are some world PvP area's, but SWTOR has failed to make them interesting in the same way WoW failed to make theirs particularly interesting.

Other points of annoyance/lack of creativity from EA/Bioware

SWTOR sometimes try to differentiate the way they do things by simply calling them different names, they call instances flashpoints. Yay, same thing, different name. Though you can only have 4 in SWTOR, versus 5 in WoW.

End game is not much different from WoW - both in PvP (though they lack an arena system as far as i know) and instances. Sadly, im usually all into the PvP - in WoW (As with many other MMO's) I can never put this down. SWTOR has failed miserably so far in making PvP something i'd like to do. Dont get me wrong, it does one thing WoW does not and evens out characters in a game (i.e. at level 35, i'd join a game and boosted up to level 50 - at least in terms of health/effectiveness of skills). This I really do like. WoW makes PvP so much about the gear your in, and i find that somewhat trajic to be honest. SWTOR has taken a large part of the gearing away from the PvP combat. The problem I have with this is that PvP is one thing I never really tire of. MMO's will usually win or lose with me based on my ability to compete directly with other people in the game (in a fun way - i dont really care much about winning, just enjoying PvP) because unlike PvE, PvP is entirely unpredictable.

If you buy the game, you cant just play it (unlike many other MMO's). EA/Bioware force you to either enter a credit card or a game time card BEFORE you can even login - I personally think thats one of the worst things any game maker has done. Considering you get 30 days of game time with the actual game, it SHOULD NOT force you to "sign up" for a plan and you should get your 30 days of game time without having to add more.

I can just imagine little johnny saving up his cash to get SWTOR, buying it thinking "yay, i have 30 days of game time, hopefully by then i'll have enough saved for a game time card" then plugging in the game and finding out that his 30 days start ticking the second he signs up and unless he either has a Credit card or a game time code he's shit out of luck. Personally I find this part a little pathetic on EA/Bioware's part, something they should be truly ashamed of themselves for.

The next thing I don't really like about this game is how EA/Bioware seem to have had a hard time ditching their single player mentality and remembering this is an MMO - multiplayer is king. There are many parts of the game where you'll be unable to interact with other players and the game feels like a serial, made-for-single-player deal that no MMO every should be. Large parts of the maps have been locked off behind shimmering red force fields, unavailable to you until you either have the quest required that allows you in, or because your simply the wrong class/faction. Ultimately, thats a real gameplay killer.

Crafting itself is good, and the more you concentrate on it, the better the things you can produce - however one of SWTORs biggest failings is its ability to sell things between players - the auction house (or Galactic Trade Network as they like to call it) has practically the worst interface in the history of interfaces. Much like WoW also, there are several GCN's, some of which are cross faction (sith/republic). Ultimately, the auction house system however is pretty garbage, ripe for improvement.

Cost of things in the game is really unbalanced (IMHO). You often get new skills everytime you level which you need to go buy (Almost exactly the same as WoW) - for some reason, the cost of them basically cleaned me out everytime i got a new level. It was really painful.

Unlike many MMO's, some of them quite old now, you can't extend the UI/client with mod's. This is a huge shame cause UI mods like WoW has are so fundamentally useful. They add functionality EA/Bioware just cant be bothered adding, and on more then one occasion i've wished there are things I could do the client is incapable of doing. For example, as a healer, i'd like to be able to switch my hot keys as soon as I enter an instances with other players so that 1,2,3 etc keys are all about healing while when im running around questing solo with the lizard, 1,2,3 keys are more appropriate to that situation. There are lots of examples of this - customising the UI is one thing EA/Bioware NEED (not want) for this game to go anywhere.

After a while, I found the things that EA/Bioware DIDN'T copy from WoW to be as big a disappointment as the things they did. For example, they copy instances pretty closely, but theres no queueing system nor a summoning stone. Your essentially reduced to having to manually find people, and in alot of cases that can be a real painful exercise. Sometimes you wont get to do that 4-player quest-based instance on one planet, but once you leave the planet you basically leave behind any chance of finding other people to do it with. Thats a real shame.

Other reviewers - my final say

The gaming industry, much like many others, has people who review it (for the purpose of informing its consumers). Unlike many other industries, reviewers (im beginning to believe) are at the mercy of the gaming industry. Consider movie/restaurant critic's. They'll happily give a movie/restaurant a bad review with little consideration for the owner of the thing they are reviewing because theres little the owner could do to either get back at them or stop them from publishing a negative review. The gaming industry does not appear to follow the same rules, and im beginning to believe what alot of people say that reviewers are scared to give negative reviews of games simply because they'll not be allowed to see the game early on like they can now.

SWTOR is a good example of this (though MW3 is possibly another). If EA/Bioware didn't like a reviewer then they could (if they wished) stop that reviewer from getting access to the game prior to its release (i.e. access in early beta stages which many reviewers do get). This is very bad scenario cause it really puts reviewers under the stranglehold of an industry that shouldn't be able to control those that would rate them.

When MW3 came out, if you looked at Metacritic you'd be shocked at the disparity between user and "professional" reviews of the game. The pro's ranked the game (on average) around 80%, the users on the other hand gave it 2/10. Now in truth it doesnt deserve a 2/10 (IMHO), but not 8/10 either. Its a good game, but I found unlike most other Call of Duty titles I stopped playing about 2 months after the game came out - ultimately It was too much like its previous versions and I got bored of it pretty quickly. Alot of people said at the time that reviews scored it so high for the reasons i mention above - fear of reprisals from the game makers. I didn't believe it at the time, but im slowly starting to.

SWTOR is similar, "professionals" rated it 85%, while the average user review is 6/10. You really have to wonder how much of what the "professionals" opinions are is based on their fear of ticking off the producers of the game they're about to review...

Either that, or "professional" reviewers are simply dumb and just love shiny new things, even if they look exactly like some other shiny thing they've seen before. Alternatively, they see games like this and apply a "im only considering this game, not others" much like I mentioned about my score early on in this blog entry. As I say though, when a game like SWTOR so exactly copies components from another game, rating it on its own is unfair to the people they've stolen ideas from and there are MANY that SWTOR has taken from WoW - things that you wouldn't just happen to think up unless you'd been doing it for a while - much like how WoW has evolved based on things blizzard have learnt.

Android GPS Emulator

2011-12-22T16:56:00.000+11:00

Emulating the GPS under android I found to be quite a tedious job. The DDMS component of the ADT plugin is pretty limited. I needed, speed, gps accuracy, heading, the works... these are something that the DDMS cant do, it'll simply step through a bunch of points.

There are tools out there that let you emulate a GPS by telnetting to port 5554 and so forth. I didnt like any of them.

So heres mine, it'll follow a gpx track as they exist on OpenStreetMap's traces site, of which i've contributed a few. These ones work great cause they follow actual routes taken by actual vehicles (in my case, my own car). Im using my own traces, but you can download some from the openstreetmap site and use those.

The emulator is written in PHP, so you'll need php, and it'll work on any OS (afaik). Theres nothing special being used in the php, so you shouldn't need any special modules.

But, Download it here, enjoy.

You call it like this:

$ php fakegps.php 5554 ~/Downloads/gpxfiles/sydney-1128470.gpx 20 10 10 5

and What this means is connect to port 5554 (your emulator port), maximum speed of 20, minimum speed of 10, every 10 seconds sit on the same location for 5 seconds. Speed is in knots.

Its really simple. Heading will change randomally up to 10 degrees in any direction. Speed randomally goes up and down between the 20-10 and is zero during the pause.

If you use it and like it or have a suggestion how I can change it, drop me a comment.

The Adonis Jot Android/IPad Stylus - WOW! - Never Wanted A Tablet Before.

2011-12-15T15:37:00.002+11:00

I've been looking (on and off) for a stylus for my phone for a while. In truth, theres not alot of need for one, except when it comes to drawing.

Now, im not great at drawing, but i do find writing out my ideas sometimes to be useful. I've never looked that hard cause everytime i do i see these ridiculous sausage things that have the accuracy of a ... and they just generally suck. I've "tried" a few including the ever-popular pogo sketch (why on earth is this popular?), the wacom bamboo and the nomad brush. I even went out of my way to find someone with the nomad brush cause of this review/comparison.

Garbage, all of them.

Then I stumbled onto a review of something called the adonis jot (and jot pro).

I VERY HIGHLY recommend this product if you want accuracy. I've tried it on my Samsung Nexus S, the HTC desire and SGS2, it is brilliant on all of them. I also have a friend who's tried it on the ipad and iphone, loves it on both.

Interestingly what spawned me travelling down this path is because of Androids 10billion apps download promotion which included Adobe sketchbook, and it works astoundingly well under it.

Now, here's the thing that I did not expect. I want a tablet. I've never ever wanted a tablet, ever. I mean, i like the idea, i just cant see a use for one. That is, until now. A tablet with a stylus is a nice little piece of a4 paper i can draw on properly.

To be honest, I never saw the need of a tablet until now. This really has me intrigued that a $25 stylus could prompt me to want a tablet worth $200-$700.

Coding My First Android App - My Experiences.

2011-12-10T01:00:00.001+11:00

Well, its not really my first attempt, but its the first attempt where I've had a real goal in mind, one that was my own, something that im doing for fun, and its been quite an eye opener in alot of ways. If you want to Skip my diatribe about previous java experiences (alot of them) being something of a struggle, go down to the line that says "And Now, Android".

But, to sum up my android experiences so far:

Android's made me love java again, im coding for fun, and fun is what im getting.
The IDE integration is the BEST i've ever seen (informative, intuitive, useful and functional rather then an obfuscating black box)
Googles ability to document and provide examples is shockingly bad.
What google lacks, the community makes up for - the community is brilliant.

I have a reasonable amount of Java experience, mostly on the J2EE side and by that I mean if you've coded a Java Servlet (say tomcat or google web services), thats about 1/10 of what encompasses a J2EE application (in terms of capability). For the most part a servlet isn't much different from how you code most web, request-response based tasks. J2EE adds an extremely large layer on top of what servlets can do, J2EE isn't simply about a web application any more its about a data lifecycle. To that end, some of the services you code in a J2EE environment may have a life that has nothing to do with when and how web requests occur.

My point is, J2EE isn't a simple thing, servlets on the other hand (to some extent) are, and coding J2EE was quite a leap from java servlet containers when I first started doing J2EE. It also taught me alot about Java. I have coded some desktop java apps and some web-embedded ones (including some of the newer webstart based ones), but its mostly been either J2EE or pure web services based stuff.

Ultimately though, I can sum up my experiences with Java in the world so far as a single word - "STRUGGLE". My first ever java coding experience was in 1.0/1.1 and I dont remember what it was, but I do remember my first real "pain" exersize with Java and that was coding a java servlet running inside a Netscape Enterprise web server (Netscape Entperise Web server was the predecessor to iPlanet which then became Sun One) in 1999 (i think) on a Java 1.2 platform. Boy was that painful. It was entirely on Sun equipment, but there were annoyances. Threads would go to sleep and not wake up, garbage wouldnt be collected, interfaces were porely documented, the list was endless.

Now for all the leaps and bounds Java has made, and its quite good now in 1.6 land, even today when i occasionally rip out a java app, it still feels like a struggle - even though that struggle is different. It still feels like Im trying to beat java into submission rather then it being a co-operative partner in my endeavours. Some of that comes down to a very painful "you do things this way and only this way" approach that java tends to instil, some of it can be lack of (or convoluted) documentation, and sometimes it can be about implementation interpretations (i.e. websphere's interprettation of what WebService.ServerPort() should return is different to Sun's version of it).

Dont mistake me, i've done java for fun because it can be fun, but it never makes me excited about coding it.

And Now, Android

So, back to android, how has this translated to the android world? Java has become enjoyable again. I mean, i'm coding this application ultimately for myself and for fun, thats all - and I find myself looking forward to spending time at the keyboard punching out code in a way I haven't felt for a while.

Now part of that (in fairness) has been the fact that your ultimately coding for a platform that encapsulates things to a degree where java's write-once, run-anywhere phillosophy actually does work for a change (plus googles tweeks to the jvm here and there that iron out some annoying bugs). You might have to deal with different screen resolutions, and maybe even API versions, but the API helps you deal with all these quite well. I've aimed at 2.2 API and i've deployed my app on 6 different phones now, multiple screen resolutions and every version between 2.2 and 4.0.1. I've even used it with a phone that has a hardware keyboard (and made use of that keyboard).

But mostly, its been a pleasure to do so. When coding for the keyboard, for example, I didn't feel like I was pushing poo up the proverbial hill to make it work - and I wasn't thinking "well I hope this works on another phone" either. It took me no more then 10 minutes to go from "so how do I intercept a key press" to it being installed on a phone and working how it should.

Its not all good though. The experiences with J2EE have got my mind set into certain things. My first version of my app was simple an on-screen application that read some data, did some stuff with it an spat it out to the screen in different format depending on what you wanted. Now at the time I knew i'd move the bulk of the data grab-and-process into a background, independent process (a remote process as android calls them). While coding v1 of the application I had assumed (although there was some reading involved) that interfaces like this would function much like a j2ee interface would. To some extend it did, but it was sufficiently different that I realised I may as well re-code from scratch.

Sadly, I also upgraded my eclipse to indigo from helios and thru an unfortunate sequence of events deleted both the code from eclipse and the git repo i was working in... whoops!. So that spured me to recode from scratch anyway with the lessons i'd learnt.

Next, I wanted to implement menus and preferences - and my "v2" app has now gotten to a point where I feel I need to re-code from scratch again cause it just didn't come together in android the way I expected. In reality, thats what will happen, i'll create an entirely new application and code from scratch learning from what I already know.

This is fine, I expect this to happen at this point in my android coding lifecycle. I just dont necessarily know how things are meant to happen in android and when they go sideways form expectations I realise my code just aint going to wrap around the differences well enough. So you start from scratch rather then just hack it back in.

In some ways, I find that the android SDK does indeed let me down in some ways. For example, menu's are trickly little buggers. You press the menu key and a menu pop's up. You code (in xml) a description for the menu, and in your app when the menu key is pressed, you load that xml and it draws it. Trying to make an item *IN* the menu sensitive to context is very very difficult without actually coding a menu in java, and that somewhat sucks.

Preferences are another example of what I find lacking in the android SDK also. I had assumed preferences would work in one of two ways:

I describe a set of preferences in some format (say xml) and it does the rest - figures out where to save them, load them, etc. All I do is go PreferencesManager pm = new PreferencesManager(MyXMLResourceName). This includes popping up a "standard" preferences dialog when i ask for it.
It gives me a handle to a location I can save data and I do it all myself.

In reality its neither - in truth it should operate as 1 and it could except someone at google smokes crack.

To implement a preferences dialog, using their standard preferences "model" you have to extend one of their classes which simply loads the xml file, it'll do saving and loading of data thru its interfaces all for you, but creating the class and then utilising it is convoluted an annoying when they could have easily just allowed you to do something like this (and still allowed you to overload their class):

// initialise a pref's dialog

PrefsManager pm = new PrefsManager(myxlmresource);

// show the dialog for the user (modal)

pm.showDialog();

// get and set example

mypreference = pm.getBooleanValue("mypreferencename");

pm.setBooleanValue("mypreferencename", true);

etc etc... im sure you get what I mean here.

Which brings me to my other beef with the android SDK - googles convoluted documentation and examples. Im intimately familiar with interfaces in J2EE land, yet reading about "services" from google's SDK documentation left me bewildered. I had no idea where to start, no idea what half the terms meant, and the examples were complex messes that left me with even more questions. Sadly, this is often the case with many API's, the documentation is horrific. It may tell you what you need to know but understanding it in the first instances is a nightmare.

There is one shining beacon of light that almost everyone should aspire to, and I *HATE* to say this, but Microsoft API documentation and examples are truly useful and understandable, unlike just about everyone elses (I include in this, java, gnome, kde, qt, boost, stl, websphere, sunone, apache... how about just about everyone?). Palm was also good in this area from memory and php is quite decent too. Python and ruby are somewhere in between and mileage various greatly between python api's. Google's golang is good at the start and the end, but there's a giant chasm in between where trying to get information can be daunting.

Then I stumbled onto a brilliant example of how they work right here, a clear and consise example of how remote (google use a poor name for what really isnt a remote) service. As well as examples of local and remote services (and various other very well documented little bits of information about how to achieve very pointed - and yet useful and important - goals).

And this is where the android community has really excelled - between stackoverflow and various blogs like Sai's, i've gotten to understand how to do things in android, THEN I find I actually understand the google API doco, even if the examples still remain useless cause of their ridiculous complexity.

But ultimately, i've really enjoyed coding my first real attempt at a purpose of my own inside an android application. I feel like my rather large goal for my project is actually achievable and I can see how i'll get it done. I also feel I'll get it done in a timely fashion.

Another thing android/google should feel proud of is the ADT plugin for eclipse, this things is beautiful. It builds perfectly onto the existing components that eclipse brings to the table for coding java (like debugging etc) and integrates the emulator so very well (and interfaces to real phones just as well). Its been a pleasure to use - even though their API documentation (again) dont really do anything with any type of reference to it. Often when coding inside IDE's (such as visual studio) I have had no concept of what the IDE is doing to generate the ultimate application itself from my code (i mean i know its going to call "cc"/"ld" to compile/link code, but the rest is a mystery). With ADT/Eclipse its the exact opposite - no IDE has every made me feel like that. It makes me feel like i can switch straight to the command line and do it all manually myself if I have to.

That, my friends, is my last beef. The eclipse integration is brilliant, yet android/google never really show you how to use it in any of their examples or documentation. Unlike alot of IDE's (visual studio for example), i don't feel like i'm coding a black box. It doesn't feel like there's stuff going on under the hood that I don't understand. I do get how it generates its "R" class and why and for the most part the layout of the resources in the IDE match how you use them.

So the eclipse integration is both very well done and informative rather then obfuscating. I've never seen such a heavily integrated IDE that doesn't mask things from the user with lots of "under the hood" mechanics, but in the ADT google/android have managed it. From that perspective, alot of people should look at ADT as a shining beacon of light for how an IDE can be functional, non-obstructive and instructional. I just really wish google would write some of their examples/documentation with reference to it.

Ultimately, google need to look at their documentation and go "so, someone wants to do x, how do we achieve x in eclipse with ADT" and write something that does X and ONLY x. For example:

how do i add a menu to an app?
how do i add a preferences dialog?
how do i implement a local service?
how do i implement a remote service?
how do i respond to events?
....

Clear and consise ones much like Sai's. Once you get some of the basic concepts (such as those above) you'll find the rest of the google documentation alot easier to understand. Ultimately, those first 2 things are 2 of the things 99.99% of apps are going to want to do (just about everyone is going to want to add a menu and pref's dialog to their apps), but the doco is so badly worded that you'll struggle doing them where a simple example will work very well.

Nexus S + ICS (Cyanogen Mod 9)

2011-12-06T05:05:00.001+11:00

So, ice cream sandwich came out and Google have revived my love for them (somewhat). When google stopped publishing the AOSP with v3 I was exceptionally disappointed in them (especially the excuses behind it). However, they brang it back in a big way by releasing 4.0 even before it was in the wild and what has this done for us?

Its done (at least partly) exactly what android needed, a kick from its hacker crowd. If you sale on over to xda-developers you'll find a number of quite active porting attempts from the members of its community. To say that its impressive is an understatement of massive proportions. Google released 4.0 with only support for the pandaboard and the new nexus prime (not that I have even a slight problem with that). But to fill the gaps for those of us not really wanting a Prime (like me, well actually i do want one, i just dont really find it all that easy to justify given i've gotten a SGS2 and nexus S on hand).

However, the ICS ports to the nexus S are truely extordinary. The one im currently using on my nexus S is the one provided by the very well known Cyanogenmod team, specifically a person by the name of Koush. His work on ICS has truely been quite spectacular but he's not alone and i'd be remiss in not mentioning both Kwiboo (another well done ICS port) and franco (who's working on the kernel side of ICS for the nexus S).

ICS is quite impressive (to say the least). I've always found the AOSP interfaces to be a bit lackluster compared to the HTC Sense interface (i own several htc phones), though i find samsung's interface rather dissappointing so far (specifically the SGS2, its fast, but its big and has a crap resolution). The nexus S seems to be almost spot on the money when it comes to resolution vs screen size even if it is a big "fat" compared to other phones.

ICS however has a brand spaning new interface (and API) that i've found to be a pleasure to use and it seems actually faster then the previous 2.3 line of AOSP. I know the Cyanogenmod port from koush probably does include some speedup's and its alpha (though supprisingly better then alot of stable custom rom's i've used) but it feels very snappy (interesting how whenever i've tried a new version of windows the reverse always seems to be the case). The ICS interface itself is very sharp on the nexus S, im not sure how or why it feels so much cleaner then it did with 2.3, but looking at the phone i can barely even see a pixel and the definition of things on the screen seems almost razor sharp.

I also have a htc desire (The original one) which actually has the same screen res as both the SGS2 and the nexus S and when i look at it, its very difficult to see any actual pixels (its a smaller screen size then both) yet it doesnt look anywhere near as sharp.

Personally I think the "lets get really big screens" ideal of the SGS2 and the like is a mistake and the dpi of anything greater then the nexus S is a bit of an error in judgement from those who designed it. As I say, the SGS2 is fast, but the screen is too large for the resolution its sporting and I really dont much like the Samsung interface.

Alot of what ICS is, however, isnt in its feature set however its more in the way they've dealt with the interface. The new controls (text inputs, slliders, etc) are very very nice to use and look quite fantastic (still not a fan of their checkboxes and i dont overly love their new "switches"), but the settings page is a pleasure to use.

I'll finish on what i like/dislike about the new features of the ICS port so far:

Video gtalk (yes, it was available with 2.3.6+, but this should have come a while ago) is now quite usable - though I would have been happy with just voice tbh.
The new layout of controls + controls themselves
The home screen - very slick even with the minor changes they've made
The definition of the display - just seems sharp
Data usage monitor - its quite nice
Speed - it just seems faster
The API - numerous additions to the API that add some functionality we (Developers) have wanted for a while
And of course, the AOSP published code.

What I dont like:

face unlock - they warn its not great, they're not kidding
Checkboxes and switches.
Google's voice recog still blows (compared to say vlingo).

What I would have liked:

Better XMPP support (the ability to integrate the phone properly with not just gtalk)
A unified messenger (I've assumed since android 1.0 they were going to make IM/sms a fairly seemless transition, which they havent so far).
A unified call mechanism - i pick up and dial fred who's on gtalk/xmpp and I get him thru any available channel - again something i would have thought they would have had since 1.0 as it would be easy for one android phone to know fred is connected to gtalk via his actual phone.

Cheapest NTP GPS Server EVER (Wyse S30 + linux + PPS GPS)

2011-11-24T17:21:00.001+11:00

Note: i'll provide better notes on what i've done to get this to work once i'm happy it works properly.

Some time ago, I picked up a wyse winterm S30 really cheap (around $40 AU). Its interesting little box - has a amd geode processor. Its possible to run linux on this thing and for a time its been sitting there in its default state just being what it was intended to be, a terminal client (typically rdp'ing into my linux server).

However, i've always wanted a decent GPS NTP. Standard GPS is pretty garbage really and what you need is a GPS that supports PPS. I've always wanted to get one, but the cost of something like the Garmin 18x LVC is a little more then what I want to part with for a "just for kicks" project that really wont have huge advantages for me.

However, I recently stumbled onto this, and it blew my mind somewhat as not only is it in AU, but its also a GPS with PPS and cheaper then anything i've found anywhere. Typically things in AU like this done often fall into the "cheap" category.

Getting the Wyse Winterm S30 to boot linux is not a fun chore - there are various places you can go to get advice, here's a few places I found useful:

ParkyTowers
Kishy's blog
Fijams blag

USB hard drives are a problem for this box.

But heres how I got mine booting ubuntu 10.04 from a USB sata hard disk:

Plug USB hard disk into my laptop and install ubuntu 10.04 onto the drive using Qemu.
Install OS
After the reboot, i logged in and ....
disabled GDM
removed all plymouth from /etc/init
disabled consoles on tty3, 4, 5, 6 (in /etc/init)

Next, I plugged the drive into my winterm and told it to boot usb and... nothing... bollocks.

So I boot tinycore linux via pxe from the thing - this works and it can see my hard drive, but not the internal flash drive.

Next, I copied ubuntu's initrd/kernel from /boot on the USB drive to my pxe server and booted the winterm via pxe onto the local USB drive (this isnt too hard if you know pxe) and it worked (the first time).

The next boot it died on boot and I couldnt figure out why. Did a reboot and it came up. This happened consistently, first boot it would fail, second boot it would work. Very frustrating.

Eventually, after trying to install ubuntu onto a usb flash drive (which it would boot, but still had consistency problems) I'd noticed ubuntu can see the internal ide flash drive and figured "hey, why dont i use that for boot".

So, next I fdisk up the /dev/sda drive (internal flash), create one fat32 partition, copied the kernel/initrd to it, setup a syslinux.cfg file and installed syslinux. Last I changed the bios to boot off ide first - this fails.

So i boot my system via the pxe method and realise i've forgotten to mark the partition on the flash drive as bootable, reboot again and VIOLA!

Since doing this, the system has been very stable. It'll boot consistently every time, and i've had zero issues with the box.

My new GPS should arrive tomorrow so im looking forward to seeing how it'll perform when that comes, ntp is working fine on the box and a "standard" serial gps is indeed giving me what i'd expect and working with ntpd also.

All up, this system will have cost me under 100$ AU - which is phenomenal if the soon-to-arrive gps works properly with its PPS and with this little bit wyse terminal.

The Vertical Wind Generator

2011-11-23T03:21:00.001+11:00

To say I am into "alternative" ways of doing things is something of an understatement. Electric cars (and conversions too them) was something I was into for some time. Never actually did one, but certainly did much research about them and I was interested in building a solar/wind powered generation station for it.

Alternative power is also something i've been into. Part of this comes from a perverse interest in most things computing and electrical and part of it comes from the more recent "green" technology. Solar for example was interesting, however, I always find this a bit of a "not-so-useful" tech being that its only available during the day.

Wind generation on the other hand was something I got into many years ago after the first time I went to the US and saw the massive wind farms around San Fran? (I cant quite remember where they were). Keep in mind, this was prior to 1990 and long before the world got into "Green" in any serious way. Most interesting of these for me was the vertical ones. These were big pear shaped things sitting on the hills above whatever city in the US that was. For example here is one.

Now where I live currently, im kind of on the top of a hill and it can get somewhat windy up here. With Aust going down the grid-connect alternative power systems we seem to be mostly focusing on solar, and in fairness that would actually work ok where I am, but I believe wind would be better here.

Consider, a 500W wind turbine runs in the ballpark of under $500 here. For the same size solar plant i'd be looking at a minimum of $2k. The obvious disadvantage of wind is that there is possibly a noise factor that gets involved, of that im not entirely sure.

However, back to the generators. Vertical generators come in all sorts of versions, again, google image search for "vertical wind generator" and you'll see quite a few, but there's one design i've always wondered if it would work and wonder about the efficiency of it. The type of thing im talking about is a sort-of wind-valve technology. When the wind blows against it, one side (the side you want "driven") the valves close against the wind and provide the inertia, the "undriven" side however, the valves open and let the wind through, kind of like this:

I know exactly what your thinking right now, and its "wow, he's so AWESOME at drawing" and your right. Sadly im limited to gimp at the moment cause I cant be bothered finding a decent drawing program for linux (even though there are several, I just cant think of the name of one).

The flaps themselves would be just cheap plastic slats (easily replaceable) with a small rubber (or something similar) stopper on them to stop any noise they generate when opening closing.

Well, anyways I think that'd be a somewhat more efficient way of doing vertical wind generation and if I was half-decent with my hands i'd give it a shot, but its just not going to happen!.

Oh, i also had alternative design for this thing that involved a cam that would push the flaps up as they moved into a certain position, but that ones less wind-direction agnostic.

MY NoSQL Database.

2011-11-22T13:54:00.001+11:00

Not that im about to write one, but i'd love to write a NoSQL database (or key/value store if you wish to call them that). But how would *I* write one?

Yes, thats write, its an architectural blog.

I love XMPP, i've blogged of my love on many occasions and this one is no different. Personally, I think one thing that tends to hold alot of "things" back is the ability to create their own protocols (for comms). It ends up requiring implementation at both ends of the link, which in an age when we have a protocol for almost all occasions seems a little pointless (re-inventing the wheel so to speak). So, for my architecture, lets look at XMPP for the comms channel.

It provides a number of advantages, and by that I mean the protocol, not necessarily the implementations of it. However, I will talk about somethings as they have been implemented not necessarily how they are meant to function.

Consider though, there are a numerous implementations of XMPP as a client library (i.e. something you can hook into from code) in just about every language, and one of these being of course php (which like or hate it, is a pretty important language when it comes to accessing a data store).

So, to the advantages:

Numerous client authentication methods (including none).
SSL encrypted (but not required) comms channels.
Server-Server and Client-Server comms.
Multi-directional conversations (i'll explain this bit later, im not talking about "conferencing")
Buffered messages (i.e. client disconnects and messages get stored, not dropped)
Generally clusters pretty well

The disadvantages:

The DNS/domain integration can be a bit of a frustration - but its not necessarily all that needed.

Now, each physical (or virtual) server that runs an xmpp server we'll call a "comms node" from now on, remember that bit. There are no requirements on our database that will revolve around the DNS domain of the XMPP machine, it can be anything so long as its known and DNS entries (i.e. _xmpp-server._tcp) are assumed to NOT exist. I.e. the client connects to an ip address (or hostname) with a username and a domain name, but doesnt find the server via dns entries.

So, thats the comms. Up next, the language i'd code in...

Its gotta be native binary, parallel, etc. C, C++, ... etc.. theres a few. However, for this (given we're talking about something that would need alot of parallelism) I was considering Google Golang. Im not a huge fan of managed code (generally), but this one does actually work reasonably well and being a systems language it should be quite suitable. It even has an xmpp implementation! YAY.

Now, im going to gloss over the actual database itself as the way it'll store information is pretty straight forward here. This is more about how you get data into it, how you communicate with it, making it redundant and scalable then anything else.

So how does it all come together, client, server and store.

Well, glad you asked. Each data store would exist in a domain (not dns domain, just an area that defines a storage - much like a database is used in mysql - confused?)

What I mean is that mysql is a database (engine) which stores databases (a singular chunk of data containing tables - i.e. a schema + data), when I say domain in my KV database, thats what im refering to, one distinct set of data.

So each "store" (which is the binary blog that manages the physical on-disk data) would service as many domains as you like.

In this example I'll say each physical (or virtual) server that runs a store is called a "node". Each "store" though can service multiple domains. For each domain, each store would present itself as a client to the xmpp servers that are services this domain. Bored yet? But, each node would use a pre-defined number to identify itself along with the store domain its servicing to create its "username" on the xmpp server. For example, assume our store (database) name is "mybigstore" and we assign its unique number as 10, it'll connect to the xmpp server using a username of "mybigstore.10.store" (or something like this). Now lets assume there's 10 nodes offering stores, each node knows how to contact the other nodes directly via its name - simple!.

Now on top of this, we need redundancy and scalability, a single XMPP server isn't going to cut the mustard. So i'll introduce a new component layer called the controller. Controllers will run on comms nodes along side the xmpp server and they are designed to facilitate communication between nodes, clients and so forth. The controllers would also appear as clients to the XMPP server and would be expected to know everything about where stores and clients are connected. Their usernames on our XMPP servers would be along the lines of "controller.". In this case the would be 1 for the controller running on the same server as the XMPP server, but controllers would also connect to any other XMPP server they need to communicate with in our set of infrastructure. I'll talk about this more later.

So lets get onto how all this hangs together. Clients can use anyname, they really dont matter, they're just a username/password combo - though obviously they cant be the "controller.*" or "*.store".

A client wants to talk to "mybigstore", in the client we say "connect to hostname with username and password ". So the client does this, heres how the connection is initiated:

Client makes authenticated connection to XMPP server (ssl or not does not matter).
First thing client does is initiate a message to "controller.1" (The controller running on that XMPP server) and asks three questions "What xmpp servers are there and what nodes exist for mybigstore".
Controller responds with ip addresses for all XMPP servers it knows of. (note: this also tells it the name of all the controllers connected) It also responds with all known nodes that are connected for that store.
Client now talks directly to nodes for moving data around.

Now I expect all my nodes to be capable of redundantly storing data and operating in a multi-master mode. But (with large distributions) I dont want every node to hold every chunk of data. So when a client asks for (or stores) a bit of data with a key of X, the node can respond with "talk to node x", as each node would also know where each key was stored - to a degree.

On top of all this you would also have nodes communicating with each other in order to full the requirements of redundantly storing data (if it was a requirement) and for exchange information about what keys they know of.

Controllers are responsible for creating and managing accounts on (their local) XMPP servers and maintaining access information. So if a client asks a node to store or retreive data, its up to the node to then ask the controller "is he allowed to do this".

Adding XMPP servers and nodes should be trivial, i.e. you tell one controller to create an account for a node (which tells all the other controllers to create accounts) and connect the node, the node then asks the controller "what other nodes are storing data for my domains" and then nodes talk directly to each other for exchanging data.

Nodes and controllers both connect to every single xmpp server they know of and we end up with a few chicken and egg scenarios here. Lets say we create 2 xmpp servers, the controllers on each know (and manage) how to connect to their own XMPP servers, but not to the other one - not entirely sure how that would get fixed as such, but it would be nice if it was semi-automatic i.e. controller 1's talk to each other and create controller.2 accounts on their local boxes for the second controller. Or perhaps all the controllers use a common password for the accounts from controller.2 onwards? im not entirely sure. Perhaps controllers themselves would be controlled via a web site and when you want to connect the two they communicate over a soap exchange to introduce themselves to each other. Given that controllers would be responsible for managing auth/access you may need the web gui anyway.

Nodes are responsible for maintaining data, both within themselves and within each other. Its expected a node would know to some degree what all other nodes have stored. Plus, be able to redundantly store data across multiple nodes (if required). Now, with redundant data, its expected you should be able to modify/read data from any node that says "i can store that key" and the nodes would then communicate with each other to make sure the data was consistent across themselves.

Ultimately though, controllers are responsible for maintaining comms between nodes, clients and themselves while also maintaining the xmpp server itself.

What we end up with here is if you say had 10 physical/virtual servers for storing your big kv store you could say install a node on every single one, then have 4 xmpp servers (or less if you wish). This gives each node 4 separate comms channels to every other node, each controller has 4 other xmpp servers it can have chats with, etc etc. Clients also can be spread across the comms nodes and indeed if a client connects to "comms node" 1, the controller might even say "please talk to comms node 2" and the client auto-reconnects to another node.

It gives us a fairly large margin for failures to occur, we already have clients, its something i'd love to code - if i had the time and sadly i do not. But i would love to see xmpp used for more general purposes then It does and I think as the comms channel to a KV store it would actually be fairly good and give lots of flexibility to your infrastructure.

Metacritic - A Sad Site Story - Or MW3 Nightmare?

2011-11-22T12:01:00.001+11:00

For those who have been watching, something interesting happened on metacritic - MW3 got an absolute raping by users.

Given my recent hatred of the battlefield 3 (as a former battlefield fanboy who looks slightly down on, but enjoys CoD) I find this rather interesting. Every critic rated the game quite highly and yet users are hating it. Im not trying to defend MW3 either, i've not played it.

Or are they?

Reading around on the forums after the release of MW3 (and I still haven't started playing it yet) it doesn't really sound much worse then when MW2, MW, Black Op's or any of the rest came out. Alot of people said (back then) very similar things as they are saying today. Or at least It would be fair to say the general level of "i love it because..." and "i hate it because..." is roughly the same.

Personally, I really enjoyed Black Op's and thought it was one of the best releases in the CoD line for a while, it wasn't overly inovative but it did try and solve some of the MW2 problems and I really didn't enjoy MW2 a whole lot (compared to previous CoD's). Black Op's changed markedly from the previous CoD titles while retaining the general flow of the game, and in multiplayer especially it really god thumbs up from me cause it seemed to improve the over-all mechanics.

(im going to bang on a bit here, bare with me).

So back to reading the forums, generally people are saying "MW3 is an MW2 map pack" or words to that effect. It may very well be. But personally I think the general feel on the various forums out there for MW3 is similar to every other CoD release, I dont think there are more or less lovers or haters then there were before. Theres more players and hence alot more forum traffic, but the average feel on them is roughly the same in my opinion.

Now, back to black ops - its got a pretty low metacritic user score and yet I dont really know many people who actually hated it that much. To the contrary alot of people I know like it for the things it added and/or did differently to MW2. On top of that, the number of people playing black ops was huge.

So why then is metacritic like it is? I believe it comes down to three things:

Generally, anyone who succeeds invariably gets a following of people who love to hate them. By that I mean people who want to knock them off what they perceive is a "high perch". Commonly known in this area of the world as "Tall Poppy Syndrome". And CoD is certainly the tallest FPS poppy.
People have discovered how much "fun" it is to mess with metacritic - this is sadly too easy.
Fanboyism - People who are worried their favourite title isnt getting the attention it deserves because of MW3 (im not talking exclusively about BF3 crowd either).

But my point is that metacritic's user scores have become garbage. There are more people interested in screwing with the system by either boosting or dropping a games score then there are people who are seriously interested in providing an actual valuable opinion of the game thats involved.

Sadly this has been the case for quite some time with metacritic and why I also really dont pay much attention to the site. The site itself is a good example of how not to derive statistically derived data. People login, go either 0 or 10 and see their impact on the overall score.

Don't believe me? Take a look at some of the actual user reviews on metacritic for MW3. Specifically the ones that rate it 0 - I mean click on their profile and have a look at the reviews they've given. The people who are doing this fall into 3 distinct categories:

People who have rated MW3 0 on every single platform its available for - and no other reviews.
People who have rated MW3 0 on every platform AND rated some other game (commonly BF3) 10 on every platform - and no other reviews.
People who have a number of reviews that vary in magnitude from 10 to 0 and generally appear to have disliked MW3.

Now obviously 1 and 2 are just trolling/boosting/bombing the game. Do you really believe they've gone out and bought MW3 (and some other game) on EVERY platform its available for, played it, and then given it a score? I mean thats just ridiculous. 3 are obviously the serious (and sadly made useless) users who tried the game and genuinely didnt like it.

This is why metacritic is a problem, 1+2 make up 90% of those user reviews, 3 are become a very very small minority on the site so the actual user ratings of games on metacritic have become a love v hate struggle. In essence it feels like metacritic user scores are not much different from a online game, you vote 10 or 0 and the number of people on your side wins. The very few useful reviews have become pointless.

What makes this even worse is that there is no real way to solve the problem, people get to then vote on other user reviews so obviously the people in 1+2 vote for the other people in 1+2. It really doesn't matter what the people in 1+2 are writing about the game.

Now what sucks about all this is that metacritic has become useless for user reviews. Now we have to turn towards actual game reviewers reviews of the titles to get some idea of what the games like and sadly they've shown how useless THEY can be as well.

Ultimately metacritic is a good example of where both democracy and free speech can really end up just creating useless white noise rather then provide something valuable and that's a real shame.

How would I fix metacritic? There's no simple way because people who bomb/boost vote for each other so a simple karma system (Such as on slashdot for example) isn't going to work. However, if you combined it with say some form of "veteran" weighting system (i.e. the more games you've rated + the longer you've been rating them = higher weighting) it might actually work.

So now, where do you turn to find out what a game is like before buying it? sadly, its youtube. Not many people will go out of their way making a video and all that on youtube unless they're going to spend time doing it, and the people in 1+2 aren't. You'll often find things on youtube that'll give you some in-sight. But look hard and long or just go and buy the game and judge for yourself.

Battlefield 3 (PS3) - In A Word "Disappointing"

2011-11-18T05:00:00.001+11:00

This is a bit of a rant... but it has a purpose - bare with me.

To sum up:
EA have created one of the most painful experiences in online gaming and a single player campaign that is utterly forgettable (one of the worst this year in my opinion). If your tossing up between MW3 and BF3, MW3 will mostly likely be a "put the disc in the drive and play" experience where BF3 is a "login to three different sites, then go to 3 other sites while holding down ctrl-shift-6 and sacrificing a virgin" to play MP - I am not joking - this alone has me deciding to push my game off to ebay, code unused.

Now, onto the rant...

One question before I start - is there ANYWHERE you can buy a version of the game thats not "limited edition"? Im reminded of someone who once said "you know what limited edition means? its limited to the number they can sell". In this case, I suspect it means its limited by the fact its the only version you can buy. Sad but true.

Gone are the days where on consoles all you do is throw in a cd/dvd/bluray and you play a game... Battlefield 3 has so far been one of the worst experiences i've had with a console game. PC Games can be a little convoluted sometimes, but this time it just blows me away how pathetic this game has been and I assume most of it comes down to two things:

Blood money - games makers are sick of people buying second hand games and they want a piece of the action and that is truly pathetic.
Piracy - You can mod chip consoles, so be it...

Half the reason I do actually like consoles is that they generally work well. EA has managed to make BF3 a truly remarkably bad experience. The sad thing is I believe the poor environment of BF3 has more to do with the blood money then with piracy. Sony have jumped onto this band wagon too with "firstpass". Its pathetic what companies will do to their own products to make a buck.

Get with the program MORONS AT EA AND SONY. Someone bought your game and they could play that game forever, it doesn't matter if they dont, you got the cash for your game and whatever they do with that game there after is their choice. If they sell it to a friend so be it, the game should still work, fully-functional as intended.

But BF3 has been a nightmare. I bought it some time ago and decided to wait and finished Uncharted 2 before playing it (quite a surprisingly good game really) and yes, i bought it second hand. I put it in my console, played the single player campaign (which does go on for quite some time) and then went online and played multiplayer. Its actually tonnes of fun. Gone, those days are. Any NEW game coming out will be subject to "firstpass". So say you buy a game second hand (BF3 for example). Want to play online, well now you get a code with the game, its one-time and tied to your account (in EA's infinite nightmare this includes EA, Origin and PSN from what I can tell). Now when YOU get the game second hand, you need to pay for another code. im not sure what this is going to cost but i've heard 20-30$. Pricks.

Got two kids at home and they both want to play the game, doesn't matter that you only have one console and one disc and so forth, you're going to be subject to PSN's and EA's blood money for your second child. Nice and family friendly too eh? What a pathetic bunch of losers EA and Sony are for this rip off.

So why is this important and why am I obviously very F**KED OFF with the pair of them. Sadly, this isn't the only reason - EA's BF series has typically had some pretty forgettable single player experiences, and this one is no exception. However, IMHO its multiplayer has ALWAYS made up for this. So far, multiplayer has always been a "click on multipler and play" kind of experience. As yet, i have no idea how the multiplayer is cause I CANT PLAY THE RIDICULOUS GAME ONLINE.

Just a note on Sony's first pass bit. As far as I know, this isnt actually active yet, this one-time code for online play comes to you courtesy of EA and EA only.

Single Player

If you've played the black ops single player your probably going to be going "woah, this is familiar" and you'd be right. Its all set in similar circumstances. Even the plot lines cross paths. If you took MW2's single player and crossed it with black ops, that's kinda BF3 in a nutshell. Its also very short, I think i've been playing about 3 hours and i've completed it.

There are numerous other frustrations with the single player campaign though. To begin with when you load up the disc you get bombarded with the horrific opening title screen that would send even the most mild epileptic into seizures within microseconds. Its annoying, you cant skip it and it'll do your head in. I'd rank it up there as one of the WORST opening titles i've ever had the displeasure of seeing in a game.

Now i've always been one to play the single player campaigns just cause they're designed to give you a feel for how to interact with the game. You generally will get a chance to try most of the "options" in the game in terms of controlling a character, guns you can use and vehicles you can drive. This ones similar. You'll get the basic idea of the game.

However, even in EA terms, this single player campaign is forgettable to say the least. Aside from the massive parallels with the CoD games its just kinda sad how quickly it ends.

Its also buggy as all f**k. Numerous times I got stuck at some point in the game where I just didn't seem to be progressing anymore wondering "what have I got to do?" and i'd login to the forums and find something like "its a bug, restart from the last checkpoint". One of these happened twice.

So, its a pretty average single player game.

Multiplayer

Yep, couldn't go anywhere near it. Why? cause of more bugs, PSN network maintenance and just generally poor game coding.

After my first stint on single player I decided i'd try multiplayer. I click on multiplayer and i get a screen that simple flashes "loading" at me on and off... and then... nothing... Left it like this for 10 minutes and still sitting there saying this. Click the PS button and im back at the XMB so i've not locked up. The game itself is still updating is little "loading" thing so I figure "well maybe its downloading something" and I turn off the TV and walk away from a bit. An hour and half later I switch it back on and im still at the same place. What the F**K is going on here? So i PS out and restart the game. I then go to some of the other options on the front screen (after my epileptic fit that is) and same deal "loading....".

So I login to the forums and find that theres a post there suggesting things are down and I wont be able to play MP anyways and decide "all right i'll leave it for tomorrow". Meanwhile im still looking at a code written on a piece of paper inside the box that supposedly gives me the extra maps wondering "when do I use this mofo?". Sadly, this code also unlocks the ability to play MP (and locks it to YOU personally). I wont be using this code.

So the next day I finish off the single player campaign and decide "hey, time for some MP action". Only the exact same thing happens. Theres no error messages, theres no ANYTHING to tell me something might be wrong just "loading...".

Now I got to the forums again and I find a thread about how to fix login problems. HOLY CR*P. To get MP working with BF3 (on a CONSOLE mind you) is a nightmare, you end up going through some extremely un-friendly stuff to activate it (http://forum.ea.com/eaforum/posts/list/7696896.page to be exact), then after all that you also have to go into the PSN and redeem your code.

WHAT A USER NIGHTMARE - simple for me, but I know plenty of console drivers who'll be sitting there going "i have no idea whats going on"... Sadly, today there is a PSN maintenance outage so I couldn't even redeem the code if I wanted to.

And that, sadly, is where my journey will end. BF3 will be going up on ebay, code un-used - for creating such a poor user experience EA deserve no less. Aside from that I also have MW3, which apparently wont have EA's "SCREW YOU ALL YOU BASTARDS FOR TRYING TO PLAY OUR GAME SECOND HAND" garbage.

This will also be the last thing I ever buy with the EA label on it.

The sad part of all this is that while I enjoy CoD, I've always preferred the BF series when it comes to the online component.

There are other factors involved here though. BF3 doesn't do split-screen multiplayer, and that is one thing I do enjoy playing with the other half. BF has always been a solo experience for me, but MW2 and Black Op's you can enjoy with your partner and that does add alot to the game.

So lastly, shame on you EA - your PS3 BF3 experience was one of the most amateur endeavours i've seen in the last 5 years and for a company your size, you should be utterly ashamed at how bad it truely is to try playing this game.

To everyone else, if your trying to decide between MW3 and BF3, well i'll start playing MW3 tomorrow maybe and I suspect MW3 will be exactly what i've come to expect from them - a simple "put the disc in the drive and click the menu options" and they'll all work.

How To Win The SAN Race

2011-11-06T02:23:00.000+11:00

My opion of how to win the SAN race (if you MAKE SAN's) - or at least, how to win a large portion of the market until your competitors can catch up.

I've worked in SAN's for a long time, so long I cant actually remember how long - but i do remember prior to 2000 I was playing with Sun a5100's (and when i say SAN, i dont mean some scsi-attached array here, i could go back a bit longer if I were talking about that). There are 3 things that annoy me with SAN's:

The space waste
The rediculous interfaces
The rediculous configurations

For quite some time, EMC ruled the SAN roost - probably about 6 years at a guess at time when Sun really started producing a bad storage story and EMC had few competitors. EMC were content to sit on the bum and up until about 2 years ago, there was very very little added features on and EMC SAN from about 2000 onwards - even today now they have "vblock" they're really a boring storage vendor and vblock really isnt much of a story - if you were able to configure an EMC SAN in 2003, you could do a 2-hour course and know EVERYTHING about managing EMC products today. But EMC are a good example of how little people both thinking about SAN's and even in the industry as a whole, while the amount of data you can push through an interface has gotten faster actual feature sets have barely changed in that time - or the way we access them. Over the last 2-4 years thats changed a bit, some competitors came out with different and defining feature sets (compellent, 3par, equalogic, etc) that have to some degree shaped the entire industry. Do you think EMC would have vblock today if it wasnt for things like compellent?

Personally, I think Fibre Channel should have died some time ago - some would say infiniband should have killed it and thats garbage. Infiniband is garbage and is mostly dead today because it always ways. IP is where we should be. SATA, SAS, FCoE, Firewire, USB - it should all be gone in favour of IP. But I digress. First I want to talk about form factor.

Today (with few exceptions) if you pick up a "SAN" its a brick of x disks poking out the front of a 3-4ru chassis and you stack these little bundles of disk together to form a SAN. It'll probably have two controllers at the top and a bunch of ports. At best, that layout will be about 30cm's deep... in a 1meter rack. So thats 2/3's of your rack space wasted and rack space is slowly becoming expensive. Now some vendors have attempted to cure the space waste but at the "small to medium" end of town there is really only one vendor i know of - nexsan. Generally nexsan's feature set is very very boring, but it stacks its disks upright onto trays that go quite deep into the rack. Ultimately they get quite a decent storage density out of it.

However, I dont think this is the path SAN's should go. Personally I think we should do away with spinning rusty chunks of metal and replace them with SSD and i think the person who embraces the SSD path properly for what it is will mark a major achievement on their "i win" calander that'll propel them into massive profits. SSD's are beautiful but how do we embrace them so far? we stick them into a 2.5" disk format and through a sata/sas/fc interface on the back. WHAT A WASTE. Yes, im aware i've posted on this before. But it annoys the **** out of me that the only thing SAN vendors have done when it comes to SSD is put then into 2.5" (or 3.5") caddies, and then treat them like very fast low latency disks. But consider if we changed the form factor of an SSD into something like a rod, say 2cm x 2cm wide, but 70cm's deep - nice and hard to wield!. In that space (if you compare it to a 250g SSD) you chould be able to fit about 750-1000G of space. Now consider how this stacks up against something like this:

Its an IBM ds3500 - it has all 2.5" disks, takes 2 ru, and goes about about 25cm's including controllers - what a waste, you can fit 24 disks in here and assuming you could get 1tb 2.5" disks for it, that'd be 25tb. Now if that space were ssd's of the form factor im talking about above, you can fit about 3 high and about 22 across... 3 x 22 = 66... woah, 66tb and the performance difference between them would be like comparing a bicycle to the space shuttle (and i dont think that even comes close to doing the speed difference justice). Btw, an RU is approx 31mm (or 1.25" for you medieval folk), and a (normal) rack is 480mm (19" for you medieval folk) across - so yes, it will fit.

Now I hear you say "but, thats going to be incredibly expensive", and your probably right - but consider how you might cut down on that expense. To begin with, we really dont need the phenomenal performance of the SSD if its packed into this kind of density. Obviously I dont know the answer to this, but if we made SSD with lower performance could we also maked them cheaper and more energy efficient? Assuming thats true (and in part it may be) and we get say 100MBps out of a disk (instread of the 200-300 most SSD's can do), that single array pumps out 6.6GBps - HOLY CRAP. 2RU, 66TB, and 6GBps in that form factor would be mind blowing.

If that were all SAN vendors did, it would be game changer for them (i really hate that term) but it would certainly be a product that was worth looking at.

But thats only the beginning as far as im concerned. Personally I think its time the world moved to IP based interfaces - everywhere. Drop USB, firewire, sata, sas, etc - make it all IP. What I really mean is not to drop the hardware, just the protocols. Now there are people who would shudder at the security implications of such a thing, but thats somewhat brain-dead. Consider a USB keyboard, you plug it in and it works - even if you know the USB protocol - you dont care and you plug it into anything and still works. If we changed that protocol to IP the security implications are void so long as you cant route thru the computer to the IP address on your keyboard - i.e. its a completely private subnet as it were. The interfaces (At the hardware layer) are great - SATA does 6gb/s - theres no reason IP would be a problem here. I say this because IPv6 has alot that could be beneficial here that I wont go into. But ultimately the protocol being IP shouldnt be a problem.

Now consider if a disks interface, even if it were sata, was talking IP off the back. We could then split up our array into subnets and those subnets only be visible to the controller thats pushing the data around - this is no different to Fibre by the way - replace WWN's with link-local or site-local addressing and talk iscsi across it. The disk need not have a full IP stack and no form of security - i.e. if you can plug into it you can talk to it (again, exactly like sata/fc/sas disk is today).

Which brings me to getting rid of FC all together - now we have 10gbps ethernet (and 100gbps is around the corner) FC @ 8gbps is garbage. FC always was though, expensive garbage too - licensed much like apple's firewire interface (i.e. give us 1billion dollars per port please because we just want money).

Now talking to your array, you still use a HBA but it's a iSCSI one. Keeping in mind one thing that annoys people about iSCSI is that its an IP link and then is treated like one (From a configuration and security perspective). Assuming you go down an IPv6 complete-automatic-config path and a minimilist approach to the IP stack, these concerns are baseless.

Well, thats all a bit of a brain dump, but thats how I reckon someone could actually win the SAN race for a time. It would be nice to see this wend its way into the consumer would too because the gulf that exists between what is "enterprise" and what is "consumer" has become something of canyon into which people just chuck hordes of money for no good reason.

The reason I say that last part is consider networking - the one true place where standards are both universal and well defined. Consider the difference between a Juniper SRX5800 (the absolute top-end of junipers enterprise firewall/router line) and a Netgear N900 wireless home router. At the core, they're almost identical things, they both do routing and firewalling, they even use the same interfaces - this is how the entire IT industry should be - networking is the one place this exists, i.e. what defines the difference between what is "enterprise" and what is "consumer" is simply scale. Networking is the only place this currently exists, but it really should be industry wide.

Linux/Unix and a new output pipe - stdobj

2011-11-05T01:38:00.003+11:00

Some background...

A little while ago I started coding a firewall/routing gui under linux - called CBRFW. Part of the reason I started it is because im really not that impressed with what is out there. Want a firewall/router on linux? im not overly impressed with any of the FOSS options. Of the linux based (and freely available) commercial/semi-commercial offerings the only one i've seen and liked has been astaro.

Sadly, I want something thats gpl, FOSS and works well. I also wanted something that can be used by both networking folk and unix/linux folk alike. Lastly, i also wanted central management and control. CBRFW has got a simple text based config (much like any fw/router appliance - loved by networking folk everywhere) but it'll do everything you expect from a firewall.

But with projects like this I always end up in a place I like to call "the linux utility parsing hell". It's a little place where if you want to say control the interface on a linux box you end up wishing you were dead - or you re-invent the wheel (which is a kind of death).

The problem is that when you look at a network interface under linux, theres numerous ways of looking at it - theres utilities (ifconfig, ip, ethtool), theres /proc and theres /sys. So you end up stuck because to change the ip address on say eth0, you have to use the utilities, such things cant be done under /proc or /sys and the problem with this is that utilites (while they are very up to date) still have no sane output you can parse easily in another bit of software. They also change a fair bit and generally make you wish you were dead. Which then leads you to turn to ioctl's - and this is another form of hell cause those interfaces change. But consider my firewall wants to control the duplex/speed of an interface - this is output of ethtool:

# ethtool eth0
Settings for eth0:
Supported ports: [ TP MII ]
Supported link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Half 1000baseT/Full
Supports auto-negotiation: Yes
Advertised link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Half 1000baseT/Full
Advertised pause frame use: No
Advertised auto-negotiation: Yes
Link partner advertised link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Half 1000baseT/Full
Link partner advertised pause frame use: No
Link partner advertised auto-negotiation: Yes
Speed: 1000Mb/s
Duplex: Full
Port: MII
PHYAD: 0
Transceiver: internal
Auto-negotiation: on
Supports Wake-on: pumbg
Wake-on: g
Current message level: 0x00000033 (51)
Link detected: yes

Parse that, bitch!, so your left with three options - 1) parse ethtool and hope it doesnt change too much 2) write something that knows how to deal with ioctl calls (this is no mean task on linux) 3) take ethtool and modify it.

3) sounds great, assuming you could get ethtool from say a git repo and then rebase everytime it modified its source (or just apply patches every release and fix your patches). But then your stuck with the situation of having to distribute someone elses code with yours - its no longer clean and it may not even be in a language thats useful to you (for eg, if your coding in python having to distribute a c-coded binary is not a fun task). 1) is a painful thing (though not absolutely painful under ethtool) and 2) well, the ioctl calls are an artform in linux to say the least, the further you can stay away form them, the better.

Sometimes if your lucky, the tool your calling may actually have an option (command line switch) you can use to get program-friendly output - this is rare, but especially so in the core unix/linux utilities (sadly).

Which brings me to the ACTUAL topic of the post - how i would love a new output pipe for command line junk. Right now every unix/linux/windows utility has three pipes it can read/write to, you all know them, stdin, stdout, stderr (in, out and error).

What I would like is another one called stdobj the purpose of which is that instead of when a bit of software outputs something to stdout or stderr it also outputs things in some programmer friendly way into a chunk of memory set aside by the program that called it - or something of that nature. When, for example, i call ifconfig and see the interface configuration output on stdout, in the background the program also outputs things in some form of memory array format (even xml perhaps) that can at least be parsed by other software. There really doesnt have to be a huge amount of "standards" involved here i.e. xml is perfectly fine, you dont need to then have an entire 500-page definition for what the tags in the xml should then look like, just a way of getting machine-readable output into some place in memory where another bit of code can then pull it back out. I.e. in a way normal uses aren't going to be bother with. It would also be nice if it were possible for said utilities not to generate the output unless the thing calling them actually asked for it.

I've hit this wall about .. im not sure how many times ever since i started coding under linux/unix which is sadly going back a very very long way. Occasionally i'll pull up the linux kernel source or the libc/glibc source and try and figure out how it might be achieved. But its not a fun place to play in reality. Im not suggesting that this be automatic either, the unix utilities would actually have to be re-coded to a degree to then implement outputing data into that chunk of memory, but at least if it were in the core of the OS (i.e. glibc/kernel) theres a chance someone might actually start implementing it in the core utilites as well (such as ethtool - which actually isnt - or ifconfig, etc).

Sadly, the problem that would most likely arise is that if said output did come out as xml there would be infinite discussion and consequent "standards development" (and flaming) about whether ifconfig should have xml that looked like:

ifconfig..
... etc

you get what I mean, people get caught up in the details and forget that all people really wanted in the first place was to know that somewhere in memory ifconfig will tell me what the ipaddress of eth0 was and i dont care if its ifconfigeth01.1.2.3 or something else so long as thats what it is everytime i run it and that interface remains relatively stable over time.

I know, i wish for too much really.

DD-WRT vs OpenWRT - OpenWRT For Me (usually)

2011-09-08T17:52:00.004+10:00

First off, let me say some of this will come across as a rather negative review of dd-wrt and it isnt intended to be as such. At its core dd-wrt works quite well, but there are many bad cludges that it has developed that openwrt has steered clear of.

I've used dd-wrt for a long long time, and i've gotten alot of love out of it. OpenWRT i've been using for a while, but not as long as dd-wrt though i do consider myself to have roughly the same experience level in both. But i've come to love openwrt for the things it does right and what dd-wrt does very very badly.

First off, the base firmware of both is quite functional. If all you want is something that can do these things:

PPPoE client (adsl router)
Broadband client (cable access or whatever)
WiFi Acces Point
Basic port forwarding

DD-WRT has an edge here. As a simple home router firmware dd-wrt is very nice and very pretty. OpenWRT is less pretty (though the gui is very good) but it has been built alot more solid and there are less cludges to get those thing you wont find in a home router working.

Ultimately if i were to describe them both in a single sentence it would go like this:

DD-WRT - pretty, functional, but very messy on the inside.
OpenWRT - Built from beginning to end and well integrated.

Now as a guy who does alot of "enterprise" network and server stuff, OpenWRT's interface is much more useful to me (though if you can drive dd-wrt for those 3 basic things above, you could drive openwrt with a similar ease).

I have (at home) 4 wifi-routers, 3 of them run dd-wrt. but the purpose of thoes 3 is simple:

Provide constant wifi around the house
Provied a 5-port gig hub at every location

DD-WRT does that brilliantly and the control you get over wifi is light-years in advance of OpenWRT. Its the things the 4th router does that important and why OpenWRT is lightyears in advance of DD-WRT in every other respect.

The most important aspect of it all is "optware" or software you install in addition to the base firmware. With OpenWRT the optware component has been built by the maintainers of OpenWRT for each and every firmware and distribution. So you know when you add optware into OpenWRT on your router it was built for your router (and its kernel) and you'll generally have few problems with it (assuming you know how to use it). DD-WRT is the exact opposite of this. The optware for DD-WRT never exists as part of DD-WRT and so its been hacked-in by various community members and the problem with it is that the optware it uses is built by other people for other hardware and other distributions. This is where it gets VERY VERY messy.

For example, on a tp-link wifi/router I have DD-WRT uses the OpenWRT optware resposity, which is good cause OpenWRT's repo's are very complete and contain lots of awesome software. Where this fails is kernel modules and web interfaces. OpenWRT's web interface has been built to be extendable (its called Luci i think) and so you have all these kernel modules and web interface extensions in the repo which are no use to you and wont work - more important when it comes to the kernel modules but often things just break in DD-WRT because of this.

There are a few other things I think make OpenWRT a much better choice in almost every other scenario:

No "DMZ" button - DMZ is evil, its basically a "forward all ports to this host on my network". Take a small office as an example, you could see someone doing this to their exchange server (only its probably a MS small business server) which would essentially be like plugging it straight into the net - poor choice. You can do "DMZ" on OpenWRT, its just you have to know what your doing to understand how.
IPSec - if your lucky, DD-WRT has vpnc which is a very limited implementation of IPSec (supports a very limited form of vpning) and originally there was a post on the DD-WRT forums that went like "you have openvpn, why would you need IPSec?" really? thats your answer?? That really rubbed me the wrong way because as a network person understanding why someone might use any and all vpn protocols is somewhat fundamental to different situations (like how I might use RIP, OSPF, EIGRP, ISIS or i/eBGP as a routing protocol - they all do more or less the same thing, but depending on the situation you will choose which one works for you). IPSec was the one that made me go "idiots" however cause its an important VPN protocol on the net where openvpn is not (yet). OpenWRT has openswan, stongswan and vpnc - plug just about every other type of VPN software you would use.
OpenWRT is well constructed and well thought out - it has a command line interface, the configuration is consistent and its been design from the ground up to do what OpenWRT/DD-WRT do. DD-WRT just always feels like 10 people when and took something that compiled on routers, made their own bit work and then someone just hacked the lot together - it feels very messy, and someone's attempted to hide that mess with a nice gui.
VLAN'ing - while not something alot of people who use something like OpenWRT/DD-WRT would use, vlan'ing is pretty important to us enterprise network folks. While not great, OpenWRT is at least understandable - but I have yet to ever configure DD-WRT successfully into a VLAN trunk or get VLAN's working at all. In fact, unless your doing just static routes, i've never really gotten any form of what could be called "advanced" networking working on the DD-WRT distro. DD-WRT's interface changes depends on the hardware also, which just adds to the confusion.
routing protocols - while i've seen some hardware on DD-WRT supports quagga, its inconsistent at best and very hard to use. OpenWRT isnt massively better in the "hard to use" basket, given that you use quagga on OpenWRT like you would on any other linux distro (and i like that personally), but theres no web interface for those who would like it. But Quagga is available on everything OpenWRT does (afaik).

Well, there ya go, thats my take. While I find DD-WRT is great as a "dont want to do much" how router and/or wifi AP, I typically think OpenWRT is much better in general, cleaner and built for a purpose. Again, my opinion!

A Guide to Getting Me To Spend Money on Android Apps

2011-08-23T03:05:00.002+10:00

I write this for all the android dev's out there trying to make a buck on the market in the hopes it'll actually be useful to you.

As a (current) android user (and i'll probably switch to iphone 5 when it comes out for reasons i've mentioned in other blogs) I do actually spend money on the app market for android. But, there has to be a reason, and i've found in general this is how it works for me:

Buying from the market:

Under a $1 - very little hesitation if i think im going to use it at all
Above $2 - very unlikely to buy unless its specific to a task i really need (so far that means none).
In between - if it has a free version and i find i play it alot and have gotten some enjoyment level out of it, i'll probably pay for the non-free version.

In-App Purchases:

Once - but unlikely to ever happen much.

Why is this? well in some ways i'd like to do it for some games where this is the only option to support the devs (Robotek HD for example). But i've yet to find an in-app purchasing thing that makes me believe im going to a real google site. First time i noticed this was with a game called "TapFish", i didnt really want to buy anything for it, i was just curious cause there was some noise about in-app purchases and patents (personally, i think thats the most frivolous patent ever, but god knows what goes through the minds of the numbskulls at the US patent office). It took me to a framed window which was just the google login and I was thinking "how do i know this is actually the google site?" Ultimately, i couldn't tell, and most of the in-app purchases are the same - i just dont have a way of trusting them. With tapfish i even quit the game, opened a browser window, logged into google and came back - still at a google login. Sorry, i just cant trust that.

Robotek HD was different, when i clicked on "support the devs" it took me straight to the browser, I was already logged into google and it was a simple click-thru. Lovely! thats something i can trust. But in this day and age, i rarely trust anything i dont intimately know to handle me logging into places that can use my money. I trust my computers web browser so long as i can see the ssl verification and i trust the google app market - but not much else.

My point is, if your going to make a app i'd want to part with cash, it generally has to be one of the following for me to go for it:

under $1 - i'll probably buy it, so long as it happens in the market
in-app purchases - take me to my PHONES web browser where im already logged in - make me believe your not phishing my login details
a free (ad-supported for example) version thats fully (or reasonably so) functional and i'll spend more so long as i do actually enjoy it and find it engaging.

Keep in mind there is little content on the android app market that is going to be more then a fickle passing fancy. Games are easy, and primarily what i find myself using most of the time. I have a flashlight app i use and a bar-coding one, occasionally a shopping list one but most of the rest that i download tend not to get large amounts of use. There is nothing on my phone that im attached to that it didn't already come with (i.e. web browser, sms/mms app, mail, calendar and contants - and thats it) and hence have no problem just uninstalling things i dont want (or things that get updated with new "annoying" bits).

Games for me have to be something i can play when i want to play - not when you want me to play them. Robo tower defense is a game that sits on the border for me. A single level can actually take a reasonable amount of time and so even though the game is fun I find I just cant pick it up, play it a bit and put it down. Farm story is a good example of a game I got rid of pretty quickly. though it had little to do with its game play and more to do with its full-screen randomally-appearing ad's that tried to take you to the market and download some of the companies other games - that truly annoyed me. Farm story was also annoying from a perspective of requiring planning, i.e. i'd plant some crops and have to come back to them in 10 hours, 12 hours, 3 days - whatever it was if i didnt get there on time, the crops die and between planting and picking i cant do anything - thats not particularly fun really.

I do like the idea of games i can "build up things" in, and paradise island and my country are probably the best examples of that, those are still not great for me. What I'd want in a game like this is to be able to pick it up, play it for as long as i want and put it down until i have time for it again. Theres a fishing game on android that actually fulfils this criteria, but i didnt find particularly engaging simply for the game play and i think it was aimed at ppl much younger than I!.

Hopefully someone out there finds that useful - im not cheap really, but probably not the stereotypical android user either.

GLCAS Moving Along Nicely

2011-08-09T02:27:00.003+10:00

There were 4 main goals I had originally set myself for the first release of GLCAS.

Intelligent local mirror of yum and apt repos
Be able to "hold" a mirror at a particular state.
Be able to hold various packages that aren't part of distros (such as webmin)
Push out packages from a central location.

1-3 are now done. I've always wanted a local mirror of the repositories i use, if only for network builds, but there are so many are they get so large it becomes tedious both from a bandwidth and a storage perspective.

GLCAS now does this very well, it'll mirror only those packages i actually use and it'll keep them for eternity. All things you cant really do with any existing software. Its certainly not been without its challanges, for eg, how do you tell if the URL your going to at a repo is a directory or a file? Well in truth there isnt one way or even any dead-set-certain way at all, there is only best guesses. At least thats true without building the directory structure of every repo into GLCAS - apt makes this impossible. YUM is beautiful from this perspective, APT is not (but then again, i could go on forever about why yum is far superior to apt in almost every way - again - but i'll spare ye).

The next thing i wanted was to be able to halt a mirror. What I mean by this is that i often find that i'll update some testing machines, do some tests and by the time i get ready to update the production machines there have been new patches in the upstream repos and i get screwed by them, doesnt happen all the time, just often enough to be annoying. Ultimately though, its nice to have your production systems replicate your test systems as closely as possible. This is EASY thankfully. Ultimately my repo proxy never actually goes and updates itself unless told to and then it knows what it needs to update. It wont every pull down a package unless a client requests it, and it doesn't matter much about what it knows about the repo. If a client has a more updated package list then my proxy and the client asks for a package that shouldnt technically exist my mirror will still go upstream to find it. However, if i have something and the client requests it, it gets it from local no matter what.

Here's where that gets interesting from my perspective. YUM repos are controlled entirely via one file called repomd.xml which then tells the client about where the package list, file list, etc (all the meta data basically). GLCAS will pull down repomd.xml everytime it's told to update itself but this isn't something a client to my mirror is capable of doing. Its done via a scheduled "cron" task (which is actually a web URL on GLCAS it calls) or via clicking an update button on the web interface. However, if a repo is "frozen", none of these ever work and it'll keep the repomd.xml it has already and never allow an update to occur. Freaking handy AFAIK!. Same occurs with the apt mirrors, it knows which files to update and only updates them as required while freezing the mirror halts this process.

There are some very interesting places this can take us later on - i refer to snapshotting. While it is nice for my test/production systems to have a common patch set, I dont also want to freeze all the other machines out there that dont have that type of dependency. This is where, some point in the future I want to implement snapshotting. Again, this isnt actually too hard (for yum, a bit harder for apt). What I want to do is keep the repomd.xml file hanging around and then say "snapshot" for a specific group of machines (yes my thing tracks machines as well now). When something in the snapshot group asks for something from the GLCAS mirror, it'll get the snapshotted repomd.xml and hence try and get the associated metadata files. This is good because all I have to do is snap repomd.xml and keep the metadata file. The metadata files themselves change name and are unique everytime. APT is where it gets tricky, cause even for a singular repo, the number of files to snapshot gets large... very very very large. Have i mentioned my distaste for APT recently?

Lastly, I've also implemented a adhoc package repo - and this is where it gets FUNKY. So I upload a package, lets say its webmin.i386.rpm (in truth, webmin is noarch, but i have a point here). I also have a list of machines and the list knows the type of machine (distro) and its architecture. So lets say a machine running debian, 64bit goes to get that package, what happens is if the "alien" package is available (yes for debian, no for redhat/centos/fedora and i dont know why, its a very useful package) it'll translate the package on the fly, so long as you ask it to. For eg, if my deb/64bit machine gets http://glcas/adhoc/webmin.x86_64.deb and only webmin.i386.rpm is available, it'll actually 302 the request to http://glcas/adhoc/webmin.i386.deb, then translate (using alien) the rpm package to a deb pkg. OR if a known machine asks for http://glcas/adhoc/webmin, it'll translate to that systems local package format (do a 302 to the appropriate url) and try and find an appropriate architecture (for eg, if a 64bit system asks for a package, it'll try get x86_64, the i386, then noarch). Sound kewl? Well hopefully it'll work the way I want, but we'll see!

After i've done some of that, and sometime in the future, i want to tie in xymon to my thing so you can easily start monitoring those unix machines via simple deployments - but thats far in the future.

Love That "Working" Feeling

2011-08-01T11:03:00.002+10:00

Theres nothing quite like the feeling of a project thats running and doing something you wanted to do.

GLCAS i've been working on now and then for the last couple of weeks. Its original purpose started cause I got sick of downloading CD's when i primarily build via PXE over the network.

But, the thing i've wanted the most is a local repository but i dont have terabytes of network bandwith to spare (space, yes, network no). This is what glcas solves for me initially. Its a local "proxy" of *ANY* yum or apt repository (actually apt doesnt work yet, but yum does). The point is that from a client perspective (build client or web browser) it looks like a normal repository. However, it only downloads packages that clients request. There is nothing out there that can really do this.

Basically what it ends up being is a very intelligent proxy (compared to squid). It knows repositories, it knows how they update and how they're laid out.

On top of that, it can also "freeze" a repo, such that if i update a set of staging servers, i can then "freeze" the repo, test my staging boxes then update my production servers with the same set of repositories in the same state. It works by just downloading repo's meta data when it needs to, then when frozen stops that process from happening. If however the client asks for a package that doesnt exist in the repo, it'll allow the download cause it assumes the client is asking for a package relating to the frozen state of the repo meta-data.

The most important part of all this is that it now works. It was tedious. the centos/redhat build installer uses (for no reason i can figure out) http ranges during the install. Trying to proxy this kind of scenario is a nightmare. The best part is that the installer is compliant to nothing and truely crap code. Truely painful working around that little piece of poorly written code.

Still, my poorly written code now works and i've built fedora and centos machines. Which makes me really happy.

Next, i need to make apt work, and i want to be able to populate repo's. If you want to check it out, its here: http://git.pjr.cc/git.php called glcas.

Google+ App's - A Word Of Caution!

2011-07-24T06:05:00.005+10:00

If (like me) your on google+ (even though you hate the whole social networking thing), and your using the google+ app for your phone, go thru its settings with a fine tooth comb.

I installed it on my "second" phone (android nexus s) and later found it uploads phone photo's by default. It wont share them by default however. Im not sure what happens on an iPhone with this app, but I wouldn't imagine its very different.

For me this is a very VERY bad behavior as I often take photo's of customer equipment while im on site. In the majority of cases if those photo's traversed the internet i'd find myself jobless and the company I work for would probably be on the wrong end of a lawsuit (irrespective of who viewed them). In my particular case, it only uploaded three of no consequence, but that is a truly horrific default.

Ultrasonic Audio Networking - An Android App...

2011-06-29T01:16:00.002+10:00

Way back when google spawned the android OS, like many google fan boys I downloaded their dev environment and set to with seeing how it all functioned.

Initially, i never liked coding much in the 1.x stack, it just really didnt feel wonderful and ultimately turned me off getting a phone based on those versions. To be truthful it wasnt even really required with the built-in emulator and the times i'd played with their phones back then made me think of them as quite clunky and slow (though i did actually end up with a G1 and android was also ported to my HTC tytn2).

Then came version 2. I liked this alot more, enough to get a HTC hero. Never really regretted it, but it was still pretty slow and clunky in reality. Then eventually came the 1ghz phones and things have been pretty good ever since. I've got a htc desire (original) and a nexus s. Both good, fast phones.

But, around the time i got a hero, i started getting back into the thought of coding applications for android. Initially I wanted to code a game, and online trading card game to be precise with some real-world territory fighting. But I suck at coding games and concentrated on things I was good at. The problem was, i just couldn't come up with something that I really liked the idea of and so most of my coding efforts never saw the light of day and that remains the way it is.

One day though, I was sitting at my computer wondering how bluetooth worked on android and it got me thinking about how information is passed around and how bluetooth (as a carrier for that) has been a total loss. Realistically, bluetooth should have allowed us all to transmit data between each other fairly securely, in reality it never worked. The sad thing is is that it could have, all it needed was a simple to use and intuitive interface for it - a universal one. That never happened and as such bluetooth's only real use these days is for headsets (or car-phone connectors). Along comes NFC, that may make things much more interesting really, time will tell. However, at the time I cant exactly remember my thought processes but it got me thinking "well, wireless is a bust, visual qr-codes aint bad, but what else is there?" and then i thought "hey you know, high-frequency audio might work"...

Well.... some time later I discovered several things:

The HTC hero just doesn't have the hardware to either transmit or receive high-freq audio (at least no high enough to get past the range where every k-nine within a click isnt going nuts).
Audio-frequency audio just sucked (too easy to interfere with and way too ugly)
High-freq audio just didn't seem to have those interference problems

It was quite simple to do really, you just listen for some carrier sound, which triggers an app looking to pull the data off the stream. At higher-then-people-can-hear frequencies you can actually transmit a fair chunk of data with very little noise loss. But you cant do it on any phone I've seen yet. The frequency capabilities of the phones just don't cut it. I did on the other hand have some very good audio hardware for PC's which let me experiment and prove the concept, but without it being useful to a phone, there was just no point at the android layer.

So ultimately i had an interesting concept but not way of really implementing it, which was fine it was a fun experiment down an interesting path. But the reason I bring all this up is because a couple of days ago I saw this on slashdot, which made me go "wow, thats what I was going to do 8 or so months ago". It also makes me wonder how they're going to make it work given my experiences with current phone hardware, imho, they're either going to annoy every animal with the ability to hear beyond what humans can perceive or they're going to add some hardware to the phone to make it capable of doing what they want. Time will tell and it'll be interesting seeing what they come up with though.

My Theories on the Universe.

2011-06-20T23:58:00.003+10:00

Everyone has a theory on the universe. Im a nerd, and ever since I was a kid I've always wondered whats "out there". Had a telescope when I was very young and so curiosity was easily born. I was also one of those kids in school that most people would call a geek or a nerd. Never been ashamed of that. EVER.

But like most people like me, I (at some point) started developing my own theories on the universe. Yeah I went to uni, studied engineering and had to do physics, and yeah i've read alot of things since then about the nature of things around us, as we believe them to be. Well here's my take on how I think things come together.

The reasons I post about this now are rather inconsequential. At some point I wanted to write a novel (which would probably never make it to the real world) of some sci-fi story based on my idea of what the universe is and how its made up. But, even if I never do that, I figure why not spurt out my ideas of what the universe is.... The question is, where do you start?

Well, let me start by saying I don't believe anything in the universe is random. Some may take what im about to say as "fatalism" and that would be partly true, except what some would describe as fate, I would describe as not going anywhere near far enough.

Big dislaimer here - these days I dont read about physics much, sure I was excited when they managed to capture an image of a planet around a distant sun, but to me that was no surprise (that they exist that is)

The Building Blocks - First, to me the universe is quite simple. Its made of stuff and its almost analogous to a binary world. The universe is basically made up matter and lack of matter, and that basic unit of matter is far smaller then we've been able to perceive so far. As in, take an atom, break it into its bits (neutrons, protons, etc), take those break those down, etc etc (do that about 50 times), thats where i think the universe begins, one minuscule chunk of energy. We'll call that "a". So you take a and another a, you get b. If you then take b and add a to it again, you end up with c. BUT, if you take 3 a's and add them together you end up with d. Point im trying to make it that the universe is basically on or off, but how you add those together changes what they make. I.e. its not just that d = 3 a's and c = 3 a's, its that c = (a+a)+a where d = (a+a+a). Ignoring the math analogy and its complete failure to actually describe what im saying, the basis of all this is that i believe the physical modelling of all this would be very simple and once you know what "a" was and the equations that goverened it, you'd know exactly how the universe came together. You could also find new ways of putting that universe together to achieve some goal. Ultimately though, theres no room for random.

How Big? How big? depends on how you define your boundaries. If you say the distance between the two fartherest chunks of matter we can "see", then thats one measurement. If you meant how far can you fly if you just chose a direction and kept going that way until you hit an edge? I believe thats forever. I dont believe the universe has a boundary, in fact i believe the universe is the sum of the matter or energy inside it. That the universe itself is simply a way of keeping track of the relation of that matter. This is why I hate the term "being inside the universe" because "inside" automatically denotes "container" in your head, thats not right in my opinion... However, my point is that if you could fly at any speed in any direction, the universe would never stop you, you could get to a point where you could look back and the universe as we know it would just be a single spot and you could keep on going.

Dimensions - Dimensions is a poor term, and by definition self-limiting. Most people could describe the dimensions we know and love, those being up/down, left/right and forward/backward (x,y,z) if you will. Many would also argue time is also a dimension (t). So most people could say that to describe somethings location you would define its x,y,z,t variables. I believe theres a few more. For starters, a time line (i'll get to this later). What else? god only knows (poor choice of words perhaps). But lets assume that to define a location for a chunk of matter in the universe we'd need to define around 10-20 variables to utterly pick that piece of matter. Lost and confused yet? I find it hard enough to actually frame the words im trying to write.

Randomness - This is a biggy to me. Assuming my depiction of the building blocks of the universe is true, then randomness is non-existent. Consider what this means for a second. If you could build a computer that could define 'a', then measure every single 'a' in the universe exactly as it was at one point in time, you could then define every single moment of the universe to come with 100% accuracy. Some would call this fatalism, but to me fatalism is what people use to describe how their lives turn out. What i mean is simply that what will happen in a hour from now is fixed. From the moment the universe came to be, every moment until its end (if there is one) was pre-defined, simple because thats the way the universe works. Is that a depressing thing? not to me, i cant define what 'a' will be in an hour and so to me the way time unravels is a mystery because we have no way of predicting every 'a' around us. But consider, if your reading this blog, that it was going to happen from the moment the universe was created (if you can say it that way). I want to distance myself from what people think of when they say "fate" here. This is not fate, this is just cause and effect. Consider how your brain works. Really sit there and think, what was it that brought you here. This is not an altruistic search for truth im asking for here, simple that what happens next is defined by what happens now because thats the way the universe functions. However, sit down one day and think about something that you recently thought of, and then work backwards. What i mean (for example) is today as I was playing a game called "infamous" because I got it free on the PSN. It made me think about being electrocuted. which made me think about death. which made me think about suicide. which made me wonder what happens to drive people to it. which made me think of going to work. which made me think of a website deployment i had to do. which made me think of ... etc. Sit down and really think about how your thoughts come together and you start to realize how you end up with something you might consider "unique" and realize its simply the sum of the thoughts that came before it. While not a good depection of how im trying to get across my point about lack of randomness - its just simply hard to describe what lack of randomness means.

The Big Bang - yeah, i believe the universe kick started with a big bang. Not one though, multiple. I dont subscribe to a multiple universe theory. Nor do i believe the universe forks into a bunch of parallel universes every time someone had to make a decision (i really hate that notion that the universe gives a rats arse about what people "choose" and then split into two based on the idiot thought patterns of someone like myself). Talk about your all-time egotistically ridiculous arguments, then thats the big one, parallel universes (at least ones based on individuals ability to choose). But anyways, the big bang i think occurred several times or at least, the way i see it happening is not "bang" and then there was the universe, but more a multi-stage event. The best analogy i can think of is throwing a chunk of water onto a very hot plate. Often that ball of water will boil away but throw off smaller chunks of water that are then more capable of being vaporized. This is similar to how i see the actualities of the big bang, that at the center of our universe (too far for us too see, or too hard for us to detect in a way we know how) theres a massive bundle of energy or matter that is seething away, expanding and contracting but too large to collapse on itself, and now and then casting off a small portion of itself with enough velocity for it to escape, contract and eventually explode into a universe itself. That is to say i also believe the universe to be significantly older then what we currently think.

Consider, billions of billions of years ago, this ball of energy casts off a portion that eventually becomes our universe (the one we can see), it takes billions of billions of years to contract upon itself to eventually explode and create everything around us. But, ours isnt the only one, so to call this thing a universe is a misnomer. It would best to describe the universe we can see as a cluster of clusters of galaxies. Now that big ball of energy somewhere out the real center is fairly undetectable. We can see the aftershock of "our" big bang, but how would you detect the parent body of energy that cast us off? who knows. Other bits of energy that have been cast off also do the same, form a "universe" (but again, thats the wrong term). But they're so very far apart light from other "universes" hasn't reached ours yet. Some of those "universes" are older, some younger. Basically, what i mean is that if you could pick where another "universe" was and you had a ship capable of it, you could get to it... That is to say its not a parallel universe, its just another island cluster of galaxies formed by a chunk of energy from some massively larger source and we're just one of them. More mundane then other theories, and well beyond my ability to prove, but an interesting theory none the less. Ultimately, the universe we see is simply an island, and all we can measure or see is that one island, even if we knew what to look for we might not see either that parent ball of energy or other "islands". Then again, the parent that threw us off might have also been thrown off in turn by an even larger body of energy. who really knows?

Aliens - yeah, i reckon they exist. BUT what really annoys me to absolute DEATH is what alot of people think when you say this. The first thing that usually enters someones head when it comes to aliens is things like abductions, crop circles and dissected cows (and your insane conspiracy theorist). That's not really what I mean and I think alot of reasonably intelligent people out there would see things the way I do. The universe (we can see) is huge, to consider that our sun is the only sun that managed to spawn a planet capable of harboring life and then the only planet which actually created intelligent life in my view is the absolute height of arrogance (quite often religious arrogance). So just taking our galaxy, I believe there'd be beings just within our galaxy capable of building ships that are capable of reaching us (possibly thousands of them). Does that mean they've been here? Possibly. But consider the reality of the situation. Our galaxy is massive, that they have found us would be quite difficult (aside from the massive amounts of radio waves we expel). Now if its true, and there are beings from other places capable of seeing us and have seen us, why would they come here? Almost all of the creatures on our planet would get away from humanity if it could, we are that bad. From an alien perspective we'd be a terrible bunch of savages who love to kill each other for the most peculiar of reasons. So why would they come here:

Resources?
Make contact?
Share information?
Slaves?

First, resources, if there is alien life out there hunting for resources then it means planets like ours cant be all that rare. If you had the choice of nicking resources from some uninhabbited planet or from earth with its possibly very problematic residents, which would you choose?

Contact - this is a hard one, why would you make contact with us? if you can travel the galaxy, you probably don't need anything we have to offer (thought sometimes I find the idea that aliens that can travel space need not be intelligent an interesting thought - more on this later). You might sit back and watch a few of us and let us know you exist, but to what end? to satisfy the curiosity of a few humans? i doubt it. Maybe, just maybe you might make contact with a few capable of describing earths origins and history.

Share Information - ultimately what are you going to share? the means to travel the galaxy? as an alien race, the last thing i'd want is humans infesting the galaxy (given the way humanity is at this point in history). Tell humans they're a bunch of ignorant savages? the ones that already know that dont need to be told, and the ones who dont know it wont believe it. Ultimately, alot of lessons you need to learn yourself.

Slaves - What use would humans be as slaves? we already have our own (i dont mean humans, i mean things like horses), and we've replaced them with mechanical equivalents. What need would slaves be except that they reproduce themselves and are perhaps cheap?

So what about the rest. Aliens crashing here? its possible but unlikely. Dissecting cows? very unlikely, we can already clone to a certain degree, cant see why aliens wouldn't do something similar. Alien abduction, even less likely, there's just so little reason to do so. Crop circles? when i think of this one, it makes me wonder that if some of these were alien-productions, are they possibly not alien kids running around in their learner-legal alien space ships gratifying a planet? If by some chance there are crop circles produced by aliens, i'd reckon that is the case, its alien kids out for a lark.

Which brings me back to another point. If an alien species were capable of roaming the galaxy, does it make them automatically more advanced or intelligent then us? On the whole I would say thats more then likely, but it doesn't mean some alien species may not have stumbled onto it by accident. So lets go back and assume intelligence isn't 100% required to develop space-fairing tech and that some race managed to stumble across it. Statistically, if some alien race managed to stumble on it, you'd have to assume that on the whole there's enough alien life out there that the majority got it through sheer intelligence. Which probably means the "intelligent" ones would contain the "lucked-out" ones, which is a reasonably safe assumption in my opinion.

The problem with trying to understand alien life is that its alien, there are some fictional novels that go a reasonable way to giving us a variety of alien life, but i believe so much of what we are is purely based on the human experience that we may never really comprehend what it means to be alien until we meet one. By that I mean, consider things like our 5 senses (poor term, we actually have more then that). How would life be different if we didn't have the sense of smell, or hearing, or sight. We may not even realize that its a dimension to our lives that missing until we've advanced far enough that we can create devices capable of measuring such things. The infra-red and ultra-violet spectrum's are very simple proofs of such things that we cant see and took us a long time to actually realize was there.

Then again, space travel may just be way too expensive no matter who you are that its really not worth it.

Thats enough for one night... more when i next get around to posting my crack-pot theories of crack-potted-ness.

Network Config Management

2011-06-12T04:42:00.003+10:00

Alot of my posts start with "a long long time ago..." and this is to be no different.

Quite a while ago (2003 i think) I was working on a client site where the method of backing up and version controlling configs on routers was (as is sadly often the case) cut and paste the config into a text file and save it somewhere on a drive (appropriately named). This whole scenario is common in the industry for network types cause router config's are simple, generally text based and not hard to deal with in this fashion. Where it tends to fall apart is two places, 1) when people forget to do a backup before they modify something and 2) the names people give the backup files can sometimes be inconsistent (including where they save them). Ultimately its a process thats heavily dependent on people doing the right thing, remembering to do it, and doing it consistently.

So going back to 2003 and this client site, I had a conversation with their network team leader and it was something unrelated, but he said to me at some point "well, you like coding for fun, write me something simple that can backup all my router config's once and hour". My response was something like "i only code FOSS for fun and not for my job" to which he said "i dont mind, you've got some spare time around here now and then, if ya get the chance have a go at it, dont mind if you make it open source at all, theres lots of router dudes out there who'll love it". So i was semi-intrigued by the prospect and that night sat down in front of my computer and hacked together a little perl cgi. The following day he was quite supprised that I had something for him, we ran it up on a spare machine (his mrtg machine from memory) and he was impressed enough to set it up for a fair number of his devices.

I never quite got around to open-sourcing that little beast, and over the years i've ran into a number of sites where this has come up and i've said "hey, i wrote this bit of code a while ago..." and to date, there's not been one site that hasn't gone "gee, this is kewl, we'll use it". In some ways it bothers me because in alot of those cases, my software holds logins and passwords to quite a few network devices out there. But over the years its been updated and fixed and even re-written from scratch once or twice.

So a couple of weeks ago, im at a site, a rather large company with many devices and their process was the old "cut and paste" routine. The network admin was actively searching for a bit of software and i mentioned the ones i've played with including the one i'd written myself. They're currently trying a few of those and who knows what they'll end up with, but the whole thing was spawned because of a deployment that shutdown part of their network and it was simply caused by a serial text console getting a glitch while a guy was cutting and pasting a config into the router and the router dropping a line of config. OUCH.

So anyway, it made me think about my code base and how its time i probably take another look at it, convert it to php and re-write again. This time however, i plan on sending it to the FOSS community. The idea is simple, it log's into a router, grabs the running config, saves it if its different from the version it already had and maintains it under some form of version control. The original verison only worked with cisco and netscreen (screenOS) but the new version I plan on making work with a variety of other devices. It also supports being able to manually update the config (where username/password are one-time password based, such as where tokens are used), emailing people when config's change and schedule-based updates. Not sure what else I can add at this point, probably ssh support (with keys), data encryption? maybe even agents that can reach deeper into a network to pull config's... who knows...

When i get it done i'll post about it here, would love to get people's feedback.

The Google Nexus S - its a mixed bag (no sd card? WHY?).

2011-06-03T01:38:00.003+10:00

I've been wanting to get one since it came out for its one stand-apart feature - the NFC chip. I've been an RFID interestee for some time and that the nexus s contained not only a 13Mhz compatible RFID card infrastructure, it was also a reader and write of the same.

THATS KEWL.

Google wallet aside, and sadly the timing of my blog is rather unfortunate cause my post has little to do with it. However, I'd wanted a nexus s for the NFC chip and because its the google flagship model, great for hacking. Ultimately, my HTC hero is just running out of ... everything. The hero was always a laggy fone, and its been "ok", but its never been great. The HTC on the other hand (my current main phone) always has - never really regretted this purchase (cept for the minor gripe of HTC being completely incapable of providing even remotely useful updates for the device). I digress....

As luck would have it, one landed on my desk two-three days ago for me to keep FOREVER, MUHAHAHAH!. So i left it in its little box and being all excited went home a little early. I plug the nexus is, give it some time to charge and then finally decide to give the thing its first run. So I pull the sim out of my hero, whack it into the nexus then pull the sd card out of my hero and... spend 5 minutes trying to find the SD card slot. Eventually I start thinking "oh, its a cheap chinese knock-off" cause I dont remember the last time I had a phone that actually didnt include an SD card slot. It looked so convincing too that I decide to google, maybe the sd card slot is hidden. No, its not hidden, the nexus S does not include a card slot.

Ya what?

So i read a few posts by people, some claiming this is good, some claiming it is bad and decide it doesnt matter either way cause in reality it doesn't have one and maybe that doesn't really matter. I want this phone to hack with, write android code and throw on a few apps like i used to with my hero. Plus, it has that fantastical NFC chip (more on this later). A day on, i've finally come to my own conclusion about the sd card (or lack there of) and whether its a good idea or not. In my opinion, its a very bad thing and let me explain why.

My HTC desire went for a long time completely untouched in terms of rooting and so forth until about 3 months ago i finally decided "I want to back this thing up". Now theres software on the market to do that, but the reality is there is nothing in the IT industry (im talking servers really) that can back things up like nandroid from a recovery rom. So I root it, s-off it and install the clockword mod recovery rom. The backup/restore of this recovery rom (nandroid) is every system admin's dream. If you wanted to backup your servers on a regular basis, you'd love to be able to just shut them down, whip the disks out, image them and then boot them back up again. But thats not plausible. Instead we end up with "enterprise backup solutions" which are all just work around to solving that problem. If you boot from SAN, its plausible you could reboot a server and before it gets past its bios, snapshot the luns on the SAN - but in reality that doesn't happen.

Nandroid however, does exactly that, and on a phone its entirely feasible simply cause it really doesn't take long. To give some context here, up until 3 weeks ago thats the only mod i'd put on that phone. It ran stock 2.2 and finally I saw a few things in gingerbread (2.3) that I really liked and had to have. So I spend a good 2 weeks downloading tonnes of roms and what i'd do is get home from work, shut my phone down, take a backup, wipe it, install a custom rom and play with it for the night (maybe 2-3 roms in some cases). Then i'd restore my stock rom. With nandroid this meant that as soon as I'd finished trialing the roms and did a restore my phone was EXACTLY the same as it was when i took the backup. This is such a beautiful thing. To be honest, the only thing i'm ever really conscious of loosing is SMS and maybe the places im up to on the games I play, but all thats fairly unimportant.

Now im not religious about it, but about once every 2 weeks I attempt to take a nandroid backup (have done since I originally rooted the phone). Then on top of this I also run mybackup (an android app) which automagically kicks off at night and backups up the majority of my phones settings. Thats good enough to make me happy really.

So what has all this got to do with the nexus S's lack of an SD card? for the simple reason that if I lost my desire, i'd buy a new one, root/s-off/clockwork mod (20 minutes work maybe) then just rip the sd card out of the old phone and throw it in the new, do my restores and perhaps within an hour i'll be exactly where I was the midnight before my phone died. The beauty of this is truely hard to fathom without actually seeing it in action. Every little setting and every little chunk of data preserved because my backups and a thing of beauty.

This is why the nexus s wont ever be my main phone. Im not about to start tyring to sync the contents of the storage on my nexus to somewhere else, cause that requires a little too much intervention and tbh, I cant really see myself doing that with any regularity. So the nexus will replace my hero as my frankenstein phone and thats fine, thats all it was ever really meant for in the greater scheme of things.

Im looking forward to sitting down and playing with the NFC chip a bit more then I have (so far all i've done is read a couple of mifare cards), the only thing that disappoints me a little about the NFC implementation is that is misses out on being able to emulate the older (and FAR FAR more dominant) 125k RFID card standard. I wasn't really expecting it would, but I couldn't find anything on the net that suggested something either way. The reason it disappoints though is simple because of the cards it can replace in my life. Right now It cant do my credit card, but thats not far down the track probably - and whats better (then a credit card) about it is that rather then just responding to anything that attempts to read it (even if the info is very secure), it can prompt you that its being read and let you choose what to respond with. But, if it had 125k cards, then the number of things it could replace in my life right now would be substantial and very useful. I've never really left home without my phone, but god knows how many times i've left RFID cards at home (yeah, i have too many of them).

However, the NFC is interesting for much more then just being a door-entry and credit-card payment solution. It can exchange data, and im looking forward to playing with it a bit. It should be fun. I also think NFC will be quite a storm coming in the industry, or google wallet will prove to be the abortion (along with paywave and pay pass, etc) or it. What disturbs me a little is that the SGS 2 does not include this (yet may include it later on on some other models). That really kinda sucks because of the fragmentation of an already fragmented market. If (as some suggest) the NFC will be embedded in SIM's (how they'll fit the thing in there is a thing of curiosity) and is as functional as the Nexus S's then its got a very good chance of continuing to live and prosper.

Personally I kinda hope NFC does take off simply for the information exchange capabilities it will bring to the table (the kind of exchange bluetooth let us down with), QR codes aren't bad too, but they are quite limited in storage capacity.

Still, other wise the Nexus S is an interesting phone. It feels light and plastic-y (to the point of feeling like a toy sometimes). The performance isnt bad, but tbh I feel like even my stock 2.2 desire rom (which i no longer run) was faster. It just doesn't feel like a hugely fast fone (its quick though), just I find im noticing the lag on it, where with the HTC desire i find i never do or did ( or at least, it feels less laggy even side by side).

Anyways, thats my 2c. Ultimately, the Nexus S aint a bad phone and if NFC becomes a day to day thing, you'll like the little things it brings to the table. Updates on the google phone are also quite inline with android releases too, unlike the HTC (@#$%#%#!) desire. Other then that, the phone isnt really mind blowing in most ways, its just good (better then the hero by a far margin, specially for speed), but still feels behind the original desire.

Rooting Android Is Bad... mmm'kay?

2011-05-29T03:23:00.002+10:00

Its hard to know where to start with this one, cause this is purely a conjecture piece. However, some time ago there was an article somewhere (i cant find it now) where google was arguing in favour of user choice when it comes to "rooting" android phones. Basically they were talking about how the phone makes (such as htc, samsung, etc) should rather then blocking access to the bootloaders and stopping people from rooting phones, they should leave choice in the hands of the people who own the phones. At the time, im pretty sure the response from the phone makers was "yeah, whatever". At the time that was no suprise, and here we are (not too much time has passed, which is why not being able to find that article is annoying) i wonder if that article from google should of ended with "... dont worry, we'll screw them over in other ways".

The reason I say that is simple. Over the last couple of weeks we've seen sony and HTC both backflip on their decision, but why? Well if you weren't aware, apparently google will have a movie market, but it wont be available on rooted phones. At first, I was annoyed but didnt care too much cause theres no way i'd be renting a movie on my phone anyways (not to mention it probably wont be available down here in AU anytime soon). But now, i've changed my opinion.

Consider, one of the reasons phone makers dont want you rooting your phone is cause the telco's dont. Alot of them dont like the thought of you tethering your phone for mobile data access (or didnt, in AU its never been an issue so i dont know if thats still the case in the US and the like). Now google themselves have set a dangerous precedent when it comes to how people can "treat" rooted android phones.

This is where it gets ugly. Originally I was annoyed cause google were very good at fighting for the rights of its users and generally respected/supported it hacker community. Then came android 3.0 - not to be released as open source. Sure they have every right to do that, but that was in reality a very different direction for them to take and not a good one. Now we have this "not for rooted phones" coming directly from google themselves and so anyone else is free to do the same with impunity.

The shame of it all is that it all comes back to google being very good at standing up for us. They were literally the company that (mostly) "did no evil" and you could actually believe that line coming form google. Now, its a lot harder to believe that line anymore.

Consider though, there are alot of people out there who would love to deny you services on your android phone based on the fact that it has been rooted but probably dont because of the popularity problems it would cause them. Now google have done it, its going to be (possibly) a free for all attack on rooted phones.

Shame on you google, you've done quite alot of evil recently and sadly you probably already know it.

Which Phone Next?

2011-05-17T03:34:00.002+10:00

Lately (for the past couple of months), i've been hunting around looking for a phone to replace my phone-of-choice, the htc desire. Its a good phone - still is a good phone actually. It says something about the acceleration of android that im already considering my next phone purchase. The Desire is fast, works well and does everything I need - so why a new phone? Cause I can. I've been with htc for at least the last three phones i've had and currently i run around with my htc hero (which is my screw-alot-with-it phone) and my desire. The hero is by no means a wonderful phone anymore and in reality never really was. Its slow, its clunky, etc, but it got the job done sufficiently.

However, I still cant help look at my desire (the original one) and wonder why it is I want a new phone. Well, there are probably 5 features coming out (or have cone out) that have made me go "oooo".

1. 3d - this is least important, or at least, playing with the nintendo 3ds has made me feel that perhaps this isnt the best thing in the world. It didn't give me a headache like some people are saying they get but I could feel it messing my eyes in a not-quite-unpleasant-but-certainly-noticable-and-not-good way. The effect is good and quite impressive, but not necessarily something thats of prime importance to me

2. Dual-core, and faster cpu's - this is somewhat important. In reality i've yet to find anything on my desire that makes it feel "slow", but that doesnt mean more speed will go unused. There is currently one phone availble in this placeholder (and i only recently realised) and thats the LG optimus 2x (the galaxy s 2 is also around, but the optimus beat everyone for that).

3. Quad-band 3g - only available on the Samsung Galaxy S2 (as far as i know), this is a very welcome feature as it means your not locked to telstra or "the others" out of the box. Basically, there are no phones out there (cept the S2) which can do both 850 and 900Mhz 3g.

4. NFC - near field comm's is and has been in use in the world (realistically) for some years now in the form of RFID access cards. People keep pushing this to the side and going "yeah whatever", but i think thats a mistake (unless, as some have suggested, sim cards start packing that tech onboard).

5. Upgradability - the ability to move to android 2.3+ and beyond with some predictability. This is something the desire has let me down for, 2.3 has been on the table for the original desire, but god knows when its coming - samsung seem much better at providing this, but google themselves (nexus s) win this little feature most easily.

In reality, it comes down to features for me, and my phone only really needs to do this:

make calls
send sms's

That is enough for a phone, and so my secondary feature set (for a smartphone) comes in like this:

Email
Browsing
Playing Games

Which the desire does.

Ultimately right now I have my mind set on two phones, the hideously expensive S2 or the LG optimus 2x because while the S2 has EVERYTHING and a nice UI (and brilliant reviews), the LG has almost the same feature set. The S2 that i'd get comes from the UK and hence wont have NFC. NFC is something i'd very much like to see, and hence i occasionally also look at the Nexus S as a possible alternative (cause its cheap). However, the dual-core nvidia tegra on the LG does indeed excite (my experiences with the nvidia chipset have been quite impressive so far).

So where does this all leave me? nowhere really. If i really sat down and thought hard about it, the S2 would be dropped either in favour of the Nexus S or the LG, but both those phones lack something i'd really like. Which leaves the next generation of phones coming out, some have NFC (not many sadly), some have 3d (HTC, LG), etc etc. Im ready to jump off the HTC ship for a little while and see how the waters are, but i'd really like a phone that has all the things I want. The S2 is close, but the lack of NFC and the large price are certainly detractors. However, with the next work refresh (which I could already do), maybe the AU version of the S2 will pack that NFC chip in. Who knows? If the Nexus S were going to be continued to be updated for each android release (3.x and beyond) then i'd probably get one just cause they are so cheap and it can replace my hero.

Ultimately though, i think im just going to sit back and wait. The market in the next couple of months will only get more interesting coming up to june/july as more phones are released (although, in AU they often take forever to be offered).

Just a little note on why I consider NFC important. NFC offers a feature to a phone that we currently already have but probably dont realise. For starters theres the pay pass bit (from visa/mastercard) which allows small transactions to be done contactless. Supposedly NFC will allow this. Then theres building security RFID cards that they could potentially also replace (what I read leads me to believe that this is already plausible if your using standard MiFare cards and the like). Ultimately though, that NFC tech will have some very real world uses that I think people without them may end up regretting - who knows, its a young technology (for phones). NFC on a phone will however make alot of things interestingly possible that we dont yet use RFID for. Google are pushing it a fair bit and for good reason, it has some interesting uses that shouldn't be discounted.

An MMORPG I Think I'd Like.

2011-05-16T03:26:00.002+10:00

I've played MMO's for a while, they're addictive, no doubt and i started not when they first came out. Instead I took the view that MMO's were a rip off of the existing MUDS we used to play (for free) and was somewhat convinced the subscription model wouldn't pan out in the end. So I was wrong - such is life...

The first real MMO i played with Dark Age of Camelot, and it was an enjoyable experience. So since then i've been playing some MMO or other. Currently I mostly play WoW, and its not bad really, it cops alot of flack for various reasons, but ultimately it has alot going for it. i.e. It just works and the game play gives you alot of options. Its not hard to play (or master) despite what people may claim unless your the first person to do something (which you never are). It doesnt matter what you do in the game, someone else has done it and figured out the best way to do it. Whether its completing a raid/dungeon or PvP'ing in any of the world PvP BG's or areas, you'll find something out there to tell you exactly what you need to do. The only complexity that exists in WoW tends to be along the lines of accounting for less-skilled players (or players who simply cant read/dont know what to do). Ultimately, if you know what your doing and the players around you do, everythings a breeze (unless its pvp, then its solely whos best on the day).

Anyways, i've played quite a number of MMO's. From DAoC, EVE Online, Star Wars Galaxies, WoW, JumpGate, Vendetta online, the list goes on, and some i've forgotten. But way way back when I left DAOC and was searching for something new to play, i started playing SWG, EVE, JumpGate all around the same time cause a futuristic MMO seemed like a good idea, and ultimately i enjoy that theatre more then fantasy (even if i do play fantasy alot more). Part of the reason I play fantasy is the failing in the futuristic MMO's. So far, EVE online i'd rate as the best experience and kept me going for a long long time, but alot of that came back to the developers continuous expansion of its universe. The rest were good, and they all had things unique to them that made them great, but not fantastic.

However, after playing SWG and EVE for a while, I sat down and thought about what i'd want from a game. I even wrote down some design notes and occasionally over the years i've added to them. Given I suck at coding games, I figure i might as well just dump them out here in case they inspire someone cause i'll never get around to it.

So first, the setting - futuristic obviously, space based and maybe some "run around the planet" content as well. The universe must be HUGE, as large as our own, and hence procedural generated (to an extent). The idea I had was to have a universe generated much like our own, only moving about 1000 times faster (such that comets would orbit solar systems in days rather then years). Then on top of that have it squished down in size so its not too far between systems. Next, i dont want to be limited by where "star gates" (as every space based mmo seems to have) can take me, I want to go anywhere (But i do also want star gates that get me places fast). I want to be able to build space stations, and mine planets, etc. So as such this component would be randomally generated (not procedural) such that when a person first starts to mine on a planet, the game chooses what they'll find there and thats how it will be to the end of time.

Flying around - this is something i personally believe eve got right, no twitch controls (well not to an extent alot of other space based ones had), ultimately I want people to pilot their ships with their mouse. In its initial release I want people chained up to a few small solar systems such that the game mechanics would stop them from plausibly getting the the larger universe until better drives were "invented". I also want people to "navigate" to a degree and find things. For eg "whats between my solar system and that solar system out there? lets send out some problems". Which brings me to the ships drives. Theres the normal "flying around" drives, theres the whole warp concept every mmo deals with. For me, i'd want drives that Jump you around. i.e. you hit a button and depending on what speed your doing and the direction your pointing, you instantly translate from one point in space to another with a certain degree of accuracy (but also with the kind of accuracy that other people can determine where you went) and also so you leave behind some form of signature (that fades over time) so people can see where one ship was and plausibly went to.

The Ships - I want to design my own ship. I want to grab a bunch of components and assemble them anyway I want until they resemble the ship I want. No size limitation (those size would affect what your ship needs, and how it works), and I also want to be able to obfuscate my design such that I can build a design, and if I like it, i can produce a blueprint of it that i can mass produce, but the components that go into it are unfathomably hard for others to guess at and reproduce. Ultimately, If i come up with a design people like, i'll be rich (until someone figures out my formula), if not, i'll buy someone elses. I want to be able to build fast (very fast) light ships with negligible shields and almost no fire power and I also want to be capable of building ships that are the size of small moons (and beyond). The whole design would have to "add up" such that if I build a huge ship with massive guns (or massive mining equipment) it'll require massive resources and a massive power plant (and move like a truck).

Crafting - Crafting should really be two things, coming up with designs and manufacturing them. The game itself should give you some way of gauging the success of your ship based on how it performs but not in hugely empirical ways. Just enough so that people can compare their designs such that someone could become known for the designs they create. But most importantly, to gauge that success you'd actually have to build your design and test it out. So you could make money in the economy just by coming up with designs that people are looking for. You could also make money just being a manufacturing house (an automated one). I should be able to go to some guy who builds crap, hand him my schematics and him build me a ship without him knowing how my design came together.

Economy - The one economy i've loved so far was SWG's truely player driven economy - while porely executed, it was a brilliant concept. EVE did something similar and claims to be a player driven economy, and to an extent thats true, but it never went as far as SWG did. What EVE did do very right, is hand alot of control over the economy to players in the form of complexity in the building/research of blueprints. My economy would have almost zero npc input. NPC's would have the very fundamental building blocks for creating the building blocks you need to survive in the universe (along with some very basic, get-around-the-universe type ships, but nothing else), and items you need to build those blocks (ships, manufacturing equipment, etc) would come from players, and not just random chance either - the ability to secure a source of some valuable item you need to produce your thing should be important, you should not have to rely on some random mob to drop things.

NPC's - The universe should exist as a controlled sphere of competing governments (nothing new there), some good, some evil, but all of them open to players joining their personal struggle (i always wanted to join the blood raiders in EVE, but you couldn't really). And similar to eve, there'd be portions of space that are highly controlled by the government and other portions which are totally uncontrolled (lawless). It'd be up to the players to take control of such places and become governments unto themselves. NPC's though need to provide lots of different styles of interactions - quests, purchasing items, faction alignment, and so forth. Being aligned to a faction would give you some set of resources available to you that aren't available to other factions - thought not to an extent that the faction your aligned to controls the type of ships you can build or design. Some factions must compete, and hence being aligned to one would give you negative alignment to another. There also needs to be "mob's" that you shoot at and destroy, just like any other MMO, and they drop things, etc.

Death - one of the things I have never liked about any MMO is how it deals with death. Its always one of two ways and the end result is identical for the most part. In futuristic MMO's its clones, you die, you wake up in a clone. In fantasy its some mythical ghost thing that just spawns you somewhere. The penalties for death vary, but ultimately its all the same thing - you dont ever really die. Not in my MMO. Death would be very hard to achieve. Space ships would teleport you to the nearest space station (if your in range of one) before you die so you be safe. But if your not in range of one (deep in lawless territory) your dead, and your character is gone. But breath easy, cause its not over. My concept of death would be to pass on (inherit) your worldly possessions to a next of kin or apprentice. As an apprentice, they'd be training up in the things you know, and hence your new character would have much the same things you already had. The government would take its cut in the form of inheritance tax. This adds some interesting little features. Lets say you're not quite happy with the way some of your character works then you could hire an apprentice that has the features you want to improve and then when you die, you get a new character (but you cant reuse your old name). If you dont want to hire an apprentice (they would cost on-going money, but provide certain types of assistance) then it'll default to your next of kin (who's name you can still choose) but other variables of the character would be randomly generated by the game itself. Thus, death would have its penalities, but provide some forms of feature that you may well want. It would be intersting to see the "family tree" that might develop from such a concept, thats for sure.

Community was one thing I was particuarly intersted in, cause its a central part of any MMO (by design). The community around WoW is pretty dull really. There's like the people you play with and the the forums and such, but ultimately the community really doesnt have a huge amount of impact on the game itself. People might complain that class X is underpowered, and the dev's may react to it, but the community is really loosely knit in reality. You might have people with 10 chars in 10 different guilds for example. EVE's community was brilliant, and in part due to its empire-building nature. Empires got MASSIVE and so did the not-quite-so-in-game games around them, forum wars developed, and empires could be lost by simple things (like corp spys). IT was brilliant in its simplicity. Some people hated it, but there was a sense of belonging to a (player controlled) organisation in game that no other MMO has had. Then there was the player component of the game-control facilities, such as a heavy involvement by players in testing/debugging and sorting out other players issues. It was quite very cool that they invovled the community in this way - something worth taking on. My community though would extend this to a player controlled board which would have a large amount of input into the building of the MMO itself. some player might come up with a concept and eventually it gets to the player controlled board who decide whether its worth voting on and putting forward as something "they want in the game". Every 3 months, the player controlled board would get to push forward 3 new features of the game they'd like to see implemented and the game dev's would then decide which one would make it (but also veto some). For eg, the player controlled board might come up with concept a, b, c and the dev's would go "we'll implement c, veto a, and b we can do if you want, but push that again in three months. The community controlled board would also be able to vote on features the dev's want to introduce. Not to the extent they could stop things from happening, but at least have a strong say in it happening or not.

Universe Content Control - People should be able to build (and destroy) structures that become permanent parts of the game (much like EVE's later station bits). Builld a station in a good location (such as on a comet orbitting a star, if thats what works) and you get a popular station that makes you money. Who knows, but the ability to put something in the universe that might be permanent and leave your mark forever would be nice.

PvP - I loved EVE's PvP in some ways, hated it in others. Same with WoW. In WoW, the only penalty for failing in PvP only related to some lost time, in EVE, there were always serious losses involved in death (well, typically anyways). There needs to be a middle ground (or both grounds). The ability to have fun with PvP while choosing that risk level such that you can risk lots or risk nothing. If you go into lawless space, you'll get in fights that'll end with you loosing your ship (and perhaps your life), but there should also be in-game comp's much like WoW's (and others) BG's, such that you dont have to risk everything in order to have fun (EVE also had the testing universe, which allowed you to interact in such a fashion, but it had numerous draw backs really).

Other bits and bobs. The UI of EVE was very nice and sharp (Every other MMO has an MMO much like any 3d fps really, the fonts are average and the game looks like a game from the 90's, eve did not), but lacked any ability to customize it (thru addon's). My UI would retain that sharpness, but allow addons much like WoW did. I'd also want to have some standard protocols wrapped around the thing (such as XMPP) so that you could communicate and perhaps even control some parts of the game thru a standard XMPP client but require the full game in order to interact with much of the universe. It would allow people to interact with the game when running the full game wasnt plausible, but require them to login often enough to keep them "going". The veteran concept of EVE was also quite tiring, no matter how much you played, you'd never really catch up to players who'd been around longer. While that (in some ways) wouldn't matter, it often determined how functional you were (meaning, you could have fun even as a 1-month old char, but you had more options to explore in terms of fun the longer you were around). WoW on the other hand is almost a polar opposite, every time a new expansion comes out, everyone almost goes through a reset and starts from scratch - almost. Ultimately though, being as good as the guy next to you really only takes time and eventually everyone hits a "cap" (except perhaps with gold, but each expansion makes the amount of gold you make over time substantially larger).

Well, anyways, if I had an MMO, that's what i'd build - and maybe in the years i've been away from EVE perhaps its grown some of those features, who knows. Then again, just cause i'd like it doesn't mean it would even work!

Android 3.0 open source - im not sure what to think

2011-05-12T03:16:00.000+10:00

Listening to the Google I/O conference, its been a mixed bag of disappointment really. Alot of the new features sounds interesting, and some sound good (android @ home for instance). But what bother's me alot is the remarks make about androids open source nature (or lack there of).

Let me first prefix all of this with this: While i've been following honeycomb, its not been closely and so some of what i say may come from a failure to understand the true nature of the beast - if that is so, i apologize and in all likely hood i'll understand my own failure to understand soon enough.

Android to me was a hackers platform - i have two main phones and several other devices that currently run android, only one of those phone do i not "screw with" cause its my main phone i use for day to day use and is somewhat important to me. But since 3.0, android's stance on open sourcing their product has been disappointing to say the least. 3.0 is out there, running on a variety of devices, why dont we have access to it? Well, if you listen to the inane ramblings of various retards at google its cause "the phone bit is broken"... who the f**k cares? Your worried about a bunch of rom hackers trying to cram 3.0 onto a htc desire or something? what if they get it to work, is that going to be embaressing to you google?

Why the HELL are their devices out there running 3.0 and we not have access to the source? dont get me wrong, its not a license requirement for the actual platform to do so, but its a HUGE kick in the teeth to the hacker community. I want to buy a xoom then screw with it, because thats EXACTLY why i'd bother buying one. Honeycomb is tablet focussed? excellent, i'll compile it and put it on my toshiba ac100 - or i would if i could get at it.

What really ticks me off though is this particular set of comments for Andy Rubin (though, they are edited).

Everything we do ends up in the open source repository. It happens in the open source, when the first device is ready, running that platform, and let me explain a scenario to you. We're building a platform, we're not building an app. When you build a platform, the developers have an ecology of APIs; they aggregate APIs, they deprecate APIs. We're always adding new functionality…

So when we add new APIs typically, in my opinion, community processes don't work because it's really hard to tell it's done, it's really hard to tell when you're dealing with what is a release and what's rejected. And when you're doing a platform, that doesn't work because developers have to have an expectation that all those APIs are good and complete at a certain date. If it's a community process, an OEM or an operator or somebody could take an early version before those APIs are locked down, start building devices with it and those devices could be incompatible from a third-party application perspective.

So what the hell does that actually mean? The way i read it, andy is tyring to say once a product is running the version, it'll be OS'd. Which (if true) makes him a liar. Now I can get that API's will change while your developing a code base, but now that 3.0 is in the wild, that time has passed. People MIGHT try and whack 3.0 on a phone, but without the actual phone capability, no ones really going to use it. They'll put it on their phones, go "oh so this is what honeycomb is all about" once they see the interface and then revert to something thats useful to them (which still might be honeycomb).

Ultimately, google need to look at the likes of redhat and ubuntu for when it comes to "generating a platform" cause they get it right. They do the correct thing by their hacker community. Sure, i may not be able to touch unity while it was being developed, but it hit the source code repo's at a fairly early point. Was it miss-used? no, it wasnt, and neither would Android if it were pushed out in the same way. It really is a kick in the nut's to all of us out there developing FOSS software to see google "bend" the rules about their own "open" platform. Theres a certain amount of "spirit" that people just understand about what it means to be open that google have just burnt at the stake. Its not about google doing the right or wrong thing when it comes to "open source" here, its about the people out there who know what the "spirit" of open source truely means. People like me, who thought google not only understood that spirit, but even fostered and nurtured it. So screw you andy rubin, google and your pathetic little excuses.

Lastly, I wouldn't mind any of this except i really took the android platform on because it was open source (in spirit, not just name), it was what put me on the platform and now i might as well be on an iphone, cause at least they're not running around with fairly distinct double standards on that component of their platform (the app store, maybe, but not the platform) which was important to me.

ARM - the end of Intel and VMWare but not AMD?

2011-05-08T04:48:00.003+10:00

I blog about arm a bit, thats because ARM is really a very interesting technology and likely to get more and more interesting.

But what impressed me recently was someone showing me something thats "in the works" that made me sit up and go "wow". First, its no secret MS have been porting some products to the arm stack, and not just for their mobile's business.

But, what I was shown was something a little different. The Sheevaplug/guruplug are fine examples of where ARM technology is going to head eventually, and it'll be painful in alot of ways, but you'll eventually learn to love it. ARM based servers/desktops are not far away at all and they have alot going for them. First, they're small and light. They dont really suck much power and they have enough grunt to get most jobs done. They dont go for a balls and all approach to computing grunt that is rarely justified at a time when the industry is becoming very (forcibly) focussed on parallel computing (multiple cpu's and multiple cores). This is where arm's cheapness will really win it over.

Consider the size advantage of the sheeva/guru plug's. They're board is really only the size a credit card and not much thicker - except for the connectors. Which brings me to another thing ARM needs - a universal connector. Let me go on a side-track here. I said some time ago that SSD needs its own connector, SATA (and SAS/FC) just dont cut it really (and i also said seagate should stop suing people and produce a disk that has both spinny platter and SSD and work like HSM - which now exists! yay me!). But when you look at the connectors hanging off your computer these days, they just dont need to exist anymore. We have ethernet, usb (3), sata, hdmi, etc. They all provide an almost identical type of connection but are architected for their given purpose and its really just not justified anymore. What we need is a connector that can be SAS (provided the hardware is capable of it), sata, usb, ethernet and video as well as service and SSD the way it needs to be accessed. This (in part) is something I think ARM will provide. consider if we had a usb-like connector hanging off the back of our little arm server, then its no longer much larger then the board itself. It needs about 20gbps of thru put and it needs to be able to function as all the types of connectors we have. Thats not hard and its something arm based servers will need. You can think of it like the way infiniband should have functioned (except for how useless it was - or at least the implementations of it that turned it into a dead technology). Consider the following machine profiles (and the connectors you need):

Desktop - USB, 1 disk, display, ethernet
Web server - 2 1gbps ethernet, 2 disk ports
Database Server - 4 really fast disk holes, 2 ethernet holes

My point is, we need a connector that can do all that and allow us to choose. So a universal connector that provides 20gbps of thru-put that can be partitioned off into any form of connector is a massive win. Consider the possibility of being able to aggregate multiple universal connectors to one server (say a server with multiple cpus, more memory and more cores).

Consider the following setup, a 1ru server would be capable of supporting around 40 of these little server (each with quad-core cpu's and 4gb of ram), in total sucking around 300w of power. Internally the servers are connected to a managed backplane that presents to each server the type of connectors the server wants to see. Off the back i have a little lom port for getting displays over the network for each of my servers, a connection for disk and a connection for network. We then simply trunk our switch to this box and present each box with a network connection to the vlans it wants to see, and we also control how disk LUN's are presented to each ARM server. Thats where the universal connector would shine. But even kewler, I'd just like a little box where i plug my arm server in, and out the back it has a set of pre-defined connectors (2 ethernet, a display hole, 2 sata holes and a couple of usb holes) - just for when i need only one server. And maybe you can get a few of these boxes that give you connectors directly on the back of your arm server in different configurations, who knows.

My ultimate point being that in 1ru you could have 40 servers. Freak! considering alot of software these days has the capability to run parallel to itself (such as exchange, LDAP, mysql, etc) having a vast array of machines is not necessarily useless at providing scalable grunt. So long as ARM heads to a place where multiple cpu's with multiple cores exist then that place where you need tonnes of grunt is plausible capable of being serviced even on ARM. But, if that all did happen, if we got useful arm servers that were ridiculously cheap (which they are), why would you ever need virtualisation again? hence why i say VMWare is likely to go the way of the dinosaur should ARM tech really hit the server market well. In 4ru i've got 160 servers - and im probably not paying alot for power, cooling, etc.

The reality is you could take alot of smaller companies these days onto arm based servers already cause they can already provide the grunt needed to do things like run exchange, sql server, IIS, etc fast enough for small companies. They'd also take up far less space and be cheaper to run. The threat to the x86 architecture is quite real in that space.

But, why do I saw not AMD? Because AMD have proven time and again how flexible they truely are. Everytime the industry moves in a direction, they're there. We dont call 64-bit x86 amd64 for no reason. Intel tried to hoist itanium on us, when amd64 (or x86_64) was the correct way to go. AMD always just seem to have a flexibility (and common-sense approach to computing) in their architecture that intel just never seem to have. Every generation, Intel have bugs and problems (6 core memory bus problems being the best, latest example). AMD never seem to suffer the problems intel do. And recently I got a board which can best be described as AMD's answer to Intel's atom, but done right.

The board I picked up was the ASRock 350M1 - its beautiful. Rather then taking a hammer to the silicon like intel did, they (AMD) used their existing architecture to produce a fully function x86 chip that is not really any different from a phenom or similar. The intel atom was a complete redesign and a bad one at that. The AMD version has everything a normal board would have, cept the silicon's strapped to the board. But it works and its brilliant. Mine is sucking about 7W at idle (even with a spinning 3.5" disk) and rather then having a bus that cant really handle much, it has something fully-fledged like you would normally find on any machine.

Make no mistake, if x86 has a future, its with AMD. When AMD can make things like ASRock's 350M1 possible, it has a real future that intel doesnt seem capable of providing.

Intel Atom Stand Aside - AMD, You Rule.

2011-05-05T01:11:00.004+10:00

When it comes to intel vs amd debates I often find that i dont end up really siding with one side or the other, alot of linux types tend to love AMD and i've never really understood the reason for that.

Intel field a wide range of CPU's and while they've had some truely terrible offerings (hello prescott), they've had some very good ones too (core i3/5/7 for example)... However, where I tend to "appreciate" AMD is more in their architecture. They (seem) to make more flexible architecture which never really has problems of scale, where intel always seem to have to redesign from the ground up everytime the world of computing heads in a new or interesting direction (64bit - hello itanium, multi-core - hello intel's 6-core problems, etc etc).

Then Netbooks became really kewl and so intel invented the Atom chip. This was a truely gutless chip designed for low power (though surprisingly it wasnt really THAT low power), but it did serve a useful purpose. Sometime later again, intel started producing mini-itx boards with embedded atom chips presumably cause that had worked for via (And the epia series) for some time. I got one of these things (but i cant find a link for it). its a gutless board (gutless, not useless) and you can feel its slugishness compared to something with a bit of grunt - but thats not a bad thing, you dont buy this board for its power. But the sluggishness always felt there, its like a kind of a lag you feel similar to when you use a java app, it just doesnt feel responsive in a way thats hard to really put your finger on. So much so that i spent some time comparing the atom to the epia MB i had (from 2002). The epia sucked a little more power, but managed to be more then half the speed. So thats what the atom was really, not alot of progress from 2002... yay.

Well, while the atom entered this world with the heralding of trumpets, amd recently entered the same game and i went completely unaware. The CPU im refering to is the E350 and its NICE. VERY VERY NICE. The Motherboard i have it on is this one and it works very well. Note the specs tho - ati radeon card on board, pcie slot, usb3, dvi, hdmi, esata, optical audio, 16GB max ram. The 945gclf2 on the other hand shipped with usb2 (fair enough at the time), no hdmi, no esata, no optical audio, a pci slot (PCI? REALLY???) intel (whatever) graphics on board and max 4gb of ram (32 bit memory bus, so you still suffered with a 32bit OS's 4gb memory limit issues even with a 64bit os). Currently my intel ATOM board does fulfill a useful purpose however, its sitting at work being a crashplan pro server supporting about 10 users and connected to a SAN via a PCI (bleh) fibre channel HBA, so its not useless, just gutless.

Not long after those initial atom boards, other people started entering the game with the ION boards, which was essentially the same board, but with an nvidia graphics card, which would have made the atom a nice media box, but they were expensive... 230-ish (AU). This board from ASRock with the AMD chip cost less then 150 (AU) and is quite substantially more impressive in its feature set (the original 945gclf2 cost me more then that from memory). But more importantly, the cpu is noticeably faster. There are some places where a nice graphics card will make up for a crappy cpu and nerds like me know exactly where those places are, and I can FEEL the snappiness of the AMD chip in comparison to the atom.

There are other little niceties too about this board and its bios, like (not that it matters really) it has a graphical bios with some useful options i've not seen in alot of other boards (like delaying startup on the bios). The board just feels nice. The CPU, rather then being a complete redesign and cut down like the intel has all the normal things a normal AMD cpu has, a proper memory bus, virtualisation extensions, all the normal things.

So running the same Schnell_pi test i ran in the earlier comparison, i get this for the AMD:

./Schnell_pi

*********************************

* *

* Schnell_pi version 1.0 *

* by Dominique Delande *

* *

*********************************

Computation of Pi using the Gauss-Brent-Salamin AGM algorithm

Number of digits is a power of 2

Memory needed is roughly 10.375*number of digits

Please enter the power of two for which the calculation will be done

10 ( 1k digits) 17 (128k digits) 24 ( 16M digits)

11 ( 2k digits) 18 (256k digits) 25 ( 32M digits)

12 ( 4k digits) 19 (512k digits) 26 ( 64M digits)

13 ( 8k digits) 20 ( 1M digits) 27 (128M digits)

14 (16k digits) 21 ( 2M digits) 28 (256M digits)

15 (32k digits) 22 ( 4M digits) 29 (512M digits)

16 (64k digits) 23 ( 8M digits)

Calculation starts Thu May 5 02:16:51 2011

...................

Total computing time = 14.460000

Calculation ends Thu May 5 02:17:05 2011

Now printing final result...

Wow, 14.4s compared to 39s for the atom. More then twice as fast. No wonder if feels like it.

Currently, the machine is using about 12W of power under load, which is less then half what the atom used. Keeping in mind that this machine currently has a 3.5" hd, where the atom was using a 2.5" hd.

So to wrap up a little, why is this little board not getting the attention it deserves. Its better in every aspect, it has the kind of things you really want (hdmi, optical audio, esata, usb 3, a REAL video card, a REAL cpu), operates more then twice as fast, and uses half the power while being actually cheaper. People should be singing the praises of this board from the hills. Its just such a nice board.

PS Network Outage - Its Just a Little Odd Really

2011-04-27T01:19:00.003+10:00

Anyone with a PS3 (or perhaps even a psp) will know that right know the entire playstation network has been offline. Curiously, its been offline for the entirety of easter. For us here in Aust, that means an entire 5 day weekend without an online component to our console.

For me this hasnt really had much impact. I've not been playing Black Op's much lately and this weekend i've not really felt the need, thus not being able to play online (the only way i play it anymore) hasn't really been a drama. Im not a plus subscriber, I dont use the ps3 to rent movie's, download music, etc and I dont really have many drm protected games i've gotten from their network (i've not even tried to play any of the 2 games i have there so I have no idea if they're effected and dont really care that much). I did however get Portal 2 when it was released and its a great game, but theres not much in the way of online content and so it hasn't bothered me there either.

But, for Sony, this outage must be devastating. There will be an absolute boat load of people (both plus and non-plus) who will be livid (and rightly so) that they've not been able to play online for the entirety of their holidays (when they're most likely to want it). I wonder how many people this long weekend have run down the road and picked up an xbox? If there was any reason to need the PSN (and there are many), this outage would have you seriously thinking about your choice of platform. For me this simply hasn't been the case.

What bothers me about the whole thing though is the timing and sony's inability to actually fix the issue or even provide anything in the way of details at a time the community will be screaming for them. Make no mistake, people at sony will loose their jobs over this - not just cause it happened, but the way its being dealt with.

But, back to the timing of the whole debacle. The only real information sony have thus far provided is that it was an external intrusion, which is about as useful as .. nothing really. Which leads one to speculate, and the longer this goes on, the more it sound very dark and conspiracy laden. Consider the reasons for breaking into the PSN (or anything really) broadly tend to fall into a couple of categories

Denial of Service
Financial Motivation
Leverage

Denial's of service are a tricky one, they tend to either involve someone with a grudge wanting to take you off the air because they dont like you and doing so over easter would make sense because its when its going to be most disruptive. They usually dont result in extended downtimes though, usually the cause of the DoS is found and destroyed (or at least, worked around). As no real data tends to be the target of such an attack, they're rarely designed to hack a system for such purposes and indeed hacks used to DoS are typically quite different from those used to steal. The point being that a DoS attack (which may involve an external intrusion) is typically not as problematic to deal with and no one in the right mind would bring down such a vital service over the easter holidays to deal with it. Secondly, alot of DoS attackers look for credit, some people have even looked at Anonymous as possibly the reason why this has happened and they've denied responsibility. Hence, i find it hard to believe this is simple a DoS.

Financial motivations are much more broad. Lets start with competition, and by this I mean "the PSN is down? go buy an xbox/wii, we're 100% operational and will be all easter". There are other people/companies who would also stand to gain from such a maneuver, but thats too broad to list. If you were such a company looking at taking down a competitor your likely to not bother trying to steal user information/CC details and the like (they're of no value to you and you can use them). What you want to do is complete destruction and maybe steal some IP about how the PSN operates such that it may be valuable to you. You would distance yourself as much as possible from it and do whatever is necessary to make sure you cant be blamed at all, but you'd want to take it down in a way that it stays down for an extended period where its most likely to be as damaging as possible. If fits the scenario, but is extremely implausible. No competitor in this space would dare because the financial burden of being responsible for such a thing would be so catastrophic that you'd be able to pick the pieces of whats left of your company from the lint in your dryer at home.

Then theres user details. Identity theft, CC details, the works. Theres alot of information on the PSN that would be of use to you if your simply looking to steal money and user ID's. You'd get in, get the information you'd want, then try and remove all details that you ever existed to make it as hard as you can for someone to ever identify you. This could mean trashing the PSN as much as you can, or it may not. Again, it would fit the scenario in alot of ways and is possibly the reason for the outage. But would sony leave the PSN completely offline for the duration trying to diagnose and repair? Its possible, but who really knows. As an admin, restoring a service from tape usually doesn't take more then a day even in the most complex set of systems (though it does take man power). Identifying the point of entry can be a little trickier though, and so maybe thats where they're stuck now. But some duty of care would lead sony to make some announcement to the effect (one would assume) telling people to do something about their CC details or that their user information has been stolen. God only knows whats going on inside sony, and sadly they're one of the worst organisations at releasing information. What strikes me as odd however is the timing. If you could break into the PSN and get such information, you'd wait till after easter (or at least, not do it before east starts) cause thats when there's likely to be an influx of information thats useful to you (cc numbers, etc). Sure, theres alot on the PSN already, but more is better.

Lastly, theres leverage, and thats where this small part of me starts to wonder at the conspiracy theories. Consider your sony and your watching the xbox platform thrive despite the fact that your free to play games online and xbox charge its user base a monthly fee (personally, i think its deplorable that microsoft did this), xbox are making a fantastic amount of money out of this. But sony themselves are unlikely to go "ok, we'll start charging people and here's how we'll do it" by trashing their entire platform and downing themselves for the duration of an important holiday.

Which brings me to the third parties that could be involved here. I want to stress i dont really know how the PSN operates from this point of view, so take anything i say with a grain of salt. Consider though that you operate the PSN on behalf of sony. You may see a cut of everyone who buys a plus subscription or you may not, who knows. One thing for certain though is that if sony have more money from subscriptions (which is likely if they made online gaming non-free), your going to ultimately win out. You'll be able to ask for more money simply because sony themselves are making more cash out of the enterprise. This may not mean you go and pay someone to do whats possibly happened here, but perhaps you just make it easier for it to happen, it only takes one hack in a system for someone to get it. Chances are too that if your running such a system your always complaining about the hardware involved and how your running on a shoe-string budget and you almost always have a list of problems you can turn to if such an event occurs so you can say "told you so" and then ask for even more money to fix the issue. Keep in mind that if i were in such a position as running the PSN, i'd always want something i can turn to in the event of a disaster to limit my liability - not for any malicious reason, simply cause I dont want to appear like i was doing anything less then 100% of my job. The thing is, the even will blow over eventually and people will come back to the platform. Its unlikely that the even will cause more then a couple of percent of users to switch platforms so the damage (from a service provider point of view) is fairly minimal in reality.

There are other ways of looking at the leverage point of view with a critical eye too, but theres some part of me that thinks this whole thing is someones attempt at pushing sony into a paid-for model for one of the most important components of the platform.

Again, its entirely speculation on my part and im probably 1000miles off-base. But this is the blog of various ramblings and half-truth's dont forget, named as such because most of the time I use this place as a sounding board for offloading whats going thru my head. The PSN being down has had very little impact on me personally and god speed to sony in getting it back up! Sadly though, this event is only going to have a negative impact to the user base, of which I am a member.

Web Admin Gui's and CSS Templates

2011-04-25T03:24:00.002+10:00

As a guy who codes a fair bit, and one who is often coding system type stuff that has a web gui I often find peoples design choices for web administration gui's quite interesting. In my job im often fulfilling the administration role and despite being a heavily unix oriented (and most people would then assume command line driven) person, i appreciate a good gui. Often though, i find that gui's disappoint as often as they impress and most of it comes down to design.

When I code a web gui, I aim for some simple things:

speed
simplicity
functionality
easy on the eye's (think eye-strain, not "pretty")
works anywhere
low on resource usage.

For me, this means coding bare simple html with as little JS as i can get away with. I also expect that from other people who code web gui's. One of the things i find a little annoying though is that if i do code standard html, it never looks pretty and when i look at what css provides it really shouldn't be that hard to make any gui that does use boring old html a bit prettier (or at least, easier on the eyes). By this what I mean for example is take a normal table, without a border it can be hard to follow, but the standard html border is too weighty - easily fixed with css. Yet, there are few css templates out there that exist to acomplish this simple goal. So far, i've been boring the one from the webmin (linux web-based gui admin tool) to do this, and it does a fairly decent job simply because all it does is redefine the look of the standard html tags.

But while writing a gui for my latest attempt at coding a systems admin focussed tool (glcas) I realised how much i truely hate some of the web gui's im exposed to in my day to day life. So let me give a few examples of admin gui's that i think are truely great.

First theres webmin itself. The software tool is fantastic - can control almost anything in linux and copes with lots of different software configuration possibilities that each vendor (ubuntu, fedora, etc) use. Its also fast (at the web gui), uses almost zero resources at the web browser, is functional, simple and easy to follow. This is what makes it great. as an admin, any time im wasting waiting for my browser to render is wasted time and if i have to click 50 times to find the configuration item im after then someone should be shot.

Another great one is xymon/hobbit/big brother. This product is a good example of something that people have open all the time because its a monitoring application and thus tells you when things are going wrong. The web based component is briliant. As someone watching their infrastructure, when things go wrong its so easy to drill down into components with a minimum of fuss. I can get to graphs, error logs and everything within about 3 clicks most of the time. The gui uses almost zero js, is fairly boring html with zero tricks. its not a pretty page, but what it accomplishes is exactly what every admin needs. As such, xymon has very often been the benchmark by which every other web gui i've used has been measured and there's been none that have come close to it in those terms. If you've not used it, then your missing out in a way cause i defy anyone to show me a better layed out gui then xymons. You will never find yourself clicking more then 3 times to get from the front page to the most detailed graph you can find in the product (i.e. where the most useful information lives).

So to be fair, let me describe a few that i always find "wanting". Seeing as I spoke of xymon, zabbix and nagios are good things to compare to. Nagios would be nice except that the gui is more complex then it needs to be and nagios itself doesnt quite manage the same level of information xymon provides. I find nagios to really have too many menu options that just arent needed when you consider what it provides. Then theres zabbix, its capable of doing every thing xymon is, and the web gui aint bad (not heavy in the broser either) however, again it has far FAR too many menu options that make browsing around when you need information quickly a true pain in the rear. In reality, its a masterpiece of information presentation and configurability, but not well thought out in most respects. Its good (once configured), but not great. Just to explain what i mean by that, If i login to a monitoring solution I want to end up at the dashboard that tells me everything i need to know - i.e. am i good or is there a problem? I then want to one-click to the place that shows me exactly everything thats write and wrong with the host that has the problem. I then want to be able to one-click to get EVERY plausible detail for that host. This not possible (exactly) with zabbix, you click around switching between graphs and status, and get caught up in a weird trail of "how did i get here" type browsing. Which can be annoying.

Then theres gui's that I truely despise. Alot of these tend to be commercial and hence less forgivable. First theres juniper junos (my wooden spoon of the gui world). I truely hate junos, both at the cli and the web gui. It breaks numerous rules, the web gui is very heavy and annoying to navigate - but what ticks me off more is that often the web gui and the cli dont work together at all. I'll change something in the cli and not be able to see it in the gui. The cli is truely horrific (like someone found a book on poor interface design and used it as their template for designing the cli), but the gui is buggy, ugly and often renders wrong (but remains functional despite that) depending on the browser you use. Whats really disappointing about that is that juniper netscreen was the opposite. fantastic at the web and the cli, easy to nagivate, options in obvious places and you never clicked more then about 4 times to find the item you wanted to configure. Junos is a mess and I've yet to meet a single person on the planet (who's used any other product) who doesn't think in a similar vein. Juniper has other saving graces tho, their hardware is nice and very flexible. Getting that flexibility to work for you from an admin point of view though can be truely frustrating.

Please, if your coding an admin gui, think of your audience. Make it simple, fast and effective. Dont make us click forever to find what we need nor wait more then 5 seconds for a page to load. Limit the number of ways you can get to information and think seriously for every link you add to a page. Stay away from complex JS. Admin's dont really care about pretty, we just want to get the job done as quickly as possible.

GLCAS: my latest project.

2011-04-08T03:43:00.004+10:00

Taking a break from my ga4php code for a bit, i've decided to have a whack at something i've wanted for a long long long LONG time. A central administration application for linux.

Theres not alot of things i really want to be able to do, but i've thought about coding it for a while (since bulos actually). All I really want to be able to do is do package management across all my linux machines at once (at work), a few important system configurations and thats about it.

There are a couple of things that exist in this space and they all annoy me for one reason or another. Redhat Satellite/spacewalk is one such. Designed specifically for redhat/rpm/yum based systems its a nasty piece of code, hard to deploy (spacewalk, not satellite) and ultimately self-defeating. Your either all-redhat RHEL and running satellite or your all RHEL clone/derivative running spacewalk.

When i was coding bulos, I found dealing with repositories quite a simple task and at the time it made me think "Ahhh, a repository proxy would be nice". There are a few of these, and again I think they all suck. What I really wanted was a repo proxy that actually looked like a repo. I.e. i could browse it like a normal repo, see packages, etc. However, I dont want those packages pulled down unless i actually try and get them.

So thats what Im going to do. 1) write a web based gui for being able to see what package updates are needed on all my servers (and one-click update them all), 2) add a yum and apt capable repository intelligent proxy and 3) some light configuration control stuff (such as syslog, firewall on/off, selinux enable/disable).

The reasons for the proxy though are two fold. 1) i despise the way yum deals with proxies. Its shite to say the least. This is one area where apt actually got things right - the proxy config is in apt, not some garbage set of environment variables that may or may not exist depending on how the person runs their package manager. Having a proxy that appears to be an actual repo presented via http solves those problems. You simply reconfiguring the repositories on the server and *poof* your proxying away. Add to this the fact that a web based repo proxy that appears to be a proxy can be built from (i.e. pxe boot a machine and tell it the proxy is its repo) and you have a pretty kewl (useful) little system.

Im using a plugin-based architecture which means it should be quite simple to add more configuration stuff later on. Perhaps even xmlrpc to webmin type behaviour, or more direct daemon control on the servers... who knows.

Android for the desktop - i'd want it...

2011-04-08T03:35:00.004+10:00

Its been on my mind for some time, and i've always wanted to be able to do it, but i'd love to run android apps and widgets on my desktop machine.

It is already actually possible. Theres a couple of articles around about running dalvik directly from a desktop. Interesting little technology, but today i stumbled onto something called Bluestacks which is exactly that (or is apparently going to be). Only runs on windows so its not much use to me, but that would be quite nice to have that kind of functionality.

What I want though is something where i'd be able to have apps coming up in suitably sized windows on my desktop. I not talking about googles emulator here, i mean one window per app, multiple windows, native. Theres quite a few little apps i'd love to have sitting on my desktop, weather, stock alerts, monitoring software, software token keys, etc. it'd be kewl to just go up to a little drop down on the taskbar and pull down a list of apps, then execute one.

Would i want the gmail app running on my desktop? probably not, but angry birds on the other hand, bring it on! SMS applications? Well maybe with a 3g modem, that'd be quite nice in fact.

If i had the time and the inclination i'd actually go thru some of the pain of compiling dalvik for x86 and see what can be achieved. Sadly at this point I have no time, and the inclination is scarily missing.

But consider, what would compete with such a system or what would such a system replace - google gadgets. Which makes me wonder if perhaps google themselves aren't eyeing off that space already. Who knows, time will tell. So are there google android apps you'd love to see on your desktop? Im sure you can think of at least one.

Chrome drops H.264 - WHO CARES?!?!?

2011-01-14T02:33:00.003+11:00

God, all i've read in tech news over the last couple of days is "google drop blah from chrome"

MORONS IT DOESNT MATTER.

Simply put, if you want to put video files on your server and have them available via the html5 tag you (right now) have to support both IE and Mozilla if you want to support the vast majority of browsers.

This means you have to support both H.264 *AND* WebM - so who gives a freaking toss if chrome drop H.264, they've not got a huge market share on chrome yet anyway (and im a user of it).

Now if youtube suddenly decide they'll only do WebM for tag in html5, thats something worth talking about.

But for f**k's sake people get some perspective, it means almost absolutely zero for anyone at all (microsoft and ars technica retards aside) for chrome not to support it. It is in reality just a way for google to show some support for Mozilla, Opera and other independent browsers who cant afford the licensing fees for the H.264 codec, and cudo's to them for that.

If google wanted to take a stand that meant anything, it would happen on youtube cause if it only supported WebM, thats when the browsers that dont support it will suffer.

DNS - how I wish it was.

2011-01-14T01:27:00.003+11:00

This is going to be a bit of a rant cause i've spent some weeks working almost entirely with DNS and im sick of living with the shortcomings.

If you work with DNS, chances are you work either windows dns, bind, power dns or mara dns (the last three being FOSS). Sadly, when it comes to implementation, I hate them all for one reason or another. If you wanted to nail me to a wall for a preference, it would be bind but only so long as you use webmin to admin it.

Im sick to death of editing text files, im sick of incrementing a pointless serial number every time i make a change, and im sick of the needless complexity (though mara is not in that case).

DNS is a simple task for a simple daemon in reality but its made stupidly complex by the implementations. So I thought i'd say what *I* would love from a dns server.

its simple, and it can be expressed with:
1) bind views
2) dynamic updates
3) de-centralised admin
4) simplified fault-tolerant method of deploying changes.

Thats it, in entirety.

All I want is to be able to go and deploy 2 dns servers in several key locations within an organisation and add everything to all of them. Screw the rest of it. Obviously I want to be able to support every record type, but they all already do that.

What I want is to be able to say add a domain to any (one) of my servers and have that domain automagically appear at all of them. I then want to be able to add views to those records so you can control what external and internal people see. Lastly, i want dynamic updates for it all.

Bind can actually come close here, webmin is not without its flaws, but it makes dns administration on bind alot less error prone (or at least less painful). But heres what i want to do from a work flow perspective:

Adding a bind server:
1) install bind
2) tell bind about other bind servers
3) bind automatically pulls down all domain config (via some pre-shared secret key)
And that should be it.

Adding a domain:
1) nsupdate add domain domain.com
run that on ANY bind server and have domain.com appear on ALL of them - get stuffed with configuring slaves, thats annoying.

Adding a view:
1) nsupdate add view

Adding a record:
1) nsupdate add record domain.com
run on any bind server and have it populated such that sees . With omitted it populates it for a default location (presumedly internal)

Getting info:
1) nsupdate get ....
Thats self-explanitory.

Im sick of having dns server for internal, external, etc. Im sick of setting up slaves everytime i add a domain. IM SICK OF IT. There should be no "im master dns server, so bollocks to the rest of you", it should be simply a case of servers talking to one another to figure out who has the lastest config and getting it from there.

There is one exception when it comes to slaves, simply because its handy to be able to push out a config for just one domain to a "friends" dns server, so add that to the list of bollocks, but pushed i'd give that up for the rest.

It really frustrates me that of all those dns implementations theres none that really come even close to being able to do that. I dont care if my internet facing servers have records for my internal network so long as views make them impossible to see.

As i said, this is a rant, take it with a grain of salt - but messing with ipv6 and dnssec over the last few weeks makes me want to go postal. Btw, if you are a dns admin and know nothing about dnssec, you'll want to read about it very soon.

CrashPlan - nice piece of software

2010-12-29T12:46:00.004+11:00

Backup is something im kind of religious about often, and at home i run bacula and have done for a while (though i really hate the way the fedora guys compile bacula with sqlite)...

That may very well change soon though as i've found a piece of software for the purpose i dont hate.

We do alot of backup "solutions" at work, and i've used all of them (the proprietary ones that is) - tsm, netbackup, backup exec, backbone, legato... etc etc. Consider myself quite knowledgeable in most of them too. However, for the most part, bacula is the one that i truely like. Partly because its FOSS, but also because of its feature set. It can do most of what all the big ones can do as well as a few little niceties (such as a fairly impressive ability to mess with how backups are stored at a very very fine grained level). The downside to bacula is its text based config and way-too-many-options-configuration and often unneeded options too (or at least, options where sensible defaults would be intelligent).

However, the other day someone said to me "hey, can you take a look a crash plan pro for me". At first i thought "oh yeah, its a cute little toy, but its not that good". But having played with it quite a bit now, im fairly impressed... It comes in three version, CrashPlan (free), CrashPlan plus (subscription) and CrashPlan Pro (enterprise/service provider).

It backs up things quite intellegently too, supports linux, and the free version will allow you to backup your machines to any of your other machines (including ones owned by other people - i.e. friends). It does de-duping, block-level incrememntal backups and uses inotify to figure out when files change in real time (though, im not sure the free version does the inotify bit).

The gui is really easy to use and their cloud based service even allows you to seed the backup by backing up to a local device and sending it to them and restore via a similar mechanism.

Its not perfect though and there are several things i dislike about it (though, they are in a way quite minor and i can live with them). Lets start with the free/plus version.

Backup destinations suck. If your say backing up your laptop to a server you have at home, when you install the crashplan software you get to specify two backup locations - first where the machine your installing the software backs up to (another computer, a local disk, the "cloud" or a friends computer). But you can only specify file systems for backup destinations (other then cloud) and you only get to choose one for the local machine and one for everyone else.

What this means is that if you want to backup say your desktop, your laptop, your partners laptop and your htpc to your central home server, they will all backup to the same filesystem - ug, thats annoying, i.e. you can just plug in another 2tb hd and get it to span backups across both drives (well, not without doing it underneath the backup software at the OS level - i.e. raiding the drives) - same (sadly) goes for the pro version, but you can specify a different backup location for each client, which is good, but still it means the file system each client backups up to has to be able to grow, right down to the underlying storage.

But, to show you how i got it working for me, heres how i installed it. First I installed two new shiny 2tb hard disk's in my linux server at home and created an LVM for it (so i can grow it later on). I then installed the crash plan software on my server and created two file systems on my LVM (one for the server to backup to and one for "clients" to backup to). During the install your prompted to create an account which then links all your boxes together in a reasonably intelligent way. Then on the server I set the location where clients get backed up to. Next i installed crashplan on my laptop and setup some files for it to backup (all done as root so far) - make sure you increate the inotify limit on your linux OS otherwise it wont pickup when files change - this is a kewl thing and worth doing!

Next, i set my laptop to backup to my server.

Then i setup when it verifies those files for backup and how often it actually backs up...

Then click "do the backup".

Thats it... If you have the client open and modify a file that sits inside your set of backed up files, you will see the client increment the number of files its going to back up during the next backup as soon as you "touch" the file.

Its very kewl, and if you have a home server and want to do some backups of your machines at home, i HIGHLY recommend it. Its simple and it appears to work very well.

Oh, one other thing, it can backup to multiple locations. For example, lets say you had a machine at work running crash plan, and a machine at home running crash plan, you laptop can backup to both of them, and you can make it so it'll only backup to either of them when your plugged into the appropriate network (i.e. it'll backup to your home server when your plugged into home, and backup to your work server when your plugged in at work - very kewl).

The only other downside is that its designed for file level backup, not for backing up entire servers (i.e. for bare metal restore), so try not to think of it as a DR tool... but the free version is very kewl none the less.

Google Authenticator - ga4php pt 3 (java vs Mono vs php)

2010-12-29T01:21:00.002+11:00

So im quite happy with how ga4php is progressing so far. Most of the work i've done recently has been on the authentication server which led me down an interesting path. So far i've been writing my auth daemon in php with shared memory comms and that does not work very well, specially in terms of scalability. So recently I switched to tcp based comms and that works a little better, but php is just not a great daemon tool - it has no threads and isnt really designed for a parallel execution model like most daemons would operate in (think web servers).

So I decided to look at some alternatives with the driving factors being:
1) reasonable amount of cross-platform-ability.
2) perform well
3) be reasonably self-contained (i.e. dependencies provided by the base language as much as possible)

I twiddled with the idea of a c/c++ implementation for a little while, but porting c/c++ is such a nasty experience most of the time so i was basically left with three options, java, mono and a scripting language (such as php, ruby, python, perl etc). I already have php so switching scripting languages isnt appealing, plus php actually has some smarts when it comes to running as a windows service (which is a surprise).

So I did two quick implementations of the algorithm under java and mono and was rather shocked by the results, generating 1,000,000 keys was by far much faster under java, mono was almost 5 times slower, and php was nearly 7 times slower. In total here's how it went:

Java: 19.1s
Mono: 103.5s
PHP: 138.2s

Which is a bit of a shock, but reading around about mono it seems mono hasn't really gone into performance mode yet and some of the posts out there are really quite interesting to read. Some of them even go into details about the benchmarks that have been run on java vs mono tests and why they suck, but the reality is my little test just does a hmac sha1 hash a million times. Some of them claim the performance difference is irrelavent and in some cases thats fair, and i'll explain why later. But for a system daemon like a web server, those kind of performance figures essentially rule out mono, i mean why would i re-implement in mono for a minor gain over php in terms of performance when i could implement in java and gain a significant one.

Still, im not sure im even going to do it, and even if i do i'll continue the php daemon in ga4php and code the java daemon as a seperate project.

So why do i say it (performance) doesn't always matter? There are places it just doesn't and there are niceties to mono that make it worth using. My experience so far with GUI's for example leads me to believe mono is just plain snappier to respond then java is (even today, which is kinda sad). Mono also doesnt have the whole class path pain that has followed java forever... i mean "java -jar somefile.jar" is nice, when it works (which is often for alot of desktop style apps), but its directory struction and how it maps into classes (and then how that maps into a compiled application) is truely painful when compared to mono and "mono application.exe".

Well, thats my two cents anyways. Those benchmarks are all on the same machine with nothing else running (though all of them only utilize one cpu core anyways).

Google Authenticator - ga4php part 2

2010-12-10T02:37:00.002+11:00

this is a blog post about my code project on google code called ga4php (http://code.google.com/p/ga4php/). So far i've had alot of fun with it, and its at a stage where it works. Has alot of design flaws (though the base ga4php class that i originally started the project to code around is quite complete and useable).

Most of my problems have been around implementing a set of server code. I sat down for some time trying to derive an approach that would work and eventually I just said to myself - start coding and you'll figure the rest out in the (inevitable) version 2 rewrite.

Now I wonder where i should really be going. The original ga4php project was really about providing a php class you could easily integrate into existing web apps to provide two-factor. However, I also want to be able to provide authentication as a service (presented as radius and soap/xmlrpc) with a web-based management interface.

That I started to do as part of the ga4php project. I have a little auth deamon (coded in php for better or worse) and a simple web interface, both of which have some major flaws that can be fixed.

I guess ultimately what I would like to have is two projects -

GA4PHP which provides the php class for auth'ing GA (or any HOTP/TOTP token) which includes some integration plugins around ga4php for some common php app's (like mediawiki, drupal, wordpress)

A authentication server which provides a flexible way of adding two factor auth to almost anything (i.e. via radius) and can talk to almost any backend

The first is more or less done. The second is where I sit here going "which approach do i take?". Do I continue using my php authd and code around the ga4php class? Do I start coding in java and turn it into a server that can run anywhere? Do I turn it into a virtual appliance (which is where the current code really heads).

In reality the virtual appliance is really "deploy generic ubuntu and add the ga4php server component - then manage it via its web interface". Generally I prefer this approach to actually creating a virtual appliance because anything you create is based on whatever OS you used as a base at the time of its production, where if you just provide a package on top of a base OS that people install, your not flinging around huge chunks of code.

Needless to say, A re-write is coming for the authentication server and im still not sure where I want to go with it. When I look at how end users of two-factor work, I think to myself "a radius server is perfect for most scenarios" but it really does have to be stupidly simple to deploy while maintaining its flexibility.

Last but not least is my concern over what google are doing with the GA. The GA is probably one of the best software tokens available, simply cause of its ease of use and ease of assigning tokens. However the implementations are a little bit iffy and I worry a little that google may disband it. The iOS implementation is really quite nice, its simple to use and does most things you'd want (for example, time-based codes automatically update over time - though it does crash in some scenarios on 4.2), but on their own android platform its a little crappy, time based codes don't automatically roll over and it feels like the project isnt exactly moving much at the moment. Time will tell. At the moment im just having fun coding a FOSS project that may or may not find use in the real world by people other then me!

Windows Auth, AD and OATH Tokens.

2010-11-22T03:28:00.003+11:00

Today, while I was hacking away at my GA4PHP code it occurred to me that implementing OATH compliant tokens directly in windows would be an interesting concept. Yet I have no idea where to start.

Originally I had started out by working on a radius authenticator as part of the GA4PHP code, which later became a general authenticator that can support radius and while trying to decide how to store user token data it occured to me that if the user was originating from Active Directory then we should just store their token data there too. I have no idea how to achieve that yet. It seems do-able for my particular little project but it made me wonder what else might be possible.

What I mean is how possible it might be to have a HOTP/TOTP system directly tied to AD. Like you extend the AD schema to store the HOTP/TOTP attributes and then when someone goes to edit properties of a user in the AD managment snap in, users have another tab called "OATH Authentication" (or whatever) and in there is the attributes for a user token. Theres not many really, and it would be nice if it displayed a QRcode for provisioning to the phone.

It would also be nice if you could then make it use the token such that a user (once logged in) could click some icon that allowed them to provision their own token and when a user tries to login after that, it prompts them for their token code.

I wonder if such a thing is possible?

The Toshiba AC100 - not so bad...

2010-11-21T03:16:00.002+11:00

Those who know me know I'm not a fan of toshiba's laptops. I've just had too many problems with them. Both those i've owned and those i've worked on for other people. So when the arm-based nvidia-tegra-sporting, netbook form AC100 came out I was stuck in two minds. To begin with its more or less a one of a kind so far in this neck of the woods... Its an arm based netbook and its pretty generally available.

All this got me excited enough to overlook the fact it was toshiba and pony up the cash to buy one. So far its been a good experience. The thing runs the phone-targetted android OS and I personally dont think that holds it back much. But, the hacker in me was dying to get it cause its an ARM based netbook. I wanted this thing like I want ... things! That nvidia tegra chipset i've been wanting to see for AGES!

So it arrives rather (unexpectedly) quickly and I pull the thing out and boot it up. Ascetically the design is very nice, and its light as anything. But as soon as I try to use the trackpad I get my first real disappointment. IT DIDNT WORK GOD DAM IT!. So I plug in a usb mouse and no problems, that works. But still, going through my mind was "i knew i shouldn't have bought a toshiba". After a bit of playing with the external mouse, I pack it up and head home from work where it sits there doing nothing cause im somewhat disappointed about the broken touch pad. Later I boot the thing up while sitting on the couch at home and realize theres a button that enables/disables the track pad. Sure enough that was the problem. Im not sure if i pressed it while i was booting it up the first time, or the default is to have the track pad off. If its the later, then thats not a good idea and maybe reading the manual may have mentioned it.

Anyways, onto the thing itself. It doesn't have the google marketplace which is a shame. But you can download some of the alternative markets in the broswer and they all have good enough app's that you wont be left wanting. The interface itself is fine and the screen is quite good. So I downloaded the quadrant benchmark utility and gave it a spin. The result was 1939 (pretty good given my htc desire clocks in around 1100). The 3d on the thing is quite nice, I played a 3d pool game app and was quite impressed with the speed of the device.

The navigation is a bit of a weird one, and what primarly left me guessing a bit. The keyboard doesnt have f1-f12 like a normal keyboard, where those keys are is a tonne of laptop-specific buttons (like the trackpad enable/disable, wifi, volume, etc). I suspect these keys actually ARE f1-f12 but they've been mapped around in android, which is kewl. But given that its android which is an OS designed for roaming around with your finger i kept wondering how you do little things like alt-f4 to close apps or "how do i bring up the url bar in opera". Its not hard to figure out, but its not obvious when you consider your likely to sit in front of this thing thinking "computer" not "phone" and thus use the keyboard accordingly. The keyboard also has a few other keys where things like the windows key's usually are, one of them is like a home key you'll find on android phones as well as search, settings, etc. The specialist keys on the thing: applicaiton menu (the little one that comes up at the bottom when your in an application), home key, search key, esc (which is back), 4 media keys, external screen off/on, brightness up/down, wifi on/off, touchpad on/off, volume up/down, mute, screenshot, email, browser, application switch (to switch between running application - alt-tab dont work like that).

So all in all, i can see myself sitting on the couch a fair bit using it to browse the web. Its a 1ghz cpu, but its quite impressively fast. Though it does lack flash (i guess cause its android 2.1? though 2.2 is due for it sometime soon - since october actually). It has no fan, and doesnt generate much heat, its light and compact (and I DO mean thin, its razor thin) supposedly it can stay in standby for days, which is very kewl but brings up an interesting thing. Im used to shutting down laptops and the first couple of times, thats what i've been doing, powering the thing off. I've realised this is not what I should be doing, i should just be putting it into stand by all the time.

So a brief list of what I liked:
1) I configured up the standard mail app to talk to my work exchange server and my home mail. This was quite kewl cause it has these little yellow lights on the side that flash whenever i get an email (much like a phone i guess) and the email prog actually doesnt feel too bad with that netbook profile (i was expecting the phone-oriented design of the mail app to hold it back in this regard, but i was pleasantly surprised).
2) The interface itself is quite good - worthy of the device I would say
3) size and design are very nice indeed
4) screen is very good

What I didnt like:
1) theres not much for this - first its about what sucks for a laptop thats trying to run a phone OS. Quadrant was a good example, it started up in a format like it would on a phone, i.e. it wanted the screen to be long and narrow like it is on a phone, and so it threw itself on the screen sideways and took the mouse pointer with it. So when i hit up and down on the track pad, i was actually going left and right on the screen - annoying to navigate. I imagine it wont be the only app that does this. I also imagine that this is one of the many issues the gingerbread android release will be aimed at fixing. On the plus side - games (or most that i've played on my htc desire) tend to operate in landscape mode on the phone - i.e. turn the phone sideways. Which will work nicely on the laptop.
2) charging - this is one of my pet hates with alot of laptop makers, they make a power plug that has non-standard connectors. I dont mean to the device, i mean from the 240v (110v) power source to the power adapter for the device - that really ticks me off. But it was also about the fact they did give it a power supply similar to any other laptop. This thing should have had either a usb charging port or some form of docking station. OR something that you just plug into power then sit the laptop on it to charge. I just dont like that it uses the same methodology as a normal laptop really.

Im really keen to try ubuntu on this thing sometime soon, but for now im gunna sit back and enjoy its little yellow flashing lights and android OS for a little while. I also gave the little device a run on my hdmi tv (the thing has a hdmi output) and it works quite ok.

Conclusion
Would I recommend this thing? Yes, absolutely if your looking for a netbook. I read a few lackluster reviews of it (which was disappointing at the time cause i wanted a reason to buy one), but as a netbook it is (imho) far superior to anything else i've seen. To me a netbook is something you browse email, browse web, maybe watch a video on and maybe read a document or two on while having exceptional battery performance - all of which this platform has in abundance. This thing does it well. But it has one thing that no netbook i've seen actually has. Fun games. Games that were designed for this platform (android) and run quite well on very low power machines - like angry birds.

That's my opinion and something alot of reviewers have failed to take into account - this thing is a netbook replacement, a very good one at that and I HOPE it doesn't fail with alot of people looking at it and get dismissed as a platform in general.

Google Authenticator for PHP

2010-11-15T12:07:00.005+11:00

Update: so my php GA class is now live, and its here: http://code.google.com/p/ga4php/

So following on from my OATH excitement I decided I'm excited enough to produce my own OTP based authentication backend.

Playing around with the various phone apps on the various markets leaves a bit to be desired though. Most of the apps that support hotp and totp aren't a great end-user experience. But eventually it led me to the google authenticator, which is itself also a hotp/totp based app.

And thus the pain begins. The google authenticator has some nice features, you can enter its key via a qrcode scan - very nice cause from there you can specify everything, the key, the key type, etc etc. Its available for android, blackberry and iphone, but thats where the "good" kind of ends.

To begin with, they use base32 for their key encoding and php has no in-built base32 encode/decode routines. There are a few implementations, but i didnt like any of them. Im also not a fan of creating dependencies on apps that fall outside the mainstream of code available for the language. Then I realized I only really want to go from base32 to hex and back again, so I wrote my own.

The next issue was the lack of implementations in the google code page. Theres a PAM plugin on the google code page but thats not alot of use. Thankfully I already have a HOTP based authenticator (quite simple too). But, I cant figure out the TOTP (time) based one. The algorithm is in draft, so maybe thats where the problem lies. But I found one class that implements totp (or is supposed to) here, http://titirit.users.phpclasses.org/package/4644-PHP-Generate-and-validate-expirable-one-time-passwords.html, but i cant seem to get it to generate the same ones as the google app. Which brings me to the third problem, the apple iphone application.

First, the scanner only accepts totp based qrcodes, so its a manual enter for the hotp based ones. To make life kewler, the iphone app appears to crash on every second scan. So like if you scan one code and its not right, generate a new one, and hit scan again, the app just closes and you are back at the normal icon menu thing of the iOS (i assume this is a crash).

Which leads me to believe perhaps I should read the totp spec and produce my own code...

But, on to the code. So prior to making my massive backend php app, im going to write a simple class that can deal with the google authenticator - and hopefully people actually use it. It should be quite easy to plug the class into something that requires authentication and just "go with it"... but we shall see... the code will be up on google code shortly (i hope, i wish they did git).

Update: Ok, im an idiot. When I first read the totp spec, i must of got a little lost but reading it again, im not sure why. Its really straight forward - totp = hotp(t,K). Basically this means that totp is a time-based step of a hotp key... how freaking simple.

2 Factor authentication - what I thought I knew...

2010-11-12T11:45:00.002+11:00

Its interesting how knowing vendor implementations (or how to implement them) can skew what what you think you know about a system you work with.

Like i've used RSA, entrust, safenet/alladin, and played with systems like Wikid and i've implemented them to do a large variety of tasks. But i realised over the last couple of days how little i know about the magic behind the scenes.

I mean, I get the basics of the way these tokens work. With RSA theres a key that both the auth server and the key itself know and they use it to generate a time-based code that you can then use.

But what I wasn't aware of is that alot of this stuff is based on standards, specifically OATH - i've heard of OATH before, but never really looked too deep at it. Theres two algorithms in use here, HOTP and TOTP - go look them up yourself, I wont bother explaining them.

And now let me digress a bit... I was interested in having something like this for home for no other reason then because I could, and at the time there wasn't much really, there was motp and maybe wikid was around - motp had potential but wikid got dismissed pretty quickly. But these days its different, in the android and iphone market places there are several software token, most of them based on either HOTP or TOTP or also motp (a FOSS based bit of software that has software tokens for MANY things).

It was mOTP that actually pushed me down an interesting path. This is mOTP http://motp.sourceforge.net/ - and its interesting. Why I ended up being interested in it had more to do with the server side implementations it listed, one of them being linotp. Looking at linotp it listed a tonne of hardware token's that it supported and among those was the safenet one. Thats when I sat there thinking, how could this be? I thought this was a proprietary system.

So after doing alot of searching and researching I found out that they are HOTP tokens and when you go get your token keys (and install them into your safenet software) from their site it downloads the key and you have everything you need to be able to plug them into a backend that knows how to generate the token code. All you need is that little hex token key and away you go.

There are in fact quite alot of tokens based on this hotp/totp algorithm. Oh, btw HOTP tokens are what you call event based, TOTP are time based. What this really means is that with event based tokens they give you a code when you ask for one (i.e. press a button) and that token code doesn't change over time, it changes whenever you press the button. But TOTP (like rsa tokens) change over time.

Now, its easy to write software that knows what the safenet tokens are producing, hence writing a backend is a relatively straight forward exercise.

The key thing is that its doesn't make safenets software redundant in anyway cause most of what they do is based on their implementation (i.e. tight windows integration). So far i'd say they have one of the best implementations that an ISV (such as I work for) can push out to a client simply cause of how tight it really is (and how easy to use). More importantly the safenet evalutation packs they throw around have a lifespan that you could use later on with something if you were bored...

Junos Olive and VMWare ESX(i) 4.1

2010-11-09T12:10:00.003+11:00

Waaay back when Netscreen first came into Australia the company I work for was probably one of the first on the bandwagon as an integrator and i've loved this since. They were simple, complete and powerful.

They got bought by juniper and not alot changed really until JunOS.

JunOS is what Juniper are now replacing ScreenOS (the netscreen OS) with on all their devices and after having a play with it, I was pretty disappointed.

But then I discovered Olive and that changed my mind simply because you can deploy it into a VMWare instance - though not with juniper's blessing.

I had some fun recently getting 10.1 into a VMware ESXi (4.1) box and so i thought i'd share my experiences.

Its pretty simple, do what you normally do for any other olive instance. Create a VM running BSD 4.11 with the good ol partition layout (i user 10g for /, 1g for swap, 2g for /config and rest for /var for no particular reason).

once installed, upload the 10.x image and do the checkpic replacement (Details elsewhere, not gunna re-iterate it here).

What you need for vmware virtual hardware itself:

1) serial port

2) use "LSI Logic SAS" card for SCSI controller, the others WILL NOT WORK (i.e. LSI Logic Parallel does not).

3) E1000 network cards - make sure you turn them into this

The Serial port - v4.1 and beyond of ESX changed how the serial port worked a bit, rather then going to an actual physical serial port you have 3 other options, log file, named pipe or network

Network is useful, use it. With network you can use server and type "tcp://0.0.0.0:10020" as the URI (for eg) and then when you click boot on your virtual machine you can then telnet to port 10020 on the ESXi *HOST* and get the serial console for the virtual box.

If your a linux user like myself you can also go the other way and type tcp://:10020 and use client. This will cause ESX to attempt a connect to you on port 10020 when the vm is booted. If you want you can start listening with netcat like so: nc -l 10020 and you then get a serial port connecting to you.

And thats it... Add the jweb interface and its like having a virtual juniper appliance you can clone lots. The only problem im yet to overcome is that tht e1000 cards dont appear in the web interface properly. Not sure what causes that.

Google Wave - what a pity.

2010-11-03T01:10:00.003+11:00

Its been a few months now since Google announced that Wave was more or less a dead duck. What a pity. It had such potential.

This is only my opinion, obviously, and as such take it with a grain of salt. But Google Wave really did have the potential to replace email as we know it and much more besides. So what ultimately killed it (IMHO) was simply based around functionality. It just lacked some key pieces and it would have been brilliant.

First, it needed (desperately) an open implementation. Im not talking about a simple protocol implementation plugin for xmpp, I mean a full client implementation. Or in the very least a usable way of talking Wave from your little private hole in the ground. A way for me to roll up a Wave server that I could then access in a functional way. It didnt need anywhere near the bell's and whistles the google front-end had, but the ability to do the main purposes on the Wave communication and collaboration components were critical - and missing.

Google turned to the community to provide these, but the community got lost. A front end client to this quite complex of protocols was no small task and to build on something like this you need to be able to start small.

As an example, take gmail (and more importantly hosted gmail). So I can plug my domain into the gmail servers and make me@mydomain.com inside gmails servers, and that has been quite successfull. Why? well its simple, i have options. I dont HAVE to use google mail to talk to other google mail people. It talks SMTP, basically one of the most important and fundamental protocols on the net. Google talk is also another good example, it uses XMPP so I can IM with anyone who has XMPP including google talk users. Yet even given the ease of implementation (these days) of plumbing up and XMPP style chat service it hasn't taken the world by storm, but at least it has a chance because of its easily accessible open implementations.

Moving on, what else did wave miss out on that would have been fundamental to its success - email. The wave was really designed to but heads directly with things like exchange and email (as well as IM, but i'll get onto that later). Again, gmail is a good example, it succeeds cause its just an implementation of a well used protocol. If wave had (day one) been capable of chatting with email users in an intelligent way (or any way at all really) it would have had a chance. What I mean is that if im creating a wave convo and I add a contact (MY) wave server should have tried to talk to the end users domain via wave, if that failed it should have bundled the wave into an email message that got sent to the end user. If the user replied the email should have then been unbundled at my end and stuck onto the wave. Sure, this is no simple thing to do but when your trying to replace a protocol like SMTP you need some form of compatibility here. Would MS exchange and Lotus Notes have stood a change in the real world if they didnt talk SMTP? (who knows really, but chances are probably not).

If it had of only had those two things it probably would have got alot more focus in the real world cause they are something worth building on. They aren't simple things and im not pretending they are, but they were more-or-less key requirements.

So what else would have made wave worth having? The ability to talk to IM clients (i.e. xmpp) would have been nice. One of my pet hates at the moment is the framentation in the IM client space. theres too many protocols and none of them except XMPP work really well. Walk into a business and your likely to see most people using MSN or ICQ or skype (or whatever else) and its somewhat luck if they're other business friends have an account on the same network. Its a pity some of the bigger players in the space (like MSN) havent adopted XMPP cause it does have serious potential for a workable protocol like SMTP is (i.e. anyone could talk it and have their own servers). More on SMTP later on...

Client plugins - outlook wave, thunderbird, evolution, mac email (whatever that is). Simple plugins that showed wave in clients people were familiar with.

Social networking - ok, i hate "social networking" myself, facebook and twitter just annoy me. BUT there are many things here wave could have plugged into nicely - such as facebook, web forums, chat groups - the works. It really did have a huge latitude for building on (things where SMTP is a complete failure at competing with).

Which brings me to Google Buzz - wave could have been a launching tool into google buzz and it makes me wonder what Rasmussen will be doing when he goes off to FB. Social networking can be a powerful tool and something I hear of (now and then) people wanting in a more corporate structure (there are even some commercial implementations of social networking in said corporates - tools that give then the ability to control what their employee's are doing where facebook scares them to death), facebook itself has a number of notable features that would (and do) work in that environment.

Now combine wave, social networking and a corporate environment and you end up with something that looks very strangely like where MS are heading with the 2010 versions of things like sharepoint and exchange. It makes me wonder if facebook might not start having a corporate hosting frame work coming out of the woodwork soon, maybe even a whitebox you drop into your datacenter that integrates into what most corporates are likely to already have (i.e. exchange/notes/whatever) because alot of the things people do on facebook have very similar analogues in the corporate environment. They dont share photos, they share diagrams and documents and they talk about them and right now, the majority of that discussion happens inside business over email.

It'll be interesting to see what might eventuate here, cause I think MS have at least seen a corner they can work in where they already have a large chunk of the market and I hope they're not the only one with grand schemes.

A couple more words on SMTP. So why was SMTP so very very important? its simple, look at any business environment and its the one communication tool you can KNOW everyone else has. They may not have the same IM client as you, but you know they'll have email. Its not just that though, there are SO MANY tools that have built up in the email space. Such as archiving, virus scanning, compliance, the list is endless. Very important things for business. Lately compliance is a massive driver in alot of things I see, the ability to run to your email server and see exactly what emails where being exchanged 2 years ago is almost critical in some places for legal reasons, how long would it have taken for such tools to exist wave? god knows. My point being that if wave could have pigy-backed email, then it would have been a driver for people to build the same tools for wave servers. Let me give one simple example, lets say im having a wave conversation with a women in my company and lets say i type "i would really like to f**k you right now"... then i backspace and change it to "i would really like to send you my timesheet" (or whatever). HUGE PROBLEM - this is a nightmare situation for a company. They want to be able to prove that I never said that, but they have no way of doing it. With email (today) they do and ultimately they want to avoid things like what happened with David Jones recently.

Anyways, thats my two cents on what a pity the death wave really is and being a fairly decent fan of what google does its a shame that they dropped it, but such is life right? I think google ultimately could have made the product work and thats a real shame. Something as potent as wave, android, google mail, google talk and google buzz has a combined potential somewhat greater then just the "search and hotmail-replacement" that google gets seen as currently in alot of places.

One of the reasons I love linux....

2010-11-01T02:26:00.002+11:00

Some years ago, I decided to splurge a bit and build a new home server. This time around I decided i'd go with actual server components (inspired by a friend of mine) and put together a nice dual-quad-core xeon machine. It had some decent components, intel motherboard, several decent sized HD's and I bought a nice Corsair power supply (a modular hx620). At the time it got fedora 9 and I migrated onto my new server.

All was great for a couple of years until one night the power supply goes "poof" and stops working. Stuck with what to do (intel server motherboards have an EPS connector thats not common on alot of power supplies) I decide i might as well just put the disk in my old dual-core AMD desktop that I no longer use. With (surprisingly) little effort the thing boots up and runs. So now my server is running on a little desktop machine.

After several attempts at contacting corsair support (1 request ignored, second request never made it past the first email) I give up on corsair (lesson learned: never buy corsair again) and decide to get an el-cheepo power supply that sports an eps connector. This has only been my second power supply i've lost (which for me is a bit of a miracle, i've gone thru dozzens of machines).

I get a few new 2tb Hitachi deskstar drives (these are IMPRESSIVE and cheap), they flog along at 7200rpm and they perform brilliantly. I install fed 13 on the machine with the intention of migrating my server to that machine. But, time makes fools of us all and after several months i've still not managed to make that jump.

But, some time ago I bought a zotac motherboard (mini-itx) and an AMD cpu. Big regret. Turns out even though the support site (At the time) said it was supported, it was not. After a very frustrating couple of weeks with their support crew (they're support crew are actually quite good) I eventually give up and flog the motherboard off to a friend (another lesson learned: never buy zotac again). So now I had a very decent AMD CPU and nothing to plug it into. After some time I decided I might as well replace the desktop machine I have with this thing and get a gigabyte motherboard with some usb3 ports on it. But again, it takes me ages to get around to installing the thing and finally last weekend I figure i'll have a play with it.

After some positive results I decide "you know, maybe another desktop machine aint such a bad idea, it can be my Xplane machine if nothing else" and decide to get a cheapo nvidia card (a gtx210 with 512mb of ram, practically the cheapest thing I could get my hands on). So far im fairly impressed. The machine now sports a quad-core amd with 8gb of ram and a 400gb hd. It runs xplane spectacularly well too. I did to a small install of windows 7 on it and it seems to run quite well also.

Finally I decide I want my real desktop case back to install it all into and sit there wondering what im going to do with my "server", I didnt really want to do the migration to fed 13 yet cause its not fun and it hit me "why dont i just turn the thing into a VM!". So I tear the disk out of my old AMD desktop, plug it straight into the xeon machine and fire up KVM. Setup a bridge network to the main ethernet and to my surprise it all just worked. The server booted up inside KVM without any mod's.

Which made me think a few things... First usb 3 external HD's running ubuntu really start to look quite impressive. I mean you could carry around your computer on a USB disk, boot your laptop, your desktop, whatever on it and it'll probably just work and if USB 3 is fast and low enough latency you're probably going to have a good experience.

But the real point is that this is one place linux excels. If you've ever had the pain of trying to take a OS disk out of a machine running windows and then stick it into another machine you'll know its pretty much impossible to do (or at least, no walk in the park). Yet linux does this with ease. My server went from a dual-quad-core xeon to an AMD desktop machine then to a vm with very little modification - so very impressive.

Playstation Home - some things just weird me out...

2010-10-10T01:31:00.002+11:00

Virtual Worlds tend to weird me out a bit. Im not talking about MMOG's and the like, I get them and play them (Eve online, wow, star wars galaxies, DAoC, etc). Places where you play a game and its open ended and you slowly build yourself up.

The kind of thing Im talking about that weirds me out are things like Second Life. A space where people "live" lives. Now I could understand it from a virtual corporate environment where it might serve a purpose, but it doesn't. People login and create crap, like t-shirts. There are other things they do, like purchase land (??) but I seriously dont get the point of it, and perhaps its got something to do with being satisfied with real life, or perhaps its just that i've missed some key aspect of this whole deal that is so important to some people.

Enter playstation home. I never bothered clicking on that in the XMB on the PS3 cause I thought it was just some website and never bothered reading about it. Until yesterday, when I clicked on it and it looked interesting. Yet what it really is is something very similar to second life itself except its very small. You have a personal room and you can purchase lots of different ones (thats right purchase, for real cash money) and they range from things like islands to space ships (maybe, i didnt look at them all).

But it doesnt really end there, see you can buy furniture, clothes, accessories, different bodies, the works... So why is it people are paying $3 of real money for a shirt in a world that really doesn't have much latitude nor worth anything in the real world? Thats what I dont get. See I could understand a social environment where people met each other on the PS3 if it were linked to the games you play. For eg, playing CoD4, MW2 and getting some of the items in the game and walking around in this world with them... Or being rewarded for game events and achievements with cash you could then spend in this virtual world, but to spend real cash on items here seems like the height of virtual waste. Have people seriously got nothing better to spend their money on?

See maybe if the items in this virtual space were linked to the games you played or perhaps things you bought in the real world (i.e. i buy a billabong shirt and it has a code i can enter into the game to get the same item in the virtual space). That would seem to have some point. But anything else just seems ridiculous.

It truly alludes me.

IPv6 Auto Config - Its really not healthy.

2010-09-20T11:21:00.002+10:00

IPv6 Autoconfiguration is a mess, but i'll explain why and how to avoid it.

This is truely one of those times when "design by committee" really has a purposeful meaning. IPv6 was designed (by morons) to work in 3 different ways when it comes to assignment;

static
Stateless autoconfiguration
Stateful autoconfiguration

Anyone familair with IPv4 already knows dhcp, so why did the ietf in their infinite wisdom push out router advertisements (RA's)? Cause they're a committee, they do stupid things for stupid reasons. Dont get me wrong RA's seem like a good idea, but they're rubish. They were designed to make assigning IPv6 in an office type space easy (with a whole heap of desktop's were you really dont care about their actual ip's day-to-day, much like we would with dhcpv4 in the desktop space). Unfortunately, no one with a brain actually worked on the concept and so we also need DHCPv6... and this is where your pain begins...

RA's basically just push out the network prefix and say "get your own god dam IP address" (this is called stateless), the machine chooses its own IP address based on its MAC address and away you go. Sounds like a good idea, however the nub's at the IETF left out DNS. While there is now a way to push out DNS prefix and address information via RA's, basically no one supports it. Which is where DHCPv6 comes in, and your pain begins. If your on a dual-stack network, just turn on RA's and hope for the best cause DHCPv6 is a world of pain for one very simple reason: support - no one does.

Well, ok thats not actually true. But if your on an IPv6 network needing auto-config, RA's are non-optional. "But you just said theres DHCPv6!".... I did indeed, which has also been designed by the same set of morons. See in order to even look for dhcp you either tell the client to use dhcp (hello, login to all the boxes on your organisation and make that change, pal) or you make the RA state that "stateful config is needed". What does this mean? basically it means your box thats doing the RA's says, "im only providing your gateway, get the rest from the DHCPv6 server", which your going to need anyways cause (as i said earlier, you cant push dns from RA's)...

So, why is this all incredibly tedious and painful? The only people who actually SUPPORT DHCPv6 (as a client) is Microsoft (not strictly true, read on)!. Thats right, linux was the first out there with IPv6 support in the stack and yet thats where all the good work ended. We poked fun at Microsoft for their lack of IPv6 support and now they have leap-frogged everyone.

So here's your scenario:

RA's can push dns details

DHCPv6 CANT push router details (your not reading wrong, DHCP cant push default routes).

Windows is the only client that supports DHCPv6.

So how do you configure an office network with mixed machines? you dont. MacOS X (including snow leopard) doesnt support it, linux (with some exceptions) doesnt support it - and by "it" i mean dhcpv6.

The client used by unix is typically the ISC DHCP implementation which - get this - does actually support the full functionality that its supposed to, but thats in version 4 and just about everyone is using version 3. At this point, the only distro that i know of that DOES have full and proper support for IPv6 is fedora.

RA's were a stupid idea for so many reasons, now we have to (for auto config) setup both our routers and our DHCP servers software and even then only windows desktop's will even get their dns info. But there are other reasons why RA's are dumb. To begin with any machine on your network could transmit itself AS and RA. Sure, same can be said for good ol DHCPv4, but RA's are pushed out LIVE and the hosts automatically reconfigure themselves. Then theres compliance, want to know what was on IP address 2002:1:1:1::1 3 months ago? good luck cause RA's never get tracked.

How f**king typical. What a mind-numbing mess. Your options are basically static config or be a windows shop if your planning on doing IPv6 by itself. People KNEW and UNDERSTOOD dhcp, its going to be a requirement why the f**k come up with a retarded idea like RA's?

And that, my friends, is why i say IPv6 is a typical design-by-committee mess. I could go on about the whole "lets make everyone a /64 network cause they're are trillions of IP's" and how we already made that mistake once already, but i wont (or did i?).

Got IPMI?

2010-09-17T15:35:00.003+10:00

For some time i've used a ipmi on alot of my servers and some of you out there may have seen it and not realised how useful it really is.

So what is IPMI? IPMI stands for Intelligent Platform Management Interface (i think). If you know the server world you'll be familiar with things like Dell's DRAC, Sun's iLom, etc. Basically they're ways of getting on a server's console (even if its powered down).

IPMI gives you a very basic version of it and just about every modern box out there that runs intel or amd (server) chips already has it. You'll probably have to jump into your bios and set it up, but once setup it gives you some pretty nice little things.

For eg, you can manage power state... you know, on, off, reset, hard reset etc... It also gives you a (text) serial console capable of managing the bios, or on machines that know how to talk to a serial port, a serial console (think unix/linux, this is very handy).

Now i've used it for a while using the openipmi (and various others) implementations under linux, which kinda restricts its general availability (IMHO) and so I've never really bothered talking about it much. I get to manage the servers at work and the one at home and thats about it.

Well, i just stumbled across supermicro's little java client for ipmi, its here, ftp://ftp.supermicro.com/update/IPMIView and its quite sexy.... runs on almost anything and so far i'be been able to talk to Dell, Sun, Intel and Supermicro servers with it (which is more than i can say for openipmi).

If you dont know IPMI its work looking at, and if you do know IPMI, that client is also worth trying.

Lastly, kudos to SuperMicro for their client!

IPv6 - its the place to be....

2010-09-05T00:54:00.002+10:00

If you know anything about networking, you've heard about ipv6... we all know ipv4 addresses are running out by the truck load, and have been since... well forever actually... then we got nat and life was bliss again...

Now they're running out again, and this time its probably for good...

To digress somewhat, i've been following ipv6 since the time when the IETF (I don't think they were called that back then) were trying to decide which protocol would take over from ipv4. That was in 95, when I was in my final year of uni and started doing work for a small company called BHP. I remember the article in Dr Dobbs about the choice being betwen IPv6, IP128 (i think?) and a few others. It wasn't long after that they decided on IPv6.

Anyways, thats how long its been that we've been "running out" of IPv4 space and who knows if we really are, but there are people now doing things about it that have the power to. Lately we've been getting alot of requests at work for people with IPv6 knowledge as it turns out that various places (mostly government connected) have started to mandate that all new things built are built with IPv6 (they can have IPv4 as well).

So it seems IPv6 may well (finally) be truely on its way, regardless of whether the IPv4 space is actually running out or if its just a bunch of doom evangelists. At the time when this started happening, I was truely the only person in the company that even knew much about IPv6. But, if you dont know about it, nows the time to learn cause it can be quite lucrative. There are people out there now being forced down the IPv6 path already and you can make money knowing it.

I've heard rumors that part of the reason we're diong all this is more to do with China and India - between them they consume only 8% (roughly) of the IPv4 address space yet account for a very vast majority of the world population, and if they got online.... woah.

But, the thing that annoys me about IPv6 is how the adoption is spreading. Consider where the address space is really being consumed. To me this surely has to be ISP's. And yet very little is being done in that space to push people into IPv6. How many home consumer routers even support it? very very few. The worst part is it would have been so very simple to start implementing it here. Everyone I know who is non-tech-savy has a little wireless router at home they get the internet from, they wouldn't know/care if they had a public IP address, let alone what protocol it was. They're all on private range's on the inside of it, so the public side could happily be IPv6 and they'd be none the wiser. The back side of the router could have even been dual-stack and the humble internet space cadet would never know.

Yet very few home routers actually support this. Even more annoying ADSL and IPv6 go together like a whole in the head (PPPoE/A and IPv6 dont go together well, so im told).

Even more annoying is how easily it would be to do at least some IPv6 connectivity to homes, welcome to 6to4, one click box on a router and the home user has a /48 subnet of IPv6 addresses that the router could tunnel straight into IPv6-ville. All that needed to happen was routers to support tunneling to the anycast address. That would have been a start.

Yet, for reasons i'm failing to understand we're pushing the place that use less IPv4 public subnets into IPv6 for no really useful reason. Such is life i guess?

Anyways, if your interesting in IPv6, best place to start IMHO is the 6to4 tunnel protocol. It requires no upstream help (as such) and will get you into IPv6 really easily, all you do is point your tunnel address at the 192.88.99.1 and your there. Every single IPv4 address has a range of IPv6 space in the 2002::/48 space just waiting for you to plumb it up and the rest happens automagically. Such a simple place to start and requires no interference with the IPv6 tunnel brokers out there.

IPv6 is not hard, if your familiar with IPv4 its essentially the same with a few different rules... you'll get the hang of it... maybe one day some schmuck at d-link/billion/linksys/etc might actually produce a router capable of doing at least the 6to4 bit automatically, but dont hold your breath.

XMPP as the IWIN protocol.

2010-03-10T00:18:00.004+11:00

XMPP was always one of those things I'd of loved to play around with, but never really got the inclination until one time at work we were looking at chat protocols and *boom*, one of those we looked at was XMPP and jabber.

At the time I was somewhat unimperssed, but bus protocols have always been something i've been interested in since about 2000 when I was playing with tibco rendezvous and XMPP is not anything too dissimilar.

But things have change markedly since then and it was really google wave that turned me to XMPP and made me realise its utter glory. First, a blog ago I mentioned how XMPP would be a brilliant protocol for replacing HTTP. But thats not all, it could replace *EVERYTHING* and I DO MEAN EVERYTHING. Read my previous blog about HTML of XMPP to understand why I think HOX would be a protocol that would make sence where HTTP is just getting more and more rediculous.

First, lets look at ftp. With a simple set of standards we could actually do some pretty interesting stuff here. But let me add a few things here. As a General protocol we have an automatic authentication mechanism that still allows anonymous use for such protocols. For example, lets say I wanted to get a web page hox://blah/page.whatever, assuming I have an account on an XMPP server somewhere I have something that can provide my identity, but I dont always want to do this. So when I go get my page, I connect straight to the server's XMPP connector as anonymous and it then says "no, i want authenticated", so now (assuming we say yes) we connect to our local XMPP server and tell it to get the page for us - wow, we're now authenticated (there is a certain amount of trust that is implicit here, but it can easily be mitigate the same way it is in normal XMPP chat type ways). And thus, we grab a page that requires some form of registration (though we didnt have to register with the end server) and life is bliss... I could have multiple xmpp accounts and choose which one to use on each site. Think about that for a second - no more registering on all those sites you have 150000 accounts for. This is no small thing. It would require practically every ISP to run up an XMPP server, but thats workable and alternatively you could actually register on their site and become an auth'ed user of the end servers XMPP end point (similar to how you would register for forums currently).

This is a major shift in the way the internet works today, but it is workable.

Ftp over XMPP would be similar, you'd connect anonymously with the end server (say by talking to the user ftp@domain.com), which would automatically happen if you connect with something like fox://domain.com - i.e. the thing would go off and do its normal dns machinations, find the XMPP port, connect anonymously and ask for a list of files. The service (ftp@domain.com) which is connected to domain.com's site as an ftp user would then return a list of files. If you try to get a file that requires auth, the server then routes the request thru your local XMPP server... this is very kewl.

But what all this lead me to realise is that XMPP really is a protocol to replace all protocols. Thanks to the additions of voice and video it could easily be the SIP replacement we sorely need (SIP is a nightmare protocol which replaced an even worse protocol some years ago). Speak to any firewall or network admin about how painful SIP is and then talk to them about XMPP and they'd probably kiss you. SMTP over XMPP is also something that excites me, but more on that in a minute. On top of all this, XMPP also natively supports SSL encryption so everything we do (weather its client to server or server to server) could be encrypted.

What I really wanted to talk about though (which is probably more interesting) is as a central authentication mechanism. Im talking about something that would replace LDAP, NIS, NIS+, AD, NDS, etc, etc. Sound pretty boring? Bear with me and you may understand why this excites me no end.

First, I have a database of users in XMPP already. Lets add a little service and call it service@domain.com, which is a simple authenticator for managing groups. We'll say my XMPP server services domain.com and anything that wants to login to domain.com, logs into me. So lets add some computers. Much like any other auth protocol I add my computer to the domain by chatting to the XMPP server and registering myself. This is something service@domain.com would manage. Lets have an example - I get a computer login to it with some account and then tell it to register to domain.com. Domain.com says "login please" and I login with my domain.com xmpp cred's. My user login then sends an xmpp message to service@domain.com telling it "yeah, register this guy" and depending on weather service@domain.com allows user@domain.com to register clients, it then adds the computer as an XMPP account (something like computername.cpu@domain.com). My computer then logs itself in as computername.cpu@domain.com and can now be a part of the domain. Not excited yet? Continue on.

So now when I go to a machine, I login as user@domain.com and auth directly to the xmpp server @domain.com, but only if the machine is actually in the domain. But this is somewhat the opposite of how things work in some ways. Simply because it would be up to the client computer to decide if it has an account on domain.com and allows logins for that domain. Normally this is the other way around, the domain.com server would decide if the computer was in the domain and then allow the user to be authenticated.

But, thats not so important. Lets consider groups. When I login to domain.com, the cpu then says to service@domain.com "what groups is user@domain.com in" and service is responsible for dealing with that side of user access. It would also give identifiers for the user such as uid and gid number's. WE would like our computer to be able to register against multiple domains too, but thats not so important at this point.

So what? another central auth scheme?

Oh yes. But consider if we now bundle clustering, http, ftp, smtp, chat, voice and video into all this. ejabberd is a good example of an XMPP service that clusters impressively. It automatically deals with the issues of replication and so forth and follows the (very important in this case) shared-nothing side of things. So lets just quietly introduce the concept of having multiple XMPP servers on our network (much like you might have multiple AD servers). But these servers now also form the backbone of a very important set of collaboration and communication protocols that are fundamentally very useful to us.

So back to my client. My client logs in via XMPP and is automatically joined to a domain which gives them access to a whole range of services. Consider, I start up a mail server and all it does is connect to one of my domain servers as "mail@domain.com". Anything that wants to do mail "talks" via normal chat protocols directly to that server. Im already authenticated as a user on XMPP and so mail@domain.com implicitly trusts that the chat message from user@domain.com actually is user@domain.com. This means our mail server could be anywhere on our network and mail could easily route through some form of edge connectors, into our XMPP cluster and then off to our mail server (s). Keeping in mind in all this, XMPP allows you to connect multiple times with the one account. This is important for simple scalability. We could have multiple mail servers that all receive the mail for example.

But our user already has access to voice, chat, video calling, http, etc, simply by logging in.

Now lets consider how web pages might in this scenario. Again, your http servers could live on an edge, but would register as www@domain.com. Anyone trying to get a page simply talks to www@domain.com. Both internally and externally. The thing to keep in mind here is that www@domain.com is simply a service that knows how to serve web pages and also knows XMPP. Its not a XMPP server itself. Same with mail and other protocols like ftp.

Now lets talk about machine admin and so forth. This would require a local service on every machine that knows who to trust, and its up to the server (or client machine) itself to decide who to trust (though often this may be delegated to some domain.com group membership, much like it is in most central auth schemes. But wait, our computer is already registerd as comptuername.cpu@domain.com. So all we need to be able to do is talk to it in some coherent manner and it needs to decide on what we can and cannot manage. Hopefully this would be done fairly smartly. For example, lets say I wanted to do something to a machine called "myserver". My user@domain.com account sends a chat message saying "manage" to myserver.cpu@domain.com, which then replies with a list of services. I then say "manage samba" (even though my client has no idea what samba is) and the computer replies with a list of configurable items (shares, domain name, permissions, etc) and they're formatted in a xml-style way. i.e. so the client can then interpret them into some coherent gui without actually knowing what samba is. This isn't too hard in reality and it would also mean that anyone could write a management gui that doesn't necessarily run on a computer that is actually connected to the domain, per-se. Its all about knowing how to interpret the data, and being able to display it to the user.

Alot of people are probably going wtf is he on about?. See the reality is, we could easily write some form of central management protocol to manage almost anything - and there is one or two out there. Ultimately any server or service can be broken down into a bunch of configurable items with names that define them for a users understanding without a client program actually having to understand the thing that its trying to configure. Take any configuration gui and have a good look. You could easily turn it into a simple set of XML that the service side knows how to interpret.

As an example, lets look at apache config (hideously complex - relatively speaking). So what I need on the service side is something that can break the config down into xml style chunklets. So first of all when I connect it says "im apache, you can configure my network stuff, and my directory stuff". the client just shows and tree with a network and directory node. I expand network stuff and a message chat goes on telling the client whats under network stuff, which might be "virtual hosts, ports, interfaces im listening on" with a list of possible entries for this. For example it might say "for interfaces, you can choose eth1, eth2, eth3, etc" and so the client presents a drop down box.

This is not too unlike how soap works. We'd define a set of types (at the server side, not client side) and a list of possible entries and the client just has to present them to the user, knowing nothing about what apache is or what it does. Still confused? thats ok, its not important just now. But lets try DNS, its a bit simpler. So my client hits DNS management and it sends back a list of options which may be:

Create New domain - text box - button
domain.com - delete button - tree expander thingy.

So I type in some text for a new domain (mynewdomain.com), and it sends that back to the server. The server the creates a domain for me and now allows me to manage it. So i hit the tree expander thingy and under "mynewdomain.com" and the server sends back another set of options:

New entry name (text box) - type of record (combo box that contains A, AAA, MX, etc)
Delete entry buttons, etc...

Under "new entry name", it would also send a list of possible extra configurable items that may appear if the user chooses A or MX or something else. This can ALL be formatted in XML in a coherent way. No longer would you require an specific local agent in order to configure a box, the server would have an agent that knows how to send possible options to the client and tells the client how it wants data coming back to be formatted.

So, do you get why I get excited about this? IF not, its ok cause it'll probably never happen. Its just way beyond my ability to implement cause im just one guy. But the idea kinda makes me go "wow, that would be so very kewl".

XMPP is just so well designed, and its extendable.

But on to the last point. Backward compatability. How do I talk mail or http to someone who is not using XMPP on their network? Its simple really, and again, this is where edge connectors come into play. All you really need is something that knows how to get data in from the old form to the new form. For example with smtp you'd just need a bit of software that listens on port 25, can connect to xmpp as some services style user (lets say smtp@domain.com) and pumps it straight at mail@domain.com. Same with any outbound. If the end point your trying to send mail to cant talk XMPP, mail@domain.com talks to smtp@domain.com (again a trusted conversation) and says "send smtp mail to user@otherplace.com for me".

Configuring smtp mail would be much like configuring any other service on the network. You simple talk to it via XMPP and say "listen on this IP address". In reality it could simply be a module for something like postfix.

Same with http. Simply a module for allowing our web requests to bounce into our www@domain.dom user via XMPP chatter.

Oh I wish I had the time and man power to do something like this cause I think the world might actually sit up and listen if they saw it....

two-factor auth on the cheap...

2010-02-27T15:20:00.002+11:00

Now and then I get involved in places where strong-ish authentication is required and everyone shouts two-factor!

Dont get me wrong, I like the tokens and stuff and have deployed numerous installations using them. But what I've always tried to figure out is "how do you do this in FOSS" or at least, make it a fairly viable solution without requiring some hardware token or without killing some of its functionality.

A bit of a back story first on two-factor. Two factor basically means a password and "something you are, or something you have"... With most current main-stream solutions this often means carrying around some token that has a series of digits on it that change over time. Basically the idea is you login with your password, then your prompted for what is on your token. This is good because if someone steals your token your going to know about it (eventually) and the only place that data exists is on the token and on the server that handles the auth requests. Basically its impossible to replicate that without physically taking the token and hence your hacking attempts are noticed and the token can be killed off at the server side.

Similar things exist for things like biometrics, user enters a password and set of biometric data. Obviously he/she is the only one with that biometric data... ultimately there are a tonne of options that swing around two-factor (by the way, this is not what people are refering to when they are refering to strong authentication).

There are loopholes in alot of these systems that reevolve around man-in-the-middle attacks and pretending your able to get the token/biometric data, but this is not the subject of this post. What Im interested in is a reasonably strong auth method that doesn't require the token. If you can do this, you can build a good system simply cause your foss hackers arent doing hardware bits like this too easy (or cheaply). Most token systems revolve around 100$/user and the token are lovely and small, try to do that with a generic hardware and your really not going to be able to achieve it.

So for about the last week i've been trying to come up with a concept that works, similar to what people are calling virtual token (personally i'm not a fan of the virtual token concept cause its not really two factors, its two pieces of replicable information (generally speaking). There are two I took a look at, one was a php based thing revolving around grid token (its basically a card with a bunch of a numbers on it that the server asks you to spit out in a specific order - easily photo copied). Then there was Wikid. I've only had a brief play with it, but it seams to suffer from the same sort of problem, ultimately you come back to a piece of data that can be replicated.

So what cant be replicated and isn't static? Glad you asked. Well its in your head actually. Consider a solution with 3 things, a key (or just a number even) thats stored on a device (such as a mobile phone), a username/password combo and lastly a code word. To authenticate I login with my username/password and then get prompted for my second factor. I pick up my mobile phone and run my token program, into that I type some of the digits (in no logical order) of my codeword. This then generates a key I can send to the authentication to provide the second factor... Confused? heres an example.

My username is "user", my password is "password" and my codeword is "ramblings". I connect to a web site that prompts for my username and password, this detects im a two-factor twit and talks to a server that knows both my codeword and my software key (the thing in my mobile phone that is used by my token software to generate a token value). I bring up the token software and type three letters "rbl", which gives me a code of "12345678" (maybe), i then type rbl12345678 into the login screen and im authenticated. The token server knows rbl is three letters from my codeword and the correct response is 12345678.

The next time I want to authenticate I must use 3 different letters from my codeword (such as "ami"). Now if i use a letter thats not part of my code word, the token server suspends my account (either until someone can fix me or for a set period, depending on what the administrators are wanting to do). Then i could have another codeword that means im in trouble, in this case "telivision", if the user types in something that could be construed as being from that word (such as tlv) then we still allow the user access, but we know our user is in serious trouble.

Now, im mister hacker and I generally have three options for breaking into someones account:

really good guess work
spying on the user (either physically or virtually)
Physical violence against the user.

So If i use 1, well good luck.

2. isnt much use either really, I may spy on the user enough to get his codeword, but im never going to be able to get his key.

3. Well, i have a way of telling people im in trouble...

Ok, well there you go, theres my idea. I assume its either 1) fundamentally flawed or 2) already in use and probably patented but I cant find anything that suggests either.

XMPP a solution for HTTP?

2010-02-12T02:15:00.003+11:00

I remember when http and html first came into existence... it was an interesting time surrounded by protocols that did similar things (or at least, aimed at a similar context), such as gopher.

I've coded too many web apps to mention (or that I can even remember really) in just about every language designed (and in some cases not designed) for the purpose. I've written web servers and i've even written a c++ application server. But, the whole http protocol has been twisted beyond all recognition to do things it was never really intended to do and now we have rather idiotic notions like RESTful.

Lets face it, HTTP sucks. But...

Ever since google started banging on about applications delivered over the web it got me thinking about how that could be made alot more functional. For a while I was thinking that you could turn the whole web browser from a page loading engine into an event driven application engine, much like a normal windowing system on any OS. The idea being of course much like googles web toolkit in that you would design an event driven gui that runs a javascript client that mangles data between the web browser to get it into a web request.

This is not ideal, but its workable... What I wanted to do though was change a bit about how this worked in that i'd turn all websites into a application presented through a javascript client with an event driven framework. Dev's wouldnt need to care too much about how to render pages, but rather would care about the look of their data frameworks... much like coding a gui. But what I also wanted to do was make it so that it was renderable within an application, i.e. you'd have a client app that would render your application into native code (not transform the code, but turn as much as possible in terms of events into something process-able at binary speeds).

As an example, image a document editor like open office's write or microsoft word. You'd kick up your client (be it native binary gnome, kde, windows, java or a javascript one inside a web browser), alot of it being cached locally whose application logic is defined in javascript with a mix of local and remote datasets (i.e. documents). There have been a few things that have come along since then. One that gained my interest for a while was mozilla's own xulrunner (why don't they make more out of this??). Its basically a javascript application engine (firefox itself is actually written in xulrunner). Quite a neet concept...

Later there have been things like gnome seed, but ultimately none of them quite do what I wanted, which is a mix of local application and the google method of "hit our webserver". It could make cross platform something more useful than java itself did. IF you could add a javascript native compiler, then client would only need to download the code when it changes and be running at native binary speed. Im hugely over-simplifing here, but you get my drift.

But I digress... Let me digress a bit further. In my younger years, prior to the web, I was very into irc (oh dear god you nerd!), which is something alot of people probably don't hear of much anymore. But its still quite interesting, something that kinda is a text-only (though with file transfers) cross between twitter, faceboook, web forums, etc. At its core, its an interesting protocol in alot of ways.

Much later in life I became (thanks to much work in the j2ee arena) aquanted to things like tibco rendezvous and MQSeries (message busses basically). The idea behind them is dead simple and is really used for business systems (i.e. backends), not by people as such. Take for example a mobile phone buying website, you click on a link to buy a mobile phone and create an account with some telco. The Telco systems running the web front end send a message (via the bus) back to a central system (possibly even mainframe) that process the request and deal with things like stock levels and "is this customer credit worthy" and things such as that. Then another message may be fired at a stock control system to tell it to deliver the device to the customers address. Eventually another message is fired back to the actual web servers to say "display a message telling the customer the transaction was successfull" (or not)... Simple concept and used for one reason - its asynchronous (well thats not the only reason). One of the problems with this approach is that the web is not asynchronous - we can fake it (and we do), but ultimately we're just mangling http requests. Typically the ways its faked is we send a continous stream of requests from the web client to the web server saying "hey, are we there yet?". Web push does kinda exist to solve this problem, but its really not that heavily implemented and bug prone.

Back when I was first introduced to messaging systems they generated some interested internally within me and I started coding something similar based on my experiences with IRC. It ended up being sometihng I worked on for about 3 months and I stopped working on it just past the "its stable and its working, but its probably got tonnes of bugs" phase. The idea was simple, clients would connect and ask to join a "channel", didnt matter if it existed or not, the server would allow it to exist. Another client would join and send something to that channel (didnt have to join though) and the first client would see it. The server side was the part i was mostly interested in though (actually determining a way of routing messages around). The servers would start up and you might connect server a and b... Later you'd connect b and c. The servers would be smart enough (though you could disable it) to connect a and c automatically (this gets really complex with multiple servers). Any client connecting and subscribing to a channel would cause a cascade of data to flow through the network until ever other server was aware that anything destined for that channel had to get to it eventually.

It was a wonderful bit of coding experience around routing algorithms and the like and more then a little fun.

But i'd digressed alot and i've not even mentioned XMPP yet have I? Why am I crapping on about all this stuff? Cause it recently all coallesed in my head into one solid well known fact. HTTP is a crap protocol designed for entirely the wrong purpose for where the world is heading. IT was great back when we just wrote a few dynamic pages, but now we're becoming more data focused and we're just mangling http more and more.

Given the focus on an event driven framework however, xmpp is a near-perfect protocol and this is what people are mangling http to do. Take a look at almost any website in existance, you click on a drop-down box and choose something and some other part of the web page is updated via a javascript mechanism (which ultimately leads to a GET or POST to a website). Ultimately we're back at mangling http. XMPP has its own problems in the information arena though - its data is transient and really designed for a message bus style protocol. However, this isn't a problem. Lets take the example of getting a web page over an xmpp link vs http.

In http, I type in a URL, the web browser finds the address and connects to port 80 (or 443 for ssl) at the server side. The page is retrieved (and all its resources such as images, possibly via multiple short-lived connections) and the connection is terminated (im ignoring keepalive here cause that gets switched off more often then not).

In XMPP, I type in a URL, the "browser" finds the address and connects to the port described by the DNS's xmpp server field. It then says "can I have the page xyz". Using XMPP messages, the server then sends the data down the link (with the page resources) and the browser doesn't need to close connection at all. In fact, keeping it open is a fundamentally useful feature. (i'll talk more about xmpp authentication in a bit...).

So thats the simple case of getting a page, i.e. thats 1990's. Pages are much more then they were. Any online store is a classic example, alot of them use javascript extensively to deal with little bits of information the user inputs (like calculating shipping once the user types in an address without having to refresh the page). But theres so much more that can be important in here that isnt taken into account. For example, I go to a store and click on their "hard drives" page and I get a list of hard drives, but what if those harddrives are updated from new stock that arrives? Well now I have an old page with old data. With XMPP, the server could tell the client about the added stock (and only send that piece of info) directly and my page could update in real time - hello XMPP, your beautiful.

This isnt the only example - consider news sites, facebook, twitter and that rubbish. Transient connections continuously updated via mangled http requests. XMPP can easily solve those kind of problems.

But, XMPP is also designed fairly heavily around identifing the person at the other end (i.e. the client) quite exensively (look at the various jabber implementations for an example). So how would we do this in the xmpp space? Well first its important to remind ourselves that XMPP was originally deisgned mostly as a chat protocol - and as such the client end is authenticated. But, lets say I want to browse the web, I absolutely dont want every website I go to to carry my id with it (hello spam!). So for a web-style content link the browser could generate a connection ID (similar to how session variables are thrown around in web content as cookies - though RESTful would tell you to get rid of that for no good reason) we use to identify ourselves to a server. Much like a cookie we could tell it to hang around or disappear after the user disconnects from the site. So lets go back to shopping. Now given that I (hopefully) have an XMPP ID on some jabber server somewhere I might create an account on the shopping site and link it back to my XMPP ID for my jabber server and in some ways this kinda does things like that open ID does, though I was thinking more along the lines of "this site now knows me as userx@mydomain.orgcom and so i'll send that as my session ID from now on" It may be even the case that you send your session id to the server but have totally different information added to the servers database - for example shipping address, password, etc.

Ultimately though, what im trying to get at is that when your browsing the general "xmpp web" you'd be sending random ID's generated on a per-server basis that essentially identify your session for as long as you want, but have no real context - i.e. they dont identify you any more then a session cookie does. Which means you'd be connecting to the xmpp server in an "anonymous" state.

My point is this - XMPP is a protocol that can work in a useful way where these days we're just continuously mangling HTTP further and further - in reality it needs to be dead and something like xmpp take its place.

The exciting world of ARM....

2010-01-14T04:51:00.004+11:00

Intel architecture step aside, arm is hot your rear....

CES this year has been very exciting. I've been watching the arm architecture with a certain amount of glee for a while and some of the things that it can really do are starting to really materialise. I think my first (serious) arm hacking's were on the linksys nslu2, brilliant little box that.

Since then i've picked up all sorts of gorgeous little devices, mobile phones, embedded devices, you name it. My most recent acquisition was the sheeva plug. This actually ended up being a bit of a disappointment for one simple reason, for $10 in parts you can add an eSATA port. Why in gods name doesn't it have one our of the factory??? IDIOTS!

Oh well, so be it. Theres also the openrd platform, which is even more exciting given that its pretty much a fully functional desktop.

But, what excites me most about the ARM platform is the sheer volume of silicon cutters out there. Already we have ARM chips capable of talking to standard bus types such as PCI (X,E), USB, SATA, and the rest. How long until we actually do have a real motherboard with an ARM cpu on it? Well, in reality they do kinda already exist.

I digress... The point I was trying to make is that currently there aren't many people producing x86 silicon, its really only Intel, AMD and perhaps VIA - Those in the generally available market that is. But ARM has more producers that you can poke a very VERY large stick at. All of them are aiming at different things, but they all ultimately end up with a CPU capable of many exciting things.

Make no mistake, ARM is poised to shake the living poop out of the industry as we know it. One major benefactor of all this is linux (and google but read on to understand why I say that). Linux has briliant ARM support. On my plug for example I installed fedora 12 and ubuntu 9.10 - both would be fully functional desktops *IF* the plug had a display, but it can do xrdp and vncserver.. brilliant. Of course, the openrd already has a display and thus can be a very functional desktop already.

So onto google, how do they win? Simple, Chrome OS and Android. Chrome OS is probably the more important bit because I can see this thing inevitably being more then just a browser. People will modify it and get things on there that Google hadn't planned for simply because its an open OS. No doubt, Microsoft will play catch up much like they did with netbooks (I cant even BUY a linux netbook in Australia anymore) and manage to stifle linux yet again. Its a huge shame vendors let microsoft push them around the way they have cause I simply refuse to give microsoft money for a netbook when linux is perfectly capable of doing everything you need on one.

But, this year CES and ARM have been hand in hand winners.. Watch that space cause its going to get so very exciting in the years to come.

... If you were looking for the bit below about xtreamer - its seems they have made good on their GPL requirements - briliant!...

eInk and the death of the ebook reader...

2010-01-14T04:36:00.002+11:00

Ok, so I'm being dramatic here, but I do believe that the ebook readers as we know them are a dead quantity and quite a few people will suffer because of it. Having seen the number of people jumping into that market it sad to see a list of startup's that are probably going to be dead in a number of years unless they come up with something thats not an ereader.

The reason is simple. They already have foldable, paper thin eink displays. They already have (in the form of the nook) a demonstration of an ereader with an android core controlling both a colour LCD touch screen display and a large eink display. What do these two things become?

A mobile phone.. duh... The only question is how long it will take to happen. How long until the back of your mobile phone folds out into an eink display (and not just mobile phones, netbooks, laptops, desktops)? Probably alot quicker then people think, and alot of companies are going to go the way of the dodo because of it.

Add to this the "Standard" format of epub (and the like) and you have a very bad position for alot of people (think kindle type scenarios). ePub is a bad thing for amazon cause if they dont support it, they loose out in the long run when everything starts having eink displays attached to it. Meanwhile Barnes and Noble (et al) will make a killing selling to everything that does support ePub.

So what do you do if your a company producing pretty much only eink display ereaders? Simple actually, here's what I want when I think eink:

a NOC display - how many IDC operations centers could ditch the projector in favour of a much lower power-sucking eink (assuming colour) display of a large size (think 60" or more)
White boards - we already have those annoying boards that people can print - add touch screen and a bit of colour and these things should be able to replace them.
A generic display, something that plugs into a usb port of a laptop - there are SOOOO many applications for this. Think of something that plugs into your HTPC and displays news for example.
Any public display - think airports, train stations, bus stops, etc etc.

Who ever comes up with that sort of thing, wins in my opinion.

Google app engine - how i wish it was....

2009-12-15T03:33:00.002+11:00

When google first went into the app engine market with their cloud I thought it was fantastic, but these days I look at it and think "what a missed opportunity".

I work for a consulting firm (as I may have mentioned many times), and so I get to see what alot of clients (from massive corporates to 2 people in a shack) and hear what they have to say (and what they know) about whats out there in terms of what they are trying to achieve. Seeing what Google has done with their cloud is frequently making it "not really the choice people wanted" for many reasons.

Now cloud computing has become somewhat of a buzz word in the IT world and those companies that depend on their IT, but dont necessarily have huge IT resources (in terms of staff, not computing) are often dependent on going to other people to tell them whats best. They often have read enough to understand the fundamental differences between offerings but don't necessarily feel equipped to make the best decisions, or in the very least don't know what other options are available.

The company I work for often works with clients and hosting providers and every hosting provider we work with now says they do cloud computing. In reality, none of them do. Well I say that, but what I mean is they've just re-badged something like colo or VPS-style services and thrown the word cloud in there. Which kind of sucks in my opinion, no real innovation involved.

But, with the mid-to-large clients they are often VERY aware of what Google's cloud is all about and they are afraid of it. Not because of the reasons you may thing though. A recent example a client in manufacturing decided they wanted to push all their computing back into a hosting providers data center and they were curious about cloud computing. They had read all about what Google's cloud was doing and were frightened of what it offered. The place they wanted to go said they were doing cloud computing and so they were afraid of that too - until they found out it what they were really being offered. There were three simple reasons:

We don't want out data mingling with other people, and we want it as secure as possible.
Same again with network
We don't want to have to custom build alot of apps.

I hear this over and over again (number 3 especially). Then when clients sit down in front of the hosting providers we know and find out that their cloud is simply the same colo services their used to (perhaps with some virtualisation thrown in) they breath a sigh of relief.

So what (in my opinion) did Google do wrong? Its very VERY simple. Its a closed cloud.

By closed I mean software wise. You can only run a Google app engine app in the Google cloud and this is (imho) a huge fundamental error on Google's part. They now have Google Chrome OS (or at least, its in the works - i do indeed have it running on my netbook). But they should have done the same with the App Engine cloud. "Hey, download this iso, install it on your server and you have your very own little Google app engine cloud". Add a simple web based gui for joining machines together, deploying apps, managing storage/network and suddenly the cloud has a secure storage and network component your in control of. By the way, I am aware of various efforts that involve running Google app engine apps outside the cloud but none of them are appropriate in a corporate environment.

After this, they could have done some very simple things to get people moving into the cloud. Simply take this private cloud gui and give it a button that says "take this app and shove it into Google's cloud" and this is exactly what people would do because they would actually gain some confidence in the Google app engine as a result. They'd also realize that not all of their apps are running super-sensitive data that need be protected to the nth degree.

This would also foster some very serious development efforts in terms of Apps available to push into Google's space. Why? I hear you ask.

Consider this. Im a company that decides to give this thing a shot - and perhaps I look at my accounting software and go "we could do this bit in a local Google app engine cloud". But the downside is "this is going to cost us some cash to develop" and "we're going to have to migrate out data into our new app" - in most cases thats some big money. Now if Google also had another button that was simple "please push this app into the Google App Engine market place and i'll sell it for $x per ". Suddenly companies are realizing the dev efforts that they may spend on creating App Engine apps could have a pay-back and a worth-while one at that. Perhaps they could even sell their developed app to some development company who would continue keeping it up to date. I.e. "this app is for sale in the market place" - not as an app you can use, but as an app you can then buy, develop, then resell yourself and ultimately you'd now own it outright.

On top of this, other people would be out there going "what apps would you like to see in the Google App Engine market place?". Corporates don't care if its java or python. They just want an app they know they can trust, and If they can then click on their local cloud and say "please deploy the app from Google App Engine that does accounting for me and here's my credit card details" - WOW, the response would be quite amazing in reality. Its such a shame Google couldn't or wouldn't do something like that because the app engine is quite an amazing innovation in many ways, but it'll stay out of the hands of many corporates cause of those simple reasons.

There are other things you could do with this that has various tie-ins back to Google. Take android and Chrome OS for example. Suddenly Corporates are moving their data onto their local private app engines and slowly realize "gee, we really dont need these massive desktops, Chrome OS would do" (ok, thats a long long way down the track, but you see where im going here right?). There's also authentication to take into account, and look to the wave as a good example of solving this (reading this may sound confusing so hopefully I can explain myself well).

Lets say Im running Google Chrome OS and I want to get to my corporate infrastructure, but im on the internet. Traditionally you'd need a vpn, but the wave is an example of how you might solve this. Think of the wave in this sense as a gateway in some ways. You'd boot up and login as "me@mycorp.com", which would then go off and find the auth server for "mycorp.com" (consider how wave/xmpp servers federate for example). You could then add cert-based or two-factor (fob keys) type auth over ssl to the server for mycorp.com. You then just route data over your secure connection through a wave-style server to the place you want to be internally. This also holds for generally authing to your local cloud - i.e. when you hit that app engine login screen and type "me@mycorp.com" it looks up dns for finding and auth server rather then just going straight back to Google.

Oh well, perhaps one day?

A few more points I think where Google got things wrong. The Java servlet engine. So good and so evil at the same time - "Lets develop our own standard for java servlets that no one else has". This isn't a bad thing in reality but its a shame Google didn't do something a little more fundamentally useful - anyone who's dev'd or admin'd j2ee knows the pain I'm referring to here. Their mod's to the java servlet spec for the App Engine are actually quite good and possibly resolve some of the annoyances with j2ee. Its a real shame they didn't just decide to go to the powers that be in the j2ee world and say "hey, we're going to produce a new Java Servlet Standard and call it j2ee-lite". Theres nothing wrong with new standards, so long as they are standards. Sun, IBM, Oracle et al would probably have embraced this (over time).

Similar too with dalvik in android. J2me is a lousy piece of work in reality, done more to harm development efforts on phones than anything else (well maybe not, but it certainly hasn't fostered them). Where if Google had decided to go to java and say "we want a new mobile standard focused on PDA-style phones", again the java powers would probably have embraced it realizing their other option is "we're getting the java source and re-purposing it in a way you cant use".

In truth, im not a huge java fan so it would possibly have been nice if Google had just invented a newer language for powering all this (no, i dont mean Go) - maybe even something based on c# or perhaps a more intelligent java? who knows.

All I Want Is A Linux Firewall!!!

2009-11-25T01:43:00.006+11:00

You would think such a thing would exist, but no, not really. This is not a rant, just a statement of fact. I've been looking around for one and here's my review of some of the options available... This does not include the commercial ones, im only intersted in trying out the FOSS ones. I've tried Asataro, Gibralta, etc and they are all quite nice.

Something I should point out, as this post comes across a bit negative. All of those below I found to be wonderful examples of what FOSS manages to achieve and they all do have a very useful set of functionality (there were others I played with that really were quite bad and so I didn't comment on those), but none were particularly viable replacements for a firewall.

One thing i've wanted for a while (especially in the age of virtualisation) is a router/firewall combo that could produce the basics of what these devices should do. To me, if your going to build such an appliance it should be:

Web controlled
Under 200mb
Be able to add and remove interfaces and configure simply
have a single login (well, multiple users, but one user database)
It should be able to to firewall things ( from ip, to ip, drop accept, you know)

Nice to have, but not required:

Routing - (I mean OSPF, BPG, RIP, etc) Protocol based routing abilities.
VPN - either point-to-point between firewalls and/or road warrior type things.
Load balancing - Do i really need to explain that one?
IDS - Intrusion detection is nice when it works
Packet accounting - Tell me how much traffic is going where and when.
HA - a simple heartbeat failover between two routers.
Proxy/HTTP filtering
QOS - I want that link going slow!
VLAN support - one interface will do me thanks!

An install for such a beast should go:

insert cd
Answer 3 questions -
Which interface should I use for management?
Whats its ip? (netmask, gateway, dns)
What harddrive do you want to install onto?
Go to web interface and configure the rest.

In the many years i've "done" unix, i've built several of these things and it always ends the same way:

Install "normal" distro (ubuntu, fedora, etc)
Install webmin

And alas, this is still the best option so far... But here's what I've found after downloading several and having a hack around on them. All these can be found linked off wikipedia, here.

Coyote/Wolverine - requires a IDE hd... seriously? what century are we in again? But really quite good

Both have a nice simple install
Both required you to configure a WAN and LAN interface on startup, but you couldn't choose what interface was what it just uses eth0 and eth1 - was very bizarre.
Does a reasonable mix of things - vlans, NAT, firewall, QOS, VPN (only wolverine does VPN)
Firewall rules are simple, but overly so, you cant get "complex".
Seems to be a bit spurty in terms of dev - big gaps between activity.

IPCop - Sadly not too bad actually. Where it fell apart

had to choose from a list of possible network interface config's during install (strange naming convensions) and couldn't add another interface later on (at least, not through the gui)
Seems more interested in working for little home-router style configs
Had to choose 3 passwords during the install, one for web admin, one for console (root) and one for backup. One password is enough thanks!
Has the whole Red, Green, Orange, etc interface notation of last century.

RedWall - Terribly convoluted:

The "install" was a nightmare, and im really not sure if I made it all the way through or not
The documentation is ... not good
The install is so terribly convoluted it really was far FAR too complex

ZeroShell - Probably the best

The install on this one was great
The thing is sadly still beta - but quite impressive none the less.
The firewall interface for adding rules was heavily complex, way more then what is needed - what it could have been is simple "src ip/iprange", "dest ip/iprange", "interface - if specified", "src/dst ports" and "accept, drop, etc" with an "advanced" button for those tricky things - this is primarily what a firewall needs to do
Does a firewall/router really need IMAP, POP, Kerberos, etc?
Interface config was fantastic, supporting vlan's.
Some things were complex, some things were dead straight-forward

Smoothwall - Again with IDE? But very decent.

Required an IDE harddrive - or at least, didnt understand VMWare's SCSI harddisks.
Based off the same IPCop install and its very annoying
Had 4 network interfaces (choose 3 during install), couldnt add interfaces after initial install
Has a really nice web interface
Did alot of the "nice to haves" listed above
Firewall rules config was great.

eBox - Big and bulky

Its not a firewall, its an everything.
Based off ubunt/debian with the same installer - its HUGE. Leave aside an hour just for the install.
Does lots of things and not really useful as simply a firewall/router replacement (i.e. filesharing, chat, dhcp, the words).
If you want a box that does almost everything you could want on a lan, this is it.
Not bad at all for what its aiming to be - its well worth having a play with.
Firewall configuration interface is "annoying"

IPFire - ipCop/Smoothwall, Again

Same annoying config as ipcop/smoothwall/etc - interface sets defined at install.
Web interface is very complex in places.

BrazilFW - coyote based thingy

Its not english - or its english in places, but not enough to get whats going on.
im sure its great if your brazilian....

pfSence - BSD, nearly the best of the lot

The isntallation is quite odd - you go through and setup the interfaces after booting the cdrom and you can do alot there, or you can go to the web interface - which then forces you to do it all again. I dont understand why they dont choose one of the other. forces you to configure a wan and lan interace (at a minumum) then forces you to do the same again on the website.... very bizare.
Firewall rules border on over-complex but its still quite good.
Web interface can be a little confusing in places - menu's could be better organised, and items could do with more text.
Supports VPN's, Firewalling, multiple interfaces, vlan's, load balancing, rip
All in all, its pretty complete.

Untangle - Didn't get very far

Takes FOREVER to install
Forces you to create an account on untangle (i didnt do this)
uses that eternal/internal/dmz network interface config for seemingly no reason (does detect extra cards, calls them "ethx" etc which is fine by me)
im sure its nice for someone, that someone is not me, and its not a firewall/router replacement. Its trying to be an "everything" server a soho might want.

ClarkConnect - Community Edition - not too bad

Install was nice and straight forward, but thats where the good times kinda ended
After install, trying to get to the web interface was a nightmare - had to keep logging onto the console of the box and dropping its iptables - why cant i get to the interface from inside???
Can define what "group" an interface belongs to - i.e. "internal", "external"
Sadly, had the same idea of "incoming" and "outgoing" in that you'd say "let port 80 in" rather the allowing a useful firewall config - i.e. allow connections from a to b on port x
Does vpn'ing and ids
Is another "does all these things, so im really a single-box soho".
Community edition doesnt do 1-to-1 natting (how bizare, its not like linux doesnt it do it out of the box)
The interface was quite nice, but still not really what we're looking for in this blog post.

Vyatta - Community Edition - Quite nice but the web interface is truely horrendous

Not a great install, but not bad.. The thing boots straigh to linux login prompt and you sit there going "ok, and now..?"... then you run install-system and sit there again going "ok, and now...?" - that could be done much nicer than that.
Has both a cli and web, which is nice, the cli has obviously had some input from cisco-types. Cisco people wont immediately know how to use it though
Has this annoying vyatta logo on the vt which means when you do something like "ifconfig -a |more" the screen pager loses the top 2 lines - frustrating!
Runs on a NORMAL web port (i.e. port 80 and 443) with ssl. For some reason, alot of routers do not do this - dont ask me why, its very bizare.
The web gui isnt fantastic... All things considered, i'd say the navigation of it is incredibly tedious, but the amount of required info for doing any one action is suitable.. i.e configuring an interface you have to dig down several layers of menu, then the main interface page is simple... you can then configure more things on the interface is deeper menus rather then having to do it all on one page.
The firewall part of the interface (i.e. where you set rules) is truely horrible, your forever digging forther and further into the interface. Was the only truely tragic part of the interface really.
The interface is very raw, like someone had an xml config file and decided to wrap a gui around it as a side-though. Like, if you go to an interface and click on "vif", all you get is a dialog that says "vif: "... then if you type in an invalid ip address it says "this is not of type uint32"... right, good ol uint32.
Supports almost everything a router/firewall replacement should support - Firewall, vpn (multiple types), bridging, vrrp, vip's, load balancing, OSPF, RIP, BGP, clustering, ipv6, some services (proxy for eg).
The interface does have this little yellow ball thing that tells you were you might have problems, and that was quite nice.

And that is pretty much all of them.. see now why i say standard distro + webmin is actually an improvement on most of them? the iptables interface in webmin is actually not bad at all. Its a pity too cause linux and bsd both have such strong networking backbones, with alot of support software. But if you want to replace an old router, or perhaps just add one into a testing system its either get fedora 12 or ubuntu 9.10 (or any other current distro) and hit the webmin install button, you'll be much better off really.

linux async replication - dm-replicator?

2009-10-31T02:47:00.004+11:00

My last blog post, i was talking about how i wish I had the time to implement a replication scheme based on lvm/dm.

Well, a while ago i stumbled on a thing called dm-replicator which was presented at a conference around October 2008... but that was the last i'd heard of it. That was until the last week when suddenly lvm/dm got some patches as did some kernel level patches suddenly appear.

Im quite excited, but details are very thin, and reading the code kinda disappoints me.

From what I can tell, its going to be another targetting-the-remote-computer style config's, which is a shame because keeping it local-only would simplify things and probably avoid alot of patent trouble (there are alot of storage vendors that hold patents around remote replication).

Secondly, anything you could do locally you can do remotely (think iscsi, vblade/aot, network block device, etc)... Consider the following situations where local replication is good (at the lvm level):

1) I have a usb drive that i plug into my laptop occasionally - sync logical volume root into volume group usbdrive
1.1) I have a ata-over-ethernet device sitting at home that i want my laptop to sync to whenever im on that network.
2) I have a iscsi brick sitting in a data center a long way away, there are no servers - lets talk directly to the storage brick.
3) im using shared LV's with clvmd, how can i do ordered replication between multiple hosts talking to the one LV? - woot, lvm-level replication solves this problem
4) keep it simple - a network layer to replication makes it complex - see drbd.

Im still sorely tempted to actually write a replication device - not within lvm - but that is lvm intelligent.

I really hope the lvm dm-replicator gets a simple form of async, local-only replication cause if its remote-only it'd be a sore loss of functionality to what is (in my opinion) one of linux's greatest storage software.

I'd love to ask some questions on the mailing list, but its hard to know which one to ask on because the code has mostly been synced into lvm's cvs rather them dm's yet most of the code is actually kernel level device-mapper code...

Linux Replicaiton, one of those things i wish i had the time for....

2009-10-15T02:42:00.002+11:00

Asynchronous Replication is a wonderful technology - when it works right (actually, usually it always works right, its implementers that screw it up). But linux has never really had a good solution for it.

There is drbd, and thats about all there is. Now its quite pretty and all, and does what you want to some degree, but it lacks (rather annoyingly) some functionality. DRBD is designed such that you create a block device that you pass-thru for replication. It them transmits the changes to a remote machine also running drbd that then replicates the changed blocks onto the remote block device. Essentially, thats what replication should be about and drdb has some configurability around that - like synchronous replication.

Let me explain that a bit better from an OS perspective. When a program writes to the disk, it opens a file name, does some writes and then closes it. Now depending on the FS, the device its writing to and a number of other factors, this may or may not occur straight away, however the program (when it calls close()) gets a return from the kernel saying "your writes are complete" (well, not really how it happens but essentially it means your writes are "commited" to the disk).

Now, when you setup a mirror disk (i.e. LVM or meta devices) what happens is that the write isn't complete until both disks have committed the write. This is synchronous replication, i.e. both disks have the data. Async means that only one disk commits the write and the other disk can commit the write at any time.

"great!" i hear you say, "i'm going to have a mirror thats out of sync, what the piss is the point of that". Well, glad you asked because its really quite useful. Lets start with a simple example of where it isn't used but could be. Lets say you have a computer with two really fast 15k RPM 300Gb SAS drives and a fair of 7.2k RPM 1TB SATA drives. What you could do is setup a mirror of the 15k drives and then asynchronously replication on to the SATA's. Sure, my SATA's wont have the most up to date data, but if my SAS drives fail they'll at least have most of it. Essnetially, what your trying to do is protect your data with cheap disk without impacting the performance of your fast disk. Now lets expand this some to places where it is used.

A very common scenario is DR (disaster recovery). Typically this will involve two sites and usually centralised storage (replication is very rare at the server level, its usually done on a SAN type system). So what I do is I have some really fast SAN that holds all my data at my primary site, but im a company of a decent size and I really dont want to be screwed if I have a complete site failure. Examples of this are:

SAN device completely fails
building blows up
natural disaster wipes out building

Bsaically, anything that has the power to turn my primary site into sludge. So what I want is a backup site where I can replicate my data, but you also want it a decent distance away from my primary site (after all, a bomb that hits my primary site could very well hit my secondary site, or think earthquakes, floods, tidal waves, all sorts of natural disasters that buildings wont stand up to).

Now you essentially buy 2 SAN's and put one at each site, however in order to mirror to my remote SAN I need some serious cash because the kind of through put on a SAN would usually require serious fibre links between the two sites capable of handling the entire workload (at its maximum) and that can cost you significantly more then your actual SAN (ouch). So what you do instead is cheap out. You buy a SAN at the remote site with less impressive specs (i.e. remove those 15k rpm, 300G expensive Fibre Channel drives and replace them with sata drives at the remote site) and provision a cheap link between the sites. Then you setup asyncronous replication. Practically any SAN on the planet can do replication (in fact aside from the dirt-cheap ones, I dont really know any SAN that would be used in a company that cant do it). The idea is simple, you watch the data changes occuring on your SAN and then ship those changes to the remote SAN as fast as you can where it then writes them to disk.

Keep in mind, all this is async, so when my server writes my data, it can then go off and do more writes elsewhere while the remote SAN may not have even caught up with the first set of writes. Now this means during peek load, that slow link is probably getting flooded and the remote SAN is falling behind by a significant amount - however, at least you still have some data you can work with. During the low-load times my remote SAN catches back up to the lastest data changes.

That in a nutshell is async replication, quite widely used and very useful in terms of protection. This is also what i've wanted to code into linux for a long time simple cause DRBD is fairly nasty and large while not being able to do simple things. I also find DRBD isnt great across slow links and my idea was simple "use whats already available".

In my mind, thats LVM. LVM already exists at the right layer in the kernel to be able to do this and all it would require is some meta-data store (for storing the id's of blocks that have changed) and a command to initiate it all (lets ignore "remote" for now). Keep in mind, I had planned for something that is useful even on desktops here.

So what I do is plug in a disk (lets say its a usb) and my desktop is running a 300gb 2.5" 7.2k rpm Hd, while my usb disk is a 1tb 5.4k rpm slow-as-buggery drive. Now my desktop was installed with LVM on the main disk so I'm already half way there (lets say my root VG is rootvg and my root LV is rootlv). I create another VG on the 1tb sata called replicavg. I then do this: lvreplicate -r -s /dev/rootvg/rootlv -d /dev/replicavg.

What this command would do is tell LVM to replication the logical volume "rootvg/rootlv" into the volume group "replicavg". LVM (in the background) then goes off and creates a logical volume large enough to store my replica data and begins the process of replication. Now being a USB disk, im probably not going to have it pluged in all the time and thats where the metadata store would come in handy - it would just keep its log of changed blocks while It waits until it can see replicavg again. Kewl? with me so far? So now your desktop machine has a viable option for replicating your important data externally on a drive you dont have to carry around all the time.

This would be lovely, but could work for servers as well - but first let me explain what else I would implement. Snapshot replication. Snapshots on LVM are wonderful. But if i can do them ordered on a replica, they're even kewler. So basically rather then snapshotting my local storage I can say "snapshot my local filesystem at the replica when it catches up to this point in time". Which would be very kewl.

Now, I said "forget remote" and I meant it, I also said it'd be useful on servers and your probably thinking "whats the point of servers without remote replication and that whole dr thing?" right?. Well its quite simple actually, here's where you introduce things like iscsi or aoe (vblade). re-use the tech thats available, and KEEP IT SIMPLE STUPID!. All the LVM has to know is where its replicating to (the volume group), leave it up to the tech that already exists to deal with the rest. LVM wouldn't require too much work to do all this, it would have alot more functionality that drbd (i.e. its useful to almost anyone). DRBD forces you to make choices and can only really do remote. Take an LVM vg for example, do you replicate the underlying volumes (i.e. create the VGs passing through drbd or do you replicate the LV's - tough choices to make and they dont give you the ability to snap effectively). Ultimately LVM was the perfect place to implement a replication scenario and i've always (for years) wanted to do just that but never have the time. Now btrfs is out Im wondering if it would be a better option for doing just that? who knows... its all too hard really. On top of all this you either sacrafice the things lvm brings to the table (things like dynamic volume sizing) just to do replication.

Its not easy, dont get me wrong, there are a lot of complication. For eg, ordered data is tough.

for eg: lets say I have 5 blocks on a disk that change like so:

1a 2a 3a 4a 5a
1b 2a 3b 4b 5a
1b 2b 3c 4b 5a
1c 2b 3c 4c 5b

Then i plug in my drive. The disks on the source volume look like point 4, yet my drive is going to try and replicate in order through each point, now lets say I let it get to point 2. What it'll do is copy blocks 1 3 and 4 because thats all thats changed at point 2 (PAINFUL!), yet at the source, we have 1c 3c and 4c. So my remote disk looks like: 1c 2a 3c 4c 5a - this doesnt correspond to any point in my ordered data stream and if these are blocks in a mysql data table, your going to be pissed off to say the least because part of a table (perhaps even part of a single column/row) are at the latest data while parts of it are at the oldest. This sucketh and ultimately has the capacity to kill the entire replicated data.

This is where it can hurt performance because you either copy the block at each write point to preseve blocks that got tampered with or you just "live with it"... dont really know the answer to that in reality.

Nokia Maemo - WOW!

2009-09-30T00:34:00.003+10:00

Lately, well actually since the openmoko I've been very entertained by a phone aimed at linux hackers such as myself. Unfortunately the openmoko was slightly disappointing from a number of aspects that included things like not-quite-open hardware and lack of connectivity.

Then along came google with android and suddenly the field changed a bit. Google certianly courted the linux community with a phone OS that had some serious possibilities. I really wanted one but was turned off somewhat by the thought of coding in java (again). I dislike java. Still, the android phones were quite interesting to behold and when the HTC hero came out I found myself very much wanting one.

That was, until the cyanogen disaster - so here's a bit of back story. I nearly had a Hero. Sometimes the worst type of frustration (at the time) can turn out to be the biggest lucky break. I was going to buy a Hero from a local distributor but a set of annoying email exchanges (and a fair amount frustration) gave time for the cyanogen mess to unfold and for me to go from a google loving fan boy to a "why would google attack its own hacking community" then finally "I can see no good excuse for this"... Even MS done do things like google did, and thats really saying something.

(you could also mention the iphone, but its just not a great hackers phone when it comes down to it).

Then along came the maemo and a couple of friends who I'd deem linux hackers were quite intersted in it. Finally it peeked my curiosity enough for me to take a good look at what this maemo actually was and heres what impressed me.

For starters, whenever I go to the nokia site I think "symbian" and locked-down-to-carriers and heavily-locked-developement phones with a very highly corporate and dumb-consumer focus. What I found astounded me. For starters the Maemo site looks very pretty, very nokia in some ways, but not in others. Then I discovered the "hacker" side of things. Links to things like irc channels on freenode and the like, and that one thing really made me sit up and pay attention. IT may sound a little shallow to base the whole thing on one link to a freenode irc channel, and thats not what nailed it for me but it is what made me really sit up and pay attention.

That nokia is aimed at "me". Its aimed at guys who mess around in ubuntu and fedora (or any other linux distro really) cause they love it and yet it has the ability to be a really impressive phone as well. In the very least it makes me realise its something worth waiting for.

Now a bit of phone back story for me. My best experiences have been nokia's, but nokia's have always had a feel of "im a mobile phone, dont screw with me cause you cant" unlike some of the other phones I've had. My first step away from nokia was when I moved from from vodafone to three and 3g (this was many years ago when three started up in AU - i.e. the very begginings) and I got an NEC e606 (not a great phone in reality, but it put up with alot of punishment). Nokia didn't have a phone worth getting back then (or at least, they were so expensive in the 3g range they just weren't worth considering). My next phone (some years later) was a Sony Erricson - the most disappointing phone i've ever owned in most respects. It put up with alot of punishment but was slow as a dog, it did however have a decent battery. My next phone was a Dopod D810 (my first windows mobile) and that wasn't a terrible phone. It was stolen about 4 months later and I ended up using my Sony phone again for a bit. Work started getting new phones and I ended up getting a TYTN2 phone. That was about 18 months ago. This was a thoroughly frustrating and disappointing phone to begin with (especially after the dopod). It was slow and bulky (stupid keyboard). But later I discovered the xda-developers hacker community for the phones and my interest stepped up a thousand notches. Ultimately it made the phone "fun" and so I kept at it for a while (consider: im a linux guy who is very anti-MS and anti-apple).

Later, people started porting the linux kernel to the phone and even various android (including the hero) interfaces. Those were the things that made me interested in Android again (the playing i'd had with android 1.0 was mostly disappointing, but it was fun playing in their dev environment). OF course, porting the linux kernel to the tytn2 had some interesting consequences such as being able to run other mobile-focused environments on the phone (opie, gpe, openmoko, etc). So the tytn2 has certainly made my life fun in ways I didn't expect straight out and I still do enjoy it.

I wonder who can really be thanked for the sudden uptake of "open" in the phone business? Openmoko maybe? Who knows...

Cyanogen and Android: How could Google possibly be hurt by this?

2009-09-28T14:51:00.002+10:00

Android Developers Blog: A Note on Google Apps for Android

If you follow android like I do, you have heard about the Cyanogen mess. And perhaps Google's response.

What I don't understand about it all (after reading about it for days and days and trying to get what is going on) is what Google could possibly be thinking. What would Google possibly have to lose with people distributing their closed source apps on custom roms?

I seriously am asking that question - there's piracy and then there's what brain did I leave at home today. I wish someone could answer that question.

See the thing is i've got a HTC TytnII phone and it kinda sucks as phones go. But when linux got ported to it (and later android) it made it worth using (or at least playing with)... even more importantly though, once the magic and hero came out, I wanted one. In some ways its was a shame that i'd finally decided i'd get a magic and then I found out about the Hero and now im kinda stuck between a rock and another rock cause of the Hero not really being available in AU, etc etc etc.... Anyways, i'd finally decided i'd go with the Hero (despite the fact that I could get screwed by compatibility problems if the company I work for switches carriers) because it just looked pretty dam awesome.

I played with the hero port on my tytn2 and its good (not 100% usable, but quite impressive). Impressive enough that I really wanted one.

Enter cyanogen mess.

I was literally 10 minutes from buying the hero when I saw the slashdot article in my rss reader.

Now lets take a bit of a detour. On my tytn2 i've gone to the xda-developers website (alot) and downloaded the 6.5 rom's that are available, all without much thought to the fact that technically thats piracy right? (or is it?) My phone comes with a license for 6.1, not 6.5. Yet MS have never raised a finger (that I know of) about these modders. The rom's include lots of MS software too.

So someone, anyone please explain to me how google lose out from someone distributing the google marketplace application?

I mean, my tytn2 has obviosuly never had a license for google marketplace yet how do google loose from me having it? If I buy software online do google not get a cut or something?

Im truely lost as to what drove google to this decision, and even more importanly I've lost faith in them, at least for now. It's been a long time since i've felt that kind of disappointment, its the kind of "loss-of-innocence" thing you go through when your grow up - something you don't expect when you hit 30+.

Additional:
Just so people understand where im coming from, I could understand if the person google went after was some rogue manufacturer of handsets who took the closed source apps and started producing an android phone. This was not, this was an enthusiastic developer with a large following of other enthusiastic developers. Traditionally this has been what has driven alot of what google do, so now Google wants to bite the hand that feeds it? So be it.

Even if google were FORCED to do this by their partners in the Open Handset Alliance, it could have been done much better without attacking people. If google had printed an "open letter" to android hackers saying "please dont distribute our apps", the community would have responded well.

In reality, google probably owe cyanogen money, cause I know at least a few people who bought android phones after seeing the cyanogen hack.

Contextual ToDo - an interesting idea...

2009-09-26T04:32:00.002+10:00

Now and then I think up something I don't believe yet exists (though it possibly does), and here's one.

I like cutting code but I often find that I dont have heaps of time to do it. For example, right now I would love to have a todo list that doesn't suck, and there are a few, but none of them inspire me to do anything. Currently I use Tasque on gnome and its not bad but it falls out of use often. I realise that alot of the reasons for that is that its often not visible to me so I forget what it is I have to do.

Now, back when I used to use KDE for my desktop (back when it was usable at 3 and not an ugly space-wasting mess) it had a wonderful RSS app that produced a scrolling list of RSS items in the panel (FANTASTIC!). I've always wanted that app so badly under gnome, I miss it greatly. However, I always thought that replacing the notification area with something like this would be much more useful then the current notification area.

Basically, whenever anything wanted to publish to the area it would add it as a text scrolly and it would stay there until it was acknowledged (or something else occured to remove it).

But, back to my task list app. What I had in mind was first an area like the above, where your current (applicable) tasks scroll across so you've always got the reminder there. If you scroll over the edge of the bar, a full list shows up (top to bottom). But i've also always wanted context with some simple rules and various task lists. For example if I'm on an IP address of a.b.c.* then I'm at work and I want my work task list, if im on x.y.z.* im at home and want that one....

Personally, i'd love to see something like that and If I had the time I'd make a project out of it.. but Alas, it wont happen anytime soon!.

Avahi - another great example of a missed opportunity (so far)?

2009-09-09T02:54:00.003+10:00

So I (dont ask me how, I dont remember) stumble across a little piece of software while looking for something linuxy called "avahi"... Its been there for a long time and i've never really bothered to learn much about it.

Until now. I cant remember what it was I was searching for, but I ended up at a wikipedia entry and was mostly stunned at what avahi actually is.

Consider I've been using linux since '92.

Avahi is a (so far) miss opportunity for linux (imho) now that I understand what it is. It is essentially a mdns service discovery tool for publishing available services from a machine to the local LAN (it can be broadcast across segments, but thats not so important).

At first I thought "Ahh, its so gnome can grab local machine resources easily" - not so, and when I realised what it did I soon discovered the service discovery applet for gnome. Wow that is very interesting. So you kick the applet in the guts and it sees all this stuff thats running on your network. In my case it turned up every server running ssh, a samba share and a few other non-essential things.

At least, thats what It did when I told the service discovery applet to look for everything.

Now imagine this in an enterprise - avahi can publish to real dns so that your servers could easily publish their available services to clients.

Now, there is the argument that "so anyone plugged into your lan can see the services available".. If this is what your worried about, you dont know security.

Still, What annoys me most is that its been around for a while and I never really heard anything about it. On top of that, the implementation is very limited at the client while the server side is probably the best server side implementation going.

Very annoying.

Various Ramblings and half-truths.

Linux Permissions - Time To Start Again....

GLCAS pt 2 - PXE Build Services

Star Wars: The Old Republic, My Review.

Android GPS Emulator

The Adonis Jot Android/IPad Stylus - WOW! - Never Wanted A Tablet Before.

Coding My First Android App - My Experiences.

Nexus S + ICS (Cyanogen Mod 9)

Cheapest NTP GPS Server EVER (Wyse S30 + linux + PPS GPS)

The Vertical Wind Generator

*MY* NoSQL Database.

Metacritic - A Sad Site Story - Or MW3 Nightmare?

Battlefield 3 (PS3) - In A Word "Disappointing"

How To Win The SAN Race

Linux/Unix and a new output pipe - stdobj

DD-WRT vs OpenWRT - OpenWRT For Me (usually)

A Guide to Getting Me To Spend Money on Android Apps

GLCAS Moving Along Nicely

Love That "Working" Feeling

Google+ App's - A Word Of Caution!

Ultrasonic Audio Networking - An Android App...

My Theories on the Universe.

Network Config Management

The Google Nexus S - its a mixed bag (no sd card? WHY?).

Rooting Android Is Bad... mmm'kay?

Which Phone Next?

An MMORPG I Think I'd Like.

Android 3.0 open source - im not sure what to think

ARM - the end of Intel and VMWare but not AMD?

Intel Atom Stand Aside - AMD, You Rule.

PS Network Outage - Its Just a Little Odd Really

Web Admin Gui's and CSS Templates

GLCAS: my latest project.

Android for the desktop - i'd want it...

Chrome drops H.264 - WHO CARES?!?!?

DNS - how I wish it was.

CrashPlan - nice piece of software

Google Authenticator - ga4php pt 3 (java vs Mono vs php)

Google Authenticator - ga4php part 2

Windows Auth, AD and OATH Tokens.

The Toshiba AC100 - not so bad...

Google Authenticator for PHP

2 Factor authentication - what I thought I knew...

Junos Olive and VMWare ESX(i) 4.1

Google Wave - what a pity.

One of the reasons I love linux....

Playstation Home - some things just weird me out...

IPv6 Auto Config - Its really not healthy.

Got IPMI?

IPv6 - its the place to be....

XMPP as the IWIN protocol.

two-factor auth on the cheap...

XMPP a solution for HTTP?

The exciting world of ARM....

eInk and the death of the ebook reader...

Google app engine - how i wish it was....

All I Want Is A Linux Firewall!!!

linux async replication - dm-replicator?

Linux Replicaiton, one of those things i wish i had the time for....

Nokia Maemo - WOW!

Cyanogen and Android: How could Google possibly be hurt by this?

Contextual ToDo - an interesting idea...

Avahi - another great example of a missed opportunity (so far)?

MY NoSQL Database.