According to the blog’s first post - I began in 2007. Vast majority of the published ( and those that died in draft form ) were a result of trying to fix things I came upon, and the associated learnings: be it automating WebSphere deployment 😜 or a GPS logger’s integration with Lightroom.

Here are a few new things I learnt week.

Wordpress Migration

I killed my WP hosting sometime in early ‘18, and just archived the content. Upon a number of conversations early on with Grant - I contemplated a conversion to Jekyll. But did not have the drive at the time to make the leap. So the WP archive to Jekyll ‘.md zip’ sat there for ~6 months.

A few days ago over lunch I had a quick perusal of static generators, and their associated ecosystem trajectories. Picked Hugo as the path forward.

The Good

• Hugo’s build, and validation is ⚡️ FAST, and Go’s Templating Language reminds me of HandlebarsJS. Fact that Hugo has limited theming & customisations is a boon to focus and productivity
• Markdown everywhere 😍

The Broken Still In Limbo

• Hugo’s Customisation
  • Albeit limited in themes and out of the box iintegrations ( a matter of time ) means in some instances like SEO, when you hit roadblocks - you have to design or add them yourself. Plus I really should have got the redirects sorted out prior to changing the structure
• Authoring Experience
  • Should not have match the quality, and fidelity of what Medium.com has created - but it has set the standard of what is possible in the browser for content creation and curation
• Decisions on Image optimisation
  • ie. LQIP, Progressive JPEG re-encoding
  • CDNs vs DAMs vs localised imagemagick
• LD-JSON content extension and integration
• Figuring out when to stop.. do I really need to write a custom Hugo shortcode to display Apple Watch HR data ?

Netlify

Immediately upon stumbling across Netlify, they piqued my interest. They have some notable pedigree from Github, Docker, Serverless but most importantly I love the user experience of their product: a streamlined barebones application platform. They have thought through the landscape of customer demands, competitors’ ability, and are now navigating accordingly. Just when you started to think that application hosting, and delivery is a done deal with same set of tools throughout, Netlify re-applies the pattern atop a highly optimising storage, and service coupling back-end to provide a very differentiated out of the box experience.

tmux & Kubernetes

After an immersive week of Kubernetes deployments, and some Wavefront, it was starting to be a pain to easily monitor the K8s state whilst trying to script several scaling and failover scenarios. Constant Terminal.app switching was inevitable. Jetisoning my love of screen it was time to adjust.

After a little bit of prodding tmux with panels for logs, hosts, deployments, services .. ugh ..

or I could have just used kubebox 🤷‍♂️

Visualising Apple Health Data

TL;DR Version

Add chart.html to your layouts/shortcodes folder.

Within your content use:

{{< chart id="meh" width="800" height="500" title="keh" data="heart.csv" >}}

The Step-by-Step

After a quite poor run with James, I decided to see how I went on the trusty phone, since Apple ⌚ tracks all movement. Quickly realised this is an opportunity for a post-dinner mini project.

Lets walk through how to create the headline graphic with ChartJS in Hugo.

Get The Data

The team at Test-Driven Data Analysis does an excellent job of showing how you can get the data off the watch, but most importantly convert it from XML to something more usable.

The fun starts once you have the CSVs.

Listing the generated apple_health_export at this point should present you with :

1.3G export.xml
385M export_cda.xml
318M ActiveEnergyBurned.csv
189M BasalEnergyBurned.csv
140M DistanceWalkingRunning.csv
89M HeartRate.csv
72M StepCount.csv
17M AppleExerciseTime.csv
7.9M DistanceCycling.csv
4.1M FlightsClimbed.csv
397K HeartRateVariabilitySDNN.csv
79K SwimmingStrokeCount.csv
77K DistanceSwimming.csv
63K RestingHeartRate.csv
62K WalkingHeartRateAverage.csv
12K VO2Max.csv
5.3K convert.py
971B BodyMass.csv
940B UVExposure.csv
497B Height.csv

Create the Hugo Chart JS Layout Shortcode

The inside of HeartRate.csv looks like this the following, multipled by hundreds of thousands of rows. So pick a subsection.

Charts themselves are quite straight forward, but you will need to map the data first

Recommended some dirty cut or awk magic, and cleanup if you’re going to be sharing / exporting, as naturally it does have your Watch’s identifier included.

Using lodash, map it to distinct dataPoints, and labels Arrays for injestion by ChartJS.

data = $.csv.toObjects(response);
labels = _.map(data, 'endDate');
dataPoints = _.map(data, 'value');

What works for brown-bag & “throw your screen up”-type demo-versations does not always translate onto a stage in front of hundreds & thousands of people.

Together with some brilliant technologists earlier this year, we had the privilege of showcasing VMware’s ability to enrich and accelerate the app development lifecycle at our yearly TechSummit’s Keynote.

What constitutes steps towards a half-decent technical demo ?

Arguably many of the below thought spurs can and should be applied to smaller spaces and any presentation. Yet the ’theatre’ on the stag necessitates considerations which in some case make it difficult to impossible to compare to its coffee table or Meetup brethren.

Feedback from the room

Outside of few, very vocal seated individuals, there is little presenter-audience interaction.

The near blinding flood-lit stages, although at first disorienting - they do a great job of disconnecting you from the audience. And if executed well, provide the feeling of being alone on that stage; great for those with nerves 👋, but not so good for getting body language queues from the audience. Especially since you’re no longer presenting to individuals but to a single crowd.

You cannot compress preparation time. Rehearse, rehearse, rehearse, yet be cautious of balancing the content rehearsal, without maiming the personalities. As with all presentations .. do enjoy them.

Validate the entire run-through with those :

• In the auditorium that are able to provide some honest and pointed course correction. Have them sit and provide feedback, whether they follow your story-line and and aren’t getting dizzy with scrolling confused with all the window changes.
• That are colourblind
• That represent your target audience - ie technologists, to validate the narrative so you can do any necessary fine tuning

Dress the content for the room

Dark or white background, typeface selection to match eligibility, aesthetics and again - intended audience. Be cautious and selective about each, and screen whether each window and click is aiding the immediate narrative and goal of the presentation.

CLI & Shell Demonstrations

Aside from this being a topic deserving its own post, you should need a really GOOD reason to dive into command-line during a presentation - and it better be legible. What is a simple scroll on your screen, translates to terminal velocity at a distance.

Upon typing a number of commands - the output will dominate the screen as the cursor is now at the bottom. Ask - are those in the back still able to see what you are narrating ? Try and keep to 1–2 commands per terminal screen invocation. CTRL + L is your friend here.

Later in the day, during lunch - I had a Solutions Architect from the US, thank me for using the same font whenever any code was shown on the screen. Wait - someone paid attention to that ?

From the slides, to the terminal to the text editor font size - the night prior, during ( our first rehearsal with all participants ), all of those were adjusted specifically to fit with the room, and the distance people would sit away from the screens. Perhaps a little 😅 OCD, but not only was the work justified after the comment from our stateside comrade, but these small, seemingly inconsequential changes and mannerisms make an enormous difference to helping people follow and understand what is being shown.

Challenge the Audience - with Respect

Paraphrasing Bruce’s frequent advice to the team - ‘do not forget the origins’. Effectively - try, and design the presentation flow with an explanation of the fundamentals first.

We often fall prey to our own wishes to push forward, and dig into the perceived most exciting, important and relevant topics, only to discover that we left the audience at a disadvantage. Name the changes, highlight the shift, and always point out the reason of why significant & relevant changes enhance their world.

Only by explaining the history, will the majority be able to truly appreciate and respect the delta of any advances.

Limit the Scope & Focus

“Would it not be great if we could .. " was not a good start. One of the inherent limitations to larger stages is the time constraint of making sure you finish on time. This requires the team to be able to say ’no’ to many opportunities to add more material.

Perfection is achieved, not when there is nothing more to add, but when there is nothing left to take away.

 - Antoine de Saint-Exupery

James’ pragmatism always brought us back to our focal area - “at the end of this session we wish to demonstrate” :

• what - pitfalls exist in a development process
• how -  an increasing level of feedback improves quality and project velocity
• products - across VMware’s Software Defined Datacentre portfolio that can enable Application Platform Capabilities within an enterprise

When a topic is dear to your heart, it is easy to go too deep or off on a tangent - especially with some over excited adrenal glands. This is where having gone through the rehearsals you create muscle memory of triggers for the story line and bring you back on point.

Whatever you do, try not to come up with an analogy on the spot that you think will resonate or humorous. // guilty

Preparation | Live vs. Recorded

There comes a time ( hopefully early ) where you must decide whether you will be doing a live demonstration, narrating a recording or as with some ‘click throughs’ try and fake live usage.

Literally upon landing in Hong Kong, two days before showtime - David discovered that the entire environment around which we built the demos around has carked itself. After spending some time in debug mode ( read : laptop being taken to a bar ), it was decided to just redeploy the entire showcase to another provider all together. Given the flexibility today in defining everything through code from underlying infrastructure, to application infrastructure like Kubernetes deployments to entire CI / CD pipelines which include Wavefront Dashboards.. made the whole ordeal so much more manageable.

 - 

Erring on the side of a technical audience, and whenever am watching - I prefer to spend copious amounts of time preparing for live delivery over recorded. James - the voice of reason, asked for recorded demos to be available during the keynote, so at the very least, the night prior we did the screen captures.

Wrap Up

The whole team travels quite a bit. It was not a dramatisation of us doing for the first time the complete physical rehearsal the night before. There were days we would hear nothing from some of the members, but when they came to the checkpoint meetings - they brought the goods.  What this experience has taught me, through the ebbs and kicks of technological dependancies, relying on the right team that has a sense of purpose of what and why they’re doing something - allows for exceptional experiences.

With that, an enormous THANK YOU to James McAfee, Grant Orchard, David Lloyd and Clement Wong for delivering an outstanding session with many lessons that we can all take into the future.

The Team

References

• Macaque -Mark MacEwen/NPL
• OODA Loop - D!gitalist

Aside from Betteridge’s law for the headline, one of the reasons for which I embarked on pursuing an Executive MBA is that I wanted to bridge my engineering and technical background with that of commerce, economics, corporate finance, considerations for strategy and its implications on building an enterprise; ‘the business fruit salad’ … I wanted to learn.

Having now completed a few courses, for me, the old adage of “perception equals reality” is not a truism across all courses (more on this next time). I will preface that the following experiential verbiage may be different to that of other schools, but anecdotally via colleagues, friends, and people that I met whilst studying on planes and airport lounges that offered their advice / experience — the issues am about to describe are de rigueur for an MBA education here in Australia.

Problem

The most prominent disconnect is between original thought and espoused validity of the content being delivered. But first lets understand the structure and workflow of the quasi-learning process that Australian Executive MBA’s look to provide.

At the beginning of each course, you are couriered an A4 binder, dense enough that were it dropped from a metre above ground, it would result in instant death for any medium sized pet playing catch underneath. Not to get too captain-Greenpeace here, but many many trees are hurt in the description of Porter’s model; and I believe anyone investing $4,050 AUD per subject, would have the IQ and bank reserves to be able to print portions of the material if necessary. Opening the said binder, undoubtedly provides much glee to the course coordinators who meticulously foraged the latest licensed articles from Harvard Business School’s — HBR for remotely sounding affiliations with their subject, with carefully centre-justified quotes whose longevity would be more impressive as cursive tattoos just above the derriere; and whilst good curation is a valuable skill, mere copy+paste regurgitation of another business school’s collateral is not.

In lies the crux. Beyond paraphrasing (well referenced of-course) business tabloid thinking of matrix organisations, conflict resolution, team norming mantra and adding page numbers .. very little original thought is actually added to these class notes. The learning methodology employed here is loosely wrapped around the following formula:

Grade = Assignments + Class (1.5h per week) + Workshop (5h) + Exam (3h)

Using the binder as the guide, you typically end up spending around 10-15 hours per week per subject. Which I believe is very reasonable.

The classes, and workshops are driven by a facilitator, but as expected, unfortunately not all are made equal. Some compliment the material with their successful experience and insightful points on framing topics and their application whilst others use it as a soap box for their acute, ‘one size fits all’ approach.

I queried the studying of not simply the past and the forcing of various industries into very specific moulds, but new and counter-routine ways in which some businesses are being structured today, i.e. holacracy, relevance of Porter to co-opetition…, these were immediately dismissed. It was made clear that only successful and valid opinion that has experienced academic rigor is to be discussed, not some quack notions; which I do, honestly believe is a very fair argument. Yet my bringing up of ideas outside the curriculum was not to simply anoint the latest trend as the “right” way and be a thorn in the facilitators proverbial, but to leverage the intellectual power of the immediate class and discuss what they will face when finishing their MBA, and note issues that will inevitably creep into their workplaces in the near future.

After a heated exchange, I was unequivocally told that the :

purpose of the class is not to view and analyse existing and possible future approach to business thinking, but to “enable me to pass the exam”.

Surprised, I asked whether “…this is an educational facility, or a business of generating tick-box MBA’s”. Perhaps naïve, or marketed-to well, I wholeheartedly hoped it was the former, but the lack of an audible response and a stern gaze from the facilitator clearly signalled otherwise.

Not a light (… or cheap) decision.

Took a few years to think about it - but finally made the leap. Am now enrolled in the Executive MBA at AGSM. This will immediately not be a review of their programme, but will morph into it as the months ( and years ) progress.

After numerous talks with friends and colleagues that have gone through an MBA, those that started and some that bailed. It was imperative for me to understand the reasons behind each decision.

Contrary to majority of those with whom I’ve spoken, and a number of those in the current class, am not doing it to change my job. I love what I did yesterday and what I do today. To me it is simple. As the few posts here would indicate, coming from an engineering background, I found it important to not be so obtusely focused on technology alone. But rather the manner in which I can help make it as transparent to those consuming and interacting with services underpinned by it.

The greatest value of the MBA so far has been a concoction of several attributes:

• Discipline of attendance and the need to learn new things. Some call it study, but for me that has a negative, wrote-learning connatation, that I highly discourage. Studying ≠ learning
• Group discussions - whoa! I’ve arranged flights around my classes so that I can learn through osmosis and where I can, add back. These are the best and most vital in my opinion opportunities to get an insightful view from someone elses experience

School

There’s a plethora of brainwashed prestige-orientated school rankings available from FT, Economist and Businessweek as well as many others. Criteria that were important to me were:

• no GMAT exam - say what you will, but am not sitting again for the HSC, or equivalent
• location - am not always in Sydney, so an ability to attend a class elsewhere is important. Make sure that the institution you pick, provides a facility to walk into a class that is most likely to be in a location you’ll be traveling to often. This is easier said than done, especially if your role requires you to cover a wide geographic region
• flexibility - for the times that you’re unable to attend  class at all, you want to make sure that you have access to all content that is possibly going to be covered
• your experience - one of the reasons I held back from starting the MBA earlier is that I couldn’t add much to the class that I would be attending. Inversely, I too would be getting very little from it; as the course-work is a framework for your experience
• experience of attendees / pedigree - as I mentioned earlier, you will learn more from those around you, than you will from pure notes. This is where it really matters of who is in your class, and where the prestige/ranking unfortunately matters
• digital acceptance and leadership - not all schools are alike. This is one area where AGSM is not just behind the eight-ball but significantly lacking, and I didn’t realise how badly until I started. This will require its own post, but in summary - AGSM believes that supplying all material for study as PDF’s is ticking the  ‘flexibility’ and ‘quality’ boxes

Tips

This won’t carry too much weight, as at the time of writing this, am just half-way through the first subject. But having seen a number of other successful students and how they take to the course coupled with what’s worked for me:

• study group - yup. The gatherings of the un-cool. Do it. Organise one immediately after orientation day. Make sure it has a good cross-section of ages, opinions and importantly - roles. If you are an engineer, and there’s another one looking for a study group - run immediately in the opposite direction. I started the MBA to get an understanding of the breadth of business dynamics and its applications. Not to see your favourite gcc argument combo.
• time management - perhaps the most obvious one. Put away at least an hour each night not for catching up on House of Cards but to cover off material and anything raised during class discussions / study sessions
• be ahead - whatever you do, don’t fall behind. Chances are, you’ll be working at the same time, and you never know when your job will kick you in the nuts. Be ready for it, as study will be the first thing to take the back seat in importance. Have the ability to take a week off, without being affected

The point I made above which relates to the cost, please don’t think of it as purely financial. There’s a significant personal commitment of time, energy and sacrifices you’ll have to make with personal relationships to see it through. I settled on AGSM, as it stood out more than the others in the vicinity mapped to what I needed from the University. So far, not bad.

That’s it for tonight.

Interestingly the most visible SSD proponent and database service is none other than Amazons AWS service DynamoDB. Launched earlier this year, a database service which can be scaled at the turn of the knob, whilst offering performance, availability and the NoSQL feature-set. Considering the whole premature klonking-out with small-block, highly-randomised IO that hits the SSD controllers, you’d think a very ambitious undertaking.

SSD’s are great, but not a panacea for all workloads, in their raw state; especially for write-skewed database workloads. One of the largest causes of headaches for SSD Controller manufacturers and those looking to optimize workloads on SSD’s is Write Amplification, otherwise known as the variable amount of operations or re-writes of the original size that have to occur before a write is actually processed / stored by the disk.

The conversation reminded me of a post by Todd Hoff who attended an AWS session of “DynamoDB for Developers”, and wrote :

Initially they ran into performance problems due to SSD garbage collections cycles

This is inevitable. In my VMUG talk from last week, I refer to the Controller, and rather its FTL (Flash Translation Layer) as the black box of the device, where all the secret sauce of the unit lives. This isn’t by mistake, as all the IP around the type of algorithms to use for wear levelling, garbage collection or any internal process resides within.

A single SSD within a single host, can cope with a lot of write-abuse, as given even a few minutes of ‘idle’ time, can effectively “defragment” its block + page layouts, and thus provide a buffer for the next onslaught. Although en-masse, where AWS operates, this simply will not work. As you will have many thousands of users and transactions in the hundreds of millions (+speculation), the drive will not have time to rest and carry out the vital to its survival, levelling and Garbage Collection.

As the folk over at Tintri have shown, when SSD’s are pummelled to a point of saturation OR effectively outside the boundary of when the internal algorithms perform cleanup, the performance or rather the predictability of said performance is significantly diminished. Below you can see latencies of nearly 300ms for an SSD device.

When you’re operating a on a significant scale, you can’t have “unknown” hold you back, so that’s why (+speculative mode) Amazon worked directly with the manufactures of their SSD suppliers to have a say in:

• When and how GC kicks in
• RAID design across individual cells (similar to vRAID within Violin)
• Granularity of block mapping
  • How blocks are read+cached+written, and aligned DIRECTLY for DynamoDB’s block-sized, in order to not necessarily alleviate, but minimise re-writes
• Locality of writes

The last being most interesting, and I believe an absolute must for the future of SSD to have an enduring life in the enterprise. Applications, or rather file-systems and thus API’s should allow for the ability to direct or at least specify the inter-relationship of data being written. Simply dumping blocks and forgetting about them is simple, but has dire consequences for the life of the storage medium in this case. Surrounding writes, or data operations with meta about them allows for solutions such as DynamoDB to exist and flourish. This requires a very tight integration between the software and hardware. There’s a reason Apple is one of the largest NAND / SSD suppliers (direct to consumer) out there, and why they bought Anobit, the Flash-controller firm last year. Go ponder.

Not saying it’s impossible, but uploading, converting, VMDK/ OVF/AMIs, starting them, adding IP’s to loadbalancers… brr - is a joke at best.

— Roman Tarnavski (@romant) April 17, 2012

Working with a heavily enterprise-led customer base, I was excited to see an article looking specifically at how large corporates can extend their workloads into the cloud, within the industry this is commonly referred to as cloud bursting and unicorn spotting.

Last night I read a post by Aditya over at Savvis that looks at “horizontal application scaling into the cloud”, and what such a solution will encompass.

I take an issue not with the article, but the concept itself in the current incarnation when its applied to the area of enterprise applications.

In order for a workload to be able to ‘burst’, and as helped by the dictionary it must be able to “break open or apart suddenly and violently”. When was the last time you’ve seen the tightly-coupled layers of any application be able to be broken apart without induced downtime? There are interdependencies not only surrounding applications but the infrastructure supporting them.

The utopia that seems to dominate the cool-aid drinking cloud echo-chamber is an all-public consumption model, where public-only / single-provider is the method for all IT service delivery. Reality is, in privacy, process and control-centric government and enterprise environments some data will always remain in a highly controlled and self-managed environments.

For me, CB’s most realistic use-case is going from logical-local or private premises to an external provider, what Forrester terms ‘virtual-private’.

In order for CB to be possible, I see two camps developing:

1. The interface between infrastructure, management and monitoring must be very clearly defined to allow for tight integration between all components and thus provide a higher awareness of actions being carried out. Such as performance re-balancing, or disaster mitigation process instantiation

• This is where over the last 2 years we’ve seen companies playing the converged infrastructure card, and spend copious $’s on R&D, integration, testing and product delivery

1. Applications must be designed with an “expect-to fail” approach - a discussion which if memory serves right was between @reillyusa and @mreferre. In summary, by taking care of failure at a higher layer, specific to the knowledge of your application, it’s cheaper than having to design an extensive-feature-rich, all use-case covering container

• Single most potent poster-child of this is Netflix with their Chaos Monkey, and continual & purposeful destruction of systems to test, and thus prevent un-expected failures

Both have their merits in todays market, and for shops which are not tech-savvy and have extensive legacy baggage or who’s main purpose isn’t IT differentiation, the first choice is a step in the right direction. On the other hand, if you are developing applications for the new world, creating scalable services and are full of zeal to produce the best possible architecture in your niche, then the latter choice is IMO a good approach.

Whatever the correct choice for your bussiness, one constant remains, you need to begin at the application.

The paper, written by Laura M. Grupp, John D. Davis and Steven Swanson does make some very valid points, and more importantly it is backed with empirical data - which I adore.

in summary

The latency will be affected by the diminishing size of the manufactured die, and as we approach 12nm, the performance as measured in read response time will increase from being ~20µs to ~25µs, or have a delta of 0.005ms.

Most significant write throughput decrease has already occurred between the NAND addressable capacity increasing to 256GB. When you start looking at 1TB to 4TB transition, we’re losing less than 250MB/s. Easily alleviated with multiple smaller cards.

Most evident sentence which should be used to ground the academic nature of the article is:

fastest hard drives can sustain no more than 200 IOPs, and the slowest SSD configuration we consider achieves over 32,000 IOPs

So if I were to lose say 15/20 or even 30% of that performance in IOPS per addressable card, and the response time was a hideous 30µs even … am not sure that’s quite such a bad trade-off for a 4TB of NAND love.

clusters to the rescue

The gloom was very well painted at the beginning of the paper, but as you get beyond prophetic statements and consider the generational shifts that have appeared between tape, myriad of rotating disk, and now Flash for primary storage; you start to realise a single constant. Flash technology is not here to replace spinning disk, but to augmented the delivery of application IO.

As an application stands today, it still hasn’t changed its consumption model of the underlying block IO, so for the time-being, fundamental principles of smaller, purpose-chosen storage-devices working as part of a larger RAID-set will continue to result in a significant performance uplift. There are a number of Flash drive and array manufacturers today already designing clusters at not only intra-device at the chip-level, but also across multiple cards in the OS. Each with its own benefits, and each surpassing the metrics Laura et. al claim will slow the adoption of Flash.

For those that read The Innovator’s Dilemma may remember that “performance oversupply is an important factor driving the transition” - which we’re flooded with on a monthly basis with IOPS records. Only difference now, with excess idle computational capability within the arrays, coupled with smarts around Hierarchical Storage Management (HSM), we can deliver that IO faster than ever before with Flash, but also tap the blended sets of primary storage based on old variations of spinning disk. All provided to the application as a single, seamless stream of superfast bytes.

Lets not split hairs, Flash is here to stay, and in its current incarnation will continue to evolve, reduce latency to microseconds, and drive throughput beyond what a practical amount of duct-taped-together spinning disk can amass.

References

1. The Bleak Future of NAND Flash Memory - link

When we start looking at availability of mechanical drives or most components inside a server, we first refer to the MTBF.

The consumer-grade disks are generally rated at several thousand hours, whereas enterprise-grade drives typically see 1.3million+ hours of operation.

Due to the differences between spinning and SSD disk, it isn’t always appropriate to use MTBF as a measurement of the drives availability.

The NAND memory cell within the SSD doesn’t act in the same fashion as a spinning-disk block-abstraction. It must first be erased, prior to a write being carried out; unlike the spinning medium where a simple overwrite is possible.

The bits within a NAND cell are registered by electrons moving through a membrane/tunnel-oxide. It is responsible for keeping those electrons in check.

Gradually, over many program/erase operations the oxide starts to wear out. It is this event that should be avoided, as it will slowly start to cause single-bit errors, and cell locations that expect a 0 will be met with a friendly 1.

For this reason, the Flash Transformation Layer, responsible for all the internals of NAND management & carries out wear-leveling operations which evenly spread the load across hopefully all NAND cells.

Whilst looking at SSDs, there are SLC, MLC, and recently eMLC based flash memory. Difference being the amount of bits that are stored in the individual NAND cells. SLC has a single, whilst MLC/eMLC store two bits. In a lot of cases they’re actually exactly the same physical die, and what separates SLC from MLC/eMLC is the aforementioned FTL, amount of reserved space for the load balancing, and capability of the onboard ECC. This results in tunnel oxide in an SLC-rated drive having a typical life of around 100,000 program/erase cycles and 10,000 for MLC.

Now that you can see, that -most- of the wear on the drive occurs during cells actually being written to, and not idle time, given lack of moving components. With nothing changing, the larger the drive coupled with better FLT, the longer the drive will endure.

The reality check comes in the form of manufacturing processes, and as AnandTech points out, with the reduction in die size, results in extra sensitivity required to read each bit - there’s an inevitable hit in individual endurance of the cells. Yet the performance has been clearly increasing with each generation of die-size reduction, which is a result of the gain achieved in the controller power and thus any ECC coupled with greater reserved capacity on the disk has a net effect of - higher performance, and total better longevity.

NAND Endurance

Instead of looking at a mechanical, time-driven metric, the best way to calculate endurance of the drive is stress the oxide tunnel, and see how many times it can be trespassed, which translates to the amount of bits that can be written.

A standard that only several manufacturers have started to adopt is the JESD218 which gives an Endurance Rating in Terabytes Written (TBW). It also nicely classes client vs enterprise workloads where something as simple as the Power on state is measured (among others… see references).

Client

• 40ºC
• 8hrs/day

Enterprise

• 55ºC
• 24hrs/day

based on a use scenario in which the SSDs are actively used for some period of time during which the SSDs are written to their endurance ratings

Most importantly the standard is not using just time as a function by which to measure, but the actual endurance level. Therefore when you see a 20PB “TBW Enterprise” rating on a drive, you can quickly determine that the endurance of the drive over a 5 year period offers an ability to write 11 Terabytes per day [20*1024/(5*365)].

Next time you’re picking a drive, be weary of the correct metric being used to show you how long the unit will actually last. Knowing that each application is different, with its own workload profile - you will be able to easily figure out how long the said drive will last.

References

1. SSD is Not the Best Way To Use Flash Memory in Storage

2. Micron’s ClearNAND: 25nm + ECC, Combats Increasing Error Rates

3. Solid-State Drive (SSD) Requirements and Endurance Test Method

4. SSDs heaven or hell? The pros and cons of using NAND flash memory

As my eyes were pried open, I uttered a somewhat able ‘hola’ to the newly found adjacent bunk-bed companions. To which one of them nicely reciprocated - “You look damaged”. Now that’s how you say “hello”!

All started with having nearly missed the flight in Buenos Aires, it was a pleasure to embark on a long-ish flight to Lima. The service was superb, food enticing, and surprising of all - metal cutlery. It also helps when the flight crew is pleasant to look at. Arrival at the hostel was post the mandatory airport-taxi haggling. Another pleasant surprise awaited; the Kokopelli hostel is situated in the Miraflores district, which for the un-initiated is one of the largest tourist traps and entertainment areas within Lima or as I refer to as cleaner version of “Sydney’s ‘Cross”. The whole place was alive with music and energy as a result of a few thousand back-packers; with fitting entertainment within an area not larger than a square kilometre.

The first stop was a pizza-sponsored dinner. Upon walking through the little street of restaurants and clubs, the abundant maître d’s from each establishment do their best to hustle you inside. Having had the pleasure of enjoying a so-called ‘pizza’ in Buenos Aires, I was hesitant to subject myself to it again, to be fair, if you want 3cm of nothing but a cheese with an oil volcano on the plate, then Argentina has the best pizza in the world. Fortunately Peru didn’t miss the memo on the preparation, and appropriate calorie level of this fine culinary specimen, and delivered on a somewhat passable pizza.

Following destination was a salsa club, about 5 doors down. Here I witnessed feats of impressive flexibility and gracefulness of middle-aged men dancing with such zeal, that it was simply a pleasure to watch. Some time has passed, during which further blood dilution occurred, which had a direct correlation with my self-confidence, and supposed ability to perform same moves. The locals were very helpful in instructing a gringo in the art of the Salsa for the duration of the stay. The bed-crash at the hostel occurred at an estimated 5am.

Meet Christoph, the transport economist and Nora, the dental explorer. Once Chris explained to me what he does, whereby he deals with a branch of economics with a mixture of civil and economy-incentive engineering, I simply wished our own CityRail employed these ‘German Efficiency Engineers’. Chris, how about you come to work in Oz? Our government needs you.

Nora on the other hand, spent the morning picking the most minute flaws in my teeth. It was a great way to incentivise me not to open my mouth for the rest of the day.

Once the ritual of breakfast was concluded, we all banded together to have lunch; so we headed into the city. Being easily impressed, I was delighted that the bus into town, didn’t have to share the same road as the rest of the horn-loving, lane-ignorant road citizens of Peru. The bus got its own, dedicated lane, separated by median strips; a simple way of guaranteeing QoS. For lunch, we had the famous raw-fish Ceviche, some ill-hydrated urine looking warm beverage that was quite sweet, and surprisingly tasty. Whilst everyone went for the safe bet, chicken + rice, I ended up having some what I later discovered not overly well cooked local octopus.

Now that we had some sustenance to go on, we proceeded to walk about town. Having decided to spend a small fortune of Peru Sol’s on some tramp stamps also known as material country flags for being vagrants in other lands, we then had a democratic discussion as to the location of each flag upon our bags, especially the USA one, perhaps I can stitch it out of sight, to the bottom.

We witnessed a holy day celebration of Our Lady of Mercy outside the local church, and officially marked it a success and concluded the day with a 1 Sol wine tasting session amidst some underwear displays.

According to the local coffee shops raggy edition of the Lonely Planet’s “Chile”, Torres Del Paine has approximately 100,000 flower tramplers per year. An astounding number of wide-eyed gringos for any destination.

Especially if you consider that the main walk[s] consist of half a dozen variations to a set track, the paths themselves in most places are no more than 1m in width. Pushing through a hundred smelly backpackers as you traverse the peaks of this beautiful terrain would be a horrible horrible thing … that’s why the best time to visit is pre-season! (shh. . don’t tell anyone)

There exist many pros on which I’ll concentrate first - in exploring the Torres Del Paine Mountain range pre-season. First is the lack of Germans with their walking poles behind every bush. History has shown, its never good to have them band together and march great distances through a foreign land.

The next really depends on how luxuriously you wish your stay to be. There’s a number of hotel$ (privately owned), camping sites at literally every trek intersection, as well as a small number of refugios. Which are nothing more than … well, a bed, roof, kitchen, + fireplace to air out your socks, these are also privately owned although the stay is malleable starting at $30odd USD per person. Simple benefit for the pre-seasoned traveller is space. In a typical high-season, bookings seem to be essential due to the firehose of willing participants. We ended up staying in a bunk-bedded room with a Korean gentleman who I can only assume was full of Godly praise for his 2-minute-noodles as he devoured them slurpingly and Roman-awakedly.

First night we stayed at the campsite alongside Hosteria Las Torres, as they wanted to charge over $300USD for the night, per person. Not to be put off, the campsite is well trimmed and looked after, so spending the night away beneath the Torres was actually very comfortable. By venturing into the Las Hosteria for a sip of the local brew, we met Ignacio, the “Aussie Lover”. Formerly a stock broker from Santiago, followed by oyster shucker in Broom, and resident of Bondi; is now a guide within Torres Del Paine.

The first days trek of the W-course is around 11km by foot, we elected for a different approach. Horses! Throughout the ‘off-season’ the Gauchos (people who beat these beautiful animals into submission, aka cowboys) let the horses roam free throughout the park. They started collecting them only days prior to our arrival. It would be remiss of me to not refer to them as ‘rock climbing horses’ or as Rob put it “mountain ponies”. The feats these wonderful beasts performed were incredible. Throughout the ‘horses-only’ trek, which is closer to the edge of the river, and in my opinion a significantly better view than achieved by plebs on foot; our companions were able to scale near perfectly flat rock faces of nearly 30º (I had a protractor with me…) with surprising ease. Have never seen a horse do anything remotely close, mind you, this was officially my second ever horse ride.

The vistas available within this part of the route (Las Torres -> Los Cuernos) spoilt us considerably as for the hours after dismounting, the rain and wind conspired together, and decided to test our gear; everything held up extremely well, apart from my atrociously designed Salomon Cosmic 4D GTX. Bad bad shoe. They’re very pretty, just don’t buy them, at least not if you want to actually use them.  Having finished the walk, the views available on this leg were still the best part of the whole week. Make sure you spend considerable time, and camera memory here. Unfortunately I couldn’t hand hold my camera, whilst attempting to detract my four-legged “sure-foot” (name credit to Rob) from sniffing the butt of the horse ahead, and not smacking me into the lower branches of trees. Photography-wise, the plebs win here.

By the time we arrived at Campento Italiano, I could have cooked several packets of spaghetti from the water in my shoes. Mainly this is a campsite, with a single wooden shack with a dirt floor, to shelter the pilgrims on their way through the park. We found a very friendly, fast talking, brightly coloured (I thought my jacket was bad!) French-man with A.D.D., we proceeded to borrow a bunsen burner he had in hope of heating up some canned ravioli, to somewhat offset the cold gust of wind and now splattering of rain.

Upon drying my socks on the rocks next to a fire, and clamouring into the tent before the next wave of ice-cold water deluge from the trees hit me, I rested.

The stormy night, was followed by a more impressive morning. Donning the daypack filled with camera gear, we rushed up to Mirador Frances, to see remnants of a glacier from the lookout, although after witnessing the mighty Perito Morino the week prior, it quickly underwhelmed. Am not sure as to why it bears ‘French’ in the name of the valley; there is not as much dirt, didn’t see any dog faeces or foreigner-disapproving, baguette-wielding locals pretending not to speak English as you ask them for help at 2am… but I digress. Even the French-man from the night before was unimpressed.

It is said that during the bloom of the season, the French Valley is one of the most beautiful sights due to its microcosm of flora from around the park.

Continuing along the path, this time towards Refugio y Compamento Pehoé, we were once again greeted by some of the friendly wind. Upon arrival, only slightly wet this time, we found solace in seeing walls, and an actual floor, a roof and to further top it off - a fireplace! It appears about 10 more people were sold on the same idea at $35USD, so an evening of pasta, tuna, chocolate, and sharing of travel stories with the fellow wet-feeters ensued.

The W-trek can be started at either side, right->left, or in reverse. At the end of our W, was the administration building; highly discouraged by other travellers due to being a trip of boredom, although quite an easy 18km walk.

As we left the protection of the refugio, en route to the administration, we spent over an hour within about 100 metres of the refugio itself; the weather finally turned, even if for only two hours, to let us capture some of the beautiful landscapes that exist within the park. Most notably right near the refugio; the owners sure know how to pick a good spot!

Arriving at Administracion CONAF, the wind almost pushing us out from the park. Continued to belt at full strength. Lack of open-season, resulted in this case the local hotel/hostel to be shut, with the only option 2mm of nylon + polyester tent to protect us at night from the whistles and face-shredding wind.  Had no choice, but to whore ourselves out to passing cars making way in the direction of Puerto Natales. Luckily for other 99,998 gringos that will visit Torres Del Paine this year, not only are the local drivers, but other tourists also happy to give you a lift. Unfortunately hardly anyone is travelling in the same direction, and duration. After just over 40 minutes of excitedly running up to stopping cars, with eyes of a 6 week-old hungry puppy; finally managed to snag a lift - on the back of a ute. Not only was the owner driving his whole family, but he still slowed down, and helped us. I take my many hats off to the driver.

Those are some of the more positive things about coming to the park before the season officially opens. What any visitor has to realise is that one of the most unpredictable parts, is the weather. Not only overnight, but within 30 minutes can go from postcard-like scenery to clouds that attempt to ruin the holiday by pelleting strategically placed rain drops into the open crevices within your poorly designed hiking boots. Other point to consider is food. If you visit when none of the refugios nor hotels are open, you will have to carry enough to sustain your ever-eager metabolism over the duration of the trek.

As an aside, the day following our return to Puerto Natales, was the Chilean Independence day. Complete with a parade, BBQ’s at every corner, and festivities for kids within the fire station. It was great to retreat to one of what became the warmest place to lunch when in town, Cafe Toore with its own “Patagonian Consultant”, Sergio, and as you can see in the photo, his daughter.

Consuming meat for breakfast, lunch and dinner at every opportunity, Dieciocho (Chile National / Independence Day) on September 18th was close to my stomach. Devouring a couple anticuchos, followed by some Mote con huesillo, it was an amazing conclusion to the first leg of this South American adventure.

The journey around Torres Del Paine, and adjoining parks is to take nearly two weeks. The vast majority of them will be on foot, thus consideration must be given to the weight of the total pack(s), as there are (unfortunately) going to be no sherpas.

Weather report for this time of year, idled within 2-3ºC below 0 for the night, and a few degrees higher for daytime. Which necessitated the number of thermals throughout the trip, coupled with the self-imposed weight restriction as well as possible lack of showers; more thermals = less smelly experience.

There are many lists online of what people have managed to bring with them along for their trek. To add to the noise, I wanted to provide an OCD technologist’s view.

Main pack is the Genesis 70, which comes with a very useless daypack, or one you can use to place your hopes and wishes into. Hence you see me bringing the Kata bag instead. Not only is the latter going to be used for housing the camera equipment + laptop, but also re-purposed as a daypack.

Here’s the complete list of the crap thats coming with me. Most is self-explanatory:

Containers

• backpack 1 - MacPac Genesis 70
• backpack 2 - Kata Sensitivity V
• medical kit
• 2-man Tent
• Mattress
• Sleeping bag - Mountain Hardwear Phantom

Clothes

• 1 x Merino - 280 GSM Midlayer
• 2 x Merino - 150 GSM Underlayer
• 3 x Merino Long-johns
• 1 x compression socks (for flights)
• 8 x sets of thin socks
• 8 x undies
• 1 x Goretex wind/water jacket
• 1 x REI Waterproof pants
• 1 x glove inserts (for warmth)
• 1 x gloves, thin but water + wind resistant
• 1 x boardies
• 2 x bonds top
• 1 x jeans
• 1 x t-shirt
• Boots, Salamon Cosmic 4D GTX
• Runners, Ascics
• Sandals, Keen
• 1 x Bose QC 15 (not for the hike itself…)
• Polarised glasses & hard case
• 2 x Pocket Handwash - Sea To Summit
• 1 x Body Wash - Sea To Summit
• Sunblock & Insect Repellent in one - Sawyer
• Multifunction Knife - Leatherman Kick
• Firestarter - exotac nanoSTRIKER
• 20m Parachute cord
• Katadyn MyBottle
• 1 x Micro-fibre Towel
• 25L waterproof bag - used to house all the clothes inside the backpack
• 6 x Gladwrap sandwich bags for waterproofing some electronics
• 1 x QuickDraw Set
• 1 x Whistle (hey… you never know)
• 1 x Washable Electric Shaver - Braun 7 - highly recommend it

Technology

• 2 x HDD 320GB iomega
• 2 x HDD 500GB Western Digital Passport
• 1 x HDD OCZ Vertex 2 SSD, great for the Lightroom catalog, and currently working-on photos
  • Workflow:
    1. Each evening, the photos from my and Rob’s cameras get copied to one of the marked hard-drives
    2. Then copied onto the replica drive
    3. Only then are the CF / SD cards cleared within the cameras
    4. Any work, sorting, adjusting of photos is then done in Lightroom against one of the drives
    5. Upon completion, the drives are once again synchronised
    6. The two drives remain completely separate from there on, at no point do they sit in a single backpack, or room (apart from when I’m there)
    7. Fortunate to have a base in Buenos Aires to go back to, so the iomega’s get dumped to the WD’s upon return. One of the WD’s gets taken with, other remaining
  • No one likes losing photos, especially me
• USB Cables 2.0 + 3.0 (+duplicates of all)
• Power cables (+duplicates of all)
• Power adapter (AUS->US)
• Apple Charger
  • One trick is to get rid of the standard two-prong cable, or extension from the brick, and re-use the one that comes with all the Canon battery chargers (… as long as you get a duplicate!).
• MacBook Air 13”, i7, 250GB SSD
• 1 x USB->RJ45 (Ethernet adapter for the MBA)
• 1 x iPhone cable
• Headlamp
• 3 x BL-5c, GPS Logger QStar BT-1000P - reviewed it here

Batteries

• 4 x Canon Batteries, LP-E6
• LP-E6 Charger
• 2 x 4 x AA
• 2 x 2 x AAA

Camera

• Canon 50mm f/1.4 USM
• Canon 17-40 f/4 USM
• Canon 70-200 f2.8 IS USM MKII — heavy!!
• Canon 5D MKII
• Canon 430EX II Flash
• Off-Shoe Flash Cord, OC-E3
• Sigh-Ray Filters
• 1 x ND-2G-HS
• 1 x ND 2G-SS
• 1 x ND-3 Rev. Graduated
• 1 x ND-3G-SS
• LEE Filter holder
• Lens cleaning solution
• Lens & Screen cloth
• LENSPEN
• Lens cleaning tissue paper
• 4 x 8GB CF, SanDisk Extreme III
• 1 x 72mm Circular Polariser, Hoya HD
• 1 x 72mm ND400, Hoya
• GorillaPod - had to leave my Manfrotto 055-XPROB at home, as I didn’t feel it could accompany me comfortably. Suggestions welcome for a light and manoeuvrable tripod for trekking
• Black Rapid RS-4 camera strap
• Book, The Last Don by Mario Puzo - during those down times such as airport transfers, and lunch breaks. Would happily replace it with a Kindle if I had one - only due to the weight of course!

Then add food, water and toiletries and you end up with a very nice ~20kg pack.

Growing up in Ukraine, I’ve seen my fair share of snow, blizzards, and every temperature from -40ºC through to 40ºC, although upon arriving in El Calafate, which is a small town in the Patagonian region of Argentina (that should be oft-described as a tourist trap… - on that later) - the first stop was to be the famous Perito Moreno Glacier which is a magnificent spectacle that is difficult to explain and take in.

Interestingly enough, the best view is far away from the the glacier itself, and on one of the Los Glaciares National Park’s viewing platforms.

By far the most enjoyable part of the journey to the glacier was donning some crampons and spending just under two hours walking around and having the history, growth spurts and inevitable ‘shock’-stories being told by our guide Diego.

After the completion of the walk, we were surprised to be greeted by a table full of glasses, some local chocolate, and a bottle of whiskey. To top it off, the ice for the tumbler, was chipped from the glacier itself.

I love to travel.

Am fortunate enough that in my role am able to do two awesome things: talk about technology and see new places. Yet exploration goes beyond taxis, hotel rooms and new office blocks; and has not been towards Argentina in detail.

Every now and then, the need to re-charge, or in this case deplete the batteries is an absolute must. I will be heading to South America for a total of 52 days with my mate - Rob and of the places I wish to visit, many have commented that I require at least 6 months to do it “properly”, I completely agree. The last thing I want to do, is to turn it into the trip many post-pubescent Australian’s take; the Contiki tours. 1-2 days in a city, get mashed, throw-up over self and some historic statues, then back on the bus, to do it all again and share the Australian ‘spirit’ with yet another unsuspecting European city.

I have two goals for this trip to be a success. First is visit Patagonia. Ever since spending a month in Buenos Aires in 2002, I’ve wanted to go further south, and see the glaciers, sleep in the mountains and hopefully, paddle through one of the lakes in a kayak. Lastly, drive through the Salt Flats in Bolivia, and a recent addition has been to do some mountainbiking around La Cumbre to Coroico. Anything else in between is a bonus.

Now to finish off the packing process. Although I can’t find too much space for socks and sweaters am certain that I will find a number of mountains in Patagonia that have never had a MacBook Air visit them.

All began a couple of months ago, with EMC Inform on the horizon, I had two sessions in front of me, one a pure technology play - ‘EMC/VMware Integration’, and second being significantly business oriented - ‘Hybrid Cloud & Workload Federation’.

Personally prefer to be sitting in the front row of any presentation, I seek validation of a technology, process or an idea, and for someone on stage to be able to explain it, and more importantly demonstrate it; as a colleague likes to say, the “proof is in the pudding”.

I put this out to the twitter follow-ship:

Working on a presentation for next week. As a PARTICIPANT, do you prefer live or recorded demos? Leaning towards the former.

@romant - 24th May, 2011

… which after all the responses, was kind enough to offer absolutely no definitive answer.

Really wanted to show instead of just tell; after-all, if you are in the audience, as both Eric and Damian point out - how do you really know something works, and there isn’t some recorded-voodoo going on? Given Adam’s feedback though, anything that takes considerable processing or wait time, would record, everything else - live.

The hardware off which I was to show off my touch-typing skills resided in Sydney. The conference spanned 3 cities, connectivity was provided in only a single location - Wellington. Un-surprisingly, it worked, even across the pond. Two remaining cities, had to be covered off with a bluetooth-tethered iPhone. In Melbourne’s convention centre, the connectivity was flawless, and the demo portion of the talk didn’t skip a single beat.. over 260 happy souls.. confidence for the largest venue, Sydney, was at an all-time high. What I didn’t realise, is that Sydney’s Exhibition Centre is built like a quasi-‘Faraday Cage’; which really helps if you’re after reaching the world through 3G.

Unfortunately, after 20 minutes of pure slides, a realisation dawned, that the mobile provider wasn’t up to scratch to make everything look as seamless as was planned. I had to revert to some miming and a description of pixels that would be drawn on screen had things progressed as planned.

The inability to attain more than 2 bars of reception love on the iPhone, rendered the experience simply unbearable.. back to the slides.

Regardless, all up, it was a great event. was able to get through most of the message in Sydney, and for the rest had some canned videos ‘just in case’, which covered the necessary points.

After the presentation, I was asked whether a live demo would ever be attempted again, the answer is simple - yes. Obviously the problem will be differently approached next time, with significant pre-checks in all locations, specifically around connectivity, and having an alternate telephony provider if necessary.

Of the sessions I was able to attend, my favourite presentations were by far, David Lloyd’s on ‘UIM and what it can do around the managements of standardized infrastructure’ - namely Vblock, there was standing-only room left., and the Q&A from Michael Bookey’s session on ‘VMware + Mission Critical Apps’. Not only was he able to bring some personality and zest to a typically mundane topic, but every question thrown to him, he answered with incredible depth.

At the end of it, enjoyable, and glad to have participated.

It is important to note that within a discussion on cloud, several attributes are always considered: trust, availability, flexibility, inevitably - cost; but most importantly the vital tenant, scale.

Arguably most of these are actually better catered for within the cloud-sphere. You have complete stateful, and migratory capability of a security policy which can reflect business rules and be as dynamic as required. Technologies like VMware’s HA, and FT can provide guarantees around operational state of the OS, integration with Intel’s TXT can further provide assurances around GRC and around execution on foreign tin; thus introducing capability the original OS + App need-not design and maintain. Lastly, through significant consolidation and utilization adjustment, an organisation can drive significant cost savings to a pure hardware-OS-app stack. How do we then grow this environment?

The problem of scale within an enterprise is normally solved vertically with mainframes, and horizontally with oft-too-expensive clustered approach. What happens to all your old apps that are mission critical? How do you ‘cloudify’ them? Typical response is re-architect. Reality is that the lifecycle of a typical enterprise-grade application is around 15-20 years (or considerably more if you look at Sabre). Thus any suggested changes to these monoliths will undoubtedly not be greeted with open arms by the application and business owners.

Therefore re-architecture is simply not an option. We have designed an elegant abstraction for each of these silos, called Virtual Machines. Which transform the old and senile into something more malleable and agile, or so you would think.

At the end of the day, we’ve just injected some functionality into the sarcophagus of an ever-decaying technological silo. Now we have to figure out a way to move it around. How do we optimize this further? How do we migrate these workloads and expand the stacks’ capability? Surely there are service providers out there that can help?!

There do exist “cloud-providers” that can cater for, and ingest your applications to be run within their own DC’s. Whether you’re moving to or from their premises, they will support you; in a very manual fashion. Herein lies the issue, under the guise of ‘cloud’; they’re no more than a hosting service provider with a brand new brochure.

In order for a provider to be truly cloud-enabled, and allow for Enterprises to scale outside their physical boundary. They must support the same abstraction-architecture as you use internally, and federate your application and data in a seamless and live manner, thus delivering on ’true-cloud’ promise, and not the same way that you have always attacked this by replication without state.

Technically, the problem of migrating live workloads across distances has been solved. We encapsulate the OS+App within a VM, all networks and their configuration for a group of VM’s that is further packed within a vApp; is provided through vShield. Inter-DC communication can be achieved with Cisco’s OVT or stretched L2 networks (ugh), thus giving us the mobility of the state. Lastly, the moving of the storage can be facilitated through with VPLEX by further abstracting the underlying storage arrays to achieve the fully virtualized, completely federated, live workload migration.

The unfortunate state of affairs is that whilst we continue to have the baggage of old-school applications, for a solution to be Enterprise-ready, it must provide for ‘warts and all’ encapsulation and transmission. Until that is the case, and organisations as well as the service providers standardise on a mechanism and technological cocktail, we can’t really be calling some clouds, clouds.

Amazon did a great job of not calling it ‘amazon cloud services’ when it first launched, this way they stay away from having to define it [cloud]. Once we remember that ‘cloud’ is really an abstraction of abstractions (even from definitions), we soon realise that apps written for tomorrows consumption don’t have to be tied to processor architecture or even a specific OS to run.

Now AWS are going up the stack and developing their platform play atop the IaaS they already have. Most importantly not concentrating purely on a single facet as performance (like the freshly announced Australian startup OrionVM), but useability and management through an extensive set of API’s. The differentiator is a little further up from a mere hypervizor.

Along the same path of execution in the past year we’ve seen VMware reach … up, and look to fill the gaps in Virtual Infrastructure Management+awareness solutions through vSM/Request-Manager for vCD, and more recently vCenter Operations.

The wonderful thing about technology is that by focusing on the value-add, and services further up, you can actually replace the foundation without the need to take down and rebuild the whole offering; as Apple did with Rosetta in their migration away from PowerPC to Intel, shows us you can provide a platform which negates the tight-coupling of CPU > OS > App of the yesteryear. In the current incarnation of cloud deployments across all worthwhile platforms, we can see a single constant - base compute.

. . . just throwing this out there, but it is not only possible, but most probable that within 24 months, not only AWS but other players will launch an Atom or even ARM-based service publicly, which will do away with offering infrastructure services and concentrate from the get-go on platform; all through the same management interface.

First up as you can see we have the fresh install, and nicely enough, with just an OS upgrade, there’s an increase in the disk throughput. More importantly though, is the encrypted performance. Averaging out PGP vs Lion, we witness a doubling of the MB/s capability within the latter. On the one hand, its a shame I have a license for PGP that I spent ~$175AUD, on the other, it served its purpose, and is now going to be replaced by something that comes free with the OS, and performs better. Symantec has a talent for buying dying horses.

What

At its core vShield Edge is a very lightweight appliance mostly based on LinuxFromScratch and the rest comes from VMware’s BlueLane acquisition.

If you want to skip the verbosity, skip straight to How

We are hearing more about vShield recently, because it is a prerequisite component of vCloud Director. It facilitates the critical piece that makes virtual datacenters work; network connectivity. Unfortunately it is still a young product, and as such feature-poor.

In the last two posts I covered two preludes. The first is how to bake your own NAT forwarding router and use it inplace of vShield within the App Network. Second being how to get inside the vSE appliance.

… which is where we’ll start today.

Why

The unfortunate repercussion of ‘rolling-your-own’ is that it simply doesn’t scale, and that’s what vCD + Cloud is all about. I simply can’t deploy my own NAT solution with every vApp, and have it integrate with vCloud API’s. So I started at the other end. Customizing vShield Edge itself.

For this piece, I wanted to add only two extra features

1. DNS resolution Simply put, this will allow for local DNS resolution within each vApp/Organization. More importantly can be made to work with the currently unused DNS Suffix, when creating networks.

2. Dynamic DNS Registration I have a dream… that one day, when you deploy a resource, it will be able to resolve and ‘see’ others within the same vApp/Organization or vSE ‘Network Boundary’.

How

Summary

• Get inside vShield Manager
• Search for vShieldEdge.ovf
• Modify it until you’re satisfied
• You’re done

/common/em/components/zones/vShield.ovf
/common/em/components/ni/ovf/vshield-ni-infra.ovf
/common/em/components/edge/vShieldEdge.ovf

Note that the other vShield products are also available to you.

Am not going to go through every command required to get this to work. If you’re stuck at any point whilst attempting this, feel free to drop me a line.

First, lets fix the dhcpd.conf generating file - /root/vSEdge/vse_dhcp_config.pl

Search for "ad-hoc" and replace that line with:

$cmd = "include \"/etc/rndc.key\";\nddns-update-style interim;\nddns-updates on;\nddns-domainname \"wcm.lab\";\nddns-rev-domainname \"in-addr.arpa\";\n";

This will do several things for us:

• Setup trust between the dhcp and named daemons via the rndc.key
• Set the zone names and allow dhcpd to update them

For bind/named itself - since we’re dealing with LinuxFromScratch, you can get the code from the LFS project page

Compiling named is done via:

./configure --prefix=/usr --sysconfdir=/etc --localstatedir=/var --enable-threads --with-libtool --disable-shared && make

You can then find the executable in ./bin/named/named

Unfortunately I haven’t been able to figure out how to get it to work with the current libraries present within vShield, so you’ll also have to copy some libs across as well.

On vShield:

$ mkdir -p /var/named /named_libs

On your compiling machine:

$ scp /lib/{ld-2.5.so,ld-linux.so.2,libc.so.6,libnsl.so.1,libpthread.so.0} root@vse:/named_libraries

Once that is all set and done, you’re just left with having to configure named like on any other machine, which means you will require at the very least:

• named.conf
• named.ca
• rndc.conf + key
• zone files

I will be using the ones I’ve prepared earlier for other projects.

… and execute it using:

$ /named_libs/ld-2.5.so --library-path /named_libs/ /usr/sbin/named -g -c /etc/named.conf

I wrapped the above and placed it into /etc/inittab.

After you’re done, you overwrite the vShieldEdge.ovf within the manager VM, and have vSE Manager provision your own, custom vShield Edge appliance.

The above is meant for those that like to get their fingers a little dirty trying to improve things, this sometimes will break expected functionality.

Some gotchas, and debugging tips:

• If for some reason vSE isn’t starting, and by looking at the console, you can see that ifconfig doesn’t return any interfaces with an IP, run /root/vSEdge/edge_configure.pl - this will nearly always tell you where you went wrong in your vse_ script modifications
• To find linked libraries for any other packages you wish to inject, use export LD_DEBUG=<type> before running a binary where the <type> is referring to [versions | files | libs]
• Don’t forget to disable/modify iptables, otherwise you’ll have trouble getting inside vSE
• For anything else, there’s my contact page

Initially started writing this as a static blog-post, unfortunately too many screenshots + and explanations would be required. Hence I simply recorded some of the steps.

Caveats

• There is no possibility that this is in any way supported by our VMware friends, please don’t raise SR’s on a vSE that you’ve Roman’ed up!
• In the last step, I replace the root user password. Be sure to restore it from the backup immediately upon ssh’ing back in. As vCloud Director uses that password to interact with vSE, and will be unable to do so.

“Error creating Edge VM”

If you look within the debug logs, the excerpt of the stack-trace will look something like this:

com.vmware.ssdc.util.LMException: Error creating Shield network appliance.
...
at java.lang.Thread.run(Thread.java:619)
Caused by: com.vmware.ssdc.util.LMException: Error creating Edge VM.
... 17 more
Caused by: java.lang.NullPointerException
...

The cause is a misconfigured ESX Host that is sitting in the same resource pool as others connected to the dVS against which vCD is trying to create the network group.

Solution being to either remove the host, or alternatively verify that all hosts are connected EQUALLY to the dVS. This is where Host Profiles come in very handy.

I apologise for the lack of a vCD screenshot, I fixed the problem before realising that it was one; at least there will always be logs. Once you do see this again, feel free to send in a screenie.

Related Local Files - logs/vcloud-container-debug.log

Direct Network

This is by far the most … straight forward way of providing your VM’s with an outside connection.

Routed Network

The elegance of vShield Edge allows you to configure DHCP, Firewall, and NAT IP Mapping through an easy to click-through UI. Be it for your Organization or vApp. Here I’ll show you how to setup a NAT-Routed vShield backed internal network.

Custom’ish vApp Network

There are environments that have already invested a lot in customizing their own DHCP server rules, Dynamic DNS registrations, reverse lookups, firewalls… the list goes on. In essence, here I can show you how you can re-use your current soft-router, and have it used within the vCloud realm.

| ERROR | Start Level Event Dispatcher | StartupUtils | Error starting application: Could not bind network port: 80 on host address: 172.16.225.131 |
java.net.BindException: Cannot assign requested address
at java.net.PlainSocketImpl.socketBind(Native Method)
at java.net.PlainSocketImpl.bind(PlainSocketImpl.java:365)
...

One of these days I’ll allocate some static IP’s to my whole Fusion-bound vLab. Until then - I will put up with DHCP assigned IP’s.

To fix, first stop vCD with vmware-vcd stop, then modify cloud-director/etc/global.properties to reflect the IP changes for:

vcloud.cell.ip.primary  
consoleproxy.host.https
vcloud.cell.ips 

.. or anywhere you have the old address[es]. Some love from sed will do the trick here

The timing couldn’t have been better. My first three days on the job, I spent in Vblock training with personnel across the VCE alliance. My background not being in networking, and having heard great things about those with CCIE certs, it was a pleasure to witness these guys in action.

Cisco UCS

The first time I saw one was at vForum 2009 in Sydney, an ugly piece of semi-gray/teal enclosure with several blades packed to the rafters full of RAM. Dismissed it immediately because it seemed like an immature solution for a company that didn’t have the pedigree of building servers.

On my second day of training we started an overview of the chassis, fabrics, blades themselves… something clicked (no, not just kool-aid). Problem is, when procuring hardware and you have a choice of vendor, you decide on the ‘immediate bang for buck’, hence inevitably look at Memory + CPU configuration before all else, without giving much consideration to something just as important if not more, ’ease of management’ and deployment ‘at scale’. Thus is the racking + stacking of a single blade in a single chassis as simple as doing it across 40 chassis and 320 blades?

There are some stunning features within UCS Manager for scalability. Most notably service profiles, which capture things like SAN+LAN connectivity as well as BIOS+Firmware levels into a quasi-template. Simply put, you can define a policy filter for a certain workload that is to be performed by the blade, and UCS Manager will tell you which blades comply in Memory, Connectivity, CPU and IO-Cards and which can be easily configured to the profile with the push of a button. Some great videos which distill some of the configuration + management benefits are here.

After those initial three days in training on EMC Storage such as CLARiiON + Symmetrix families, UCS and VMware to see how it all fits within the Vblock offering, I only had two days in the office. In that time I attempted to digest the email firehose of Vblock configurations, customer requests and explanations. My manager promptly and kindly offered my services to help break down some vSphere 4.1 features around core-mapping. As any new starter, then had to spend time on setting up phones, VPNs, cards and travel - have you heard about this VMworld thing? 🤞

• adapted from flickr user striatic

The cause is that the packaged OVA is actually a vApp extract from vCenter, and standalone hosts (not under management of a vCenter) are not able to accomodate vApp.

From VMware

vApps: Ensuring seamless application movement and choice between clouds

It seems a single host isn’t “cloud”-enough. 😉

An OVA that is extracted from vCenter which contains only a single VM is extremely close in its structure to the OVF apart from a tuple called … *drum roll* … <VirtualSystem ovf:id="cake"> in the single, and <VirtualSystemCollection ovf:id="cake"> in one that encompasses multiple machines. So a hosts ability is limited to not being able to parse this collection of machines and discriminate between their properties, that’s why you need to spend $ on vCenter!

This is all great, but my colleague is now stuck half-way around the world with a set of machines that need to be deployed on a single host, and all of them are saved within this OVA.

An OVA is just a wrapper for an OVF (think VMX) and some VMDK’s {anyone VM/OVconfused yet?}.

$ tar tf cake.ova
cake.ovf
cake-disk1.vmdk
cake-disk2.vmdk
cake-disk3.vmdk
cake-disk4.vmdk
cake-disk5.vmdk
cake.mf

We can see there’s a collection of VMDK’s, the manifest which inside looks like this:

SHA1(cake.ovf)= 03AD765EC45B90E13BC22D0115088F08021F2AE5
SHA1(cake-disk1.vmdk)= 92BB519FD1926D4F3C170C727C037E2D5D79775B

… and more importantly the OVF, which describes the OVA motherload.

My thoughts were at first to just create a VMX for each of the VMDKs and then I can go to bed. Unfortunately one of the machines has two disks, so the first issue is with figuring out how the OVF nests the rasd - Resource Allocation Setting Data (as per the DMTF OVF Specification V1.1).

Thanks to Mr. Pekka for writing xmlindent or you could just use the online version

I was quickly able to see to which VM disks 3+4 belonged to, and create the appropriate VMX skeleton. Now if only there was a parser…

There are three components in this setup:

1. Host
2. router/pfSense
3. NAT Client

Host

A great read for beginners and those refreshing is the VMware Virtual Networking Concepts whitepaper

Now lets create a network that our NAT’ed VMs will be using.

When prompted under Connection Type, select a ‘VM Network’, as this is for the typical traffic within the Virtual Machine (not IO or management of your machines).

Lets create a vSwitch that doesn’t connect to anything, a dud, a blankie. This will be our NAT’ed environment. It’s quite important that you DON’T connect a network card to this vSwitch to prevent any inadvertent DHCP leakage. Make sure you have nothing selected.

Give it a name to differentiate.

Once you’re done, click finish and you will have something two network available to your VMs:

• VM Network
• NAT Network

Time to setup pfSense. Once you’ve downloaded and extracted it. You have the option of either copying it directly to your datastore and then adding directly to inventory, or importing via the Standalone Converter. I find the latter is always faster.

router/pfSense

Incase you’re converting pfSense first (like I did whilst re-doing it for this post), I recommend you disable the network interfaces until you’ve finished setting up the host networks. We’ll enable these in a later step.

Once the conversion is complete, time to configure our virtual router. pfSense is provided with two NICs out of the box. One for the WAN interface (which is your internal LAN), and one for ‘its’ LAN - the one on which it will be servicing DHCP requests.

Mark down the last 4 digits of the MAC address, these will help to validate the following step.

Start the pfSense VM. You will be guided through the mapping of the interfaces, and just to make sure - check to see the MAC addresses matching to the VM Network (in my case 67:3c) and NAT Network (67:46).

Upon following the wizard, and if you’ve followed everything accordingly (or rather I documented the steps properly) you will be shown the interfaces within pfSense, their mapping (WAN vs LAN) and IP addresses.

NAT Client[s]

You are now ready to assign clients on this host to the NAT Network and have them pick up addresses dished out by your shiny new appliance.

The whole setup takes just under 5 minutes from start to finish to complete.

When you look at the mostly fragmented cloud and ancillary services market, who do you pick?

Amazon, the incumbent in providing no-frills, pay as you go raw crunch-power has carved out the niche over the last few years with its EC2 offering which has been very popular with corporates dipping their toes in the cloud space. Due to the non-exhaustive list of governance, privacy, flexibility, compliance and classification - organisations are finding it difficult to dive completely in, thus the ‘private cloud’ model is cast.

In order to partake in the race to the bottom within cost of IT operations and service agility so well presented by the cloud pundits, organisations (the more intelligent ones at least) started to re-think internal infrastructure. The simple question was posed - “how do we make what we have now - better ?”. First step with any problem worth solving is to break it down into its elementary components: as such organisations by their very nature will have numerous divisions, departments, sub-organisations and overhead associated with managing not only the division of these business entities, but mainly their computational dependence - this stuff is COMPLEX!

… or is it?

With what Amazon has achieved within the IaaS and Salesforce in S/PaaS in the recent years is a testament to the fact that true multi-tenanted solutions that can scale and be at face value simple to interact-with and build-atop are possible. The only problem is that to date, no one has managed to do this effectively within the confines of an enterprise.

In a recent article Scott Drummonds talks about the necessity in transparency of costs associated with spending between various Lines of Business in the organisation. I wish to see that taken further, and not only must transparency exist in consumption, but also in design and most importantly management of the core infrastructure and the systems that uphold the business.

From Citrix, VMware, Microsoft, Oracle or RedHat - there is still not a single public offering which allows you to manage, automate, delegate and provision datacenter technologies on the fly. The best means to competitively differentiate private cloud vendors will not be done in the hypervisor, but within the middleware. You can wheel as many CloudBursts (am looking at you IBM) or Vblocks (VCE) into an the datacenter and try and call it “cloud”, but when it comes to true agility with the various datacenters something is missing.

In comes Nimbula, the love child between two former EC2 executives and a development team in South Africa. The aim of the unreleased product is to “combines the flexibility, scalability and operational efficiencies of the public cloud with the control, security and trust of today’s most advanced data centers” - sounds great! The Nimbula Director is slated for general availability in Q4 2010. What is most interesting is that one of the investors is … *drum roll please* - VMware. They haven’t even started, and already have an exit. Wow.

According to TechTarget, VMware themselves are coming out with a Service Director product, which aims to do exactly what Nimbula just announced.

Given Nimbula's pedigree they will be going with the Xen hypervisor, and building their management suite on top. Coupled with the fact that VMware is now one of the main stakeholders provides us with some assurance of compatibility and hopefully a solution that is truly hypervisor agnostic.

Let the management games begin!

• adapted from flickr users maisonbisson | dominik99 | zeno77

Adding a new port (i.e. a vMotion interface) to a vSwitch on vSphere/ESX leads to this lovely error message. If you check your vpxd.log you’ll see something the image verbalised.

[2010-06-04 15:30:52.411 03152 info 'App'] [VpxLRO]; ERROR task-54417; host-36589; vim.host.NetworkSystem.updateNetworkConfig: vim.fault.PlatformConfigFault:
(vim.fault.PlatformConfigFault) {
dynamicType = <unset>,
faultCause = (vmodl.MethodFault) null,  
text = "SysinfoException: Node (VSI_NODE_net_tcpip_plumb) ; Status(bad0017)= Out of resources; Message= Instance(0): Input(4) if=0 portset=VMkernel macaddr=00:50:56:76:16:67 tsomss=65535 ",
msg = "Error during the configuration of the host: SysinfoException: Node (VSI_NODE_net_tcpip_plumb) ; Status(bad0017)= Out of resources; Message= Instance(0): Input(4) if=0 portset=VMkernel macaddr=00:50:56:76:16:67 tsomss=65535 ",
} 

Key here is the Out of resources; message. The reason for this is none other than the default number of ports for the vSwitch on ESX is 24, and if you have VM’s or other interfaces using up ports (such as AppSpeed probes), you will quickly run out. Switch this easily by going to Configuration -> Networking -> Properties [for the vSwitch in question] and up the value up to and including your growth requirements for the future.

Note

After you setup a new host, set it to more ports than you require, as you’ll need to restart the host for the ports to be provisioned; best to do this immediately after installation.

I think a better message would be to explicitly say “Out of available ports on vSwitch - [blah]”; instead of the semi-cryptic one presented.

Here are three mkisofs commands that you can use for each OS, and to ease your search a little.

You will require to be in the root of each ISO directory Remember the lone period “.” at end of each statement

Linux

mkisofs -q -V VOLUME_NAME -b isolinux.bin -c boot.cat -no-emul-boot -boot-load-size 4 -boot-info-table -R -J -T -o /LinuxISO.iso .<br />

Solaris

mkisofs -b boot/grub/stage2_eltorito -c .catalog -no-emul-boot -boot-load-size 4 -boot-info-table -relaxed-filenames -l -ldots -r -N -d -D -V VOLUME_NAME -o /SolarisISO.iso .<br />

Windows

The Bootable_NoEmulation.img normally lives in the [BOOT] folder. I tend to move it.

mkisofs -q -b Bootable_NoEmulation.img -no-emul-boot -boot-load-seg 1984 -boot-load-size 4 -iso-level 2 -J -joliet-long -l -D -relaxed-filenames -N -V VOLUME_NAME -o /WindowsISO.iso .<br />

What I require is to add the VMware Video drivers to the base Windows installation. This will work for XP/Windows 2003. You can easily integrate the drivers once you’ve found them with the use of nLite. The trick is in finding them.

First thing you’ll need is the VMware Tools ISO that we discovered a couple posts back.

NOTE: I rename VMware Tools[64].msi to vtools[64].msi

The next step is to extract the vtools64.msi which you can accomplish with:

msiexec /a vtools64.msi /qb TARGETDIR=c:\driver\

After a few seconds, you’re done. By navigating to the Common64\VMware\Drivers\Video folder, you will find the vmx_svga.inf that you need for nLite as well as the vmx_svga.sys which is the driver. When it comes to nLite - just point it to the directory.

VMware video directory listing

vmwogl32.dll	  
vmwogl64.dll	  
vmx_fb.dll	  
vmx_mode.dll	  
vmx_svga.cat	  
vmx_svga.inf	  
vmx_svga.sys

We have an admin that once installed a couple of ESX hosts atop other host’s boot LUNs. It’s pointless blaming the admin, as there’s a basic rule that must be followed at all times that will prevent this from happening.

Note - Don’t let the host see boot LUNs for any other host than itself.

This can be accomplished with:

• LUN Masking (on the host HBA)
• SAN Zoning (on your fabric)
• Port Grouping (on your storage)

Horror story aside, lets have a look at a basic kickstart for the ESX. When you install ESX by hand, it creates a ks.cfg file in root’s home. Inside you will find the host configuration at the time of install.

ASSUMPTION: You have already provisioned the necessary space on your array for your host, and have mapped the host access to that LUN.

After a few modifications and some cleanup, I ended up with the following

accepteula
keyboard us
auth  --enablemd5 --enableshadow
#this will work on all 'fresh' LUNs. If you want to an
#existing vmfs store, then add --overwritevmfs
clearpart --firstdisk
install cdrom
rootpw --iscrypted $1$MoIxblff$td1t5Oe4Kq8KZqyByh5Bc1
timezone --utc 'Australia/Sydney'
network --addvmportgroup=true --device=vmnic0 --bootproto=dhcp
part '/boot'  --fstype=ext3 --size=1100  --onfirstdisk
part 'none'  --fstype=vmkcore --size=110  --onfirstdisk
part 'HOSTNAME_boot'  --fstype=vmfs3 --size=9232 --grow  --onfirstdisk
virtualdisk 'esxconsole' --size=8232 --onvmfs='HOSTNAME_boot'
part 'swap'  --fstype=swap --size=1228 --onvirtualdisk='esxconsole'
part '/var/log'  --fstype=ext3 --size=2000 --onvirtualdisk='esxconsole'
part '/'  --fstype=ext3 --size=5000 --grow --onvirtualdisk='esxconsole'
%post --interpreter=bash
# ntp settings
esxcfg-firewall --enableService ntpClient
chkconfig ntpd on
cat > /etc/ntp.conf <<EOF
# ---- ntp.conf ----
restrict default kod nomodify notrap nopeer noquery
restrict -6 default kod nomodify notrap nopeer noquery

#replace with your ntp server address
server x.x.x.x

hostname HOSTNAME
sed -i -e "s/PermitRootLogin.*/PermitRootLogin yes/g" /etc/ssh/sshd_config

EOF

I set out to install onto the FIRST available drive. In my case, thats always LUN 0, and if by some accident there’s already a VMFS on it - installation will cease (you can accomplish a force through by adding --overwritevmf to the clearpart line.

Once the setup completes you will be greeted with a “Press <enter> to reboot” - in a normal RedHat install you can get around this by adding reboot --eject just before the %post statement. Unfortunately this doesn’t seem to work for ESX. I’ll update the post once I find a way.

You may notice that I keep referring to HOSTNAME - that’s because the ks.cfg is part of another script which puts together a custom ISO for each of our blades.

To get them, head over to “/vmimages/tools-isoimages” on the host and you’ll be greeted with:

If you’re just after the rpm then rip open the ISO and you’re done.

greater virtual performance over physical

Not every environment is large enough to dedicate a whole 8/12/40/96 GB of RAM Blade to a vCenter. Its simply a waste; the premise on which people dive into virtualisation is that green feeling, saving the environment or the saving the USD.

So businesses tend to ‘donate’ a physical box that will be the nucleus of their virtual offering. Yet this tends to be an underpowered, not-far from decommission door stopper.

As a Virtual Machine, you’re able to utilize the benefit of prompt resizing of the hardware, and at the ability to easily place it on higher-powered machines.

That’s the benefit, although what REALLY happens when you decide to move your vCenter around?

vMotion Mission Statement:

Storage VMotion lets you relocate virtual machine disk files between and across shared storage locations while maintaining continuous service availability and complete transaction integrity.

To me, that reads - “stuff will happen while we move your data”.

Unfortunately that isn’t always the case. Here’s the scenario.

Host

• DS1

• DS2

• DS3

vCenter was living on DS1 (slow-LUN), and I wanted to put it across to DS2. Since vCenter is very important, it’s continuous service availability is crucial. Read: {perfect candidate for svMotion}.

What happened next is counterintuitive to the credo. The vSphere client freezes, and no new connections can be made to vCenter. If you start monitoring the datastores (kudos to VMware) the VM is actually being transferred.

Once the migration completed, the first time I had to reconnect the vSphere Client; I wasn’t happy with this outcome. So I attempted a DS2->DS3 (both fast-LUNs). Not only did my connection to vCenter not drop this time, but the whole process was seamless and worked as advertised.

I ended up moving the VM in which I house SQL Server (vCenterDB) in the same fashion to the faster LUNS, and everything proceeded to operate as designed.

Lesson learnt: get decent storage.

You are going to have black caviar (highly recommended). You were provided with 50 grams; you figure you can fit at most 45 grams onto the piece of that delicious dark-rye. So what do you do? Do you disregard and thus throwout the 5 grams? … Well it’s not important what YOU would do, its important what VMware does!

Mr. ESX tends to look at the excruciatingly expensive, sustenance-providing caviar, and throws out the majority that it can’t handle, and opts for the remains in the hard-to-reach crevices of the jar.

What does this all mean for the geek?

There’s a decrepit limit of 2TB minus 512 bytes for each LUN that you can present to ESX. Anything larger, it has no love for. So if you were to present it with a 4TB LUN, you would naïvely assume that you would get the bastardised version of 2TB and the rest would be lost in the ether. I guess that would be somewhat logical.

Lets try it:

There you have it. Instead of actually using up as much as ESX’ly possible (~2TB) from a LUN that has been allocated, VMware chose to only pick up the left-overs (~500GB).

“Database job [Past day stats rollupvcenter] was created by another user. Please use the same user to setup your DSN or remove the job. ODBC Error: [Microsoft][SQL Native Client][SQL Server]The specified @job_name (‘Past Day stats rollupvcenter’) does not exist.

When vCenter is first installed, it schedules jobs with the help of the system DB - MSDB. What’s left to do, is just remove the jobs created by the previous dbo of your vCenter data. You achieve this by first listing what jobs are created on the SQL Server.

SELECT [job_id],
[originating_server_id],
[name],
[enabled],
[description],
[start_step_id],
[category_id],
[owner_sid],
[notify_level_eventlog],
[notify_level_email],
[notify_level_netsend],
[notify_level_page],
[notify_email_operator_id],
[notify_netsend_operator_id],
[notify_page_operator_id],
[delete_level],
[date_created],
[date_modified],
[version_number]
FROM [msdb].[dbo].[sysjobs]

As you can see, four scheduled operations exist. Thankfully you don’t have to worry about just clearing this table, as there’s a stored procedure that comes within MSDB → sp_delete_job

Run it for each of the jobs, and you’ll be ready to continue installing vCenter.

For what I’m building, I needed access to the API provided by VMware, and unfortunately, it is only provided as a SOAP interface. The documentation is a given, although it is of quite a poor standard. Since case-sensitivity is a must, not to mention correct parameters.

So how do we go about creating an easy to use PHP page which would consume the .NET WebService?

<?php
$namespace = "http://vmware.com/labmanager";
$soap_dat["username"] = "user";
$soap_dat["password"] = "password";
$soap_dat["organizationname"] = "org";
$soap_dat["workspacename"] = "cake";
$client = new SoapClient("https://server_name/LabManager/SOAP/LabManager.asmx?wsdl");
$header = new SOAPHeader($namespace, 'AuthenticationHeader', $soap_dat);
$client->__setSoapHeaders($header);
$config["configurationType"] = "1";
$result = $client->ListConfigurations($config);
print_r($result);
?>

After using some of the methods, I decided to write a wrapper for the LabManager’s API, so I could use it within other applications we develop inhouse. You can then call this directly from the command line, and integrate it with your bash/perl - and not have to re-invent the wheel.

As such - I present to you phLabManager - “Simple, lightweight wrapper for LabManager 4.0 SOAP API” written in PHP.

Once you’ve grabbed the labmanager.php, you can write quite succinct calls directly to the methods, without worrying about the implementation. There’s an example on usage available.

I haven’t finished implementing all the methods, and in the coming days will endeavour to do so, alternatively - you can do it, and commit into the tree. It would be great to get some help in optimizing my rough implementation.

This is due to a file which hosts all the chart settings (an XML-based key value pair collection) becomes corrupt, and is actually cut off.

Its quite easily found at : c:\Documents and Settings\user_name\Application Data\VMware\server_name-charts.xml

From here, you have several choices:

• revert to a backup
• delete it
• clear it (just place <ChartSettings /> inside of it)

That’s all there is to it. Happy vSphere’ing

1. Training + certification revenue
2. Tools that are clobbered together over years of being driven by big-customer requests/demands, and not a defined architectural vision.

One such example is installing and uninstalling cumulative fixes for WebSphere. My first step was to download the PUI, followed by its extraction and then I had to load the extremely cumbersome and slow Java GUI. Then to use it, I either dropped into X11 forwarding the session from my Mac, or for my unfortunate colleagues - they have to start a VNC session first and then use the client. Stupid (not the colleagues).

So I came up with something that wrapped the updatePortal.sh in between a purpose built interface, accessible from the command line. I bring you, the CLI Wizard for Portal CF’s.

Lets disassemble the necessary steps first, and start with uninstalling.

Portal places all the CF definitions into PortalServer/version/PK{number}.efix, these are just xml files which describe the PK. What we need is the PK number and the description.

After a few lines of bash, on a server with several CF’s installed the output will look similar to this:

Not very useful though, yet. So we now allow for actually selecting one of the CF’s through the great bash select statement.

Having the PK number, we are easily able to cut/awk and pass this to the trusty updatePortal.sh to do its uninstalling deed.

The installation is much the same, although you must first obtain the PK jar. In the office we have a build server that spits them out in (in zips with the readme). So the script first of all gives a listing of available CF’s to install, and then when you choose the appropriate zip, it will download it and install.

You can download the complete script (no longer available), and let me know if you have any issues. I thought of distributing this on Google Code, so others that deal with Portal can extend it, as Portal’s Java GUI leaves much to be desired.

Instead of installing and managing software that monitored CPU/Memory/HDD/Network utilization,  I merely converted the whole machine to a virtual one - and offered it as a replacement under the same IP to all the users. This was done with a quick P2V conversion of the server into a VMware image pushed to a dedicated IBM Blade server running ESX 4.0 - which had far more grunt than the originating server. A few tweaks on the DHCP server to assign the right IP to the new born, and we were away. NOTE: the usage shown is of the machine running on the Blade, not on its original hardware; I’ll come back to this later in the post.

Before I continue, I’ll just tell you more about the software stack that is deployed on this specific machine. There is WebSphere + AppServer as well as a local instance of DB2. Mostly as you can see, its Java based, so memory usage will be fairly straight forward as it all gets eaten up on app start, and the variations that we see are from DB2. (I could use a remote DB and alleviate the memory variation all-together, which is something I will be doing with all deployments from now on).

Firstly now, the CPU Usage, throughout the day with constant use of the machine, averaged out to ~5%. Am certain this isn’t unheard of, for those that actual look at their data centres and the individual/un-optimized servers. Disk usage, aligns quite nicely with the Memory footprint, which once again points to DB2 doing its job.

So what can I improve here? -  I can do several things:

1. remove local DB, and deploy into a contained VM cluster, or just point it at a dedicated machine (which will also be a VM)
2. if I offload the DB, I can increase the heap size within WebSphere for increased startup time, and overall performance
3. put more machines onto this single Blade

Realising the first two points only after looking at the graphs (to write this post), I still haven’t implemented them. I did though - do the third.

Very quickly you realise that although there’s an abundance of processing power, there’s just not enough RAM on the host blade. The ration of Ghz to Gbyte’s of RAM is about 3:1, instead it should be at the very least 1:1. Keep this a consideration next time have to put a new purchase order for more machines.

In the beginning of the post, I mentioned that the current blade host although more powerful than the original home of this software stack, it is far more scalable, and that is one of the reasons why I wanted to transfer and consolidate the numerous x345’s we have onto several blades. Most of the x345’s that were hosting servers are now turned off, the remaining few, I’m building a storage cluster with, to act as a iSCSI target server for the ESX’ed Blades.

What was satisfying was some people commenting on how fast the servers responded, little did they know that they were virtualized. Lesson: don’t buy 20 cheap and nasty servers, but a blade centre and start populating it as your budget permits.

.. is the extremely helpful message that VMWare converter displays when it fails.

Digging deeper, within the logs we can see that there are multiple instances of

"Warning: failed to create directory"

and

"Warning: failed to clone directory tree".

The simple work-around is to convert the vmdk disk to a monolithic-sparse.

You can do this by issuing:

$ vmware-vdiskmanager -r original.vmdk -t 0 destination.vmdk

This will clone the disk image as well as modify it from being composed of 2GB files for the entirety of your VM to a single vmdk referred to as a ‘monolithic-sparse’ (merely referring to the fact that it will increase in size automatically to encompass the the VM partition).

After completing the cloning process, you should have no problems in restarting the conversion process, and it should complete as advertised.

Have you recently sent an email from your beloved iPhone and have it delivered - empty? Then when you look at the sent on your phone you get the lovely: “This message has no content”.

It was simple - the problem was my signature. Which constituted of a simple ‘•’. An elevated full-stop, also known as bullet-point.

There appears to be a bug in the iPhone that is as follows:

• If you send an email with a • as the LAST character, your sending bar at the bottom of Mail.app will continually say ‘Sending’, and the email will not even show up in your sent
• If you send an email with words, then • and then some more words - not a problem - it will be delivered

If I didn’t know better, I’d say that upon Unicode conversion upon sending, the email becomes corrupt through a rogue parser.

Quite an oversight on Apple’s behalf, and has only appeared since the 2.0.2 update.

Moral of the story - don’t have a bullet point as the last character in your emails, let alone your signature.

Have you noticed any strangeness in your iPhones behaviour?

•

Think of all the bills, invoices, warranty cards, tax refunds and bank statements that you file away, day after day. The problem here is that paper is bulky, and takes up a lot of physical space, especially if you’re an owner of a filing cabinet.

My hope for this task was to achieve a state where I can be light on my feet, so should I require to move places or send off a tax returns, I can now do so within a few minutes and not hours, and perhaps even to save myself a bit of sanity

Goals

• quick - no one wants to spend 5 minutes processing each document that comes in the mail
• searchable - putting in an account number of any kind, should bring up ALL the relevant document. So OCR here is a must
• manageable and easy - I don’t want to wade through hundred of folders to get the document that I want. Plus should I wish to group a set of files together for a submission, that should also be easily achieved

What will you need:

• document scanner
• OCR software
• some time to plan-out your storage strategy

Quick

Speed is an important factor when deciding on picking the right document scanner. I looked at several brands, but being an Apple user, am unfortunately limited to a certain domain of equipment with drivers that will work with OSX. I went with a Fujitsu ScanSnap S510M. The ‘M’ n the model denoting ‘Mac’, so naturally the only difference between that and the non-mac version is that its white and comes with apple drivers. Fortunately for me and you, it competes very well against the competition such as Canon DR 2050C

Searchable

The ability to find a scanned document by varying characteristics such as tags and creation dates and not merely by the hierarchy of parent folders, I find quite a necessity. Several applications that I looked at, had each their good points and down-right annoying.

DEVONthink Pro Office apart from the dated interface, lack of tagging ability {think folders}, $150 dollar price tag - is quite a lot like Lightroom for your documents. It has a lot of nice features, such as word occurrence calculators, which helps you pick out the most unique descriptions for each document. Devon keeps a database that you must also backup, of all the structure and data that you enter about each of the documents, so not the most agile of concepts.

Yep by Ironic Software is one of the more popular solutions. Thanks in-part to their clean design and ability to tag each document. It too relies on an database {XML-based plist} for the storage of its own meta-data. After spending quite a bit of time with Yep, I very much like the speed and almost ‘freshness’ of the app. Its drawbacks are also its strong points, it doesn’t have an OCR engine, which is great - since it leaves it up to you the user to figure out which one you wish to use. I tried both ABBY, and Acrobat; prefer the latter due to speed, and ability to do some compression with help of AppleScript before spitting out the resulting PDF.

I did notice a few bugs on first use

• by setting the ‘original document created’ field, the date doesn’t get saved to the file itself, and thus isn’t reflected in Finder’s ‘Info’.
• ability to create multiple smart collections of documents with the same name {screenshot below}
• when searching for tag occurrence within documents and you aggregate several tags together, an empty result-set is generated

I sent the above-mentioned issues to the developers, and I have yet to hear back from them. Will update this section when I get a reply.

Despite the technical flaws, am a big fan of the workflow that Yep presents, yet their testing practices leave much to be desired.

Last but not least, Papers, an application that specializes in the storage and research of scientific papers. I would leave it at that, I didn’t find it useful for the task at hand, that’s not to say it isn’t good at what its designed for.

Manageable

Although tightly related with the previous section, manageability of your documents is something that will underpin how well you’re able to search for items once you get all those documents into some sort of a repository.

Unfortunately all the applications I’ve looked at, keep their own database of the meta-data. This is unfortunate for two reasons:

1. Switching cost - between applications is increased, as you’re more likely to stay with an app once you’ve imported a copious number of documents and succumbed to the apps way of thinking.
2. Backup - of not only the documents but also the database has to be thought of. This is where Devon makes it easy, by allowing you to place its database file into any location you desire, such as with your documents. What’s worse is that Yep doesn’t give you that freedom, and keeps its library within your Application Support folder. I received an email from ‘Ironic Support’ informs me that the only way to keep documents and data-store in one location, is if you keep all documents in your Application Support/Yep folder.

The Scanner

An important consideration for this whole task is the scanner itself. So what are you looking for in a document scanner? It doesn’t really matter what I say here, as price inevitably dictates peoples choice. Personally I saw the time it takes me to collate, organise, and file my .. err files extremely wasteful. It ended up being about how much my time is worth to me in the long run.

After doing some research the decision was between two models, the Fujitsu S310M and the S510M. The baby {given its diminutive size} of the two, S310M has one of the most attractive features in a document scanner, it can be powered merely off the USB ports on your laptop. So if you travel a lot, and its important to scan research papers {if not available in PDF-form already} or contacts, then you can’t go past the S310M. I chose to go with the S510M hoping that the extra $100 will result in a better quality scanning ‘head’ if I may call it that; so your choice should also depend on how many documents you’re going to be scanning. If you’re merely going to be doing a couple of pages a day, then grab the S310M alternatively if you are swamped with papers on a daily basis, or are going to be re-scanning your complete paper filing cabinet it might be an idea to go for the big brother S510M.

The shots below are to allow you to see comparatively how big or small this document scanner really is. I didn’t want to use the metric system, nor the imperial system - to prevent confusion. So I picked a unit everyone is familiar with. So the scanner is about 1.3 iPhones tall, with a footprint of about 1.1 iPhones, with the length of just under 2.3 iPhones.

Both scanners do duplex scanning, thus double sided documents are not a problem. What impressed me most is how well the paper feeder actually works. I’ve had the experience of working with a Canon DR 2050C and if you mis-align a page, or put something of obscure size, such as a shopping docket through - you were bound to get it jammed, not so with the S510M. I threw about a hundred sheets of A4, A5, dockets, scrunched receipts from the corner PC store, throughout the duration of the day and without a single twitch it generated a double-sided PDF for me; with the help of Acrobat 8.0 Professional and some AppleScript it OCR’ed and knocked out backgrounds for me too. In instances where I was a little eager and mis-aligned the paper, the software automatically corrected any skewing and rotated the page upright - nice!

Conclusion

When I first embarked on setting myself free from dependence on the myriad of documents, I was very hesitant to lay down the $’s for a scanner. Albeit since it arrived, it has been a blessing. As soon as I scan a document, back it up {on that in the next post}, I shred it. Hence to anyone that values their time, and ability to streamline some of the mundane tasks of paper management I couldn’t recommend the idea of a paperless office more.

Have you made your office paperless?

Having recently purchased the unit, especially for the purpose tagging all the photos for an upcoming trip, I flew away extremely happy, witnessing the outstanding performance of this little device. You can see a screenshot of Google Earth displaying the track below; its the takeoff from Sydney towards Tokyo.

What sets this unit apart from other loggers and GPS units on the market, is its ability to not only acquire a signal within the boundaries of an airplanes fuselage, but keep that sync as the airplane is banking. This latter achievement is the result of the -158dBm sensitivity rating of the unit.

Geotagging

In order to couple the data off the unit with your images, first of all you must make sure that the date+time is the same on both the unit, and your camera. Then based on the time of each point recorded and the time a photo was taken, the closest data point becomes the location of where the photo was taken.

There’s nothing you can do about the GPS, since its sync’ed with the satellites and stored in UTC. What is important though is to adjust the cameras time to the Time Zone of your location, this will save you much heartache later on when you have to adjust the times.

For my needs, I thought I’d try out HoudahGeo, and GPSPhotoLinker. I very quickly dismissed HoudahGeo, because I was unable to try it ‘properly’. By which I mean use it unrestricted for a period of my journey. I don’t believe its possible to evaluate certain software unless you can actually try your complete workflow with the software, and see if it fits, HoudahGeo unfortunately provides you with a handicapped version.

In comes GPSPhotoLinker, which is actually a free app, and gets the job done surprisingly well. What you’ll notice is that once you combine your photos with the exported data from the GPS, it will actually look up the City and Suburb data as well and embed that into the EXIF along with the GPS location. This is extremely useful for my Lightroom catalog where I can pick things based on the City as looked up by GPSPhotoLinker - brilliant! This is extremely helpful when you’re island-hopping, as is inevitable in Greece, or are merely in a new place.

As it stands, I think its a wonderful little unit that has served its purpose extremely well. I would highly recommend it, and will not be leaving the house without. It now lives at the top of my camera bag.

• log data to internal memory
• make sure there’s LOTS of memory
• and lots of battery

After looking at the Sony GPS-CS1, Wintec WBT-100 and GlobalSat DG-100, I concluded that the BT-1000P is by far the best unit for my needs.

• First of all, the GPS chip supports up to 51-channels, with sensitivity down to -158dBm, which allows me to get a satellite lock even at the university concrete monstrosity of a car-park.
• It has integrated bluetooth, so I can use it as a replacement for my ageing Holux-236.
• Most importantly, it can log around 200,000 points, with my current configuration of strings, only 122,398 (@5 second intervals, that’s 7 days of 24 hour logging before I have to clear it!)

What pushed me to purchasing after looking at the specifications though is the battery.

• The same as you find in most Nokia phones [BL-5C], which can be picked up cheaply off eBay
• Delivers a claimed 32 hrs of logging, although I was able to get 26 hours without any problems, before the little red battery starts to flash. Mind you - I didn’t fully charge it, and just started running it soon after receiving it in the post.

Now I needed the unit to work on a mac, which the BT-1000P doesn’t do out of the box. The good news is that there’s a project called bt747 that lets you interact with the BT-1000P.

Upon connecting the unit via USB the rest of the steps are quite simple:

1. Download and install the USB->Serial driver
2. Download bt747
3. Switch the BT-1000P to ‘LOG’ mode
4. Copy RXTXcomm.jar and librxtxSerial.jnilib into /Library/Java/Extensions
5. Run run_rxtx.sh

You should get an extremely ugly GUI

Just hit ‘Connect Port Nbr’ and you’re ready to go. You should get something similar to: After this, all you have to do, is run around the block a few times with your unit in LOG mode. Come back home, run the app again then: Under the ‘File’ tab, set the Output directory to a location that has world write access, then hit ‘Apply & Set the above values’ Alternatively create a directory in your /var and sudo chmod 777 <directory>.

1. Now -> ‘Log’ tab
2. Hit ‘Get Log’ button {this might take a while, depending on how long your jog was}
3. After its complete, feel free to select the output format, either CSV, GPX or KML.

If you chose to export a KML, and you have Google Earth installed, you should get a nice looking track. Unfortunately it is evident that the BT-1000P doesn’t filter its points prior to writing them out, resulting in data that isn’t representative being recorded, hence you can observe the spike.

Now to get ready for my last exam tomorrow morning, and then to start packing for the trip. Once I’m back, will report on how the unit goes in the field, and I’ll include some geocoded images.

UPDATE

Upon the comment below by Benn, I remembered a feature that this unit has, that every time you take a photo on your camera - you can also hit the big red button on the tom of the unit. This marks the POI in the data stream. Merely allows you to come back for any manual intervention and find interesting parts easier.

Everyone agrees that WebKit is the faster browser engine. Plus it’s my favourite browser/engine in OSX

Problem is that it doesn’t update itself automatically - and every night there’s a beautiful new build that comes out - which I simply must have.

So in order to save you time - here’s a little script that I put together this afternoon that will download the latest build at the push of a button.

Initially I wrote it with wget, but realised not everyone will have it installed - hence here’s a version with curl

#!/bin/bash
#  getWebKit.sh - Obtain the latest WebKit build
#
#  Created by Roman Tarnavski on 2008-04-13.
#  For an updated version head to www.romant.net
#
#
#  Distributed under a Creative Commons Attribution 3.0 License
#  For more details on the license: http://creativecommons.org/licenses/by/3.0/
var_dir="/var/tmp/"
echo -n "Downloading WebPage ..."
location="http://nightly.webkit.org"
curl http://nightly.webkit.org/index.html -o ${var_dir}index.html > .get_webkit.log 2>&1
nightly=`cat ${var_dir}index.html | grep ".dmg" | cut -d '"' -f 2 | uniq`
echo "Done"
filename=`echo $nightly | cut -d '/' -f 5`
echo -n "Getting WebKit ..."
curl $location$nightly -o $var_dir$filename >> .get_webkit.log 2>&1
echo "Done"
if [ $? ]; then
	echo -n "Installing WebKit ..."
	rm -rf /Applications/WebKit.app
	hdiutil attach -quiet "$var_dir$filename"
	cp -R /Volumes/WebKit/WebKit.app /Applications/
	hdiutil detach -quiet /Volumes/WebKit
	echo "Done"
	exit 0
fi
#cleanup
if (( $? )); then rm ${var_dir}index.html && rm $var_dir$filename ; fi

Let me know how you go

MacBook Pro

Dell 2408WFP

You can evidently see that the Dell shows a purple instead of a blue. Very disappointed was I to think that a screen on which I just dropped some hard earned, can’t even display “BLUE”!

and then..

I just thought I would open it up the images in Firefox - just to check.. and voila


Unfortunately after a few more tests like the above - it seems across OSX is the same thing - ie. Finder when displaying the above photos - merely shows it in shades of purple as well.Now the only reason I even came across the fact that it could be an application specific error - is when I look at the photo in Lightroom 1.4, depending on which screen I have the image on - it changes the colours to suit the screen. So my question is - if Lightroom can do it - why can’t OSX?

Update

After re-calibrating both monitors with the Gretag Macbeth i1 Display 2, I have composed some windows together to show the difference between different applications and their adherence to the needed colour management.

The applications I used for this test, and in this specific order are:

1. Safari3.1 (5525.13)
2. Lightroom 1.4
3. Firefox 3.0b4
4. Preview 4.1 (469.2.1)
5. Photoshop CS3

Dell 2408WFP

The peculiar thing about the Dell, is that Lightroom actually showed a very strong purple instead of blue, and then after about a minute - without me physically interacting with the computer - it simply changed to the correct colour. Which is depicted in the shot above.

MacBook Pro

Analysis

It seems that in the initial shot with the 2408WFP, Safari obviously does the worst, showing a blatantly purple where there should be blue. Other applications vary slightly in the shades of blue, but otherwise display a fairly closely matched colour.

When it comes to the MBP to display the same set of images, Safari, Lightroom, and Photoshop - all display a shade of purple instead of blue.

The annoying part is that whilst writing this - if leaving the  specific application on either screen - it adopts the correct colour profile, and displays the correct colour.

As such - I have no choice but to personally conclude that it seems that OSX isn’t informing the application that it has been moved to another screen in a timely manner.

There have been reports that the Dell uses a new backlighting system that the i1 is finding difficult to actually meter, yet to me it seems there’s more software problems at play here under the bonnet of OS more than anything else.

Can’t believe I stumbled onto this shortcut, as its exactly what I wanted to do. You see, I didn’t want to disable the multimedia keys as defined by default in OSX - as it would lead me to having to hold Fn to do one-key thing such as volume. So I went about pressing everything I could - and by mistake I came upon the following shortcut.

Fn + Tab

Which hides both - your left and right panes.

Quite useful for both development view [at least until Adobe decides to provide secondary monitor support], and grid view.

Before:

.. and After:

So how does one provide the appropriate backup solution to safeguard their cherished memories? My general approach to backing up the photos is by no means simple, yet it does the job.

Just as a disclaimer - the main application that I use is unfortunately Windows only - although a similar setup can be achieved using rsync.

Here goes:

1. Download the images off the camera using Lightroom, which simply stores them in the yyyy-mm-dd format
2. SyncBackSE then automatically mirrors the newly added images onto a separate hard drive in my T60’s expansion slot, for a pseudo RAID-1 solution
3. When I’m at home there are two states for my laptop, either docked or free and about. In either case, my network drives get mounted. Once this occurs, the local SyncBackSE copies the images straight onto my server.
4. The server’s two main backup drives are configured in a RAID-1, using Adaptec 1210SA adapter
5. I then have another copy of SyncBackSE installed on the server, which once a week copies all the files from the RAID-1 drive, onto a third drive within the server. This gives me some time to recover shots if I did delete any.
6. Once a month or there about – I also have an external drive which I merely copy manually and then store at a friends house.
7. As mentioned in the previous step, a third drive is given 1 week of lead time prior to being updated. Now in order to solve the geographic problem of having all the backups under the one roof - I have a Site-to-Site VPN with a relative, for whom I purchased a 500GB drive, onto which I then backup once a week.

The great thing about this setup is that as soon as a file is added by me or I modify tags within Lightroom of a photo, the change is automatically replicated across 4 drives - just in case, and overall – each image exists in 8 places.

Pros

• Quick and automated backup solution
• When I’m travelling – I still utilize the complete backup cycle – by connecting via a VPN directly to the server using my laptop in an internet cafe – and provided they have enough UP bandwidth – I’m done uploading in 20-30 minutes, and my server then takes care of everything.

Cons

• This occurs within a one physical domain (provided my laptop is at home at the time) - so should a fire occur, the complete setup is futile - as everything is burnt.
• There’s a finite amount of space available on my laptop, and although it wasn’t a problem when I shot JPEG with the 300D, now with the ease of RAW processing within Lightroom and ability to shoot it easily with my 40D - one days outing ends up being about 1-2 GB.

Solution to the first problem is in the next step of my backup workflow, and to the second problem I’ll cover in a later post.

In order to help distinguish between my drives, I have a naming scheme for them. I grab the names out of Greek mythology, sometimes even attempting to match the name to the task of the drive.

Here’s a diagram of a solution that works for me

If anyone finds this useful/too-much – am interested to hear your opinions. Feel free to share how you do it – and any suggestions you may have.

Since TFS manages source code, as well as documents, it is an extremely convenient repository of data for your projects; yet - by putting all your eggs in the one basket, you may be unpleasantly surprised when you decide to rename your project. In Team Foundation Server 2005, this ‘feature’ was absent, if you were hoping to find it in 2008 – stop looking, there are too many DB references to update. We can only hope that Microsoft will bring about a tool to do this as part of a service pack.

Now you can jump for joy - as right now ACR supports 40D

Also note that on the page it lists Lightroom 1.2 as available also, yet this has been reportedly false. which is available for download over at Adobe

If anyone has attempted to import from 40D into Lightroom, they might notice that all the images are rotated 180%.

Windows picks up the correct way up, yet Lightroom simply doesn’t.

I hope Adobe will be quick to remedy this problem, as well as update the RAW converter to accommodate the 40D

Update

Having a further look into it - I noticed this only occurs when you connect the camera directly to the computer, as opposed to extract the CF, and using it within a reader.

Also windows didn’t automatically rotate the photos when looking at them through explorer, so possibly this is a Windows Vista problem, and not Lightroom.

If anyone in XP or OSX can reproduce this - do post.

Update 2

After trying this with Lightroom 1.3 - the issue still persists.

Not fixed yet.

You can get plenty of complete previews/reviews off other sites

I will merely mention several things.

I purchased the kit, which included the 17-85mm f4-5.6 IS USM lens.

Initially I was considering of purchasing the 400D, although after having a handle of my friends 350D [which practically has the same physical dimensions] – I decided that the camera felt .. soft. Not the build of it, but more so that actual ‘lack-of’ grip. It was simply too small in my hand.

To overcome this – I thought of getting the battery grip, and upon a visit to the local CameraHouse – I didn’t like that either, as the seams of the battery grip didn’t meet up properly with the base of the 400D. This resulted in the camera simply not feeling right, plus it was smaller than my 300D!

Then I envisioned purchasing the 30D, but that thought quickly faded, as I’d be investing in old[er] technology. So I set my sights on the 40D.

Upon driving 90km to the store I ended up a happy camper. So how did it fare?

Body

Holding this puppy makes me feel like Kim Jong-il with his finger on the button. The finish of the body can be described like Vista – without the sarcasm – “Wow”. It is quite solid, and I have yet to drop it, hopefully I won’t be testing that feature.

The LCD is simply stunning, large [3.5″] and the LiveView is quite nice, if ever-so subdued and seemingly dull.

The buttons and their layout will be familiar to all previous Canon SLR users.

Lens

The kit lens is nice and bulky so it has a right balance with the body. The USM is very quick to focus, and the IS helps you get those photos you previously discarded due to being too blurry.

One gripe, and the main gripe I have with the lens is its ability to focus. I hope I just got a bad copy of this beast, but it simply cannot focus properly. I end up with images that are way too soft to be appreciated. At first I thought it was the camera, yet upon donning the 50mm 1.8 II and seeing the exceptional images it produced I simply decided to take the lens back, which I’ll be doing once the store opens up on Monday.

So far – that is it. There are too many features to mention that this camera contains, which is most likely more than an S-Class Mercedes.

Overall

Extremely happy with the camera, not so much with the kit lens.

UPDATE

I did a few test shots, to outline what I believe is a pretty bad copy of the 17-85, or perhaps as some have suggested the new focusing mechanism in the 40D playing up with a lens that can’t go f2.8..

Lens - 50mm 1.8 II

Focal Length: 50mm

Exposure: 1/160

FStop: f/2.2

Focal Length: 50mm

Exposure: 1/60

FStop: f/3.5

Focal Length: 50mm

Exposure: 1/50

FStop: f/4.5

Focal Length: 50mm

Exposure: 1/30

FStop: f/5.6

Lens 17-85 IS USM

NOTE: Check the focal length

Focal Length: 17mm

Exposure: 1/60

FStop: f/4

Focal Length: 17mm

Exposure: 1/30

FStop: f/5.6

Focal Length: 17mm

Exposure: 1/13

FStop: f/9

Focal Length: 35mm

Exposure: 1/50

FStop: f/5

Focal Length: 35mm

Exposure: 1/30

FStop: f/6.3

Focal Length: 53mm

Exposure: 1/40

FStop: f/5.6

Focal Length: 53mm

Exposure: 1/25

FStop: f/6.3

Focal Length: 85mm

Exposure: 1/25

FStop: f/5.6

Focal Length: 85mm

Exposure: 1/15

FStop: f/7.1

As you can see, the lens simply isn’t usable around 17mm, yet it does perform quite nicely at 35 and over. Understandably this isn’t L glass, but I don’t think it should be THIS soft. I will be returning the lens to Canon for re-calibration.

I like music, and as someone that has a few computers around the house, coupled with a Vista Media Centre - I want to be able to have access to mood altering music throughout.

My music application of choice happens to be iTunes™. Understandably it isn’t everyone’s favourite, and I must admit - it does seem very sluggish on Windows™ compared to its OSX™ counterpart.

The point of this post is to setup streaming of music to iTunes™ utilising mt-daapd [aka Firefly], Ubuntu, VMWare and obviously the mp3’s themselves - hosted on a Windows™ share.

Many people will have a ‘server’ of some sort, be it for the hundred of gigabytes of family photos and videos, a common repository for house mates to exchange files or simply in my case - all of the above.

I have a server that runs Windows™ 2003 Server. My laptop synchronizes with it every time I dock it, thus providing me with a cheap NAS-like solution for my photos, university lectures and assignments.

Also, since Internet radio is something else I like to dabble in, and my ISP providing me a nice list free from bandwidth charges - and ultimately wishing to purchase a Wi-Fi enabled radio streamer, I decided to enable streaming of music across my house.

You might be asking your self - why am I running a VMWare Server + Linux in order to just stream music held on a Windows™ share? Well there are alternatives, and apps such as Tangerine that will run quite happily on Windows™ itself and serve music - but I do not want to run anything that isn’t necessary on the server itself - I prefer to leave it to virtual machines, as it allows for a much quicker re-deployment, coupled with the fact that I need a few other services to run on the VM for the network.

So here’s a guide on how to get streaming mp3’s with the use of mt-daapd, linux and about 30 minutes of spare time.

I will use names for usernames and server configurations that I have used, change as needed.

On the Windows box

1. Create a username “buntu” [assign a password]

2. Create a directory which will host the root of your music. So if it’s present in H:\Backup\Files\Personal\Music, then share and make sure you share Music, thus it will be accessible via \quack\music [quack is the name of the server]

   • Make sure the security as well as the sharing settings allow for “buntu” to have ‘read-only’ access

3. Download + Install VMware Server

4. Download Ubuntu Desktop/Server [or a distro of your choice - I will only run through Ubuntu, as it’s based on my farourite Debian, only prettier + more user friendly]

   • I recommend you download the *alternate* ISO, ubuntu-7.04-alternate-i386.iso as with VMWare server, you can get a “no screens found” error, and the alternate install will merely remove the GUI whilst installing Ubuntu.

If you’re creating the VM - 2 things.

1. When you’re going through the ‘New Virtual Machine Wizard’ - make sure you select “Use Bridge Networking”, as that way you’re making it easier for your new virtual machine to have access to the current network and thus the network shares on another computer.

2. Hard drive space allocation - if you’re going to be using this for something other than just streaming, and have other services in mind for this VM - such as perhaps a dedicated development environment, you might consider giving it more than the default 8Gb.

On with it.

Assign the ISO you download as the CD drive.

As I mentioned above,

Proceed through the menus and install Ubuntu, for most - just accept the defaults.

Optional

In order to have the VM start when the host boots up, select the appropriate options after going to the VM Settings and under the Options tab - you will see - “Startup/Shutdown”, with the respective options being now open to you on the right hand side.

Then

apt-get install smbfs
mkdir /music
mount -t smbfs -o username=buntu //quack/music /music
password:`

• The above should go through, if you have all the permissions set up correctly.
• Now you have to make sure it is mounted on boot. Edit the /etc/fstab and append the following:

note the space between the source and the mount point

//quack/music /music cifs username=buntu,password=music,user 0 0

A quick check:

mount /music
ls /music

Should return no errors, and you will now have /music mounted each boot.

Time to set up the streaming.

apt-get install mt-daapd
/etc/init.d/mt-daapd start

Now if you installed something with X, launch your browser of choice [firefox], and point to: http://localhost:3689 (Default username:password -> admin:mt-daapd)

or

alternatively modify the /etc/mt-daapd.conf manually

Here you are able to specify any parameters as well as monitor who’s leeching [scratch] listening to your music.

At the very least, I recommend you modify (under Configuration on the site):

• Server Name [as per the file itself - ’the stuff that comes up in iTunes’]
• Admin password [used to log into the the site]
• Music Folder, set it to ‘/music’
• Hit ‘Save’

Now on the status page, hit ‘Start Scan’ - and you’ll see the little counter for ‘served’ increase quite rapidly. Any iTunes™ instance on the network, will be able to see Quack under shared music.

Within the Western society, it is the norm to go through the sifting process and hopefully end up still floating when HR finishes washing their lists, to make sure that the ‘key words’ that they were after, are actually on your resume.

Now that you’re up to stage two, you tend to go through several more:

1. Meet HR - have them toy with you and get a feel for you.
2. Technical Test - if you’re going for a programming role, you’ll once again be sifted until they make sure you know what OO is, and where you put a tilde to form your destructor.
3. Group Excercise Now that you’ve passed the previous stage with flying colours, you’re invited back into a group formation where you can be studied like a primate “assessed on your thinking ability, general knowledge, communication,argument structuring, confidence, presentation skills as well as creativity” (Lifted from Accenture Recruitment Guide)
4. Meet the manager At this point, as far as HR is concerned, you’re as good as hired, since not many people make it to this stage - and the only reason you wouldn’t pass is primarily due to your lack of personality . Here you are evaluated on your ability to actually ‘fit in’.

With this “fool-proof” system, every company still ends up with a multitude of useless workers that merely are able to meet the basic requirements, pass the aptitude test and now be a leech on the system. Chances are - it will cost the company more to support this worker that it would have been to spend more time to find the right candidate in the first place, especially considering the inflexibility of labour evidenced by the plethora of unfair dismissal lawsuits.

So back to the Japanese gentleman I sat next to on my flight to Frankfurt a few years ago. He informed me that the company he worked for, had an unwritten policy that if they’re going to be spending over ¥10 million JPY - then the hiring manage must conduct a careful analysis of the person, which in this case meant - take them to a golf course for the day!

Wow, a whole day of playing golf and relaxing.. Well, not exactly. As through the day the manager is able to see your traits that you would normally leave out of an interview, such as how competitive you are, and how you actually deal with stress and your diplomatic ability.

These subtle hints which will provide the employer with a clearer image of the real person, is far more useful than a two hour psychometric test, that after 30 minutes you are just clicking on anything just to finish!

A day long interview - does it cost too much? Well if you want to employ someone for a few years at the least, and they are not there to clean the toilet or make a cup of crap coffee, then you simply can’t afford not to spend the time.

At the moment my set-up consists of:

2x Samsung 940B’s

1x IBM/Lenovo Advanced Mini Dock

1x T60 [ATI Mobility Radeon x1400]

Works quite well, as soon as it’s docked, it recognises the two monitors (one plugged in using VGA, other DVI), and off it goes.

In Windows, it’s quite simple - plug in another monitor, and through Display Settings - select where each screen is relatively to the others.

Unfortunately, even in Windows Vista - 5th Version of the monolithic, GUI - OS [95,98, 2000,XP, ->Vista), it is has yet to cater for a spanning taskbar, one that would cross all monitors.

That and other shortcomings are fixed by a nice program called UltraMon. It allows for a multitude of features concerning multiple monitors. One’s that I use:

• the spanning taskbar
• Windows Title Bar

Puts icons near the close button of the window, that allow you to quickly with a click, move the windows to another monitor, or stretch it across all monitors

• Wallpaper

Finally - a tool which looks at setting distinct wallpapers for each screen, that is easy to use, and is persistent across reboots as well as docking/undocking my T60

Now with UltraMon 3 Beta - you can finally use it in Vista.

Several problems though:

• Docking the laptop - Causes multiple taskbars to be rendered on the secondary display

• Freezing whenever some applications are launched - ie. FileZilla

With all these, am still a hapy camper, and absolutely love the app. Just hope they hurry up, and release the final.