romex-system-incoming-message-flow-crop4blog

Fixing Fragile Dependent Processes

The Client had windows services that processed messages from subscribed devices.  There were two services: one which did the main processing of messages that primarily processed details of the message type and time, and the second service processed GPS positions associated with each message.  For example: a message might indicate that a journey had ended, and the first service would calculate the consequences for ending a journey, and the second service would figure out where the journey had ended and the consequences of that.

In general, messages came in no more than once per minute per device, and the result was that most of the time, processing a message had plenty of opportunity to be completely handled by both services before the next message came in.  However, devices sometimes failed to send in messages in a timely fashion and might end up sending in many at once.  In these circumstances, even though individual messages were always processed in the correct order within a given service, the pair of processes could not guarantee the correct order to process all aspects of each message, so outcomes might vary from one day to the next.  The result of these complex interactions was odd results to processing that could only be understood when considering the potential progess of two different services through a number of messages at a given point in time hours or days ago! Continue reading

map-replay

The One Missing Feature

Sometimes the success or failure of a new system function can end up depending on one feature; and if that feature or function is missing the whole reason for the system existing can be undermined.

This system we are looking at today is one which can track vehicles, and one feature is the ability to plot journeys on a map in a web-browser; it’s actually very cool, but the plotting of the route was a little slow. Continue reading

Who Are We… I Mean… Who Am I Kidding?

Some of the earlier posts on this blog use ‘we’ and ‘us’ to hint or suggest that 42 IT Solutions has a team of like-minded people working for a better future in IT.  I’m sure there were any number of clues that this was ‘enterprise-speak’; but as things stand 42 IT Solutions is effectively only one person: me, Nigel Rheam.

Hello!

Continue reading

The Dangers of Duplicate Data

Received an email from Amazon today, this email explaining that delivery of my parcel was delayed demonstrates the dangers of duplicate data, I think:

29/09/2016 is actually a Thursday!

29 September 2016 is actually a Thursday!

The duplication I am thinking of is that in addition to providing the date; 29/09/2016, they have also chosen to provide a refinement; ‘Friday’.

Continue reading

Software Inventory – Joel on Software

While the Software Inventory post by Joel Spolsky is a thinly-veiled advertisement for Fog Creek’s new (at the time) ‘project management’ tool Trello; the points Joel makes are nevertheless deeply meaningful to me. Points like this:

“The trouble is that 90% of the things in the feature backlog will never get implemented, ever. So every minute you spent writing down, designing, thinking about, or discussing features that are never going to get implemented is just time wasted.”

Or this:

“… the desire never to miss any bug report leads to bug bankrupcy, where you wake up one day and discover that there are 3000 open bugs in the database, some of which are so old they may not apply any more, some of which can never be reproduced, and most of which are not even worth fixing because they’re so tiny.”

…though you probably won’t need any more than 20 bug reports to find outdated or incorrect tickets!

I’ve just had another reason to hunt-out and refer to this blog post professionally, in an effort to encourage a client that having a bunch of years-old tickets assigned to someone but ‘on-hold’ when the original ticket requester has left the business is a sure-sign that really those projects are never going to happen and are probably not a good idea anyway!

– Make it consisent – Make it consisent – Make it consisent

We made a mistake recently, breaking one of our own rules; Be Consistent.  Now, of course it is not always possible to ‘be consistent’, sometimes because you are doing something truly new; but often because one incorrectly sees differences – when you may be better off seeing patterns and similarities (and thus implementing something to fit an existing pattern)!

Continue reading

Premature Simplification – Allowing UI Display Formats to Drive Data Storage Formats

What is Premature Simplification – other than ‘Allowing UI Display Formats to Drive Data Storage Formats’? It will be easiest to start with an example: A company receives records from many devices, and decides that the end-user of the system web site will never want to view detail at finer grain than 1 second… so they decide that all time formats should be stored without milliseconds (or ticks) – that is to say, timing data is rounded or truncated to seconds. Continue reading

Two Hard Things

This post by Martin Fowler quotes Phil Karlton:

There are only two hard things in Computer Science: cache invalidation and naming things

(and Martin adds the derivative quote: ‘there are two hard things in computer science: cache invalidation, naming things, and off-by-one errors’) which is quite nice. Today’s post was originally about naming things being hard… but I think I can extend it to something close to the two topics in the quote. Continue reading

Rejuvenating a Stalled Project

The client had a stalled project with many changes to an existing system; there were several areas of new functionality, but they were not completely trusted. There was considerable concern regarding the reliability and performance of the database, and in some senses the database was seen as the whole problem.  Not only had this set of changes stalled, but minor fixes had also stacked up waiting to be released and all were dependent upon the main project.
Continue reading