Breaking Down and Architecting Epic Features

One of my recent experiences on a large project rewrite got me thinking how a project team would break down large feature sets. One day, a specification document for a very large ‘module’ built from an existing flag ship product landed on peoples desks that was over 100 pages large. When dealing with that kind of large information, without an Agile mindset, the natural reaction of the team was to read through that 100 page specification and ultimately hand it to the development team when one is comfortable with what it says should be done. You’re hoping here that the development team will swallow that same epic amount of information and simply do what’s asked – what could go wrong?

Image

With my Agile experience, my head was spinning on the constraints this was going to put on the project at this point. Time to pop on the systems thinking hat.

Let’s assume for a second the product team are happy with that document, and off it goes to the development team. If this lands in the hands of a developer, you’re running the risk that this developer has the Agile mindset and will breakdown this information appropriately, designing the code to handle everything in this document with a sprinkle of future proofing and just enough architecture to handle what’s in there. If you’re lucky enough to have a large team with an architect who can break that down before it gets to the development team then great. Now, I know what you might be thinking at this point, it stinks of waterfall, and you’re right, we’ll address this in a moment.

Here is the first key point I’m trying to make when dealing with large amounts of information in projects that ultimately need developers to act on and deliver. Breaking down information when it lands on the developers desk isn’t the only place breakdown should occur. There is no such thing as a product rewrite from simply one technology to another. People change things and want to make things better from previous generations of software – changes will occur. By the time the developer has chewed through that same document to implement it, the business/customer may and probably will change its requirements.

Breaking down information/features should happen as early in the software process as possible.

It’s not until the changed features are in development and the end user explores the new interface/platform that they begin to have ideas about what it is exactly they want to change in the new product. As developers at least, we know this, this is what Agile has taught us or even our own experience. We know people change their minds when they start working and demoing the software. To optimise this way of working throughout the whole process, we spread the idea of breakdown right on the first day of seeing that 100 page specification document. Information being kept small means it goes through your entire software process faster, giving faster feedback, and more opportunity to readdress and redirect.

How Breakdown Affects Architecture

When the information is broke down so all elements of the software process are invoked more often, this creates an affect on the architecture of the application. For example, if you choose not to consider everything in an epic feature, how will you know how big it really is, and how much code scaffolding you need to implement to future proof the entire feature development? Hindsight can be a wonderful thing.

The key here is scouting the specifications for key architecture points. Design fundamentals such as an application being multilingual, multi-currency etc. absolutely need to be considered right at project inception. An experienced product team or developer will know to ask these kind of questions very early on in a project. One of the more common questions is, to what capacity does it need to scale to?

If the team don’t go looking for these kind of key points, and they are realised later on, they are really going to smash through the architecture of an application, causing a lot of rework if done very late in the product and are very sweeping design points. Absorbing these kind of design points early really depends on the experience of the team.

The line has to be drawn somewhere, and no matter where you draw it, it won’t be perfect. The future proofing decisions will be made from database structure, deployment strategies, to UI/UX design etc. If you talk over your constraints in the project and let them guide you to the decisions of what you come to know as the ultimate “must be in” features, and the less important points can be dropped and considered in the future, if they occur.

Examples of early key decisions:

  • Is it going to be multilingual?
  • Is it going to be multi currency?
  • Should this application run on both browsers and tablets as an application?

Examples of deferrable design decisions:

  • Should the icon look like X, or Y?
  • Should the transition effect of the menu display be like X, or is it better with Y?

Talking about the deferrable design decisions may cause developers to build in future proof code for good reason. For example, when the decision being made has a result in all options to produce work, rather than a result that possibly means the work is not done at all.

I hope this post helps when embarking on large project rewrites or trying to consume large feature requests. Let me know in the comments if you’ve had a different experience or disagree with anything.

Posted in Agile, Project Management | Tagged | Leave a comment

Debugging / Profiler Code in Production

So, debugging, profilers, it’s going to slow down the code! Let’s just have a moment and think about all the other things in the world that have production debuggers and profilers. We’ll cover the pro’s and con’s after:

1. Your car, hopefully has a spare tyre in the boot, along with the jack to hold up the car, and the wrench to remove the wheel nuts. How about the first aid kit or a warning triangle for traffic if you break down?

2. iPhone’s, Samsung Galaxy, and other mobiles phones, all have debugging modes,and recovery modes for firmware flashing. This makes it easy for field engineers or stores to restore firmware or fix issues without you having to send it right back to the manufacturer where they perhaps would instead have to take it apart and replace chips…

3. Manufacturing robots in blue chip companies all have pluggable interfaces for an engineer to come along with a laptop, and listen to all the sensors that are installed around the machine. These sensors allow the machine to monitor itself and predict issues or notify the users of the machines when part’s need attention.

4. Another one for cars – that ECU computer in your engine make it very easy for your local garage to plug a computer in to and diagnose issues around the car, from again, sensors installed at critical points around the vehicle.

5. Windows – Safe mode! There is also the feature when you overload the resources of the system and it starts advising that you should turn off theme support and unnecessary graphical enhancements.

6. Your computer – POST. Yep, it really is self testing. Better yet, you get a diagnostic report if your RAM is faulty or drives are not connected correctly.

The benefits of allowing debugging, performance monitoring or health checks in live systems is incredible. Would you rather spend time on the phone talking through a user experience to understand an issue, or would you rather have your application contact you with a full report on exactly what was going on in it’s environment before things started going wrong? Better yet, what if your application could monitor itself, and give the user some hints on improving or avoiding an upcoming problem?

Consider an event journal from your application. You could use this journal to replay through an automated test harness and arrive at exactly the same point your user began experiencing issues, or found a bug. This replay mechanism could also be used for performance testing changes. If it took 30 seconds to execute a journal of events in your application before a new feature was added, just re-run it and see how much longer it takes.

The visibility you can get from just recording check points in your code for performance monitoring is also very nice. Imagine a single line of code to “check in” with a monitor. This monitor could time all your DB calls, or amount of hits to a service, and warn or adjust itself if things become too chatty or slow.

Imagine actually receiving an email from your application as it immediately begins performing badly perhaps due to data access reasons. You could be working on the issue and have it resolved way before your users are aware and start raising issues with support/help desk – “Hi, we’re experiencing a slowness in the loading of this report, can you help?” – “Absolutely, we’re aware of the problem and the fix is already being applied. This will be resolved in the next few minutes.” – “Great, thanks!” (experienced reactions may vary ;)).

Now what does this cost? Usually, just a tiny bit of code noise. You may spot a line of code at the top of method that says “Checking In”, or a block with a pair of “BeginProfile” and “EndProfile” calls. Usually these calls are time stamping or recording state.

It’s important to realise that this kind of helping code has to be high performing, and not be doing anything risky/heavy. By risky, I mean, throwing data at a database that may go down which then simply starts throwing exceptions, killing your application. For heavy, you really don’t want these helper calls to be causing any delay by waiting for a server to respond or blocking your thread as it sits writing blocks of data to the local disk for example. Keep it simple – don’t get fancy.

Posted in Debugging | Leave a comment

Entrepreneurs

At my current place of employment, parking has become a bit of a nightmare. The companies success over the past few years has caused a boom in employee’s and parking space ran out long ago.

We’re located on a business park, surrounded by hotels, shopping centres, gym’s, bowling alley, cinema etc..

Most employees now use the parking spaces five minutes away by walk, in the biggest shopping centre on the business park. Today, we received a polite notice via windscreen letter that we’ve been detected as abusing the car park for long term stay. In our defence, we do use the shopping centre daily for lunch breaks, but that’s beside the point.

Back in the office, there were discussions about where we were going to find parking. One thing led to another, and from the back of a joke about a business opportunity for writing to local business who have spare parking, to rent them out, this really got me thinking. I carried on planning out a business idea with my colleagues next to me, and this idea really started to make sense!

An hour later after the conversation died down, someone came to me and mentioned that a local company had picked up on this issue some months back, and did just the idea we talked about. They were renting out parking spaces for a small fee per month in other business parking areas.

Now to my point.

I was amazed how so many of us involved ourselves in a discussion about a business opportunity, but pushed it off as nothing more than an office joke. I thought…..this is what must make the difference between an entrepreneur and well, the rest of us.

The power to believe in an idea, have the self confidence and belief that you can put it in to action is truly an amazing quality. There are so many non self believing people, but true confidence and determination will make the difference between forever dreaming of idea’s, and making them a reality.

So, to you, entrepreneurs – I salute you!

Posted in Uncategorized | Tagged | 1 Comment

Software Development And Baking A Cake

I happened to be catching up with an ex-colleague tonight over MSN who told me how small things during his learning of Agile were gradually making sense. During a moment about complexity, he said the following:

“A overly complicated piece of OO code can be just as bad a long piece of procedural VB.”

This got me thinking about the principle that any tool, methodology, fancy design pattern, can be overused or used incorrectly, creating a bigger mess than what it intended to resolve. This needs an analogy. Cooking!

A software principal, such as the Single-Responsibility-Principle (SRP), is like the sugar in a cake recipe. No sugar, and it’s not tasty, too much, and it’s sickly, and people most likely won’t eat it. Just the right amount, and you have a great tasting cake.

Any tool or principle that comes a long in software development is never a golden bullet. It’s the wisdom of the developer to know when and how any tool or principle is applied. Never buy in to a buzzword or “the cool design pattern” that pops up on your favourite software forum.

There are of course naturally occurring problems in all areas of software development. Like division by zero errors, or null reference exceptions. We solve these problems with defensive coding and exception handling. We should also equip ourselves with a set of design patterns that we know we’ll need in 99.9% of all software projects. These guard us from the more subtle growing issues that creep up in large growing projects.

Driving without a seat belt is irresponsible. The time you’ll need it on, it will save your life. Drive software development without a set of patterns and practices, and you’re asking for trouble – code that’s so hideous, no one will touch it, or maintenance of a project that becomes exponentially higher each time you work with it.

Do yourself a favour and learn the SOLID principles along with the importance of unit testing/TDD in software development. These help form your foundational principles that should be at the front of your thinking when writing code and designing software.

Posted in Agile, SOLID, TDD | Leave a comment

Importance of Business Cases

Today, I was looking at some code that for the life of me, I couldn’t make head or tail what was going on. I was trying to resolve some bugs that were the result of some events that were crossing each other in such a non-logical way that I couldn’t see through the smoke and mirrors of the events. Not even the good old “Debug.WriteLine(“I’m here.”);” across all the events, and recording them out in to a journal, helped me.

Then it struck me. I’d got so used to not writing tests, as this was a WinForm development piece, that I lost the value of what writing tests meant to me. TDD, and testing in general, placed the importance of the behavior of software, way before writing the software solution. I was trying sitting there trying to figure out the complex event model without any cases of what it was I was trying to solve or see the behavior.

A quick step back, and I wrote out the use cases from the very edge of the system, down on a piece of a paper. Starting with the most simple, I flexed the software to try and fulfill the business case. Low and behold, they failed. Then determining the exact step the case was beginning to fail, I could resolve the system to fulfill the case. Iteratively then working through each of the cases I had listed, they were all eventually fulfilled. A quick smoke test of all the features around the points changed and all was well. Boy do I miss TDD, and this is why UI development frustrates me a little. There was no way for me in WinForms to add tests to the model I was working in. I was always going slow, thinking about regression, and constantly retesting the software with a real run.

Today’s lesson – always always hold the business cases right where you can see them – in tests! If you can’t write the tests, then at least write them down in front of you, reminding yourself that what you are developing is to fulfill those cases. It sounds pretty simple, and quite obvious. But it really is easy for a developer to get lost in technical details and complexity of a system, which of course, is a code smell in itself…

Posted in Uncategorized | Leave a comment

Transaction Script

I’ve been involved in looking at a lot of middle ware “Transaction Script” style code that sent a few alarms bells off in my mind. This pattern of code is explained in Martin Fowler’s book, Patterns of Enterprise Application Architecture.

As I’ve been coming from a lot of research in to Agile programming practices, my main focus is testable code. From that, is how clean the code has to be to be testable. The other bigger values of clean code is of course, one’s ability to learn an API or maintain it – navigation.

With that in mind, “Transaction Script” certainly doesn’t fit nicely with the new Agile principles and practices I now have. The disadvantages I see with this style of programming is quite alarming to me. I’ll be quick and summarize my thoughts:

  • Huge “global” type functions make it very difficult to test and inject mocks – TDD and TAD becomes difficult.
  • OOD/OOP is usually out the window, as the style is more procedural.
  • Code duplication due to the repeated nature of logic throughout other transaction scripts.
  • Low class cohesion – big weighty functions carry all their state required to perform the one operation.
  • Anaemic domain model – “Transaction Script” is usually accompanied by DTO’s.

It’s not all bad however. Anyone can start a function and grow it with endless amounts of logic in the fashion “Do this, do that, do this, do that, done.”. It definitely is quick development in the short term and not hard to simply to come up with CRUD style operations over a set of DTO’s. This highlights that Transaction Script favours business code that doesn’t carry a lot of logic – load an object, update an object, save an object. It really shines if most of the code you have to write is simply that. Beware though, the parts of a system that start requiring more logic than usual will soon give birth to a “Tar Ball Object” – horrendously large classes.

Posted in Agile | Leave a comment

My Path

I’d just like to give you my steps of inspirational readings and my most turning moments so far.

There have been a few “WOW” moments in my software career. They went as follows:

Head First Design Patterns – This really opened up the world of OOP/OOD for me, an absolute staple.

Robert C. Martin – Clean Code – I didn’t sleep for days with excitement from this! As simple as it is, boy had I forgot what it meant to have clean code, and this put me right back on track.

Robert C. Martin – Agile Patterns and Practices – I now live my coding career based around these principles. The word SOLID goes through my mind for each and every piece of code I write. This in combination with Clean Code, I was affected so much by them that, I convinced my employer at the time to let me teach it to the entire company of around 15 developers. I wasn’t even a senior, good at presentations, but my enthusiasm made it happen.

My last “WOW” moment, was the discovery of TDD. I shared my realization of it with 2 other close friends who also had the same “WOW” effect. We were all hooked and now practice this as much as we can and always discuss the important of testing in the Software Development Life Cycle – better to have self verifying code than a team of much slower humans to do the testing for you😉.

One final point from me that I feel is very close to my heart, is that if you ever want a true fundamental understanding of a technology, such as a script parser, a REST library, a test framework etc, then write one! Just for fun! Not only will you enjoy it, but I’ve found that attempting to write the things I do not yet fully understand, gives me much more respect and a much much more deeper understanding of it, and forces me to understand it thoroughly.

Posted in Uncategorized | Leave a comment