Monday, May 21, 2018

Thinking about Software Architecture & Design : Part 9

I approach software architecture through the medium of human language. I do make liberal use of diagrams but the diagrams serve as illustrators of what, to me, is always a linguistic conceptualization of a software architecture. In other words, my mental model is nouns and verbs and adjectives and adverbs. I look for nouns and verbs first. This is the dominant decomposition for me. What are that things that exist in the model? Then I look for what actions are performed on/by the things in the model. (Yes, the actions are also “things” after a fashion...)

This first level of decomposition is obviously very high level and yet, I find it very useful to pause at this level of detail and do a gap analysis. Basically what I do is I explain the model to myself in my head and look for missing nouns and verbs. Simple enough.

But then I ask myself how the data that lives in the digital nouns actually gets there in the first place. Most of the time when I do this, I find something missing in my architecture. There are only finite number of ways data can get into a digital noun in the model. A user can enter it, an algorithm can compute it, or an integration point can supply it. If I cannot explain all the data in the model through the computation/input/integration decomposition, I am most likely missing something.

Another useful question I ask at this level of detail relates to where the data in the nouns goes outside the model. In most models, data flows out at some point to be of use i.e. it hits a screen or a printout or an outward bound integration point. Again, most of the time when I do this analysis, I find something missing – or something in the model that does not need to be there at all.

Getting your nouns and verbs straight is a great first step towards what will ultimately take the form of objects/records and methods/functions/procedures. It is also a great first step if you are taking a RESTian approach to architecture as the dividing line between noun-thinking and verb-thinking is the key difference between REST and RPC in my experience.

It is hard to avoid prematurely clustering nouns into types/classes as our brains appear to be wired towards organizing things into hierarchies. I do this because I find that as soon as I start thinking hierarchically, I close off the part of my brain that is open to alternative hierarchical decompositions. I try to avoid that because in my experience, the set of factors that steer an architecture towards one hierarchy instead of another are practical ones, unrelated to “pure” data modelling. i.e. concerns related to organizational boundaries, integration points, cognitive biases etc.

Take the time to explore as many noun/verb decompositions as you can because as soon as you pick one and start to refine the model, it becomes increasingly hard to think “outside the box” of your own architecture.

Friday, May 18, 2018

Thinking about Software Architecture & Design : Part 8

It is common practice to communicate software architectures using diagrams, but most diagrams, in my experience are at best rough analogies of the architecture rather than faithful representations of it.

All analogies break down at some point. That is why we call them “analogies”. It is a good idea to understand where your analogies break down and find ways to compensate.

In my own architecture work, the main breakdown point for diagrams is that architectures in my head are more like movies than static pictures. In my minds eye, I tend to see data flowing. I tend to see behaviors – both human and algorithmic – as animated actors buzzing around a 3D space, doing things, producing and consuming new data. I see data flowing, data flowing out, data staying put but changing shape over time, I see feedback loops where data flows out but then comes back in again. I see the impact of time in a number of different dimensions. I see how it relates to the execution paths of the system. I see how it impacts the evolution of the system as requirements change. I see how it impacts the dependencies of the system that are outside of my control e.g. operating systems etc.

Any static two dimensional picture or set of pictures, that I take of this architecture necessarily leaves a lot of information behind. I liken it to taking a photo of a large city at 40,000 feet and then trying to explain all that is going on in that city, through that static photograph. I can take photos from different angles and that will help but, at the end of the day, what I would really like is a movable camera and the ability to walk/fly around the “city” as a way of communicating what is going on in it, and how it is architected to function. Some day...

A useful rule of thumb is that most boxes, arrows, straight lines and layered constructions in software architecture diagrams are just rough analogies. Boxes separating say, organizations in a diagram, or software modules or business processes are rarely so clean in reality. A one way arrow from X to Y is probably in reality a two way data flow and it probably has a non-zero failure rate. A straight line separating, say “valid” from “invalid” data records probably has a sizable grey area in the middle for data records that fuzzily sit in between validity and invalidity. And so on.

None of this is in any way meant to suggest that we stop using diagrams to communicate and think about architectures. Rather, my goal here is just to suggest that until we have better tools for communicating what architectures really are, we all bear in mind the limited ability of static 2D diagrams to accurately reflect them.

Thursday, May 10, 2018

Thinking about Software Architecture & Design : Part 7

The temptation to focus a lot of energy on the one killer diagram that captures the essence of your architecture is strong. How many hours have I spent in Visio/Powerpoint/ on “the diagram”? More than I would like to admit to.

Typically, I see architectures that have the “main diagram” and then a series of detail diagrams hidden away, for use by implementation and design teams. The “main diagram” is the one likely to go into the stakeholder presentation deck.

This can works fine when there are not many stakeholders and organizational boundaries are not too hard to traverse. But as the number of stakeholders grows, the power of the single architectural view diminishes. Sometimes, in order to be applicable to all stakeholders, the diagram becomes so generic that it really says very little i.e. the classic three-tiered architecture or the classic hub-and-spoke or the peer-to-peer network. Such diagrams run the risk of not being memorable by any of the stakeholders, making it difficult for them to get invested in it.

Other times, the diagram focuses on one particular “view” perhaps by putting one particular stakeholder role in the center of the diagram, with the roles of the other stakeholders surrounding the one in the middle.

This approach can be problematic in my experience. Even if you take great pains to point out that there is no implied hierarchy of importance in the arrangement of the diagram, the role(s) in the middle of the diagram will be seen as more important. It is a sub-conscious assessment. We cannot help it. The only exception I know of is when flow-order is explicit in the diagram but even then whatever is in the middle of the diagram draws our attention.

In most architectures there are “asks” of the stakeholders. The best way to achieve  these “asks” in my experience is to ensure that each stakeholder gets their own architecture picture, that has their role in the center in the diagram, with all other roles surrounding their part in the big picture.

So, for N stakeholders there are N "main views" - not just one. All compatible ways of looking at the same thing. All designed to make it easier for each stakeholder to answer the “what does this mean for me?” question which is always there – even if it is not explicitly stated.

Yes, it is a pain to manage N diagrams but you probably have them anyway – in the appendices most likely, for the attention of the design and implementation phase. My suggestion is to take them out of the appendices and put them into the stakeholder slide deck.

I typically present two diagrams to each stakeholder group. Slide one is the diagram that applies to all stakeholders. Slide two is for the stakeholder group I am presenting to. As I move around the different stakeholder meetings, I swap out slide number two.

Tuesday, May 08, 2018

Thinking about Software Architecture & Design : Part 6

Abstractions are a two-edged sword in software architecture. Their power must be weighed against their propensity to ask too much from stakeholders, who may not have the time or inclination to fully internalize them. Unfortunately most abstractions require internalization for appreciation of their power.

In mathematics, consider the power of Euler's equation and contrast it with the effort involved to understand what its simple looking component symbols represent. In music, consider the power of the Grand Staff to represent musical compositions and contrast that with the effort required to understand what its simple looking symbols represent.

Both of these abstractions are both very demanding and very powerful. It is not uncommon in software architecture for the practitioner community to enjoy and be comfortable with constantly internalizing new abstractions. However, in my experience, a roomful of software architects is not representative of a roomful of stakeholders in general.

Before you release your killer abstractions from the whiteboard into Powerpoint, try them out on a friendly audience of non-specialists first.

Wednesday, May 02, 2018

Thinking about Software Architecture & Design : Part 5

Most architectures will have users at some level or other. Most architectures will also have organisational boundaries that need to be crossed during information flows.

Each interaction with a user and each transition of an organizational boundary is an “ask”. i.e. the system is asking for the cooperation of some external entity. Users are typically being asked to cooperate by entering information into systems. Parties on the far end of integration points are typically being asked to cooperate by turning around information requests or initiating information transfers.

It is a worthwhile exercise while creating an architecture, to tabulate all the “asks” and identify those that do not have associated benefits for those who are performing the asks.

Any entities interacting with the system that are giving more than they are receiving, are likely to be the most problematic to motivate to use the new system. In my experience, looking for ways to address this at architecture time can be very effective and sometimes very easy. The further down the road you get towards implementation, the harder it is to address motivational imbalances.

If you don't have  an answer to the “what is in it for me?” question, for each user interaction and each integration point interaction, your architecture will face avoidable headwinds both in  implementation and in operation.

Friday, April 27, 2018

Thinking about Software Architecture & Design : Part 4

Any new IT system will necessarily sit inside a larger context. If that context includes the new system having its own identity – from “the new system” to “Project Unity” - it will be anthropomorphised by its users. This can be good, bad or neutral for the success of the new IT system.

It does not matter what form the naming takes e.g. “Project Horizon”, “Bob's System”, “The new HR system” or even a visual identity such as "the new button", or even a tactile identity such as “the new panel under the desk at reception”. In all cases the new IT system may be treated by existing team members in much the same way as a new team member would be treated.

New systems get “sized up”, so to speak, by their users. Attributes such as “fast”, “unreliable”, “inflexible” or even “moody” might be applied to the new system. These may be factually based, or biased, depending on the stance the community of users adopts towards the new system arriving into their team.

One particularly troublesome possibility is that the new system may be seen as causal factor in events unrelated to it. “X used to work fine before the new system came along....” The opposite can also happen i.e. the new system get plaudits for events it had no hand or part in. Causality versus correlation can be a tricky distinction to navigate.

Takeaway: sometimes the human tendency towards the anthropomorphic can be used to your advantage. If you suspect the opposite may be true for your new system, it can be useful to purposely avoid elaborate naming and dramatic rollout events which can exacerbate anthropomorphisation.

Sometimes, new systems are best rolled out with little or no fanfare in a “business as usual” mode. Sometimes it is not possible to avoid big bang system switchover events but if it is at all possible to adopt a phased approach to deployment, and transition slowly, I would recommend it for many reasons, one of which is the sort of team dynamics alluded to here.

As AI/Robotics advances, I think this will become even more important in the years ahead.

Tuesday, April 24, 2018

Thinking about Software Architecture & Design : Part 3

In software architecture and design we have some pretty deep theories that guide us on our way. We know how to watch out for exponential run times, undetectable synchronisation deadlocks, lost update avoidance etc. We have Petri nets, state charts, entity/attribute diagrams, polymorphic object models, statistical queuing models, QA/QC confidence intervals....the list goes on....

...and yet, in my experience, the success of a software architecture & design project tends, in my experience, to revolve around an aspect of the problem domain that is not addressed by any of the above. I call it the experiential delta.

Simply put, the experiential delta is a measure of how different the “to be” system appears to be, to those who will interact with it – experience it – day to day.

A system can have a very high architecture delta but a low experiential delta and be many orders of magnitude easier to get into production than a system with low architecture delta but a high experiential delta. 

It pays to know what type of experiential delta your “to be” solution represents. If it has high experiential delta, it pays to put that issue front and center in your planning. Such projects tend to be primarily process change challenges with some IT attached, as opposed to being IT projects with some process change attached.

In my experience, many large IT projects that fail, do not fail for IT reasons per se. They fail for process change reasons, but get labeled as IT failures after the fact. The real source of failure in some cases, is a failure to realize the importance of process change and the need to get process change experts into the room fast. As soon the the size of the experiential delta crystallizes.

Indeed in some situations it is best to lead the project as a process change project, not an IT project at all. Doing so has a wonderful way of focusing attention on the true determinant of success. The petri nets will look after themselves.