Navigating the Seas of Information

Sunday, May 19, 2013

Publish Subscriber Pattern in ArchiMate

On the ArchiMate forum on LinkedIn, Stéphane Passignat posed the following question:

"Publish Subscribe Pattern reprentation with data-object.

How would you represent data-object exchanges from an application component to another application component using ESB topic or any kind of pub sub pattern ?"

There are a number of aspects here:

1) ("using ESB topic") How do you model a Topic with associated data?
2) ("from an application component to another application component") How do you show the application using the topic?

3) ("any kind of pub sub pattern") How do you model generic Publisher and Subscriber patterns with associated data?

What is the Publisher Subscriber Pattern?

This pattern is for an event to be broadcasted to multiple listeners. The same event may be broadcast by multiple originators. This is an Event-Driven Messaging pattern in SOA and a Publish-Subscribe Channel in Enterprise Intergration Patterns (EIP).

Enterprise Integration Patterns describes the problem as "How can the sender broadcast an event to all interested receivers?" The solution is a topic with one or more publishers and multiple subscribers.

Figure 1 - EIP Publish-Subscribe Pattern

Here it is very clear that a single message is delivered through a "channel" to multiple subscribers. I have created a very simple context for reference, however this time using the SOA template form Thomas Erl.

Figure 2 - Onboarding Scenario for context in the discussion

Here as part of an on boarding process an employee is added to a HR system. The service that configures the new employee broadcasts via a New Employee Event Service provided by ActiveMQ that the new employee exists. This message is picked up by the Create Account Service and an account is created into LDAP. I was also going to add the Training Enrollment Service as the second subscriber but one is enough to show the principle.

A Note on Syntax for ArchiMate

I am using the labeling syntax from Mastering Archimate Edition 1. It helps to identify what some of the objects are associated from other layers when not in a total view or using a tool like Archi which provides a visualization between layers while editing a particular view. It also helps in the menu to quickly grab the right object:

Figure 3 - Snapshot of Application elements from Archi

How do you model a Topic with associated data?

The first one in ArchiMate can be done through a technology layer, describing how the infrastructure provides a topic as an Infrastructure Service, an Infrastructure Interface and associated Artifacts. Here in Figure 1, I have modeled an example of Fuse Message Broker(FMB) as the MOM layer for the ESB. FMB is an ActiveMQ implementation and for distinction I have named it as an ActiveMQ infrastructure.

Figure 4 - MOM Infrastructure with service, interface and artifact

Described is a virtual server with RedHat Linux, and ActiveMQ as a System Software. The underlying systems are all aggregated as ActiveMQ Node. The Node realises a the infrastructure service "New Employee Topic". The topic accesses a "New Employee Message" which is an artifact on the system. This could be stored as a file or in an RDBMs depending on the setup but for this I have not dived that deeply. The Node also presents and external Infrastructure Interface to the Topic.

So how do you show the application using the topic?

Here I have taken the liberty of not modeling the infrastructure of FuseSourceESB. Instead I am modeling the services ('applications') deployed onto the ESB and how they access the infrastructure of ActiveMQ. I have done it through a Used-By relationship. There is a problem with this view. It does not show who is the publisher and who is the subscriber only which are using which. If there was only one publisher and multiple subscribers the Used-By relationship still would not identify the specific roles. In the Infrastructure with Usage view in ArchiMate there is no way of showing the flow.

Figure 5 - Infrastructure with Usage view of 2 service Components using the ActiveMQ New Employee Topic

Here we are at component level and we can see that the Employee data object is part of the New Employee Message data object via the composed of relationship. In this view the Employee data object is accessed by the Configure New Employee but I have not shown the fact that the Create Account component would also access it once it had extracted it from the New Employee Message. I would but I want to highlight where it originates from this this case.

The infrastructure artifact is now shown as the realisation of that New Employee Message data object. It is accessed by the New Employee Topic infrastructure service to which the New Employee Topic Interface is assigned.

How do you model generic Publisher and Subscriber patterns with associated data?

The answer that I propose to the problem is Application Interaction using an Application Co-Operation View.

Figure 6 - Application Co-Operation view using an Application Interaction to describe a New Employee Event

Here we can see the Application Interaction showing the flow of the message from the one to application to another. The "Configure New Employee" function initiates the "Broadcast New Employee Event" function which flows to the "New Employee Event" interaction.

This Application Interaction shows a 3 way interaction pattern between the publisher the channel and the subscriber as per the Publish-Subscribe Channel pattern. You can easily add more subscribers to the flow though with ArchiMate you may find allot of overlap between lines so I would recommend a second view with all the subscribers and no data object once there is one view with a data object.

I hope this helps and feel free to suggest improvements and alternatives.

Tuesday, April 30, 2013

ACS Forum: The Physics of Notation: Designing Diagramming Notations that Work review

I attended the ACS Forums monthly meeting at the Waterfront restaurant in The Rocks. The event had a great atmosphere and the people, food, and topics were excellent. I mind mapped the evening as I went through and then cleaned it up this evening, so it was a little more presentable.

The speaker was Dr Daniel Moody, Director of Ozemantics, and the topic was The Physics of Notation: Designing Diagramming Notations that Work. The talk focused on the problems of diagrams in the IT industry, and proposed a set of principals which could be used as the guide for improving the way visual languages are defined in order to improve communication between diverse groups. Dr Moody highlighted many of the current flaws with existing diagram development processes, including but not limited to,

a 90% focus on content 10% on syntax,
a lack of employing visual designers
lack of consideration for the way the human visual system works
intuitive and personal taste design over a measurable form
lack of design rationale for elements

The last one was perhaps the main focus, when an element is designed then there is little reasoning behind it. As he mentioned he was giving a similar presentation to a group and posed the question “why is the image for multiple events in BPMN a pentagon?” I must admit, I pondered it before he finished his story and assumed that each corner of the pentagon meant a direction for an event to occur thus multiple events. He then told us that the person who had designed that icon was in the room and told him “we had run out of symbols”. It makes you wonder.

The solution proposed was very interesting, in alignment with the idea of having a quality standard such as the ISO 9216 Product Quality quality model, he proposed a set of principals which could be used to give specific measures for how to create well designed visual languages. He went into detail on 3 of them, perceptual discriminability, semantic transparency, and semiotic clarity.

Perceptual discriminability is the ability to identify the differences between shapes. A circle compared to a square has a greater perceptual discriminability than a square against a rectangle. I found ArchiMate interesting, that a Business Service, Application Service and Infrastructure Service are identical in shape. I thought that colour was the key to their differences but as it was pointed out colour is not part of the specification.

I missed noting down all the variables during the presentation, so my mind map is a tad lax there however I did find it interesting that the mind can use colour to differentiate 3 times faster than shape. It would be worth seeing a study if the same occurs for contrast, as I think colour blindness was highlighted a number of times as an issue with colour.

Semantic transparency is simply the visual equivalent of an onomatopoeia word, which is a word that sounds like the thing it is describing for example “boom”. So the visual representation of a boom should look like an explosion.

I wonder how this would go with more abstract concepts. When you look at how a class is modelled in UML it can be modeled a number of ways depending on the view being used and the state of the object. For instance an active class, defined as “An active class is a class whose instances are active” has a notation defined as “An active class is shown as a rectangle with doubled vertical lines on the left and right sides.” An active class is something that is in action it is doing something, that same notation in flow charts means sub process.

Double lines on the road indicate do not cross, “=” is equals, and a quick search on google images turns up Double Line Stitch. I like the double line stitch, can we suggest it to be the new image for an active class. It continues the rectangle motif through the use of negative space, and has the double lines but has many of them and aligned facing outwards indicating an active rather than passive state.

Semiotic clarity was the last principle dealt with in detail. This principal means that one symbol means one thing. Most of the languages have ambiguity in some of the shapes. BPMN has 2 shapes which mean can mean an OR gateway, UML has lines and arrows which will change meaning dependent on view, and ArchiMate has the same issue. I do suspect there are only so many ways of drawing a line so you would need to head down the road of line/shape combinations (arrow) inevitably and if there needs to be one for every meaning it may become a very heavy language depending on what it needs to explain. The goal here however is precision, being able to communicate precisely the intended meaning.

I have noted a few others in my mind map, and I will see what I can do about getting the full list.

Daniel also has a manifesto for notifications. I have them listed here though they may not be in the right order.

Explicit design rationale must be included
Explicit testing
Evidence based
At least equal effort and attention on syntax and meaning
Adhere to the principles

From my own experience I use diagrams all the time. On my desk are sheets of A3 paper for discussions, I carry around my “crayons” (white board markers) with at least 8 colours. I regularly use and abuse UML, flowcharts, BPMN, CRC, DFD, ERD’s and am now looking at how to add ArchiMate to my list of communication arsenal. I found many of Daniel’s thoughts refreshing and insightful on the limitations of these visual languages and to look to ways to overcome them.

Thursday, April 25, 2013

SCRUM Australia conference

It has been a few weeks since the Scrum Australia conference and as I promised I would post my notes. Gathering all the information into a single post would be challenging, as there was so much information and shared experiences exchanged between everyone attending. So I have produced a summary, with a mind map detailing my notes.

My reason for attending was to get back in touch with people and what was happening in other companies with Scrum and agile processes in general. What I got out of the conference was allot of new friends, new tools and new insights into where Scrum as going and being treated. I was very glad to listen to talks that focused on how the business can use Scrum ver how a development team can use it and some heated discussions over different agile methods.

I was in some sense surprised to find that there was a growing trend of defining a very hard line of what is and isn't the correct process. One person mentioned that they had encountered a trainer who taught that a story must begin with "A user ...." and that if you used "A customer ..." you were not doing agile. I guess you get some miss interpretations and "hard liners" in any community. I doubt I will ever know the exact context that drove to that style of training.

The conference was over 2 days, April 10-11, there was a good range of talks, and an "Open space". This was my first open space which I really enjoyed talking about topics raised on the first day. I wish more conferences did open spaces.

I attended

Day 1

Opening Keynote: Economically Sensible Scrum
Scrum in highly regulated backend environments
Scrum from concept to app store
Global attitudes towards Scrum
Take it offshore! Ramping up a distributed team.
Inspire Management

Day 2

Open Space discussions (Scrum Trends session)

Benefits and Realisations, Business Value metric, Emerging architecture
Kanban
UX - How much is too much?
Product Management

Ebay. An agile journey for the worlds largest Data Warehouse
Portfolio Management with Scrum
Closing Keynote: Systemic Scrum - The lifeline of organisations

I have gathered my full notes into the mind map, it is quite large and I have provided a high resolution version so you can download and inspect the branches, and I will refer to some of the branches in my discussion.

Scrum process seems to be well adopted, and now the same questions that face every process are being addressed, such as cost of technical debt, product support mechanisms, resource and knowledge management, and measures of success. When you look at the Product branch you will see the main notes regarding these problems. What this suggests to me is that the process be it Scrum or Waterfall we are still dealing with project processes and wrestling with the operational process impacts. It may be worth looking at operational best practices and see how they influence Scrum methodologies. (Thus the buzz on DevOps)

Speakers such as Justin Urbanski and Jason Harwood highlighted how the introduction of SCRUM exposed the values of processes like continual integration, using spikes for rapid prototyping early on in projects and having custom testing tools were critical especially where high quality is required. I think the Halfbrick friday concept Jason described, where everyone across multiple teams get together and pitch ideas is of huge potential especially if you can implement those ideas.

Quite a bit of discussion was had over how to manage time, interruptions and work. I have been a fan of Pomodoro ever since Julines Liong introduced it to me years ago when I was working at USyd. It breaks time into Pomodoro units of 25min work and 5min clarity, establishes rules on managing interruptions, self improvement and is amazingly helpful if you are feeling overloaded and stressed. It is being applied to teams as Henry Wiputra pointed out, though that was the first time I had seen it applied at a team level. The figure that a person should be on no more then 2 projects was very interesting showing that more then this will cause increasingly inefficient work.

It is good to see that people are trying to determine the returns on products done through Scrum methodologies such as Kenneth Rubin in his Essential Scrum, however in discussions allot of people were reporting frustration at not being able to determine real benefit realisation or value from projects. This is something that definitely needs further investigation for myself as I am very interested in the correlation between BV figures and prioritisations. The question to ask is: "How can you prioritize with a measurement for estimated business value that can't then be measured for actual business value?"

The day before the event I had a Kanban game training session at work, which I really enjoyed, so the discussion on Kanban vs SCRUM was a great one, even though I did spend most of the time as an observer. The key items I felt came out of it was that people view it as a managers agile process and a way of taking decision making away from the team and back into the managers hand. I assume this comes from ways it has been implemented where that has been the case. I find the Kanban process interesting and from a pipeline perspective an effective way of managing blocks of work. Who makes the decisions can be a team or a manager.

I tended to agree with the outcome of the Product Management discussion branch which noted that the Business Analyst makes the best Product Owner. Its a gap bridging role as are the architect roles. Though having said that I must mention as Rowan Bunning pointed out to me, the PO role is about making the

"hard-nosed commercial decision making. It's about vision, $'s, trade-offs, value-based thinking, outcomes, being an intrapreneur and often saying "no" to the sort of peripheral requirements that many analysts tend to dig up"

Perhaps this one needs some more discussion and input at the next conference open space.

"T-Shape" skill set is something I absolutely agree with. Find a focus that your personality and skills really are best for, but ensure that you have the breadth of knowledge and skills to be able to move between positions and support others in your organisation. In agile this is essential. To simply pick up a task, execute it and then do a different task with different skill set is the key to efficient agile teams.

Slack Slack Slack. Use it, don't abuse it.

Daniel Gu's guidelines for good agile teams were very good. What eBay have done is very impressive as they have achieved very good agile teams and they do reflect many of the experiences that people related at the conference. If I was to put a team together for a project from anywhere I know who I would pick. Most of them are Scrum masters, they all have diverse specialties they are all skilled at adapting quickly to new technologies, they are all professional in their work and the sum creates a fantastic working team.

Well my pomodoro's for this blog have run out so I would also like to thank the organizers for doing such a fantastic job, and all the people who did attend for sharing all their experiences and insights.

Some good references gathered from discussions: