Service-Oriented Architecture Attributes

 Service-Oriented Architecture Attributes

After reading Messaging Patterns in Service-Oriented Architecture, Part 1 by Soumen Chatterjee {http://msdn.microsoft.com/en-us/library/aa480027.aspx} and various online patterns of Enterprise Integration Patterns by Gregor Hohpe and Bobby Woolf {http://www.eaipatterns.com or http://www.eaipatterns.com/eaipatterns.html} I have decided to try to make my terminology better conform to that of the Service-Oriented Architecture (SOA) publications. 

From my recent reading of this material it seems to me that the following classifications make sense.

First, to separate the SOA attributes into the following four categories.

I:   Those that are attributes of a component,

II:  Those that are attributes of a message topic,

III: Those that are attributes of a subscription by a component to a topic, and

IV: Those that are attributes of topic readers and writers.

For instance, an attribute of a component is how it is run or scheduled.  Whether it is run upon demand such as by an event or whether it runs periodically.  An attribute of periodically scheduled as can be further described as strictly periodic as scheduled by the operating system or aperiodic such as scheduled in a forever loop with a delay.

An example of a message topic attribute could be publish-subscribe versus point-to-point.

An example of a subscription attribute could be whether the subscribing component does so as the producer or the consumer of the topic.

Therefore, based upon my reading and the attributes that I have used in my exploratory project I propose to try to make my exploratory project more consistent with the literature by using the following attribute categories.

I:   Component Attributes

The components are independent services that are loosely coupled via the message framework.  The components can be distributed among a number of different separately built and executable applications / Windows processes.  As such, a component can be moved from one application to another.  Within a particular application a component installs itself into the message controller framework.  To do so, it supplies a number of attributes to the framework.  Each application must be built upon the message controller framework.

Components of an application that cannot have all of their topic subscriptions satisfied by other components of the application will have their subscriptions dispatched to the other applications of the distributed network to find a component that can satisfy the subscription.  These pairings will be maintained by the distributed framework such that instances of a topic will be routed from one application to another and then forwarded to the remote component.

1)  Component Name

The name of the component.  This attribute is used to describe the component and to avoid the registration of duplicate components – at least, those with the same name – and for lookup in the framework tables.

2)  Activation

The Activation criteria used to activate/schedule the component.  The exploratory project options are

·        Upon Demand – the component is only run when an event is sent to it via the publishing of an Event Driven message.

·        Periodic – the component is run by the use of the operating system to begin at intervals specified at the time the component registered itself.

·        Aperiodic – the component is run via a forever loop of the framework with a following delay prior to return to the top of the loop to run the component again.  So the interval between invocations is the time the component took to finish (including when suspended due to a lower priority) and the delay specified when the component registered itself.

3)  Component Category

The Component Category attribute declares whether the component is part of the framework or not, as well, as whether it is a time-critical framework component. The options are

·        Time-Critical

·        Framework

·        User

where User is the default.

4)  Time Interval

The Time Interval attribute declares the time interval between activations of the component if it is Periodic or the delay time if it is Aperiodic.  Otherwise, for Upon Demand, it is meaningless.  The units of the time interval depend upon the operating system being used by the framework.

5)  Time Capacity

The Time Capacity attribute declares the expected execution time of the component.  The units of the time capacity are the same as for the time interval and depend upon the operating system being used by the framework.

6)  Compatibility

The Compatibility attribute is used to determine the compatibility of the component to the framework.  As yet it is unused in the exploratory project.

7)  Stack Space

The amount of stack space to be allocated for the component – if non-standard.

8)  Initialize Entry Point

This attribute is the address of / pointer to the callback to be used to allow the component to initialize itself.  Null if no initialize is necessary beyond that done when the component installs / registers itself with the framework.

9)  Main Entry Point

The Main Entry Point attribute is the address of / pointer to the callback to be used to invoke the component each time it is executed / activated.  If the component is entirely executed upon demand via event driven topics where the Topic Subscription for each topic supplies the callback to be used to notify the component of the event, then this callback should be null since no Main entry point is required.

10) Return Attributes

The Install / Register of a component with the framework returns two attributes.

a)      Whether or not the install was successful.  The possible values are

·        Valid – The component was installed,

·        Duplicate – The component was previously registered,

·        Invalid – One or more of the supplied attributes were not valid,

·        Inconsistent – The supplied attributes were inconsistent, or

·        Incomplete – Insufficient attributes were supplied

The component was not registered if the return value is anything except Valid.

b)      The private key assigned to the component by the framework to identify the component if the install was successful.  Such as to identify the component when it subscribes to a topic.  The format of the key is not visible to the component.

II:  Message Topic Attributes

Message topics are described in a topic contract Ada package / C# class.  Such a class constitutes a contract between the producer component(s) and the consumer component(s).  It includes a number of attributes, some of which will be defaulted by the framework if not supplied.  It must also provide the format of the topic message and a validation method.  Some topic contracts are paired with both a request topic and a response topic each of which has its own attributes and message format.  Each message format has a data format and a hidden header.  The hidden header has data that is only used by the framework.  The validation method can check various data items for correctness or, at a minimum, only specify whether there is a valid pointer to a data buffer.

1)  Topic identifier and name to be used to identify the topic of a message.  The topic name is used to describe the topic and to avoid the registration of duplicate topics – at least, those with the same name – and for lookup in the framework tables.  Similarly for the topic identifier in regards to duplication and lookup as well as to insert into the private message headers of the topics that are distributed between applications of the network.

2)  Topic Distribution

This category distinguishes between general topics and those that are only to have participants that are components that are part of the framework.  The options are

·        General

·        Framework Only

where General is the default.

3)  Delivery Protocol

The delivery protocol or pattern has the options

·        Point-to-Point (PPP)

·        Publish/Subscribe

For Publish/Subscribe all subscribing consumer components can read any particular instance of the topic message.  For Point-to-Point, even if there are multiple consumers registered for a topic, a particular instance of the topic is only delivered to one of the consumers (for instance, for a Delivery Identifier message exchange topic).

4)  Topic Permanence

This attribute deals with the permanence / persistence of an instance of the topic.  The options are

·        MRP for Most Recently Published – When a component runs, it can only read the most recently published instance of the topic.  The use of this attribute means that the framework can allocate message buffers based upon the number of consumers such that a consumer can examine the most recently published instance of the topic without the need for a copy of the message to be made for it to be thread-safe.  Of course this is only if the consumers are in the same application – otherwise, a copy is made to transmit the instance to another application for distribution to its consumers.

·        Queued – A copy of the topic instance is queued to be read by each consumer.  When the component runs it can read the FIFO queue to obtain the oldest entry in the queue and can continue reading until the queue becomes empty or wait until it runs again.  The size of the queue is specified for a particular consumer when it registers for the topic.

The framework may default this attribute based upon other attributes if it is able.

5)  Message Exchange

The Message Exchange attribute specifies whether or not the topic expects an exchange between a requesting component and a responding component.  The options are

·        No Exchange

·        Delivery Identifier

·        Request-Response

·        Content Filtered

No Exchange is the default where the topic contract only specifies the format of a topic to be published by one or more components and consumed by other components. 

The Delivery Identifier is a topic contract that specifies the format of a topic to be published by one component and consumed by the particular component, if any, that subscribed for the delivery identifier specified by the topic writer.  This is content filtering via the framework since the instance of the topic is only delivered to the component that subscribed for both the topic and the particular delivery id.  If no component has subscribed for the delivery identifier of the topic instance, it is discarded.

Request-Response means the topic has a paired contract consisting of a Request contract and a Response contract and that one of N different components can publish the request version of the topic and the consuming component will synchronously respond to the request with the framework only delivering the response to the requesting component.  Therefore, the topic must have Permanence of Queued and Delivery Protocol of Point-to-Point.  If the requestor did not subscribe for the response, the framework will discard the response.  Although the component that consumes the request will synchronously respond, the requestor (if it subscribed) may have to wait for the response if it does not have Upon Demand activation.  However, if the requestor subscribes for a response, it will need to have Topic Permanence of Queued.  In many cases, the Request will be used to send a command and the Response will be used to indicate whether the command was successfully performed.

Content Filtered is similar to Request-Response except that the response is not synchronous and it is implied that the request consumer will filter the data that it treats and only publish a response for the requestor when the filter conditions are satisfied.  Since the response is asynchronous, the request consumer does not need to have Upon Demand activation.  The request will specify the filter conditions that the component needs to have satisfied before a response is sent.  Such conditions can be a trigger to start sending the response when the conditions are satisfied, to only send once when the conditions are first satisfied, to send as long as the conditions are satisfied, etc.  Different requesting components can supply different values for the request filter conditions with the request consumer publishing a response for that requestor when its conditions are satisfied.  Therefore, the request topic must have the requesting component key as part of the contracted data.  Also, the request consumer / response producer component must maintain a table of the requests along with the requesting component’s private key and supply the key when it instantiates a Writer for the response.

If other versions of Content Filtered become desirable, the above described version can become Consumer Filtered since it is the consumer component that is doing the filtering.

6)  Buffer Storage Allocation

The topic buffer storage allocation attribute specifies how data buffers for instances of the topic are to be allocated.  The options are

·        Fixed

·        Reusable

Fixed allocations of the buffers are made at subscription time whereas Reusable allocations come from a buffer pool and the buffer can be reused for writing another topic when all components to read an instance of a topic have instantiated a reader and then finished running and returned to the framework.

III: Subscription Attributes

Topic subscriptions are usually (always until now) registered by a component just after it has installed itself.  The topic subscription supplies a number of attributes mating the component and the topic in the specified way.  Some of the attributes are defaulted if not supplied.

1)  Component Key

The private key that was assigned to the component by the framework to identify it at the time it successfully installed itself with the framework.  The format of the key is not visible to the component but is used by the framework to identify the subscribing component.

2)  Topic Participant Role

The role of the component in its subscription for the topic.  The options are

·        Producer – the component is registering to write and publish; that is, produce, the topic.

·        Consumer – the component is registering to read; that is, consume, the topic.

3)  New instance Detection

How the component detects a new instance of a topic.  The options are

·        Event Driven – the component is run via a notify event.  Therefore, the component must be an On Demand component.

·        Polling – the component attempts to read the topic each time it is run.  The component can be either On Demand such that it reads the particular topic when initiated for another reason or cyclic (that is, Periodic or Aperiodic).

4)  Permanence Queue Size

The size of the queue to retain instances of a topic for a particular consumer if the Topic Permanence is Queued rather than Most Recently Published.  This is a Quality of Service (QoS) value.  If not supplied, a default value is used.  When the queue becomes full, the oldest entry will be discarded and its buffer released for reuse.

5)  Subscription Duration

Whether the instances of the topic are to be durable for the consumer.  That is, whether the consumer can detect / read an instance of the topic that was published while its application had lost power or became disconnected from the network.  The use of this attribute would mean writing the instances to permanent storage such as to disk or to flash memory so they could be delivered later.

6)  Return Attributes

The subscription to the framework of a component for a topic returns one attribute as to whether the subscription was successful.  The possible values are

·        Success – the subscription succeeded,

·        Duplicate – the subscription failed due to being the duplicate of a previous subscription, or

·        Failure – the subscription failed for other reasons.

IV:  Topic Reader or Writer Attributes

An instantiated construct is returned to allow access to the current buffer for the topic each time the component requests to read or write the topic.  The construct allows access to the validation method of the topic (a callback) and supplies a pointer to the topic data in a particular message buffer. 

1)  For a Reader, the validation method can be invoked to determine if the topic data successfully passes the common verify method provided by the topic contract and whether the pointer is valid.  If so, then the component can examine and/or copy the buffered data according to the topic format supplied by the contract.  The component only needs to copy that data that the component needs to be persistent from one activation to the next.  The validation will fail if there is no new instance of the topic to be read so the component can skip the topic for the particular activation.

2)  For a Writer, a determination must be made as to whether the buffer pointer is valid using the Valid function of the Writer.  That is, if the framework was able to supply a buffer to be written to.  If so, the component can copy data to the buffer via the supplied pointer according to the topic format supplied by the contract.  When complete, the component can Publish the new instance of the topic.  The Publish has no attributes except for being the callback for the particular Publish method of the Writer.  The published topic data is not validated when published.  It is up to the Reader to validate that all necessary data was supplied and is consistent.

Content Filtered Topic Added to Exploratory Project

Content Filtered Topic Added to Exploratory Project


As mentioned that I would on September 1, I have added the Content Filtered Topic feature mentioned in The AIDA System, A 3rd Generation IMA Platform, DIANA Project White Paper to my exploratory project.

With a pat on the back for myself in regards to my design and implementation, adding this feature took from sometime in the afternoon of September 1 to begin thinking about what I would need to do to having a running application yesterday afternoon while averaging maybe 5 hours a day at doing so. Not too bad if I do say so myself.

In this implementation, I also added an Aperiodic activation method for a component to eventually replace the pseudo periodic method. The difference is that in the Aperiodic method the component is run via the Message Controller framework so that the forever loop is part of the framework as well as the delay to continue the loop and again invoke the component after the interval specified when the component registered itself with the framework. This removes the need for the component to know the means that are necessary to provide the delay or that there is a forever loop involved.

In regards to the new Content Filtered delivery style, having the forever loop in the framework also allows the framework the ability to free any message buffers for topics read during the just completed execution of an Aperiodic component and remove them from a delivery queue for the component.

I figured that components that would be requested to filter the content that they produced before publishing what met the criteria of the requesting component would likely need to run at a particular interval rather than be event driven when notified of the publication of a topic.

Before adding the Content Filtered feature, a requesting component could only be assured that its consumer received a particular instance of a topic was if the delivery was point-to-point and the consumer was event driven such as with the N-to-One delivery method. A component that ran at specified intervals could only read data flow delivered topics where the component received the most recent instance of the topic. If multiple producers, the latest instance of one producer would replace that of an earlier producer as the one that the consumer would read. Therefore, the filter conditions of one requestor could be lost if another requestor published a request after that of the first and before the consumer had a chance to run.

Therefore, I added a delivery queue to the framework for each registered Aperiodic component that would allow the Content Filtered requests of each requesting component to be queued awaiting the execution of the request consumer; that is, the producer of the requested content.

In my implementation, if a requestor publishes multiple requests during the interval from one execution of the Aperiodic component and the next, the second is to replace the first in the queue. However, the requestor will most likely fail to obtain a write buffer for such a second request and so won’t be able to publish it until the first has been read.

The requesting and producing components must have a “contract” with each other as to what the filter conditions can be with the producing component keeping track of the various conditions and only publishing the response topic for a particular requestor when the conditions that it specified have been met. Therefore, it must maintain a table that identifies the requesting component by its private key, the topic that is to be published, and the conditions that have to be met to publish it. The request topic (Ada package or C# class) identifies the conditions while the response portion of the topic identifies the content to be included in the response. The request must identify the requesting component, the scope of the request (cancel, every time the conditions are met, only the first time, etc), and the condition pairs where each pair has the equality condition (Equal, Less Than, Less Than or Equal, etc) and the value to satisfy the condition.

Each time that the Aperiodic component reads the topic, the framework marks the oldest request in the queue as read and then provides the pointer to the topic buffer. The component can then save the request conditions. Before finding any oldest request in the queue, the framework checks for any previously read requests and releases the topic buffer of such a request and removes the request from the queue. Thus, if the Aperiodic component reads the topic until there is no instance of the topic to be returned, the queue will be emptied and each requestor’s buffer will be released for it to use again.

Both the reading and writing to the queue are, of course, done in a thread safe manner. In case the Aperiodic component doesn’t empty the queue, the framework will also check for previously read requests and release the topic buffer and remove the request from the queue when the component has finished running and control has returned to the forever loop of the framework.

It is assumed that the requestor wants its own version of the filtered topic. That is, the filtered content that it requested. Therefore, the response is delivered point-to-point to the requestor in the same manner as N-to-One. Only it is delivered each time that the content satisfies the specified conditions rather than immediately following the read of the request topic.

The contract between the requestor and the producer could, of course, be different. That is, there could be one requestor that specified the conditions that would be used by the producer as the criteria for publishing a corresponding data flow topic that could be read by any subscribing component. This could be indicated by the use of a request topic that didn’t supply the identity of the requestor. Or, a Content Filtered topic could be used as a means of providing data to an Aperiodic component if a more controlled delivery than data flow was needed.