Service-Oriented Architecture Attributes Progress Report 1

Service-Oriented Architecture Attributes Progress Report 1

The modification of my Exploratory Project to incorporate the component and topic attributes of my blog post of a week ago is complete.  That is, complete enough to execute the same set of test components and topics as previously. 

The Display Application can send events to the Display Interface component of User Application 1.  It then extracts the event data from the message and publishes a Delivery Identifier message exchange topic where the identifier of the button, checkbox, edit box, etc of the widget of the event is used as the delivery identifier.  The instance of the Widget Event topic is then routed by the framework to the application component that registered/subscribed to treat the widget event.  Three different components in two different applications so registered to treat a set of widget events.

These components then create a request-response topic as the command to be sent back to the display application.  This is routed to the Display Interface component as the component that registered to consume the request and it forwards it as a Display Command message to the Display Application which modifies what is displayed.  In addition the Display Interface component creates and publishes the response to the request.  Two of three Widget Event handler components didn’t subscribe to the response so the framework discards these responses while routing those for the third component back to it.

Another Content Filtered topic response is created and published by a component and routed by the framework to the particular component that requested it.

Certain previous attribute classifications were eliminated while other classifications were added.  This caused the need to restructure some of the framework packages / classes for consistency with the new attribute classifications.  Also, the modification of some of the logic was necessary when a combination of the new attribute options was necessary to specify a characteristic of a previous classification that had a greater range of more specific options.  For instance, N-to-One had to become a combination of the Request-Response, Queued, and Point-to-Point attribute options whereas N-to-One had previously implied all three.

Since there are now more combinations of options available, such as the ability of a periodic component to consume a point-to-point topic via polling of queued instances of the topic, further updates to the framework are going to be necessary for such combinations.  Thus further test components and topics will be necessary to debug these modifications as they are made.

A future progress post will report this.

Service-Oriented Architecture Attributes

 Service-Oriented Architecture Attributes

After reading Messaging Patterns in Service-Oriented Architecture, Part 1 by Soumen Chatterjee {http://msdn.microsoft.com/en-us/library/aa480027.aspx} and various online patterns of Enterprise Integration Patterns by Gregor Hohpe and Bobby Woolf {http://www.eaipatterns.com or http://www.eaipatterns.com/eaipatterns.html} I have decided to try to make my terminology better conform to that of the Service-Oriented Architecture (SOA) publications. 

From my recent reading of this material it seems to me that the following classifications make sense.

First, to separate the SOA attributes into the following four categories.

I:   Those that are attributes of a component,

II:  Those that are attributes of a message topic,

III: Those that are attributes of a subscription by a component to a topic, and

IV: Those that are attributes of topic readers and writers.

For instance, an attribute of a component is how it is run or scheduled.  Whether it is run upon demand such as by an event or whether it runs periodically.  An attribute of periodically scheduled as can be further described as strictly periodic as scheduled by the operating system or aperiodic such as scheduled in a forever loop with a delay.

An example of a message topic attribute could be publish-subscribe versus point-to-point.

An example of a subscription attribute could be whether the subscribing component does so as the producer or the consumer of the topic.

Therefore, based upon my reading and the attributes that I have used in my exploratory project I propose to try to make my exploratory project more consistent with the literature by using the following attribute categories.

I:   Component Attributes

The components are independent services that are loosely coupled via the message framework.  The components can be distributed among a number of different separately built and executable applications / Windows processes.  As such, a component can be moved from one application to another.  Within a particular application a component installs itself into the message controller framework.  To do so, it supplies a number of attributes to the framework.  Each application must be built upon the message controller framework.

Components of an application that cannot have all of their topic subscriptions satisfied by other components of the application will have their subscriptions dispatched to the other applications of the distributed network to find a component that can satisfy the subscription.  These pairings will be maintained by the distributed framework such that instances of a topic will be routed from one application to another and then forwarded to the remote component.

1)  Component Name

The name of the component.  This attribute is used to describe the component and to avoid the registration of duplicate components – at least, those with the same name – and for lookup in the framework tables.

2)  Activation

The Activation criteria used to activate/schedule the component.  The exploratory project options are

·        Upon Demand – the component is only run when an event is sent to it via the publishing of an Event Driven message.

·        Periodic – the component is run by the use of the operating system to begin at intervals specified at the time the component registered itself.

·        Aperiodic – the component is run via a forever loop of the framework with a following delay prior to return to the top of the loop to run the component again.  So the interval between invocations is the time the component took to finish (including when suspended due to a lower priority) and the delay specified when the component registered itself.

3)  Component Category

The Component Category attribute declares whether the component is part of the framework or not, as well, as whether it is a time-critical framework component. The options are

·        Time-Critical

·        Framework

·        User

where User is the default.

4)  Time Interval

The Time Interval attribute declares the time interval between activations of the component if it is Periodic or the delay time if it is Aperiodic.  Otherwise, for Upon Demand, it is meaningless.  The units of the time interval depend upon the operating system being used by the framework.

5)  Time Capacity

The Time Capacity attribute declares the expected execution time of the component.  The units of the time capacity are the same as for the time interval and depend upon the operating system being used by the framework.

6)  Compatibility

The Compatibility attribute is used to determine the compatibility of the component to the framework.  As yet it is unused in the exploratory project.

7)  Stack Space

The amount of stack space to be allocated for the component – if non-standard.

8)  Initialize Entry Point

This attribute is the address of / pointer to the callback to be used to allow the component to initialize itself.  Null if no initialize is necessary beyond that done when the component installs / registers itself with the framework.

9)  Main Entry Point

The Main Entry Point attribute is the address of / pointer to the callback to be used to invoke the component each time it is executed / activated.  If the component is entirely executed upon demand via event driven topics where the Topic Subscription for each topic supplies the callback to be used to notify the component of the event, then this callback should be null since no Main entry point is required.

10) Return Attributes

The Install / Register of a component with the framework returns two attributes.

a)      Whether or not the install was successful.  The possible values are

·        Valid – The component was installed,

·        Duplicate – The component was previously registered,

·        Invalid – One or more of the supplied attributes were not valid,

·        Inconsistent – The supplied attributes were inconsistent, or

·        Incomplete – Insufficient attributes were supplied

The component was not registered if the return value is anything except Valid.

b)      The private key assigned to the component by the framework to identify the component if the install was successful.  Such as to identify the component when it subscribes to a topic.  The format of the key is not visible to the component.

II:  Message Topic Attributes

Message topics are described in a topic contract Ada package / C# class.  Such a class constitutes a contract between the producer component(s) and the consumer component(s).  It includes a number of attributes, some of which will be defaulted by the framework if not supplied.  It must also provide the format of the topic message and a validation method.  Some topic contracts are paired with both a request topic and a response topic each of which has its own attributes and message format.  Each message format has a data format and a hidden header.  The hidden header has data that is only used by the framework.  The validation method can check various data items for correctness or, at a minimum, only specify whether there is a valid pointer to a data buffer.

1)  Topic identifier and name to be used to identify the topic of a message.  The topic name is used to describe the topic and to avoid the registration of duplicate topics – at least, those with the same name – and for lookup in the framework tables.  Similarly for the topic identifier in regards to duplication and lookup as well as to insert into the private message headers of the topics that are distributed between applications of the network.

2)  Topic Distribution

This category distinguishes between general topics and those that are only to have participants that are components that are part of the framework.  The options are

·        General

·        Framework Only

where General is the default.

3)  Delivery Protocol

The delivery protocol or pattern has the options

·        Point-to-Point (PPP)

·        Publish/Subscribe

For Publish/Subscribe all subscribing consumer components can read any particular instance of the topic message.  For Point-to-Point, even if there are multiple consumers registered for a topic, a particular instance of the topic is only delivered to one of the consumers (for instance, for a Delivery Identifier message exchange topic).

4)  Topic Permanence

This attribute deals with the permanence / persistence of an instance of the topic.  The options are

·        MRP for Most Recently Published – When a component runs, it can only read the most recently published instance of the topic.  The use of this attribute means that the framework can allocate message buffers based upon the number of consumers such that a consumer can examine the most recently published instance of the topic without the need for a copy of the message to be made for it to be thread-safe.  Of course this is only if the consumers are in the same application – otherwise, a copy is made to transmit the instance to another application for distribution to its consumers.

·        Queued – A copy of the topic instance is queued to be read by each consumer.  When the component runs it can read the FIFO queue to obtain the oldest entry in the queue and can continue reading until the queue becomes empty or wait until it runs again.  The size of the queue is specified for a particular consumer when it registers for the topic.

The framework may default this attribute based upon other attributes if it is able.

5)  Message Exchange

The Message Exchange attribute specifies whether or not the topic expects an exchange between a requesting component and a responding component.  The options are

·        No Exchange

·        Delivery Identifier

·        Request-Response

·        Content Filtered

No Exchange is the default where the topic contract only specifies the format of a topic to be published by one or more components and consumed by other components. 

The Delivery Identifier is a topic contract that specifies the format of a topic to be published by one component and consumed by the particular component, if any, that subscribed for the delivery identifier specified by the topic writer.  This is content filtering via the framework since the instance of the topic is only delivered to the component that subscribed for both the topic and the particular delivery id.  If no component has subscribed for the delivery identifier of the topic instance, it is discarded.

Request-Response means the topic has a paired contract consisting of a Request contract and a Response contract and that one of N different components can publish the request version of the topic and the consuming component will synchronously respond to the request with the framework only delivering the response to the requesting component.  Therefore, the topic must have Permanence of Queued and Delivery Protocol of Point-to-Point.  If the requestor did not subscribe for the response, the framework will discard the response.  Although the component that consumes the request will synchronously respond, the requestor (if it subscribed) may have to wait for the response if it does not have Upon Demand activation.  However, if the requestor subscribes for a response, it will need to have Topic Permanence of Queued.  In many cases, the Request will be used to send a command and the Response will be used to indicate whether the command was successfully performed.

Content Filtered is similar to Request-Response except that the response is not synchronous and it is implied that the request consumer will filter the data that it treats and only publish a response for the requestor when the filter conditions are satisfied.  Since the response is asynchronous, the request consumer does not need to have Upon Demand activation.  The request will specify the filter conditions that the component needs to have satisfied before a response is sent.  Such conditions can be a trigger to start sending the response when the conditions are satisfied, to only send once when the conditions are first satisfied, to send as long as the conditions are satisfied, etc.  Different requesting components can supply different values for the request filter conditions with the request consumer publishing a response for that requestor when its conditions are satisfied.  Therefore, the request topic must have the requesting component key as part of the contracted data.  Also, the request consumer / response producer component must maintain a table of the requests along with the requesting component’s private key and supply the key when it instantiates a Writer for the response.

If other versions of Content Filtered become desirable, the above described version can become Consumer Filtered since it is the consumer component that is doing the filtering.

6)  Buffer Storage Allocation

The topic buffer storage allocation attribute specifies how data buffers for instances of the topic are to be allocated.  The options are

·        Fixed

·        Reusable

Fixed allocations of the buffers are made at subscription time whereas Reusable allocations come from a buffer pool and the buffer can be reused for writing another topic when all components to read an instance of a topic have instantiated a reader and then finished running and returned to the framework.

III: Subscription Attributes

Topic subscriptions are usually (always until now) registered by a component just after it has installed itself.  The topic subscription supplies a number of attributes mating the component and the topic in the specified way.  Some of the attributes are defaulted if not supplied.

1)  Component Key

The private key that was assigned to the component by the framework to identify it at the time it successfully installed itself with the framework.  The format of the key is not visible to the component but is used by the framework to identify the subscribing component.

2)  Topic Participant Role

The role of the component in its subscription for the topic.  The options are

·        Producer – the component is registering to write and publish; that is, produce, the topic.

·        Consumer – the component is registering to read; that is, consume, the topic.

3)  New instance Detection

How the component detects a new instance of a topic.  The options are

·        Event Driven – the component is run via a notify event.  Therefore, the component must be an On Demand component.

·        Polling – the component attempts to read the topic each time it is run.  The component can be either On Demand such that it reads the particular topic when initiated for another reason or cyclic (that is, Periodic or Aperiodic).

4)  Permanence Queue Size

The size of the queue to retain instances of a topic for a particular consumer if the Topic Permanence is Queued rather than Most Recently Published.  This is a Quality of Service (QoS) value.  If not supplied, a default value is used.  When the queue becomes full, the oldest entry will be discarded and its buffer released for reuse.

5)  Subscription Duration

Whether the instances of the topic are to be durable for the consumer.  That is, whether the consumer can detect / read an instance of the topic that was published while its application had lost power or became disconnected from the network.  The use of this attribute would mean writing the instances to permanent storage such as to disk or to flash memory so they could be delivered later.

6)  Return Attributes

The subscription to the framework of a component for a topic returns one attribute as to whether the subscription was successful.  The possible values are

·        Success – the subscription succeeded,

·        Duplicate – the subscription failed due to being the duplicate of a previous subscription, or

·        Failure – the subscription failed for other reasons.

IV:  Topic Reader or Writer Attributes

An instantiated construct is returned to allow access to the current buffer for the topic each time the component requests to read or write the topic.  The construct allows access to the validation method of the topic (a callback) and supplies a pointer to the topic data in a particular message buffer. 

1)  For a Reader, the validation method can be invoked to determine if the topic data successfully passes the common verify method provided by the topic contract and whether the pointer is valid.  If so, then the component can examine and/or copy the buffered data according to the topic format supplied by the contract.  The component only needs to copy that data that the component needs to be persistent from one activation to the next.  The validation will fail if there is no new instance of the topic to be read so the component can skip the topic for the particular activation.

2)  For a Writer, a determination must be made as to whether the buffer pointer is valid using the Valid function of the Writer.  That is, if the framework was able to supply a buffer to be written to.  If so, the component can copy data to the buffer via the supplied pointer according to the topic format supplied by the contract.  When complete, the component can Publish the new instance of the topic.  The Publish has no attributes except for being the callback for the particular Publish method of the Writer.  The published topic data is not validated when published.  It is up to the Reader to validate that all necessary data was supplied and is consistent.

Durable Subscription in an Older Aircraft System

Durable Subscription in an Older Aircraft System


An older message framework that I designed seems like it must have anticipated Durable Subscription. That is, each instance of a message was delivered. The messages were delivered to components of the application in the local processor or to components of the application of a remote processor of a dual processor system.

However, in that system there was a controlling application and a service application. Only the controlling application could initiate request messages when the supplying component was in the remote processor. However, it could initiate a delayed or filtered content request; that is, it could send a request for a periodic response or one to be triggered when a given value was matched. Then the service application could send the response message back to the controlling application where the framework would distribute it to the requesting component or any others that had registered for it.

The delivery of messages to other components within an application was the same in both applications. Rather than a queue for each component, there was one framework request queue. The top message of the FIFO queue was popped from the queue and then delivered to each requesting component of the application in turn in a priority order where the priority was configurable for each message topic. After a component had run, the message was delivered to the next component that had subscribed for it.

A message topic subscribed by a remote application component could be either synchronous or asynchronous. That is, the framework of the originating controlling application would either wait for the response from the remote component before proceeding or continue without waiting. For a filtered content request topic from the controlling application component to a component of the remote application, the request had, of course, to be asynchronous.

Some Commentary on the “TALIS Airborne Architecture Recommendation Document”

Some Commentary on the “TALIS Airborne Architecture Recommendation Document”


From the “TALIS Airborne Architecture Recommendation Document” TALIS-WP5-AAR-0016, http://talis.eurocontrol.fr/pub/d5.2%20TALIS-WP5-AAR-0016-v1-0.pdf page 17


3.2.1.2 Openwings Architecture

The key value of the Openwings architecture is the ability to create systems composed of completely reusable components. Openwings is an instantiation of the Service Oriented Programming (SOP) paradigm.

Openwings provides a framework for SOP that is independent of discovery mechanisms, transport mechanisms, platforms, and deployment environments. The following figure shows a high level view of the Openwings architecture.







Figure8: Openwings architecture

Openwings is a network-centric architecture and hence the diagram rests on a network. The Openwings frameworks depends on Security, Management, Context, and Install Services. The Security Service provides code, transport, and service security. The Management Service allows management of any component in the framework. The Context Service provides a deployment environment and system boundary. The Install Service provides hands-off installation and propagation of static contextual information. The Container Service handles life cycle management of components and provide a clustering capability. The Connector Service provides protocol independence for asynchronous and synchronous protocols. The Component Service provides a simple programming abstraction for SOP and discovery independence. The unit of interoperability in Openwings is a Java Interface. Well-defined interfaces and code mobility provide the core enablers of Openwings interoperability.

On top of the Openwings framework components and domain models can be built. Openwings provides some common facilities, such as data services. The Data Service is built on the Openwings core and provides an object-based abstraction for persistence mechanisms that is independent of databases.

Each element in the Openwings Core can be used independently, with little or no interdependencies. This provides the greatest flexibility to Openwings users. In cases, where one service uses another, it is designed to integrate the service, but still run without it. For instance the container services can communicate with the install services to find new components.


Also from TALIS-WP5-AAR-0016:


5.2.2.3 Messaging JMS Domains

Before the JMS API existed, most messaging products supported either the point-to-point or the publish/subscribe approach to messaging. The JMS Specification provides a separate domain for each approach and defines compliance for each domain. A standalone JMS provider may implement one or both domains. A J2EE provider must implement both domains.

In fact, most curreeliability of queues but still allow clients to send messages to many recipients. The following figure illustrates pub/sub messaging.

                                                       Figure14: Publish-Subscribe model



My commentary on the above in regards to my exploratory project:

It would seem that my exploratory project could be described as peer-to-peer; each application is the equivalent – that is the peer – of the other. Data Flow and ordinary Event Driven notify messages are not really retained until consumed – rather the most recent instance of the message is the one delivered to the Reader. But Content Filtered, N-to-One, and One-of-N are point to point and, at least Content Filtered and N-to-One, are retained until consumed.

Perhaps I should add the durable subscriptions feature so that exploratory project could have multiple subscribers that could receive each instance of a published topic. This would be somewhat like Data Flow and ordinary Event Driven notify topics but the topic instances would be queued to each subscriber/consumer such that they could receive each instance rather then just the most recent. From Figure 14, this is N-to-M. That is, multiple producers (N) and multiple consumers (M) with the consumer able to request a durable subscription.

Of course, I don’t have multiple producers in any case but to request that the consumer provide a response (N-to-One and Content Filtered). Would allowing multiple producers of a topic be desirable? That is, other than as a backup producer where the topic would be ignored/tossed unless the primary producer ceased to function. In a general system, all producers would be equal but in an aeronautics system it doesn’t seem likely except perhaps for a monitor component. That is, perhaps one or more monitor components to receive a topic from multiple suppliers to vote on the supplier to trust and cause a switch to another supplier if the designated supplier became failed. Then, the Message Controller framework could allow multiple producers of many topics with the framework discarding the redundant topic instances from all but the currently designated component. Or the consumer components could discard topics from other than the designated producer – a kind of filtering. If the later, the components need to be able to identify the source of the message – at least to distinguish one from another or to be able to match with the current accepted producer.

Putting in the monitor / accepted selector capability would be major addition to the exploratory project. Even just allowing multiple producers with the consumers not knowing from which producer a particular instance originated is a major step.

That is, the framework now has the mechanism, for Data Flow and Event Driven topics, to have M+1 copies of a topic as produced by the one allowed publishing component with any particular consumer m (of a particular application) reading whatever is the most recently published instance of the topic at the time the consumer component starts running. The M+1 copies being the thread safe mechanism so that one buffer is available to write to while each of the M consumers could be at some stage of reading the copy that was the most recently published when the consumer began its read.

For N-to-One, there are N buffers for the N producers and the one consumer receives the instance that is at the top of the FIFO queue, treats it, returns the reply and then gets the next instance from the queue.

In either case, the reader doesn’t need to move, in a thread safe manner, the topic instance from the buffer to its own memory since the buffer can’t be modified while the consumer is examining it.

For N-to-M this would no longer the case. That is, each of the N producers could be writing to its buffer while each of the M consumers could be reading any particular buffer – the one placed in its queue first. Therefore, the read buffers have to be separate from the write buffers with the data of a particular producer copied to a new read buffer as it is queued to the particular consumer. Then, the producer can produce a new instance while various consumers have yet to process the previously queued instance with some ageing mechanism to recycle buffers of queues where the consumer is too slow to keep up.

That is, it would seem, KQofS buffers are needed for each consumer depending on how old the topic can get and still be of use to the consumer and a copy of the latest instance as published by each of the N producers. KQofS could be N for each of the M consumers but still not be enough to hold one copy from each of the producers since some could publish far more often than others. Or, KQofS could be, for instance, 1 or 2 to just retain the one or two most recently published instances of the topic no matter what the source. Each consumer could specify how many instances it wants queued.

There would still need to be the extra copy of the topic data to free up the write buffer for the producer to publish again without regard to whether the previous instance had been read by each consumer. Even if had N times M+1 buffers, any receiving component could only read the most recently published instance from each of the N producers without the copy. That is, an instance would not persist until read.

Of course, in the current implementation for persistent topics a producer cannot produce again until the consumer has finished running after reading the instance of the topic – at which time the buffer is released. But it would seem that a new delivery method should lift this restriction.

Note: These comments only apply to components that are within the same application. When the topic has to be consumed by a remote component(s), there is a proxy component that transmits it over the network. Therefore, there is a copy made for transmitting and there could be multiple remote consumers of this one proxy producer.

What does this say about the current design? That is, if every component were treated as a remote component, then part of the buffering problem would be taken care of. But there would be two extra copies to be made; one to transmit the message and one from the receive buffer to publish the topic by the proxy producer in the receiving application. This would defeat one of the original goals: that of keeping the number of copies to a minimum and the CPU cycles used while copying.

Yet, a durable subscription feature would seem to be something to look into supplying as an option for components to use. And, if allow multiple producers or even if don’t, allow the consumer to specify the number of copies that can be queued.

This then becomes another version of the Content Filtered feature that I just added. Except, rather than the request and response being paired in the topic, a durable subscription topic could be produced at any time there was a buffer available and it would be queued for each consumer that had registered to read the topic.

If the consumer’s queue has become full, it would replace the oldest entry not being read (since the consumer can supply its KQofS). A queue element marked as being read has to be retained since the component could be actively examining it. The KQofS queue length could be supplied when subscribing to the topic rather than when registering the component so that there would be a separate queue for each durable subscription topic subscripted to by the consumer component.

The consumer could be event driven or any form of periodic since there would be a queue in either case and when the consumer ran it could empty the queue or just process the top of queue entry according to the needs of the component. Therefore, each of the Scheduler methods of running a component would need to be able to check if the component had a Durable Subscription queue to release any buffers that had been read and pop their topic instances from the queue.

The consumer could likewise publish one or more Durable Subscription topics as a result of the content that it produced.

Therefore, this would be more general than the version of Content Filtered that I just implemented. This would be so since a particular component could publish a Durable Subscription topic that really contained the content filtered conditions and the consumer could then publish another Durable Subscription topic that contained the filtered content. If multiple consumers subscribed to this later topic, each would receive the filtered content. And each would receive each instance if their KQofS queue length for the topic was sufficient.

The Content Filtered feature could then be modified to be implemented in the same manner except that each requestor would specify its own filter conditions. But the delivery to the requestor would be via its Durable Subscription queue. If the requestor was event driven, it would receive a Notify event for the response. Otherwise, the topics would still be in its queue when it ran periodically. So the Delivery method would be Durable Subscription but the topic would be Content Filtered with its Request and Response subtopics.

Therefore, Durable Subscription as mentioned in TALIS-WP5-AAR-0016 “TALIS Airborne Architecture Recommendation Document” seems like a good feature to add to my exploratory project. Plus a modification of the Content Filtered feature mentioned in “The AIDA System, A 3rd Generation IMA Platform, DIANA Project White Paper” to use Durable Subscription as the delivery method.

Content Filtered Topic Added to Exploratory Project

Content Filtered Topic Added to Exploratory Project


As mentioned that I would on September 1, I have added the Content Filtered Topic feature mentioned in The AIDA System, A 3rd Generation IMA Platform, DIANA Project White Paper to my exploratory project.

With a pat on the back for myself in regards to my design and implementation, adding this feature took from sometime in the afternoon of September 1 to begin thinking about what I would need to do to having a running application yesterday afternoon while averaging maybe 5 hours a day at doing so. Not too bad if I do say so myself.

In this implementation, I also added an Aperiodic activation method for a component to eventually replace the pseudo periodic method. The difference is that in the Aperiodic method the component is run via the Message Controller framework so that the forever loop is part of the framework as well as the delay to continue the loop and again invoke the component after the interval specified when the component registered itself with the framework. This removes the need for the component to know the means that are necessary to provide the delay or that there is a forever loop involved.

In regards to the new Content Filtered delivery style, having the forever loop in the framework also allows the framework the ability to free any message buffers for topics read during the just completed execution of an Aperiodic component and remove them from a delivery queue for the component.

I figured that components that would be requested to filter the content that they produced before publishing what met the criteria of the requesting component would likely need to run at a particular interval rather than be event driven when notified of the publication of a topic.

Before adding the Content Filtered feature, a requesting component could only be assured that its consumer received a particular instance of a topic was if the delivery was point-to-point and the consumer was event driven such as with the N-to-One delivery method. A component that ran at specified intervals could only read data flow delivered topics where the component received the most recent instance of the topic. If multiple producers, the latest instance of one producer would replace that of an earlier producer as the one that the consumer would read. Therefore, the filter conditions of one requestor could be lost if another requestor published a request after that of the first and before the consumer had a chance to run.

Therefore, I added a delivery queue to the framework for each registered Aperiodic component that would allow the Content Filtered requests of each requesting component to be queued awaiting the execution of the request consumer; that is, the producer of the requested content.

In my implementation, if a requestor publishes multiple requests during the interval from one execution of the Aperiodic component and the next, the second is to replace the first in the queue. However, the requestor will most likely fail to obtain a write buffer for such a second request and so won’t be able to publish it until the first has been read.

The requesting and producing components must have a “contract” with each other as to what the filter conditions can be with the producing component keeping track of the various conditions and only publishing the response topic for a particular requestor when the conditions that it specified have been met. Therefore, it must maintain a table that identifies the requesting component by its private key, the topic that is to be published, and the conditions that have to be met to publish it. The request topic (Ada package or C# class) identifies the conditions while the response portion of the topic identifies the content to be included in the response. The request must identify the requesting component, the scope of the request (cancel, every time the conditions are met, only the first time, etc), and the condition pairs where each pair has the equality condition (Equal, Less Than, Less Than or Equal, etc) and the value to satisfy the condition.

Each time that the Aperiodic component reads the topic, the framework marks the oldest request in the queue as read and then provides the pointer to the topic buffer. The component can then save the request conditions. Before finding any oldest request in the queue, the framework checks for any previously read requests and releases the topic buffer of such a request and removes the request from the queue. Thus, if the Aperiodic component reads the topic until there is no instance of the topic to be returned, the queue will be emptied and each requestor’s buffer will be released for it to use again.

Both the reading and writing to the queue are, of course, done in a thread safe manner. In case the Aperiodic component doesn’t empty the queue, the framework will also check for previously read requests and release the topic buffer and remove the request from the queue when the component has finished running and control has returned to the forever loop of the framework.

It is assumed that the requestor wants its own version of the filtered topic. That is, the filtered content that it requested. Therefore, the response is delivered point-to-point to the requestor in the same manner as N-to-One. Only it is delivered each time that the content satisfies the specified conditions rather than immediately following the read of the request topic.

The contract between the requestor and the producer could, of course, be different. That is, there could be one requestor that specified the conditions that would be used by the producer as the criteria for publishing a corresponding data flow topic that could be read by any subscribing component. This could be indicated by the use of a request topic that didn’t supply the identity of the requestor. Or, a Content Filtered topic could be used as a means of providing data to an Aperiodic component if a more controlled delivery than data flow was needed.

Some Discussion of the AIDA System white paper as it applies to the Exploratory Project

http://www.scarlettproject.eu/news/Farnborough%20Dissemination/WhitePaper-1.0.pdf

Some Discussion of the AIDA System white paper as it applies to the Exploratory Project

In The AIDA System, A 3rd Generation IMA Platform, DIANA Project White Paper page 10, I found

“AIDA remote services design shall allow multi-supplier integration compliance with DO-297 recommendations such as ARINC 661 Cockpit Display System standard service architecture.

Modern avionic architectures request a decoupled and fault tolerant communication model in a system that is considered reliable and static. As a consequence, the AIDA architecture includes an asynchronous communication model (command / event based) that allows a location transparency of remote service implementation and the object-oriented support adapted for the AIDA needs.

Another characteristic found in avionics systems, and not usually found in other applications domains, is the extensive presence of health monitors. The AIDA proposed communication model takes advantage of that, not associating by default a status to a command.

One important peculiarity of the proposed architecture with respect to existing approaches, such as ARINC 661, is that AIDA does not impose the use of commands to update data. AIDA proposes the use of a publish / subscribe data centric mechanism with content filtered topic supported by standard ARINC 653 ports and channels for data communication support. The use of content filter mechanism will largely reduce AIDA inter partition communication required bandwidth (see the two figures below).

From an implementation point of view inter-partition data communication is supported by middleware level data-centric services (publisher and subscriber services) accessing partition memory and distributing data over predefined ARINC 653 standard channels.

In summary, the above vision does not affect the approach of keeping the high level architecture compliant with ARINC 653: inter-partition communication remains port-based even if a mechanism that enhances location-transparency and reduces system integrators effort is used. Intra-partition communication remains based on ARINC 653 API services supported by the capabilities of compliant RTOS.”

The figure shows the usual Producer and Consumer.  However, it has that the producer is only to send Temp information when the temperature is greater than 300 with the consumer having requested that condition. 

This is something that was available in a previous avionics communications controller that I designed in regards to the requestor being able to specify the frequency with which the topic was to be sent by the producer.  A consumer could request a particular topic be sent to it at a rate of its own choosing while other consumers could request the topic at other rates.

Allowing the consumer to subscribe to a topic in a way that allows it to specify the conditions under which it wants to receive the topic is something I can add to my exploratory message controller framework as well and seems like an excellent feature to provide. 

I already have this to a degree with the One-of-N topic delivery style where a consumer can subscribe to the topic to only receive it if its delivery id matches certain values or range of values.  In its case, the producer publishes the topic no matter what the delivery id and the framework checks if any consumer subscribed for its delivery id.  If not, the instance of the topic is tossed and goes undelivered.  If so, it is delivered to the particular consumer component that subscribed for it.

The kind of subscription of AIDA would be more complicated since the condition or set of conditions could vary and not just be based on particular integer values.  And it would be sent to the producer since it would have to decide when to publish since one consumer could have one set of conditions and another consumer might subscribe to receive the topic under a different set of conditions.  Or perhaps it would need to be One-to-One with only one publisher and one subscribing client.  In any case, the framework couldn’t keep track of the conditions for delivery since the possibilities would vary from topic to topic.

This would be something like N-to-One where I have N possible requestors and one consumer of the request that produces a response that is only delivered to the particular requestor.  However, for N-to-One the requestor sends a request and then synchronously gets its response.  With the AIDA feature the requestor sends its request for the topic with the conditions that are to be met to send the response.  Then the requestor gets its response when the conditions are met.  (Once or periodically while the conditions are met?  This could be another condition that is supplied such as a Frequency field in the request of 0 for only the first time the condition is met and a positive value to indicate a frequency to be sent while the conditions are met.)  The various sets of conditions would be a contract between the producer of the response and the various requestors.

This feature is surely what I should choose to do next.

On page 14

“A more complex task is the protection of data leaving their partition walls. For this purpose the MILS Message Routing (MMR) component is designed. The MMR contains a privilege table defining which applications may communicate with each other and which information an application may gather from another. In ARINC 653 the messaging component may be seen as the complement to this MILS facility. The A653 messaging relies on a configuration which is defined before start-up and describes all possible communication paths within a system. This way, it is impossible that a partition sends or receives a message without authorisation. The picture changes with applications that force users to authenticate. Now, the actor, the sender or receiver of a message, is not necessarily a neutral application, but an actor with a role in the systems interaction. The respective role may have authorisation to access certain information or may not. In such a situation, the existence of a static routing description addressing partitions, ports and messages is no longer sufficient. A real access control is needed which identifies users and grants privileges to them.

The general access control performed by the MMR can already contain detailed information like client/server-configurations, message types and access levels.  But it is not intended to enter protocol specifics of single applications. The MMR itself, being an OS component or a central system service, would become too complex if it integrated details of all possible types of applications and their communication protocols.”

The mentioned a privilege table reminds me of the loadable table I was going to have to identify the various allowed applications in a particular proposed avionics system and info on how they could interconnect.  I have a somewhat similar loadable table in the exploratory project that defines what communication methods (MS Pipes and/or WinSock) that an application supports and the WinSock IP address to use.  This could easily be expanded to specify which pairs of applications are allowed/authorized to communicate with each other.  Then an attempt to connect without authorization could be refused.

“To extend the capabilities of the information flow control, made available by the MMR, MILS provides the concept of guardian applications. These guards are applications of their own, running in separate partitions, but dedicated to the protection of an application or a set of applications. Each application or set of applications may have a guard to validate the exit and entry of messages in the given context sender/receiver, contents, purpose etc. This way, security related tasks are removed from the applications and concentrated in specialised protection software.”

I currently have some validation built in.  That is, that a received message was expected, has the correct header data and CRC, etc.  Perhaps a “guardian” package into which to place some of this code would better emphasize its purpose and allow, at a future date, additional validation of received messages and add validation prior to the transmit of a message.