Wednesday, October 16, 2019

Kubernetes Using Named Pipes and TCP Sockets


Kubernetes Using Named Pipes and TCP Sockets

This post covers two things.  The minor one being combining the Named Pipes code of the previous post (Kubernetes Using Named Pipes) with the Sockets code of the previous posts (Kubernetes Follow-On (Part 4) and Pseudo Visual Compiler Using Sockets). 

Since this wasn't much of an accomplishment I decided to add the transmit of Broadcast messages to selected components to be designated in the Delivery.dat file.  This proved a "little" more difficult.  As will be commented upon.

Besides the indication which components were allowed to receive broadcast messages, the Delivery.dat file also, as time went by, had the application that supports the component added and whether the component could publish/transmit broadcast messages.  This latter mainly to eventually allow any component to transmit a broadcast message and also eventually a message of a particular topic.  This latter will need the receiving components to indicate which topics they wish to receive – that is, to subscribe to those topics in a separate section of the file.

Note: In the framework delivery method of older posts the methodology to support message topics and to subscribe to consume particular topics was the responsibility of the component and there was program support to capture these requests into a library.  In the current approach, there is the separate Delivery.dat file where the "subscription" would take place.  The implementation of topic delivery is for the future while the ability to send and receive broadcast messages (of any topic) is preparation for the delivery of particular topics to the components indicated by the Delivery.dat file.

The combination of the use of the Named Pipe and TCP Socket delivery methods was relatively minor and won't be gone into here.  It did involve specifying in the Delivery.dat file which method the component expected.  Both for transmit and receive.  Hence, the paired component had to support the same method.  Eventually, the broadcast message producer had to support both methods so delivery could be made to a component based upon which method the receiving component expected.

For broadcast messages I assumed that I would be able to use the "To" pipe name of each paired component as the pipe name for transmitting the message just as, for components using sockets, that the component's IP address and Windows sockets port could be used.  This proved not to be the case – or at least I didn't find a way to get it to work.

Wild Goose Chase


For instance, I puzzled over a problem for quite some time.  I had Component1 receiving the broadcast message (as it should have) but it appeared to be received via TCP Sockets instead of Named Pipes as I thought that it should have been.  As can be seen in the Delivery.dat file data below, Component1 was marked in the Delivery.dat file to receive broadcast messages twice – once when it was paired with NewComponent to receive messages via Named Pipes where both components were application App1 components and a second time when paired with Component2 of App2 to receive messages via TCP Sockets. 

While chasing this perceived problem I changed the broadcast message so that it indicated which communication method was going to be used.  And as I thought, Component1 was receiving the message that was being sent via sockets.  Either before or after, I had changed the table of components that were marked to receive broadcast messages to not include a component again if it was already in the table.  So this resulted in Component1 only being in the table for its first occurrence in the Delivery.dat file where it was paired with NewComponent and to use Named Pipes.  But still the problem persisted.  So I chased it and chased a solution. 

During this time Component7 kept receiving the broadcast message even though it wasn’t supposed to and components 2 and 3 didn’t receive the message although marked to receive it via Sockets.  This I thought was strange but I concentrated on the Component1 problem.

Not until the morning of the last day did the light dawn even though it should have been staring me in the face all along.  In the Delivery.dat pairings Component7 is paired with Component3 and Component1 is paired with Component2 where Component2 and Component3 were supposed to receive the broadcast message and Component1 (the source of my mystery) and Component7 were receiving the message. 

For named pipes I had created the pipe name as 8toN where N was the component id of the component to receive the message.  This because for the paired components the Delivery.dat file contained the basic name to be used via a pair of names – one name for sending to the opposite component and one for receiving from it where the names were reversed in the record of the other component.  Since the BdComponent, a sample of a component to send broadcast messages, wasn’t to receive messages and hence not paired with each of the components to receive its messages there wasn’t the Delivery file pipe name to be used.  Hence, the creation of the name; 8to1, 8to5, 8to6 for the three components marked to allow the receive of broadcast messages via named pipes where 8 was the component id of BdComponent.

However, for TCP Sockets the IP Address and Port of the Delivery file had to be used and was selected from the entries for the components to be allowed to receive broadcast messages via sockets.  These were for Component2 and Component3.  Due to my chasing what I perceived as the Component1 problem I had been blind to the obvious – that Component1 and Component7 were the components paired with 2 and 3.  The problem being that the wrong port of the Component2 and Component3 was being selected.  That is, the port that was to be used to send messages rather than receive them.  Therefore, the broadcast messages were being received by the opposite component of the pair!  Talk about being blind to the obvious and hence the wild goose chase.

Results


Delivery.dat file
1|Component1|Pipe|Bd|COSTCO-HP|1to4|4to1|
4|NewComponent|Pipe| |COSTCO-HP|4to1|1to4|
5|Component5|Pipe|Bd|COSTCO-HP|5to6|6to5|
6|Component6|Pipe|Bd|COSTCO-HP|6to5|5to6|
3|Component3|Socket|Bd|x1.x2.x.y|8004|8003|
7|Component7|Socket| |x1.x2.x.y|8003|8004|
2|Component2|Socket|Bd|x1.x2.x.y|8002|8001|
1|Component1|Socket|Bd|x1.x2.x.y|8001|8002|
8|BdComponent|Both| |COSTCO-HP|8to1|1to8|x1.x2.x.y|8008|8008|
where x1, x2, x and y have been substituted for the actual IP address for this blog post.

Text output at the start of such output for App1:
. . .
portions of the DeliveryTable as parsed from Delivery.dat
  1 Component1 Pipe COSTCO-HP 1to4 4to1 0 0
  4 NewComponent Pipe COSTCO-HP 4to1 1to4 0 0
  5 Component5 Pipe COSTCO-HP 5to6 6to5 0 0
  6 Component6 Pipe COSTCO-HP 6to5 5to6 0 0
  3 Component3 Sock    x1.x2.x.y 8004 8003
  7 Component7 Sock    x1.x2.x.y 8003 8004
  2 Component2 Sock    x1.x2.x.y 8002 8001
  1 Component1 Sock    x1.x2.x.y 8001 8002
  8 BdComponent Sock COSTCO-HP 8to1 1to8 x1.x2.x.y 8008 8008
. . .
and the table built by the Broadcast-Client Request procedure invoked by the BdBroadcast package
Components to Receive Broadcast Messages
Com 1 Component1 COSTCO-HP 8to1
Com 5 Component5 COSTCO-HP 8to5
Com 6 Component6 COSTCO-HP 8to6
Com 3 Component3 17439
Com 2 Component2 16927
where 17439 and 16927 are the reversed byte port integers used by Sockets for ports 8004 and 8002, the first port of the pair specified for Component3 and Component2.

Problem With Using Named Pipes


There is a problem for the use of Named Pipes for broadcast messages.

With the use of broadcast messages there has to be a named pipe for each component that can receive a broadcast message and one for every component that can send a broadcast message.  That is, there has to be a match between the pipe name used to transmit the message and that used to receive it. 

And for every such named pipe there has to be a receive thread to wait for the message.  This will quickly escalate the number of threads to be supported by the application if the same pipe name and receive thread can’t be used as for the paired messages.

I tried to send a broadcast message using the receive pipe name of a paired component such as 4to1 or 2to1.  But this didn’t work.  So I went to names like Bto1.  But this only worked for one broadcasting component.  (Note: By this time I also had a Component9 of App2 broadcasting messages.)  So for a second component sending broadcast messages I had to create yet another pipe name.  That is, it created a situation like that of the paired components of broadcast-component-to-receiving-component for each possible receiving component.  So instead of one receive thread for the receiving component of the paired component a whole set of such pairs was required.  And required that it the Delivery.dat file specify which components were going to transmit broadcast messages so that the pipe names could be generated.

Results having only one server to receive broadcast messages for a named pipe


Starting App1 before App2
Now got
BdComponent to send 29 BdComponent Message for all 2
Broadcast Client SenderData ToId 1 Bto1
NamePipe-Server Receive of 29 Index 4 Bto1 4317470
Component1 received a message: BdComponent Message for P 1 2
Transmit WriteFile OK 29
Broadcast Client SenderData ToId 5 Bto5
Transmit WriteFile OK 29
NamePipe-Server Receive of 29 Index 6 Bto5 4320654
Broadcast Client SenderData ToId 6 Bto6
Component5 received a message: BdComponent Message for P 5 2
with App1, when started first, getting the broadcast message by both components marked to receive such messages.

Also
Component1 received a message: NewComponent Message for Component1 2

NewComponent received a message: Component1 Message for NewComponent 4

Component1 received a message: Component2 Message for Component1 2
NamePipe-Server Receive of 35 Index 5 6to5 4320654
Component5 received a message: Component6 Message for Component5 2
So all the paired messages were also received.

Then I got for App2,
Component2 received a message: Component1 Message for Component2 3

Component6 received a message: BdComponent Message for P 6 2
NamePipe-Server Receive of 29 Index 4 Bto3 4318206
Component3 received a message: BdComponent Message for P 3 2
NamePipe-Server Receive of 29 Index 2 Bto2 4315754
Component2 received a message: BdComponent Message for P 2 2
NamePipe-Server Receive of 35 Index 5 5to6 4320658
Component6 received a message: Component5 Message for Component6 2

Component2 received a message: Component1 Message for Component2 4

Component7 received a message: Component3 Message for Component7 2

Component3 received a message: Component7 Message for Component3 2
So each component receives its paired messages and the broadcast messages from BdComponent but not from Component9 which sends its messages.

Then, starting App2 first App1 has the results
NamePipe-Server Receive of 35 Index 1 2to1 4317470
Component1 received a message: Component2 Message for Component1 2
NamePipe-Server Receive of 35 Index 5 6to5 4320654
Component5 received a message: Component6 Message for Component5 2
NamePipe-Server Receive of 28 Index 2 Bto1 4317470
Component1 received a message: Component9 Message for P 1 2
NamePipe-Server Receive of 28 Index 6 Bto5 4320654
Component5 received a message: Component9 Message for P 5 2

NamePipe-Server Receive of 28 Index 2 Bto1 4317470
Component1 received a message: Component9 Message for P 1 3
NamePipe-Server Receive of 28 Index 6 Bto5 4320654
Component5 received a message: Component9 Message for P 5 3
OpenReceivePipe \\.\pipe\4to1
Broadcast Callback Bto5
NamePipe-Server Receive of 35 Index 1 2to1 4317470
Component1 received a message: Component2 Message for Component1 3
NamePipe-Server Receive of 35 Index 5 6to5 4320654
Component5 received a message: Component6 Message for Component5 3

NamePipe-Server Receive of 37 Index 7 1to4 4322978
NewComponent received a message: Component1 Message for NewComponent 2

And App2 has the text output
NamePipe-Server Receive of 35 Index 7 3to7 4323110
Component7 received a message: Component3 Message for Component7 2

NamePipe-Server Receive of 35 Index 3 7to3 4318206
Component3 received a message: Component7 Message for Component3 2

NamePipe-Server Receive of 28 Index 6 Bto6 4320658
Component6 received a message: Component9 Message for P 6 2
Transmit WriteFile OK 28
Broadcast Client SenderData ToId 3 Bto3
Transmit WriteFile OK 28
NamePipe-Server Receive of 28 Index 4 Bto3 4318206
Broadcast Client SenderData ToId 2 Bto2
Component3 received a message: Component9 Message for P 3 2
NamePipe-Server Receive of 28 Index 2 Bto2 4315754
Component2 received a message: Component9 Message for P 2 2

NamePipe-Server Receive of 35 Index 1 1to2 4315754
Component2 received a message: Component1 Message for Component2 2

NamePipe-Server Receive of 35 Index 7 3to7 4323110
Component7 received a message: Component3 Message for Component7 3

NamePipe-Server Receive of 35 Index 5 5to6 4320658
Component6 received a message: Component5 Message for Component6 2

So all the paired component messages are received (although the Component5 message of App1 took quite some time) and all the broadcast messages of Component9 were received by the components of both apps.

When App1 is started first its broadcaster gets the pipes and when App2 is started first its broadcaster gets the pipes.  Therefore separate servers are required for each different transmitter.

Therefore, by these results, a pipe is necessary for each broadcaster with a receiver for each pipe.  This would be manageable in the configuration that I currently have with just two components sending broadcast messages.  That is, just add another set of pipes for the second broadcaster.  However, in the long run this is a non-starter. 

That is, if any component can send a broadcast message, the number of pipes and the number of receiver threads goes up substantially.  This wasn’t such a problem when messages were between applications and then forwarded to the components of interest as in the delivery frameworks of posts of the previous delivery method .  For instance, three applications than required three pairs of pipes (to make them unidirectional).  But for multiple components each potentially transmitting to a number of other components the use of named pipes becomes cumbersome and resource intensive.

Therefore, it seems that the use of sockets is a better choice.  Instead of expanding the number of named pipes from the one extra per receiving component of the current code I returned to the use of sockets for most of broadcast message delivery leaving just the application one (App1) components to receive the messages via named pipes.  This still necessitated a second broadcast pipe for each of the two App1 receiving components which will be illustrated in the code.  This doesn’t place a great burden on the application resources since there are only two components that send broadcast messages. 

However, for the general case where any component could send a broadcast message it would be necessary for the Delivery file to specify which components would do so enabling the code to create the necessary pipe names and receive threads.

The fix to allow the receive of Broadcast messages from multiple transmitters

As mentioned before, I had thought that I could use the same named pipe path name for a broadcast message as that for a paired component when the To component was the same component as that to receive the broadcast message.  I failed to achieve that so came up with the solution of the above section where a separate pipe name was used for components that were allowed to receive broadcast messages.  But it only delivered the messages of the broadcast component of the application that started first and, hence, secured the connection.

Therefore, I created in the Delivery package a table with a double dimensioned array of pipe names.  The first dimension (the table rows) for the each component that could publish a broadcast message and the second dimension (the table columns) for the receiving components that used Named Pipes for the delivery method.

This table was then filled in with pipe names – a different one for each transmitter, receiver combination.  Then the particular pipe name was used that matched both the transmitter and the receiver.

In the current configuration in Delivery.dat this resulted in
B08to01 B08to05 B08to06
B09to01 B09to05 B09to06
where the first row corresponds to the BdComponent of App1 and the second row to the Component9 of App2.  The columns are for the 3 components (Component1, 5 and 6) that receive their messages via Named Pipes.

This could then be extended for additional components to allow them to send these unpaired component messages including those that receive paired messages.  In addition, changes should be easy enough to make to allow components to register (via another .dat file or a different section of the general .dat file) for particular topics and the component that published the topic to have the Client package select the set of receiver components to which to transmit the topic message.  (Shall I say I leave this as an exercise for the reader?)

However, like the Kubernetes project of GitHub, the use of TCP Sockets is the better choice since (with Windows) this only requires one port for each component rather then one pipe name for each From-To component combination.

Results having only a server to receive broadcast messages for each named pipe


With two transmitters of broadcast messages – BdComponent installed in App1 and Component9 installed in App2, App1 got
Component1 received a message: BdComponent Message for P 1 2 ß
Transmit sending 35 bytes WriteFile
Transmit WriteFile OK 37 4to1
NamePipe-Server Receive of 37 Index 4 4to1 4321894
Component1 received a message: NewComponent Message for Component1 2
Transmit WriteFile OK 29
Broadcast Client SenderData ToId 5
Transmit WriteFile OK 35 5to6
Transmit WriteFile OK 29
Broadcast Client SenderData ToId 6
NamePipe-Server Receive of 29 Index 6  4325078
Transmit WriteFile OK 29
Component5 received a message: BdComponent Message for P 5 2 ß
Broadcast Client SenderData ToId 3
Transmit WriteFile OK 29
Broadcast Client SenderData ToId 2
Transmit WriteFile OK 29
Broadcast Callback B08to05
Broadcast Callback B08to01
Component1 to send to Component2 35 Component1 Message for Component2 4
Client Transmit Index 1
Transmit sending 35 bytes WriteFile
Transmit WriteFile OK 35 1to2
NamePipe-Server Receive of 28 Index 3  4321894
Component1 received a message: Component9 Message for P 1 5 ß
NamePipe-Server Receive of 35 Index 5 6to5 4325078
Component5 received a message: Component6 Message for Component5 5
NamePipe-Server Receive of 35 Index 1 2to1 4321894
Component1 received a message: Component2 Message for Component1 5
NamePipe-Server Receive of 28 Index 7  4325078
Component5 received a message: Component9 Message for P 5 5 ß
So the broadcast messages now received from both transmitters of broadcast messages.

And App2 received
Component6 received a message: BdComponent Message for P 6 2 ß
NamePipe-Server Receive of 29 Index 5  4322630
NamePipe-Server Receive of 29 Index 2  4320178
Component3 received a message: BdComponent Message for P 3 2 ß
Component2 received a message: BdComponent Message for P 2 2 ß
. . .
Component6 received a message: Component9 Message for P 6 5 ß
Component7 received a message: Component3 Message for Component7 5
NamePipe-Server Receive of 28 Index 6  4322630
Component3 received a message: Component9 Message for P 3 5 ß
Broadcast Client SenderData ToId 2
Transmit WriteFile OK 28
NamePipe-Server Receive of 28 Index 3  4320178
Component2 received a message: Component9 Message for P 2 5 ß

So each component that was allowed to receive broadcast messages received them from each of the transmitters.

Delivery.dat that was used to produce the above results

In order to concentrate on the changes needed to deliver broadcast messages via Named Pipes the Delivery data file was changed to only specify the use of pipes. 

The fields of each record are separated by the | token.  The fields in order are
1 Component Id and Component Name
2 Application Id
3 The delivery method
5 Whether the component will transmit broadcast messages
6 Computer name to be used for Named Pipe delivery
7 Pipe name to transmit
8 Pipe name to receive

For components that transmit broadcast messages they must support both named pipes and sockets so they can do so via sockets if the receiving component specifies the use of sockets.  The IP address isn't identified.

1|Component1|1|Pipe|Bd| |COSTCO-HP|1to4|4to1|
4|NewComponent|1|Pipe| | |COSTCO-HP|4to1|1to4|
5|Component5|1|Pipe|Bd| |COSTCO-HP|5to6|6to5|
6|Component6|2|Pipe|Bd| |COSTCO-HP|6to5|5to6|
3|Component3|2|Pipe|Bd| |COSTCO-HP|3to7|7to3|
7|Component7|2|Pipe| | |COSTCO-HP|7to3|3to7|
2|Component2|2|Pipe|Bd| |COSTCO-HP|2to1|1to2|
1|Component1|1|Pipe|Bd| |COSTCO-HP|1to2|2to1|
8|BdComponent|1|Both| |Bd|COSTCO-HP|8to1|1to8|xx.x.x.xxx|8008|8008|
9|Component9|2|Both| |Bd|COSTCO-HP|9to1|1to9|xx.x.x.xxx|8009|8009|

Notice that Component1 and Component5 of application 1 and Component6, Component3, and Component2 of application 2 are specified to receive broadcast messages while BdComponent of application 1 and Component9 of application 2 transmit broadcast messages.

Windows 10 Problems

What with the support for Windows 7 ending at the end of the year I went ahead and got a Windows 10 laptop.  What a fiasco that has turned out to be. 

It won't allow the applications that I have been using to be installed or to run.  And the GNAT GPS Ada that I have been using for these private projects won't run in the versions that I have been using and the latest version GNAT GPS 2019 that I had to install as the only one seemingly available for download has problems that prevent it from being usable.  For instance, I did manage to get a build for the App1 application but it failed for the App2 application that is identical in structure.  That is, it built to a much smaller executable that doesn't do anything.  Perhaps I'll figure that one out but I failed to do so after a couple of days of trying.

Then there is the need for special treatment to be able to run my own created applications (that is, app1 and app2).  That I found out about and can get around.  But the need to buy all new applications suitable for Windows 10 seems to me to be a RIP OFF.  A version of Windows that Microsoft is pushing us into needing to use that is completely deficient.  That is, Windows 10 can't install or run programs that earlier versions of Windows had no problems running.

So I'll need to continue to use Windows 7 even though it’s unsupported come the end of the year.

Return to the use of Sockets for delivery

Returning to the use of Win Sockets for delivery to three of the App2 components while retaining the use of Named Pipes for the App1 components and one of the App2 components the Delivery.dat file looks like
1|Component1|1|Pipe|Bd| |COSTCO-HP|1to4|4to1|
4|NewComponent|1|Pipe| | |COSTCO-HP|4to1|1to4|
5|Component5|1|Pipe|Bd| |COSTCO-HP|5to6|6to5|
6|Component6|2|Pipe|Bd| |COSTCO-HP|6to5|5to6|
3|Component3|2|Socket|Bd| |xx.x.x.xxx|8004|8003|
7|Component7|2|Socket| | |xx.x.x.xxx|8003|8004|
2|Component2|2|Socket|Bd| |xx.x.x.xxx|8002|8001|
1|Component1|1|Socket|Bd| |xx.x.x.xxx|8001|8002|
8|BdComponent|1|Both| |Bd|COSTCO-HP|8to1|1to8|xx.x.x.xxx|8008|8008|
9|Component9|2|Both| |Bd|COSTCO-HP|9to1|1to9|xx.x.x.xxx|8009|8009|

With the use of Sockets the same port could be used to transmit to a component and the same receive callback thread could be used whether the receiving component was receiving a paired component message or a broadcast message.

This is important since I had to have a different named pipe for each From component to To component connection.  So I had to have a different receive thread to await the message for each connection.  Therefore, for larger applications with more components this would become a major use of resources.

I had thought that I could find a method to reuse the paired component connection for the broadcast messages.  But as I have mentioned I failed to do so.  I suppose I could try again but to what purpose when the use of Sockets can be used in place of Named Pipes.

Results using both named pipes and sockets to receive messages


The App1 results
Component1 received a message: NewComponent Message for Component1 1  ß the paired
. . .
NewComponent received a message: Component1 Message for NewComponent 2 ßcomponents
. . .
Component5 received a message: BdComponent Message for P 5 2 ß the local broadcast
NamePipe-Server Receive of 29 Index 2  4324078
Component1 received a message: BdComponent Message for P 1 2 ß   messages
. . .
Component1 received a message: Component2 Message for Component1 1 ß remote paired
. . .
Component1 received a message: Component9 Message for P 1 2 ß the remote broadcast
NamePipe-Server Receive of 28 Index 6  4327262
Component5 received a message: Component9 Message for P 5 2 ß   messages
NamePipe-Server Receive of 35 Index 4 6to5 4327262
Component5 received a message: Component6 Message for Component5 2 ß remote paired

So the App1 components received all their messages.  Note these all used Named Pipes. 

The App2 results
Component6 received a message: Component5 Message for Component6 2 ç remote paired
Socket-Server Receive of 29 Index 2
Component3 received a message: BdComponent Message for S 3 1 ç remote broadcast
Socket Server Receive after Listen 2 True
Socket-Server Receive of 29 Index 1
Component2 received a message: BdComponent Message for S 2 1 ç remote broadcast
. . .
Component2 received a message: Component1 Message for Component2 2 ß remote paired
. . .
Component6 received a message: Component5 Message for Component6 3
NamePipe-Server Receive of 29 Index 2  4327266
Component6 received a message: BdComponent Message for P 6 2 ß all three
Socket-Server Receive of 29 Index 2
Component3 received a message: BdComponent Message for S 3 2 ß  components rec'd
Socket Server Receive after Listen 2 True
Socket-Server Receive of 29 Index 1
Component2 received a message: BdComponent Message for S 2 2 ß  remote broadcast
Socket Server Receive after Listen 1 True
Socket-Server Receive of 35 Index 1
Component2 received a message: Component1 Message for Component2 3
. . .
Component7 received a message: Component3 Message for Component7 1 ß local paired
. . .
Component3 received a message: Component7 Message for Component3 1 ß local paired
Transmit sent using client socket port 17439
ERROR: Write to pipe failed 404 0
. . .
Component3 received a message: Component9 Message for S 3 1 ß local broadcast
Broadcast Callback B09to06
OpenReceivePipe \\.\pipe\B09to06
NamedPipe Server (Receive) Handle is Valid 412
Server connecting to client...
Socket Server Receive after Listen 2 True
Client Socket Connected to Transmit
        412
NamedPipe Server setting Connected           0
Socket-Server Receive of 28 Index 1
Component2 received a message: Component9 Message for S 2 1 ß local broadcast
. . .
Component6 received a message: Component9 Message for P 6 2 ß local broadcast

So the App2 components received all their messages.  Component6 via Named Pipes and Components 3, 7, and 2 via Sockets.

Application Code

The code will be resented in a separate post.