Kubernetes Follow-On (Part 3)

Kubernetes Follow-On (Part 3)

In the previous post I described sending messages directly between components via TCP/IP (Transmission Control Protocol/Internet Protocol) via Microsoft WinSock residing within the same application, different applications on the same PC, and different applications on different PCs.

It took me a while to get the communications between components on different PCs working but it worked fine after I got the missing links figured out.

However, there was a problem when WSA 10061 (Connection refused) errors occurred.  It was 2 strikes and you're out.  So after publishing the previous results I decided to see what would happen if after a WSA 10061 error, I immediately did another WSAStartup.  This worked great.  No longer did I get the WSA 10093 (Successful WSAStartup not yet performed) errors that prevented communications from happening.  Problem solved.

While at it I decided to try to terminate an application and then re-launch it to see if communications between the components of another application that remained running would start over.  This required a couple of changes to the code but when made the components re-connected and messages again were sent and received by the components residing in different applications.

WSA 10093 fix

To fix the WSA 10093 error after a 10061 error, I replaced the calls to the WSACleanup function with calls to a new WSARestart procedure located in WinSock.adb.

  procedure WSARestart is

    Status

    -- Result of WSAStartup call

    : ExecItf.INT;

    use type ExecItf.INT;

  begin -- WSARestart

    -- Do WSA Cleanup

    Status := ExecItf.WSACleanup;

    -- Followed by WSA Startup

    Status := ExecItf.WSAStartup( VersionRequired => 16#0202#, -- version 2.2

                                  WSAData         => lpWSAData );

    if Status /= 0 then

      Text_IO.Put("ERROR: WinSock WSAStartup failed");

      Int_IO.Put(Integer(Status));

      Text_IO.Put_Line(" ");

      return;

    end if;

  end WSARestart;

For instance,

  if Comm.Link(Index).Receive.Socket.Socket = ExecItf.INVALID_SOCKET then

    ExecItf.Display_Last_WSA_Error;

    declare

      Text : Itf.V_80_String_Type;

    begin

      Text.Data(1..9) := "WSA Error";

      Text := TextIO.Concat(Text.Data(1..9),Integer(Index));

      TextIO.Put_Line(Text);

    end;

    Status := ExecItf.WSACleanup;

    declare

      Text : String(1..28) := "Client Socket NOT created: x";

    begin

      Text(28) := to_Digit(Integer(Index));

      Text_IO.Put_Line(Text);

    end;

  else -- valid

of WinSock-Recv-ReceiveCreate of the previous post has

    Status := ExecItf.WSACleanup;

become

    WSARestart;

as well as removing the declaration of Status.  The same for the other calls to the WSACleanup function.

Reconnect after application closed

A few changes were required in order to have the ability to have a running application with its components reconnect to another application's components after it had been terminated and then restarted. 

The main one was that the WinSock Xmit forever loop had to continue to execute its Connect loop rather than exit it and enter a do nothing loop following the Accept of a client connection.  This merely involved removing the "exit Connect;" statement and then dispensing with the second loop since the code was then never executed.

That is, the Connect loop already checked to avoid the call to the C_Accept function if the Transmit connection was connected.  Therefore, there was no harm in using the Connect loop as the forever loop.  After the connection was recognized the loop did nothing so it could continue to be executed instead of needing the exit.  It thus became ready to recognize the need for the Accept function when the connection was broken due to the closing of the application that contained the component of the connection.  When the application was relaunched it was then ready to have the conditions for a client connection fulfilled once more.

Thus, the WinSock-Xmit callback forever loop just becomes

  -- Forever loop as initiated by Threads

  procedure Callback

  ( Id : in Integer

  ) is

  -- This procedure runs in the particular thread assigned to accept the

  -- connection for a component.

    Client_Socket

    -- Accepted client socket

    : ExecItf.SOCKET := ExecItf.INVALID_SOCKET;

    type Int_Ptr_Type is access ExecItf.INT;

    use type Interfaces.C.int;

    Index

    -- Index for Component for Comm.Link

    : Connection_Count_Type

    := Connection_Count_Type(Id);

    Client_Address_Size

    -- Size of socket address structure

    : ExecItf.INT

    := Comm.Link(Index).Transmit.Socket.Data'size/8;

    use type ExecItf.SOCKET;

    function to_Int_Ptr is new Unchecked_Conversion( Source => System.Address,

                                                     Target => Int_Ptr_Type );

    function to_Integer is new Unchecked_Conversion -- for debug

                               ( Source => ExecItf.PSOCKADDR,

                                 Target => Integer );

  begin -- Callback

    Connect:

    Loop

      declare

        Text : String(1..28);

      begin

        Text(1..19) := "Xmit Callback loop ";

        Text(20) := to_Digit(Integer(Id));

        Text(21) := ' ';

        Text(22) := to_Digit(Integer(Index));

        if Comm.Link(Index).Transmit.Created then

          Text(23..28) := " True ";

        else

          Text(23..28) := " False";

        end if;

        Text_IO.Put_Line(Text);

      end;

      if Index = 1 then

        Text_IO.Put_Line("index of 1");

      elsif Index = 2 then

        Text_IO.Put_Line("index of 2");

      else

        Text_IO.Put_Line("index of 3");

      end if;

      if Comm.Link(Index).Transmit.Created and then

         Comm.Link(Index).Receive.Connected and then

         not Comm.Link(Index).Transmit.Connected

      then

        -- Accept a client connection.

        Client_Socket :=

          ExecItf.C_Accept( S       => Comm.Link(Index).Transmit.Socket.Socket,

                            Addr    => null,

                            AddrLen => null );

        declare

          Text : Itf.V_80_String_Type;

        begin

          Text := TextIO.Concat( "Xmit after C_Accept", Integer(Index) );

          TextIO.Put_Line(Text);

        end;

        if Client_Socket = ExecItf.INVALID_SOCKET then

          Text_IO.Put_Line("ERROR: Server Client Socket NOT accepted");

          ExecItf.Display_Last_WSA_Error;

        else -- Accepted

          Comm.Link(Index).Transmit.Connected := True;

          Comm.Link(Index).Transmit.Socket.Client := Client_Socket;

        end if; -- invalid Client_Socket

      end if; -- Comm.Link(Index).Transmit.Created

      Text_IO.Put_Line("Xmit Callback initial loop end");

      delay(1.0*Duration(Index)); -- seconds

    end loop Connect;

  end Callback;

end Xmit;

without the exit from the Connect loop and without the following do nothing loop. 

Also, note that the Client_Socket of C_Accept function is stored in a new Comm.Link(Index).Transmit.Socket.Client location of the WinSock structures rather than overwriting the Transmit.Socket.Socket as before so that it remains available for restart if needed.

Another minor change was made to WinSock-Transmit.adb to indicate that the Receive (i.e., Client) connection was no loner valid along with the Transmit / Server connection when the Send of a message could no longer be accomplished.  Thus, that code became (using the newly saved Client Socket)

  Bytes_Written :=

    ExecItf.Send( S     => Comm.Link(Index).Transmit.Socket.Client,

                  Buf   => to_PCSTR(Message),

                  Len   => ExecItf.INT(Count),

                  Flags => 0 );

  if Bytes_Written /= ExecItf.INT(Count) then

    Text_IO.Put("ERROR: WinSock Message Send failed");

    Int_IO.Put(Integer(Bytes_Written));

    Text_IO.Put(" ");

    Text_IO.Put(String(Comm.Link(Index).Transmit.Name(1..25)));

    Int_IO.Put(Integer(Index));

    Int_IO.Put(Integer(Comm.Link(Index).Transmit.Socket.Data.SIn_Port));

    Text_IO.Put_Line(" ");

    ExecItf.Display_Last_WSA_Error;

    -- Indicate that no longer connected

    Comm.Link(Index).Transmit.Connected := False;

    Comm.Link(Index).Receive.Connected := False;

where

    -- Indicate that no longer connected

    Comm.Link(Index).Transmit.Connected := False;

became

    -- Indicate that no longer connected

    Comm.Link(Index).Transmit.Connected := False;

    Comm.Link(Index).Receive.Connected := False;

I think these minimal changes were all that were necessary.  With the final one immediately above, Receive.Connected was set back to False.  Therefore, the Forever loop of my Recv callback could re-connect without change since the loop checks for Connected before waiting for a message via the Microsoft Recv function.

Results

Debugging the two changes I started App1 and App2 at close to the same time and allowed them to run for 10 to 15 seconds.  I then terminated App2 and waited about 10 seconds and then restarted it and allowed both applications to run for another 10 to 15 seconds before terminating both applications.

App1 contains the same two components as in the past – Component1 and NewComponent that is also identified in the output as Component4.  App2 contains Component2 and ExComponent.  Component1 and Component2 communicate with each other.  Therefore, when App2 was terminated these two components could no longer communicate while Component1 and NewComponent should continue to send messages to each other.

The results are

Client Socket 2 Connected

Comm.Data(2) Available for Client ß

Receive Connected 2 17183

valid socket

Client Socket 1 Connected

Comm.Data(1) Available for Client ß

Receive Connected 1 16671

valid socket

Client Socket 3 Connected

Comm.Data(3) Available for Client ß

Receive Connected 3 17439

valid socket

Component1 received a message: Component2 message ß from App2

Xmit Callback loop 1 1 True

index of 1

Xmit after C_Accept 1

Xmit Callback initial loop end

Component1 wait for event

Component1 after end of wait

Component1 sending to Component2

Transmit Index 1 DeliverTo 2

Transmit sent using socket port 16927 ß sending to Component2 of App2

Component1 sending to component4

Transmit Index 2 DeliverTo 4

Transmit 2 not connected, returning ç above has Available for Client

NewComponent in forever loop

Transmit Index 3 DeliverTo 1

Transmit 3 not connected, returning ç above has Available for Client

Xmit Callback loop 2 2 True

index of 2

Xmit Callback loop 1 1 True

index of 1

Xmit after C_Accept 2          ß connection to local component completed

Xmit Callback initial loop end

Xmit Callback initial loop end

Component1 received a message: Component2 message ß message from Component2

Xmit Callback loop 1 1 True

index of 1

Xmit Callback initial loop end

Xmit Callback loop 3 3 True

index of 3

Xmit after C_Accept 3          ß connection to local component completed

Xmit Callback initial loop end

Component1 wait for event

Component1 after end of wait

Component1 sending to Component2

Transmit Index 1 DeliverTo 2

Xmit Callback loop 2 2 True

Xmit Callback loop 1 1 True

index of 1

Xmit Callback initial loop end

index of 2

Transmit sent using socket port 16927

NewComponent in forever loop

Transmit Index 3 DeliverTo 1

Xmit Callback initial loop end

Component1 sending to component4

Transmit Index 2 DeliverTo 4

Transmit sent using socket port 17183

NewComponent received a message: Component1 message for 4 ß local msg received

Component1 received a message: Component4 message   3     ß local msg received

Transmit sent using socket port 17439

Component1 received a message: Component2 message  ß message with App2 running

Xmit Callback loop 1 1 True

The above shows the connection between Component1 of App1 and Component2 of App2 at its beginning along with NewComponent of App1 receiving a message from Component1 and Component1 receiving a message from Component4 (that is, NewComponent). 

Then App2 was terminated and is indicated when the Send failed in WinSock Transmit.

Component1 received a message: Component2 message ß

Xmit Callback loop 3 3 True

index of 3

Xmit Callback initial loop end

Xmit Callback loop 1 1 True

index of 1

Xmit Callback initial loop end

ERROR: WinSock Receive failed 1

ERROR: WSALastError          0

Xmit Callback loop 2 2 True

index of 2

Xmit Callback initial loop end

Xmit Callback loop 1 1 True

index of 1

Xmit Callback initial loop end

NewComponent in forever loop

Transmit Index 3 DeliverTo 1

Transmit sent using socket port 17183

Component1 received a message: Component4 message  18

Component1 wait for event

Component1 after end of wait

Component1 sending to Component2

Transmit Index 1 DeliverTo 2                  ç Try send from Com 1 to Com 2

ERROR: WinSock Message Send failed         -1 ç WinSock Server Accept 01           1      16927

ERROR: WSALastError          0                ç App2 has been shutdown

Component1 sending to component4

Transmit Index 2 DeliverTo 4

NewComponent received a message: Component1 message for 4 ß (A)

Transmit sent using socket port 17439

ReceiveCreate Index 1

Xmit Callback loop 1 1 True

index of 1

Xmit Callback initial loop end

ERROR: WSALastError      10061               ç

WSA Connect Error 1                          ç Index of (1,2) pair

Client Socket 1 NOT Connected                ç

ERROR: Client Connect 1 FAILED:              ç

WinSock Receive 01                 1

Receive NOT Connected 1 16671                ç

ReceiveCreate Index 1

Then WSA errors of 10061 start to happen.  But 10093 errors no longer happen since the new WSARestart procedure does the WSACleanup and then does a new WSAStartup.  The messages between the two local components continue as before as illustrated by NewComponent receiving a message from Component1 at (A).

The 10061 errors continue but don't shutdown Microsoft WinSock since the WSAStartup is done each time.  Then, upon the restart of App2 there is a last 10061 error (before or maybe after the restart), Component1 attempts to send to component 2 but Transmit has yet to recognize a reconnection (B) and so just returns to Component1.

Xmit Callback initial loop end

ERROR: WSALastError      10061 ç again

WSA Connect Error 1

Client Socket 1 NOT Connected

ERROR: Client Connect 1 FAILED:

WinSock Receive 01                 1

Receive NOT Connected 1 16671

Xmit Callback loop 2 2 True

index of 2

Xmit Callback initial loop end

Xmit Callback loop 1 1 True

index of 1

Xmit Callback initial loop end

NewComponent in forever loop

Transmit Index 3 DeliverTo 1

Component1 received a message: Component4 message  33

Transmit sent using socket port 17183

Component1 wait for event

Component1 after end of wait

Component1 sending to Component2

Transmit Index 1 DeliverTo 2        ß (B)

Transmit 1 not connected, returning ß

Component1 sending to component4

Transmit Index 2 DeliverTo 4

NewComponent received a message: Component1 message for 4

Transmit sent using socket port 17439

ReceiveCreate Index 1

Xmit Callback loop 1 1 True

index of 1

Xmit Callback initial loop end

Client Socket 1 Connected

Comm.Data(1) Available for Client

Receive Connected 1 16671

valid socket

Xmit Callback loop 1 1 True

index of 1

Xmit Callback loop 3 3 True

index of 3

Xmit Callback initial loop end

Xmit Callback loop 2 2 True

index of 2

Xmit Callback initial loop end

NewComponent in forever loop

Transmit Index 3 DeliverTo 1

Component1 received a message: Component4 message  34

Transmit sent using socket port 17183

Component1 wait for event

Component1 after end of wait

Component1 sending to Component2

Transmit Index 1 DeliverTo 2

Transmit 1 not connected, returning

Component1 sending to component4

Transmit Index 2 DeliverTo 4

NewComponent received a message: Component1 message for 4

Transmit sent using socket port 17439

Xmit Callback loop 2 2 True

index of 2

Xmit Callback initial loop end

NewComponent in forever loop

Transmit Index 3 DeliverTo 1

Transmit sent using socket port 17183

Component1 received a message: Component4 message  35

Component1 wait for event

Component1 after end of wait

Component1 sending to Component2

Transmit Index 1 DeliverTo 2        ß B (again)

Transmit 1 not connected, returning ß

Component1 sending to component4

Transmit Index 2 DeliverTo 4

Transmit sent using socket port 17439

NewComponent received a message: Component1 message for 4

Xmit Callback loop 3 3 True

index of 3

Xmit Callback initial loop end

Xmit after C_Accept 1          ç Connection to App2 completed once again

Xmit Callback initial loop end

Xmit Callback loop 2 2 True

index of 2

Xmit Callback initial loop end

NewComponent in forever loop

Transmit Index 3 DeliverTo 1          ß message to be sent to Component1

Component1 received a message: Component4 message  36

Transmit sent using socket port 17183 ß message sent to Component1

Component1 wait for event

Component1 after end of wait

Component1 sending to Component2

Transmit Index 1 DeliverTo 2          ß (C)

Transmit sent using socket port 16927

Component1 sending to component4

Transmit Index 2 DeliverTo 4

Transmit sent using socket port 17439

NewComponent received a message: Component1 message for 4

Xmit Callback loop 1 1 True

index of 1

Xmit Callback initial loop end

Component1 received a message: Component2 message ß msg received from Com 2

Then, the C_Accept invocation of the Xmit Callback succeeds again (as it did when App2 was initially started).  And at (C) Transmit again recognizes that there is a connection and sends the Component1 message to Component2 of the newly started App2.  With the last line shown from the example, Component1 has again received a message from Component2.