Docs: fix the command dealing function and updating TCP_Server_APIs.rst (#39)

* Docs: updating TCP_Server_APIs.rst

* Docs: updating TCP_Server_APIs.rst while the handling info APIs part has been done.

* core-code: updating TCP_Server_APIs.rst

* Docs: updating TCP_Server_APIs.rst

* core-code: fix the errors of the command dealing output result sending

* core-code: fix the error of the command dealing functions (undone)

* core-code: fix the dealing command functions

* core-code: fix the command dealing functions

Author: F18-Ray

docs/Network_APIs/TCP_Server_APIs.rst

@@ -170,18 +170,29 @@ the arguments which has been defined in the
 change are ``port_add_step``, ``port_range_num`` 
 and ``is_hand_alloc_port``.*
 
-- receiving raw bytes from the socket using ``recieve_message``
+- receiving raw bytes from the socket using `recieve_message`
 - buffering incoming data until newline-terminated messages are complete
 - splitting and processing each message line-by-line
-- routing special commands beginning with ``/`` to ``handle_command``
+- routing special commands beginning with ``/`` to `handle_command`
 
-*Note: The ``handle_command`` function is defined as:*
+The `handle_command` function is defined as: 
 
 .. code-block:: python
 
     def handle_command(self: Self, client_socket: Any, client_address: Any, command: Any) -> None:
         ...
 
+In `handle_command`, there are variable conditional 
+branches to handle built-in commands like ``/help``, 
+``/time``, ``/clients``, ``/quit``, and some file 
+transfer commands. If you already added more commands 
+by the command extension API, the function will 
+determine if the inputed message matched the extension 
+commands.
+
+*Note: For more details of the command extension API, 
+and the built-in commands, please visit ...*
+
 - logging normal chat messages and acknowledging receipt
 - removing the client from ``self.clients`` and closing the socket when the client disconnects or an error occurs
 
@@ -190,11 +201,12 @@ and ``is_hand_alloc_port``.*
     def recieve_message(self: Self, client_socket: Any, msg_length: Int) -> Any:
         ...
 
-`recieve_message` is a thin wrapper around socket receive operations. 
-It reads up to ``msg_length`` bytes from the given 
-``client_socket`` and returns the raw byte payload. 
-Message decoding and newline message framing are handled 
-by the caller.
+`recieve_message` is a thin wrapper around socket 
+receive operations. It reads up to ``msg_length`` 
+bytes from the given ``client_socket`` and 
+returns the raw byte payload. Message decoding 
+and newline message framing are handled by the 
+caller.
 
 .. code-block:: python
 
@@ -214,4 +226,161 @@ These methods form the server's client I/O loop
 and ensure reliable message exchange for connected 
 TCP clients.
 
-TCP Server 
+TCP Server command API
+----------------------
+
+The server supports several built-in commands 
+and a command extension API. The main entry 
+point is the `handle_command` method, which 
+is invoked for any message starting with ``/``.
+
+We support two solutions for command handling, 
+one is input a command in the console, and 
+the other is recieving a command from other 
+clients, and you can also call the functions 
+which are defined for the commands in the code.
+
+*Note: You can select the solution which you 
+want to use by changing the args of the 
+`TCP_Server_Base` class, the arg is 
+``is_input_command_in_console``. ``True`` is 
+allow the server to input the command in 
+console, while ``False`` is don't allow.*
+
+Built-in client commands include:
+
+- ``/help``: returns the available command list and usage hints.
+- ``/time``: returns the current server time.
+- ``/clients``: returns the list of connected client IDs.
+- ``/quit``: returns a goodbye message and disconnects the client.
+- ``/file <file_path> <client_id>``: starts a file transfer request from client to server.
+- ``/file_folder <folder_path> <client_id>``: starts a folder transfer request from client to server.
+- ``/server_file_transfer_port <port> <client_id>``: internal protocol message used to coordinate file transfer ports.
+
+In `handle_command`, the server will first 
+check if the command matches any built-in commands. 
+If it dose, the `handle_command` will call the 
+functions which are defined for the commands.
+
+If a command is not recognized by the built-in handler, 
+`handle_command` will check if it matches any 
+registered custom commands from the command extension 
+API. If the command is already registered, the 
+`handle_command` will call the function defined 
+for that command. The command extension API is 
+defined as:
+
+.. code-block:: python
+
+    def register_command(
+        self: Self, command_name: Any, handler: Any,
+        where_to_run: Any, run_in_thread: Any=False) -> bool: ...
+
+The args of the `register_command` function 
+are as follows:
+
+- ``command_name``: The name of the command to register.
+
+*Note: The ``command_name`` should start with a slash 
+(e.g., ``/my_command``) to be recognized as a command.*
+
+- ``handler``: The function to call when the command is received.
+- ``where_to_run``: Specifies where the command should be executed.
+- ``run_in_thread``: A boolean indicating whether to run the command in a separate thread.
+
+This extension API allows server-side and 
+console-side custom commands to be registered 
+dynamically. Valid values for ``where_to_run`` 
+are ``"server"`` and ``"client"``. The ``"server"`` 
+means the command will be handled when a client 
+sends the command, while the ``"client"`` means 
+the command will be handled when the server input 
+the command in console.
+
+The ways to run the command will be different 
+according to the args ``run_in_thread``. If 
+``run_in_thread`` is ``True``, the command handler 
+will be executed in a separate thread from the 
+server's thread pool. If ``run_in_thread`` is 
+``False``, the command handler will be executed 
+synchronously in the main server thread.
+
+TCP Server console commands
+---------------------------
+
+The server console input thread accepts administrative commands when
+``is_input_command_in_console`` is ``True``. Supported console commands include:
+
+- ``/stop``: stops the server and closes all active connections.
+- ``/status``: prints the current connection count and running state.
+- ``/clients``: prints the connected clients and their connection times.
+- ``/send_msg <message...> <client_id1> <client_id2> ...``: sends one or more messages to specific clients.
+- ``/file <file_path> <client_id>``: sends a file from the server to a specific client.
+- ``/file_folder <folder_path> <client_id>``: sends a folder from the server to a specific client.
+- ``/multiple_file_multiple_client <file1> <file2> ... <client1> <client2> ...``: sends multiple files to multiple clients.
+- ``/diff_multiple_file_diff_multiple_client <file1> <file2> ... <client1> <client2> ...``: sends different file lists to different clients.
+- ``/help``: prints a help summary of console commands.
+
+These console commands make it easy to manage the active server and perform
+server-initiated file transfers without modifying the code.
+
+TCP Server file transfer API
+----------------------------
+
+The TCP server contains a file transfer subsystem that supports both client-to-server
+and server-to-client transfers.
+
+Client-to-server transfer flow:
+
+1. The client sends ``/file`` or ``/file_folder`` to request a transfer.
+2. ``handle_command`` starts a dedicated file-server thread using
+   ``file_transfer_server_recv_server_start_thread``.
+3. The server allocates an ephemeral transfer port with ``palloc`` and sends
+   ``/server_file_transfer_port <port> <client_id>`` back to the client.
+4. The client connects to that transfer port and sends file metadata, including
+   length-prefixed filename and file size.
+5. The server receives the file and writes it under ``received_files``.
+
+Server-to-client transfer flow:
+
+- ``file_transfer_server_recv_client_start`` is used to initiate outgoing
+  transfers from server to a connected client.
+- The server sends transfer commands to the client socket and waits for the
+  client to establish the file transfer connection.
+- Folder transfers are performed recursively, with each file transfer respecting
+  ``self.max_file_transfer_thread_num`` and the configured semaphore limit.
+
+Common file transfer helper methods include:
+
+- ``file_transfer_server_recv_server_start``: receives file data from a client.
+- ``file_transfer_server_recv_client_start``: sends a file or folder to a client.
+- ``file_transfer_mode``: performs the low-level client-side transfer handshake.
+- ``file_transfer_mode_recv``: performs the low-level receive-side transfer handshake.
+
+Port allocation API
+-------------------
+
+When ``is_hand_alloc_port`` is ``True``, the server uses manual port allocation
+and lock files to avoid conflicts across multiple server instances. The relevant
+methods are:
+
+- ``alloc_port``: allocate a port range for the server.
+- ``free_port``: release the allocated port range when the server stops.
+- ``hand_alloc_port`` and ``hand_free_port``: internal helpers used by the manual allocation flow.
+
+This mode is useful when the server must reserve a controlled range of ports
+for client-file transfers or when multiple server processes share the same host.
+
+TCP Server helper APIs
+----------------------
+
+The following helper methods are also available on ``TCP_Server_Base``:
+
+- ``broadcast(self, message, exclude_client=None)``: broadcast a message to all connected clients.
+- ``send_msg_to_specific_client(self, message)``: send a message to one or more specific clients by address.
+- ``submit_task(self, func, *args, **kwargs)``: submit work to the server's internal thread pool.
+- ``create_temporary_server(self, handler, port=None, max_connections=1)``: start a temporary TCP server for short-lived tasks.
+- ``create_temporary_client(self, server_host, server_port, bind_port=None, on_data=None)``: start a temporary client that receives data asynchronously.
+
+These APIs make it easier to extend the base TCP server for custom command handling,
+background tasks, and temporary connections.

src/command_control_extension_tcp.py

@@ -1,6 +1,7 @@
 import os
 import ast
 import json
+import copy
 import shlex
 import shutil
 import traceback
@@ -10,7 +11,7 @@
 from datetime import datetime
 server_instance=None
 client_instance=None
-command_counter=0
+command_counter={}
 command_counter_lock=threading.Lock()
 def _setup_command():
     print("Setting up server command...")
@@ -57,15 +58,19 @@ def _command_handler(sock, addr, cmd):
         clients_num=0
         clients_list=[]
         commands_list=[]
+    client_id=0
     for pair in command_client_pair:
         for msg in pair[0]:
-            command_msg="/command"+" "+shlex.quote(msg)+" "
+            command_msg=("/command"+" "+shlex.quote(msg)+" "+
+                         shlex.quote(str(len(pair[0])))+" ")
             for client in pair[1]:
-                temp_msg=command_msg+shlex.quote(str(client))+"\n"
+                temp_msg=(command_msg+shlex.quote(str(client_id))+
+                          " "+shlex.quote(str(client))+"\n")
                 client_socket=client_class[client]["socket"]
                 server_instance.send_message(
                     client_socket=client_socket, message=temp_msg)
                 print(f"Sending command to clients: {command_msg}")
+            client_id+=1
 def _command_handler_server_setup(sock, addr, cmd):
     global command_counter
     print(f"Received command from {addr}: {cmd}")    
@@ -78,14 +83,19 @@ def _command_handler_server_setup(sock, addr, cmd):
         print("Invalid command format.")
         return
     command = cmd_parts[1]
-    client_addr = cmd_parts[2]
-    with command_counter_lock:
-        command_counter += 1
-        cmd_id = command_counter
+    client_addr = cmd_parts[4]
+    command_total_num=int(cmd_parts[2])
+    client_id=cmd_parts[3]
     log_dir = os.path.join(os.path.dirname(__file__), 'logs')
     if not os.path.exists(log_dir):
         os.makedirs(log_dir)
-    log_filename = f"{cmd_id}.json"
+    with command_counter_lock:
+        if str(client_id) not in command_counter:
+            command_counter[str(client_id)]=1
+        else:
+            command_counter[str(client_id)] += 1
+        cmd_id = copy.copy(command_counter[str(client_id)])
+    log_filename = "logs{}.json".format("_"+str(client_id))
     log_path = os.path.join(log_dir, log_filename)
     try:
         result = subprocess.run(
@@ -103,6 +113,8 @@ def _command_handler_server_setup(sock, addr, cmd):
     log_line = {
         "timestamp": datetime.now().isoformat(),
         "command": command,
+        "cmd_id": str(cmd_id),
+        "client_id": str(client_id),
         "client": client_addr,
         "from": str(addr),
         "output": output,
@@ -123,28 +135,34 @@ def _command_handler_server_setup(sock, addr, cmd):
         json.dump(log_data, f, ensure_ascii=False, indent=2)
     print(f"Log written to {log_path}")
     msg="/file \"{}\"".format(log_path)
-    client_instance.file_transfer_client_recv_client_start(
-        message=msg, file_folder_abspath=None)
-    client_instance.send_message(
-        client_socket=client_instance.client_socket,
-        message="/command_done \"{}\"".format(log_filename))
+    if cmd_id==command_total_num:
+        command_counter[str(client_id)]=0
+        client_instance.file_transfer_client_recv_client_start(
+            message=msg, file_folder_abspath=None)
+        client_instance.send_message(
+            client_socket=client_instance.client_socket,
+            message="/command_done \"{}\" \"{}\"".format(log_filename, log_path))
+    else:
+        pass
     print("Dealing the command seccessfully!")
 def _command_done_dealing_server(sock, addr, cmd):
-    log_filename=shlex.split(cmd)[1]
+    cmd_parts = shlex.split(cmd)
+    log_filename = cmd_parts[1]
+    log_path = cmd_parts[2]
     log_dir=os.path.join(os.path.dirname(__file__), 'logs')
     if os.path.exists(log_dir):
         pass
     else:
         os.mkdir(log_dir)
     received_log_file=os.path.join(
-        server_instance.file_transfer_dir, log_filename)
+        server_instance.file_transfer_dir, log_path)
     try:
         shutil.move(
             received_log_file, os.path.join(log_dir, log_filename))
         print("Command Done!")
     except:
         traceback.print_exc()
-        print("ErrorWhileMovingTheLogFile: moving log file faled.")
+        print("ErrorWhileMovingTheLogFile: moving log file failed.")
 def client_setup():
     global client_instance
     client_instance=connect_tcp.TCP_Client_Base(

src/connect_tcp.py

@@ -496,7 +496,7 @@ def handle_command(self, client_socket, client_address, command):  # deal with s
                 run_in_thread = self._custom_handler_threaded[0].get(
                     cmd_name, False)
                 if run_in_thread:
-                    self._custom_executor.submit(
+                    self.submit_task(
                         self._execute_custom_handler, handler, command,
                         client_socket, client_address)
                     return "Command received, processing in background.\n"
@@ -1194,7 +1194,7 @@ def console_input(self):  # deal consule input
                         run_in_thread = self._custom_handler_threaded[1].get(
                             cmd_name, False)
                         if run_in_thread:
-                            self._custom_executor.submit(
+                            self.submit_task(
                                 self._execute_custom_handler, handler, deal_cmd)
                             pass
                         else: