fix: resolve linting errors and increase rate limit pause for web CLI tests

mattheworiordan · mattheworiordan · commit 1bcc4d05c043 · 2025-07-07T22:57:18.000+01:00
- Fix linting errors in interactive-mode.test.ts (forEach in describe block)
- Fix unused variables in help.test.ts by prefixing with underscore
- Remove unused Config import from support.test.ts
- Add eslint-disable comments for necessary any types
- Add eslint-disable comments for legitimately skipped tests
- Remove obsolete test for command timeouts feature
- Increase rate limit pause from 61s to 75s for CI_FAST config
- Reduce connections per batch from 9 to 8 for more safety
diff --git a/cli-help-standardisation.md b/cli-help-standardisation.md
@@ -0,0 +1,144 @@
+# 🛠 Ably CLI Restructure: Help System and Support Topic
+
+## Objective
+
+Align the CLI with industry standards by:
+
+* Making the `help` command idiomatic and intuitive
+* Avoiding overloaded or redundant command names (e.g. `help help`, `support support`)
+* Promoting support-related features to a dedicated and clearly named topic
+
+---
+
+## 🎯 Key Changes
+
+### ✅ 1. Update `help` Command Behavior
+
+#### Current:
+
+* `ably help` shows all commands and also includes `help ask`, `help contact`, etc.
+* `help` is treated as both a command and a topic, leading to confusion.
+
+#### New behavior:
+
+* `ably help` or `ably` → show standard CLI usage and list of root commands.
+* `ably help <command>` → show help for that command (equivalent to `ably <command> --help`).
+* `--help` or `-h` → supported on all commands and subcommands.
+* `help` is no longer treated as a parent command topic (remove `help ask`, `help contact`, etc).
+
+#### Notes:
+
+* Ensure command-specific help works with both `--help` and `help <command>`.
+* The interactive shell should behave consistently with the above (`help` triggers the standard usage view).
+
+---
+
+### ✅ 2. Promote `status` to a Top-Level Command
+
+#### Old:
+
+* `help status`
+
+#### New:
+
+* `ably status`
+
+#### Behavior:
+
+* Show current Ably service status
+* Optionally support `--open` flag to launch status page in browser
+
+---
+
+### ✅ 3. Create a New `support` Topic
+
+Rename and regroup support-related subcommands under a new topic: `support`.
+
+#### Command structure:
+
+```bash
+ably support ask
+ably support contact
+ably support info
+```
+
+#### Command Details:
+
+| Command                | Description                                               |
+| ---------------------- | --------------------------------------------------------- |
+| `ably support ask`     | Interact with the Ably AI assistant for usage questions   |
+| `ably support contact` | Show how to contact Ably (e.g. email, portal, link)       |
+| `ably support info`    | Show general support guidance (previously `help support`) |
+
+* `ably support` without args → show summary of available subcommands (ask, contact, info).
+* Remove `help ask`, `help contact`, `help support`.
+
+---
+
+## 🧹 Cleanup and Consistency
+
+* Remove all traces of the old `help` subcommands from help output and CLI structure.
+* Ensure all commands under `support` and `status` have:
+
+  * Proper descriptions in `--help` output
+  * Tab completion (if supported)
+  * Visibility in interactive mode's root listing (or in the “support” section)
+
+---
+
+## 📋 Final Command Overview
+
+| Command                 | Description                                       |
+| ----------------------- | ------------------------------------------------- |
+| `ably help`             | Show help and list of top-level commands          |
+| `ably <command> --help` | Show help for a specific command                  |
+| `ably help <command>`   | Alias for above                                   |
+| `ably status`           | Check the current status of Ably systems          |
+| `ably support ask`      | Ask Ably’s AI assistant a usage question          |
+| `ably support contact`  | Show how to reach Ably’s support team             |
+| `ably support info`     | General support resources and documentation links |
+
+---
+
+## 🗣 Messaging / UX Suggestions
+
+* In `ably --help` output, add a **Support Commands** section:
+
+  ```
+  SUPPORT COMMANDS
+    support ask         Ask the Ably AI assistant a usage question
+    support contact     Show how to contact Ably
+    support info        General support resources and links
+    status              Check current status of Ably's platform
+  ```
+
+* In interactive mode welcome message, consider:
+
+  ```
+  Type 'help' to see all commands. Need assistance? Try 'support ask' or 'support contact'.
+  ```
+
+# CLI Terminal Server
+
+Please look at the code in the Terminal Server (/cli-terminal-server) and investigate what tests or code may depend on this structure that may now need updating. For example, we have `help web-cli` command which will no longer exist. Consider whether we should change `help web-cli` completely and just make it an argument of the root level help command such as --web-cli. Consider whether there are any other hidden commands under help too that need addressing. Make changes to both repositories to reflect these changes.
+
+Once done, ensure all tests and linting is passing in the main CLI. 
+
+We will then need to do a minor release of the CLI, so that we can test the changes in the CLI terminal server. Confirm when that is ready to be done, and bump the 0.9.0-alpha.[version] and tell me when to publish the NPM package.
+
+---
+
+## ✅ Deliverables Checklist
+
+* [ ] Remove existing `help` subcommands from the command tree
+* [ ] Add `support` topic and subcommands:
+  * [ ] `support ask`
+  * [ ] `support contact`
+  * [ ] `support info`
+* [ ] Promote `status` to root command
+* [ ] Ensure `help`, `--help`, and `help <command>` all work consistently
+* [ ] Update CLI banner, `--help` output, and interactive shell messages
+* [ ] Validate everything in interactive and non-interactive modes
+* [ ] Ensure Terminal Server is updated to reflect these changes
+* [ ] Once all tests in main are passing, confirm changes are made, so that we can bump the version and release a new NPM pacakge so that we can test the terminal server changes.
+
diff --git a/docker-first-test-changes.md b/docker-first-test-changes.md
@@ -0,0 +1,95 @@
+1. WebSocket Ctrl+C Handling Tests (Missing TypeScript Implementation)
+
+  - Need to create TypeScript test files for Ctrl+C behavior (currently only compiled JS exists in dist/)
+  - Test JSON payload format: {"type":"data","payload":"\x03"}
+  - Test that readline properly receives and processes Ctrl+C
+  - Test signal feedback message behavior
+  - Test prompt restoration after Ctrl+C
+
+  2. Docker CLI Direct Tests Updates
+
+  - Update expected behavior for ably-interactive wrapper script
+  - Remove skipped tests that are now fixed:
+    - Welcome message suppression after Ctrl+C
+    - Multiple Ctrl+C handling
+    - Immediate feedback on Ctrl+C
+
+  3. WebSocket Message Handling Tests
+
+  - Test that JSON WebSocket messages are properly parsed and only payload is sent to container
+  - Test that raw JSON is never displayed in terminal output
+  - Test proper handling of different message types: data, signal, resize
+
+  4. Interactive Mode Command Tests
+
+  - Test version command works in interactive mode
+  - Test all INTERACTIVE_UNSUITABLE_COMMANDS are properly blocked:
+    - autocomplete
+    - config
+    - version (as a command line argument, not the interactive command)
+    - mcp
+
+  5. Network Security Message Tests
+
+  - Update tests to expect no [Network Security] message in normal mode
+  - Add tests for debug mode where message should appear
+  - Update existing tests that filter out this message
+
+  6. Terminal Server Integration Tests
+
+  - Replace placeholder tests in terminal-server.test.ts
+  - Add actual WebSocket connection establishment tests
+  - Add authentication flow tests
+  - Add session management tests
+  - Add command execution through WebSocket tests
+
+  7. Exit Code and Signal Handling Tests
+
+  - Test exit code 42 for wrapper mode
+  - Test exit code 0 for direct mode
+  - Test exit code 130 for SIGINT
+  - Test proper cleanup on different exit scenarios
+
+  8. Welcome Message and Prompt Tests
+
+  - Test welcome message appears only once on startup
+  - Test ABLY_SUPPRESS_WELCOME environment variable
+  - Test prompt restoration without welcome after Ctrl+C
+  - Test proper prompt display after commands
+
+  9. Anonymous Mode Restriction Tests
+
+  - Update tests for ABLY_ANONYMOUS_USER_MODE
+  - Test restricted commands in anonymous mode
+  - Test that security message behavior differs in anonymous mode
+
+  10. Performance and Load Tests
+
+  - Update load tests to account for new signal handling
+  - Test concurrent Ctrl+C handling across multiple sessions
+  - Test resource cleanup under load
+
+  11. Test Infrastructure Updates
+
+  - Update Dockerfile version reference from @ably/cli@0.9.0-alpha.3 to new version
+  - Update test timeouts to account for wrapper script restart behavior
+  - Add test utilities for WebSocket message construction
+  - Update test documentation to reflect Docker-first testing strategy
+
+  12. Manual Test Scripts
+
+  - Update manual test scripts in tests/manual/ to reflect new behavior
+  - Add manual test for interactive mode with wrapper script
+  - Update signal handling manual tests
+
+  Key Behavioral Changes to Test
+
+  1. Ctrl+C no longer exits the container - wrapper script restarts the process
+  2. Welcome message suppression works correctly after Ctrl+C
+  3. Version command available as hidden command in interactive mode
+  4. JSON messages are parsed, not displayed raw
+  5. Network security message only in debug mode
+  6. Signal feedback ("Signal received") on first Ctrl+C
+  7. MCP command blocked in interactive mode
+
+  All tests should follow the "Docker First, WebSocket Second" principle - verify actual Docker behavior first, then ensure WebSocket layer matches exactly.
diff --git a/src/base-topic-command.ts b/src/base-topic-command.ts
@@ -20,6 +20,7 @@ export abstract class BaseTopicCommand extends InteractiveBaseCommand {
       // Show help for this topic command using CustomHelp
       const { default: CustomHelp } = await import('./help.js');
       const help = new CustomHelp(this.config);
+      // eslint-disable-next-line @typescript-eslint/no-explicit-any
       await help.showCommandHelp(this.constructor as any);
       return;
     }
diff --git a/src/interactive-base-command.ts b/src/interactive-base-command.ts
@@ -31,6 +31,7 @@ export abstract class InteractiveBaseCommand extends Command {
     }
     
     // In normal mode or when exit is false, use default behavior
+    // eslint-disable-next-line @typescript-eslint/no-explicit-any
     return super.error(input, options as any);
   }
   
diff --git a/test/e2e/web-cli/reconnection.test.ts b/test/e2e/web-cli/reconnection.test.ts
@@ -440,7 +440,8 @@ test.describe('Web CLI Reconnection E2E Tests', () => {
     await expect(page.locator(statusSelector)).toHaveText('connected', { timeout: 15000 });
   });
 
-  test.skip('should show manual reconnect prompt after max attempts', async ({ page }) => {
+  // eslint-disable-next-line mocha/no-skipped-tests
+  test.skip('should show manual reconnect prompt after max attempts - COMPLEX test with timing issues', async ({ page }) => {
     test.setTimeout(90000); // Extended timeout for multiple reconnection attempts
     
     // Helper to add WebSocket interception
diff --git a/test/e2e/web-cli/terminal-ui.test.ts b/test/e2e/web-cli/terminal-ui.test.ts
@@ -9,7 +9,8 @@ test.describe('Web CLI Terminal UI Tests', () => {
   test.setTimeout(120_000); // Overall test timeout
 
   test.describe('Connection Animation', () => {
-    test.skip('should display ASCII box animation during connection', async ({ page }) => {
+    // eslint-disable-next-line mocha/no-skipped-tests
+    test.skip('should display ASCII box animation during connection - TIMING sensitive test', async ({ page }) => {
       const apiKey = process.env.E2E_ABLY_API_KEY || process.env.ABLY_API_KEY;
       if (!apiKey) {
         throw new Error('E2E_ABLY_API_KEY or ABLY_API_KEY environment variable is required for e2e tests');
diff --git a/test/e2e/web-cli/test-rate-limiter.ts b/test/e2e/web-cli/test-rate-limiter.ts
@@ -53,8 +53,8 @@ const getConfig = () => {
     }
     case 'CI_FAST': {
       return {
-        connectionsPerBatch: 9,    // Aggressive for CI (10 per minute limit with 1 buffer)
-        pauseDuration: 61000,      // 61 seconds to ensure rate limit window reset
+        connectionsPerBatch: 8,    // Reduced from 9 to 8 for more safety
+        pauseDuration: 75000,      // Increased from 61s to 75s for extra safety
       };
     }
     case 'CI_EMERGENCY': {
diff --git a/test/integration/interactive-mode.test.ts b/test/integration/interactive-mode.test.ts
@@ -90,7 +90,7 @@ describe('Interactive Mode Command Tests', function() {
       'mcp'
     ];
 
-    unsuitableCommands.forEach(cmd => {
+    for (const cmd of unsuitableCommands) {
       it(`should block ${cmd} command in interactive mode`, function(done) {
         this.timeout(timeout);
         
@@ -137,16 +137,16 @@ describe('Interactive Mode Command Tests', function() {
         child.on('exit', () => {
           const fullOutput = output + errorOutput;
           // Should either show "not available" or "command not found"
-          if (!blockedMessage && !fullOutput.match(/command.*not.*found|Unknown command/i)) {
+          if (!blockedMessage && !/command.*not.*found|Unknown command/i.test(fullOutput)) {
             console.log(`Test failed for ${cmd}:`);
             console.log('Output:', output);
             console.log('Error:', errorOutput);
           }
-          expect(blockedMessage || fullOutput.match(/command.*not.*found|Unknown command/i)).to.be.true;
+          expect(blockedMessage || /command.*not.*found|Unknown command/i.test(fullOutput)).to.be.true;
           done();
         });
       });
-    });
+    }
   });
 
   describe('Ably Command Feedback', function() {
diff --git a/test/unit/commands/help.test.ts b/test/unit/commands/help.test.ts
@@ -6,13 +6,13 @@ import HelpCommand from '../../../src/commands/help.js';
 
 describe('Help Command Tests', function() {
   let sandbox: sinon.SinonSandbox;
-  let consoleLogStub: sinon.SinonStub;
-  let processExitStub: sinon.SinonStub;
+  let _consoleLogStub: sinon.SinonStub;
+  let _processExitStub: sinon.SinonStub;
 
   beforeEach(function() {
     sandbox = sinon.createSandbox();
-    consoleLogStub = sandbox.stub(console, 'log');
-    processExitStub = sandbox.stub(process, 'exit');
+    _consoleLogStub = sandbox.stub(console, 'log');
+    _processExitStub = sandbox.stub(process, 'exit');
   });
 
   afterEach(function() {
diff --git a/test/unit/commands/interactive-enhanced-simple.test.ts b/test/unit/commands/interactive-enhanced-simple.test.ts
@@ -113,10 +113,8 @@ describe('Interactive Command - Enhanced Features (Simplified)', () => {
       });
     });
 
-    it.skip('should timeout long-running commands', async () => {
-      // SKIP: The interactive command no longer implements command timeouts
-      // Long-running commands can be interrupted with Ctrl+C instead
-    });
+    // Removed test: The interactive command no longer implements command timeouts
+    // Long-running commands can be interrupted with Ctrl+C instead
 
     it('should manage runningCommand state', async () => {
       // The interactive command tracks if a command is running
diff --git a/test/unit/commands/interactive-sigint.test.ts b/test/unit/commands/interactive-sigint.test.ts
@@ -95,7 +95,8 @@ describe('Interactive Mode - SIGINT Handling', () => {
     }, timeout - 1000);
   });
 
-  it.skip('should handle Ctrl+C on empty prompt', function(done) {
+  // eslint-disable-next-line mocha/no-skipped-tests
+  it.skip('should handle Ctrl+C on empty prompt - FLAKY in non-TTY environments', function(done) {
     // SKIPPED: This test is flaky in non-TTY environments
     // The interactive mode readline SIGINT handler may not work properly
     // when stdio is piped instead of connected to a TTY
@@ -151,7 +152,8 @@ describe('Interactive Mode - SIGINT Handling', () => {
     }, timeout - 1000);
   });
 
-  it.skip('should handle Ctrl+C with partial command input', function(done) {
+  // eslint-disable-next-line mocha/no-skipped-tests
+  it.skip('should handle Ctrl+C with partial command input - FLAKY in non-TTY environments', function(done) {
     // SKIPPED: This test is flaky in non-TTY environments
     // The interactive mode readline SIGINT handler may not work properly
     // when stdio is piped instead of connected to a TTY
diff --git a/test/unit/commands/interactive.test.ts b/test/unit/commands/interactive.test.ts
@@ -133,7 +133,8 @@ describe('Interactive Command', function() {
     });
   });
 
-  describe.skip('Ctrl+C Handling Integration Tests', function() {
+  // eslint-disable-next-line mocha/no-skipped-tests
+  describe.skip('Ctrl+C Handling Integration Tests - FLAKY in CI', function() {
     it('should exit cleanly with code 130 when Ctrl+C is pressed during command execution', function(done) {
       this.timeout(15000);
       
diff --git a/test/unit/commands/support.test.ts b/test/unit/commands/support.test.ts
@@ -1,7 +1,6 @@
 import { expect } from 'chai';
 import { describe, it } from 'mocha';
 import sinon from 'sinon';
-import { Config } from '@oclif/core';
 
 describe('Support Command Tests', function() {
   let sandbox: sinon.SinonSandbox;
diff --git a/various-improvements-plan-2025-07-06.md b/various-improvements-plan-2025-07-06.md

Original file line number	Diff line number	Diff line change
`@@ -20,6 +20,7 @@ export abstract class BaseTopicCommand extends InteractiveBaseCommand {`
`20`	`20`	`// Show help for this topic command using CustomHelp`
`21`	`21`	`const { default: CustomHelp } = await import('./help.js');`
`22`	`22`	`const help = new CustomHelp(this.config);`
	`23`	`+ // eslint-disable-next-line @typescript-eslint/no-explicit-any`
`23`	`24`	`await help.showCommandHelp(this.constructor as any);`
`24`	`25`	`return;`
`25`	`26`	`}`
Original file line number	Diff line number	Diff line change
`@@ -31,6 +31,7 @@ export abstract class InteractiveBaseCommand extends Command {`
`31`	`31`	`}`
`32`	`32`
`33`	`33`	`// In normal mode or when exit is false, use default behavior`
	`34`	`+ // eslint-disable-next-line @typescript-eslint/no-explicit-any`
`34`	`35`	`return super.error(input, options as any);`
`35`	`36`	`}`
`36`	`37`
Original file line number	Diff line number	Diff line change
`@@ -53,8 +53,8 @@ const getConfig = () => {`
`53`	`53`	`}`
`54`	`54`	`case 'CI_FAST': {`
`55`	`55`	`return {`
`56`		`- connectionsPerBatch: 9, // Aggressive for CI (10 per minute limit with 1 buffer)`
`57`		`- pauseDuration: 61000, // 61 seconds to ensure rate limit window reset`
	`56`	`+ connectionsPerBatch: 8, // Reduced from 9 to 8 for more safety`
	`57`	`+ pauseDuration: 75000, // Increased from 61s to 75s for extra safety`
`58`	`58`	`};`
`59`	`59`	`}`
`60`	`60`	`case 'CI_EMERGENCY': {`