TECH_DOCU.MD

Technical Documentation - Inputs Emulator

1. Overview

This document provides a technical overview of the Inputs Emulator project, a macOS command-line application designed to monitor, record, save, load, and simulate keyboard and mouse events.

The application leverages macOS Core Graphics and Application Services frameworks for event tapping and simulation.

2. Architecture

The application follows a modular design:

• Main Application (main.cpp): Handles user interaction via a command-line menu, orchestrates the overall workflow, manages the lifecycle of the InputMonitor, and initiates saving, loading, and simulation processes.
• Input Monitoring (InputMonitor): Responsible for capturing system-wide keyboard and mouse events using a Core Graphics Event Tap (CGEventTapCreate). It runs the event tap on a dedicated CFRunLoop managed within a separate std::thread.
• Event Logging (InputLogger): Stores captured events in memory (std::vector<std::unique_ptr<Event>>). Provides functionalities for accessing, sorting, clearing, and managing these events. It collaborates with FileLogger for persistence.
• File Persistence (FileLogger): Handles serialization and deserialization of event data to/from files. Currently uses a JSON format for storing event sequences.
• Event Data Structures (Event, KeyEvent, MouseEvent): Define the structure for storing individual input events, including timestamps, event types, and event-specific data (key codes, mouse coordinates, button states, scroll deltas).
• Simulation (Simulation, Keyboard, Mouse): Takes a sequence of Event objects and simulates them programmatically. The Simulation class manages the timing and dispatching of events to the Keyboard and Mouse classes, which encapsulate the low-level logic for creating (CGEventCreate...) and posting (CGEventPost) synthetic events.
• Utilities (Util): Provides helper functions, such as time calculations and cross-platform sleep functionality.

Data Flow

1. Monitoring: InputMonitor creates an event tap -> macOS sends events to InputMonitor::eventCallback.
2. Callback Processing: eventCallback determines event type -> Creates KeyEvent or MouseEvent -> Logs the event via InputLogger.
3. Storage: InputLogger adds the std::unique_ptr<Event> to its internal vector.
4. Saving: main triggers save -> InputLogger provides event data -> FileLogger serializes events to a JSON file.
5. Loading: main triggers load -> FileLogger deserializes events from JSON file -> InputLogger stores the loaded events (potentially clearing existing ones).
6. Simulation: main triggers simulation -> Simulation class receives event data from InputLogger -> Simulation calculates delays and calls methods on Keyboard/Mouse -> Keyboard/Mouse create and post CGEvents.

3. Core Components

3.1. InputMonitor

• Event Tapping: Uses CGEventTapCreate to install an event tap listening for key down/up, mouse down/up/move, and scroll wheel events.
• Threading: The CFRunLoop required by the event tap is run on a dedicated std::thread (monitorThread) created in the start() method. This prevents blocking the main application thread.
• Callback (eventCallback): A static C-style function passed to CGEventTapCreate. It receives event details, reconstructs the InputMonitor instance pointer (refcon), creates the appropriate Event subclass object, and logs it using the associated InputLogger.
• State Management: Uses std::atomic<bool> isMonitoring to track the running state. This is crucial for thread safety between the main thread and the monitor thread.
• Stopping Mechanism: The stop() method signals the CFRunLoop to stop (CFRunLoopStop) using an atomic flag (stopRequested) and CFRunLoopPerformBlock. It then waits for the monitorThread to join, ensuring resources are cleaned up properly. compare_exchange_strong is used on stopRequested to prevent multiple stop attempts.
• Escape Key Handling: The eventCallback now checks for the Escape key (kVK_Escape) press and calls monitor->stop() directly, simplifying the logic in main.cpp.
• Throttling: Basic mouse move event throttling is implemented in eventCallback to reduce the number of logged move events based on distance and time intervals.

3.2. InputLogger

• Storage: Employs std::vector<std::unique_ptr<Event>> events to store event objects polymorphically.
• Management: Provides methods like logKeyEvent, logMouseEvent, getAllEvents, clear, sortAllEvents (using std::sort with a custom timestamp comparator).
• Persistence Interface: Uses FileLogger for saving (saveAllEvents) and loading (logAllEventsFromFile, which takes ownership of the loaded events vector).

3.3. FileLogger

• Format: Serializes the events vector into a JSON format. Each event object is converted into a JSON object containing its type and relevant fields (timestamp, keyCode, isKeyDown, eventState, button, position, scrollUnit).
• Library: Uses the nlohmann/json library (included via third_party) for JSON parsing and serialization.
• I/O: Uses std::ofstream for writing and std::ifstream for reading.

3.4. Event Hierarchy

• Event (Base): Abstract base class defining common properties like timestamp and eventType, and virtual methods like getEventTypeString, toJSON.
• KeyEvent: Derived from Event. Stores keyCode (CGKeyCode), isKeyDown (bool), and potentially keyString.
• MouseEvent: Derived from Event. Stores eventState (enum: PRESSED, RELEASED, MOVED, SCROLLED), button (int, relevant for clicks), position (CGPoint), scrollUnit (CGVector, relevant for scrolls).

3.5. Simulation, Keyboard, Mouse

• Simulation::runSimulation: Iterates through the provided event vector. Calculates the delay between consecutive events based on their timestamps and uses std::this_thread::sleep_for to pause execution, simulating the original timing.
• Event Dispatch: Calls appropriate methods on Keyboard or Mouse instances based on the event type.
• Keyboard/Mouse: These classes contain methods (keyDown, keyUp, mousePress, mouseRelease, moveTo, scroll, etc.) that:
  • Create the corresponding CGEvent using functions like CGEventCreateKeyboardEvent, CGEventCreateMouseEvent, CGEventCreateScrollWheelEvent.
  • Important: Include NULL checks after event creation calls to handle potential system failures gracefully.
  • Post the created event to the system event stream using CGEventPost(kCGHIDEventTap, event). Event taps like kCGHIDEventTap allow posting low-level events.
  • Manage event resources (e.g., CFRelease(event)).

4. Build System (makefile)

• Uses g++ as the compiler.
• Sets CXXFLAGS to -std=c++14 -Wall and includes header directories (-Iinclude, -Ithird_party).
• Links against the required macOS framework: -framework ApplicationServices.
• Uses find to locate source files (.cpp) recursively.
• Defines separate directories for object files (obj/) and the final executable (debug/ or potentially release/).
• Includes targets for all, clean, and directories.

5. Key Design Decisions & Considerations

• Event Tap Threading: Using a separate thread and CFRunLoop for the event tap is essential for responsiveness and preventing blocking the main UI thread.
• Atomic Flags: std::atomic<bool> is used for isMonitoring and stopRequested to ensure thread-safe communication between the main thread and the monitor thread.
• std::unique_ptr: Used extensively for managing the lifetime of Event objects in the InputLogger and during file loading, preventing memory leaks.
• Event Simulation API: CGEventPost with kCGHIDEventTap provides a robust way to inject low-level input events.
• Error Handling: Added NULL checks after CGEventCreate... calls to prevent crashes if the system fails to create an event. Basic exception handling is present in file I/O and simulation loops.
• JSON Persistence: JSON provides a human-readable and relatively easy-to-parse format for saving/loading events.

6. Future Enhancements / Areas for Improvement

• More robust error handling throughout the application.
• Support for simulating modifier key combinations more explicitly.
• Refined simulation timing (potential inaccuracies in sleep_for).
• Support for different file formats (e.g., binary for efficiency).
• More sophisticated event filtering or transformation options.