Accessibility Scanner – Update Notes Version 4.0.2

High-Impact Architectural Improvements

Fixed a small bug before it burst out of control.

• Core Engine Refactor for Stability: The previous architecture, which utilised separate clusters for crawling and scanning, has been completely refactored. The new, streamlined model integrates scanning directly into the crawl process, eliminating the resource-intensive second cluster. This change significantly reduces memory overhead, simplifies parallel processing logic, improves reliability during long scans, and prevents crashes related to complex cluster management.

• Advanced Anti-Bot & Adaptive Remediation Engine: An Adaptive Remediation Engine has been introduced to actively combat bot-blocking measures in real-time. When the scanner detects timeouts or blocking status codes (e.g., 403, 429), it automatically escalates its strategy through multiple levels:
  • Level 1: Increases the crawl delay.
  • Level 2: Enables Basic Stealth Mode to mask the automated browser.
  • Level 3: Activates Enhanced Stealth Mode with advanced browser fingerprint spoofing. This is now more dynamic, randomizing hardware details like hardwareConcurrency and deviceMemory to better mimic real user profiles.
  • This allows the scanner to successfully navigate and analyze sites that were previously inaccessible.

• Critical Timeout and Navigation Fixes:
  • Crawl Page Timeout: The default timeout for loading a page during a crawl has been doubled to 60,000 ms, making the scanner more resilient when analyzing slow-loading or media-heavy websites.
  • Navigation Strategy: The default page load wait condition now primarily uses the more resilient networkidle2, preventing premature timeouts on modern web applications that have persistent background network activity (e.g., analytics, live chat widgets).

New Features and Intelligence Enhancements

• NEW: Interaction Scanning (Deep Scan): This optional mode goes beyond static page loads by simulating user interaction to find issues in dynamic content. After the initial page scan, it automatically finds and clicks on interactive elements like buttons, tabs, and accordions. It then re-scans the page state after each interaction to uncover accessibility issues hidden within components that are only revealed dynamically. This provides a much more thorough analysis but may significantly increase total scan time.
• NEW: Smart Crawl & Trap Avoidance: A brand-new "Smart Crawl" feature prevents the scanner from getting stuck in common "crawler traps." It intelligently analyzes URL structure and path depth to identify and avoid patterns like infinite calendar pages, complex URL filters, and repetitive path segments, saving significant time and resources.
• Proactive Bot-Blocker Detection: The scanner now proactively inspects page content and titles for keywords indicating bot-blocking mechanisms (e.g., CAPTCHAs, JS challenges, "Access Denied" screens). This triggers the Adaptive Remediation Engine instantly, rather than waiting for a network error to occur.
• Smarter Task Retries: The executeWithRetry function is now more intelligent. It categorizes different error types(e.g., network timeouts vs. bot-blocking status codes) and adjusts its retry delay and strategy accordingly, making the entire process more resilient and adaptive.

User Experience & Feedback

• Enhanced Real-Time Frontend Feedback: The scanner now pushes a wealth of detailed status messages directly to the user interface. Users have a live, "play-by-play" view of the scan, including adaptive actions being taken, retry attempts, and smart crawl decisions, providing complete transparency into the scanner's operations.
• Smarter Out-of-the-Box Defaults: To ensure a better experience for first-time users and provide a powerful baseline for all scans, key features like Smart Scan and Adaptive Remediation are now enabled by default.

Code, Stability, and Usability Refinements

General Stability and Reliability:

• Robust Process Cleanup: The application's shutdown process on a critical failure has been hardened. It now uses execSync to forcefully and reliably terminate any orphaned browser processes, preventing system resource locks and ensuring a clean state for the next run.
• Race Condition Prevention: The screenshot naming process has been fortified by adding a unique, high-resolution timestamp and a random hash to each filename, preventing file-writing collisions during high-concurrency scans.
• Deadlock Prevention: Timeouts have been added to the pause/resume operations to ensure the server remains responsive and does not get stuck waiting for a cluster that fails to respond.
• Memory Leak Mitigation: The logic for parallel tasks now ensures that all browser pages are reliably closed, even if an error occurs within the task, preventing memory leaks during large scans.
• Streamlined Startup Process: The experimental update_npm runtime command has been completely removed, resulting in a cleaner, more stable, and more secure server startup process.
• Enhanced Screenshot Logging: Progress and error logging for screenshot captures have been made more descriptive, making it easier to diagnose why a specific screenshot might have failed to generate.