Web crawler tool

[bluera-ai]

Overview
Introduce a Universal Tool allowing to crawl the web locally using crawl4ai.

Requirements

• The crawler should be reliable, targeted and its output should be cleaned up by default.
• Tests must prove compatibility for all of all changes.