Panic parsing Google 4th of July homepage

[MorganBennetDev]

Using blitz 0.2.1, blitz_html panics when attempting to parse the Google homepage with the message:

thread 'main' (179535) panicked at ~/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/blitz-dom-0.2.4/src/document.rs:550:13:
to be able to resolve /web/20260703180215cs_/https://www.google.com/xjs/_/ss/k=xjs.hd.jILEt-ueYLk.L.B1.O/am=AAAEQAAAAAAAAAAAAAgAAAAAAAAAAAAAAAAAAAAAAAAAAQAAgEABAIAMAAAAAAAIAACgBhEAgAAAAAAAAIQHAAAMAAAAAAAAAIAAAAAAAIAAAIAAAAAAAAAABAAAAAAAAAAAQAAAAAAAAAEAAigAAIAABQAAAAAAgAAAAAAAAAAAAAICAAAAAAAAAAAAgAAAAAAAAAAYAMEAIAAAAAAgDAAAAAAAAAAAAAAAAAAAAAAAiARAAAAEAAAAAAABAAAAAAABAIQQBAABCiAAAAAAAACABQAAAAAdQAAEBAAAAAAAAAACAEEgBBAAAAAgAAAABAABAAEAAQEARtAAACQqEAAAogMAAAAIAAAAAIAAAAAAAAAAAAAAQAQAAAAAAACwAABDQCAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAQAE/d=1/ed=1/rs=ACT90oHwnyY8rZ_swysHVUPbUAsB_rKaAw/m=cdos,hsm,jsa,mb4ZUb,cEt90b,SNUn3,qddgKe,sTsDMc,dtl0hd,eHDfl,YV5bee,d,csi?cb=121509378 with the base_url: Url { scheme: "data", cannot_be_a_base: true, username: "", password: None, host: None, port: None, path: "text/css;charset=utf-8;base64,", query: None, fragment: None }

It also produces the same panic on the archive.org snapshot. I've included the code I used to produce this panic below.

Details

use blitz::dom::net::Resource;
use blitz::dom::DocumentConfig;
use blitz::html::HtmlDocument;
use blitz::shell::{create_default_event_loop, BlitzShellEvent, BlitzShellNetCallback};
use blitz::traits::net::Request;
use std::sync::Arc;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    tracing_subscriber::fmt::init();

    let event_loop = create_default_event_loop::<BlitzShellEvent>();
    let net_provider = create_net_provider(&event_loop);
    let (_, bytes) = net_provider
        .fetch_async(Request::get(url::Url::parse("https://www.google.com")?))
        .await
        .unwrap();
    let html = std::str::from_utf8(&bytes)?;

    let doc = HtmlDocument::from_html(html, DocumentConfig::default());

    Ok(())
}

type EnabledNetProvider = blitz::net::Provider<Resource>;

fn create_net_provider(
    event_loop: &blitz::shell::EventLoop<BlitzShellEvent>,
) -> Arc<EnabledNetProvider> {
    let proxy = event_loop.create_proxy();
    let callback = BlitzShellNetCallback::shared(proxy);
    Arc::new(blitz::net::Provider::new(callback))
}

This seems to be due to the parser attempting to use a data: URL as a base and failing.

Side note: Is there a setting I've missed that disables blitz attempting to resolve URLs during parsing? This seems like something that should either be configurable or not part of the parser.

[nicoburns]

The regular link doesn't show the 4th July content for me as I'm not in the US, but I am able to render the archive.org link with out panicking with latest Blitz main (note that 0.2.4 is a 9 months old release at this point):

[nicoburns]

Side note: Is there a setting I've missed that disables blitz attempting to resolve URLs during parsing? This seems like something that should either be configurable or not part of the parser.

If by "resolve" you mean actually fetch the content, then you can pass your own NetProvider and run completely arbitrary logic to decide whether to fetch content or not (or if you just never want to fetch then pass None).

[MorganBennetDev]

I am able to render the archive.org link with out panicking with latest Blitz main (note that 0.2.4 is a 9 months old release at this point)

The latest version on crates.io seems to be 0.2.1 (https://crates.io/crates/blitz), and 0.2.4 doesn't seem to be listed in the versions tab. I just used cargo add blitz to install with no extra options.

If by "resolve" you mean actually fetch the content, then you can pass your own NetProvider and run completely arbitrary logic to decide whether to fetch content or not (or if you just never want to fetch then pass None).

Sorry, I think I used the wrong terminology by accident. I meant converting relative URLs like /a/b.html to absolute ones like www.site.com/a/b.html.

[nicoburns]

You'll find 0.3.0-alpha.6 published 11 days ago https://crates.io/crates/blitz/versions. It's published as an alpha because we've been syncing "full" versions with Dioxus releases, but I'd definitely recommend it over 0.2.x at this point.

Sorry, I think I used the wrong terminology by accident. I meant converting relative URLs like /a/b.html to absolute ones like www.site.com/a/b.html.

Ah. Yeah. That's something I've thought that perhaps we ought to do differently. But I wasn't quite sure how it ought to work. How would you like things to work? And what is your use case?

[nicoburns]

I'd also probably recommend using the individual crates like blitz-dom, blitz-html, blitz-net, etc directly. The blitz crate hasn't seen much use and consequently isn't as well tested.

[MorganBennetDev]

I'd also probably recommend using the individual crates like blitz-dom, blitz-html, blitz-net, etc directly. The blitz crate hasn't seen much use and consequently isn't as well tested.

Noted for the future.

Ah. Yeah. That's something I've thought that perhaps we ought to do differently. But I wasn't quite sure how it ought to work. How would you like things to work?

My assumption with something called a parser is that it only turns the input I give it into a tree without modifying anything unless I tell it to. This may be a misunderstanding since I've been programming professionally for under a year so I'm not 100% solid on terminology yet.

And what is your use case?

I write a lot of web scrapers for work and I was experimenting with Rust browser stuff to see if there was anything useful I could make. The two main things I'm interested in are: layout information to detect tabular data that isn't properly formatted in HTML (i.e. the elements are seemingly unrelated in the markup, but visually you can see a table), and rendering SPAs without spinning up a whole Selenium instance. The first one is what I'm interested in blitz for; the second is out of scope for blitz and more of an unrealistic goal for me but something I'd still like to try and solve.