Failure to parse URL that doesn't start with scheme but contains a query that contains a URL with a scheme

[mlhdeveloper]

Here's a simple example URL that fails to parse:

somedomain.com/?g=http://somedomain.com/

If you add a scheme to the front, it then parses properly:

http://somedomain.com/?g=http://somedomain.com/

[mlhdeveloper]

I think the solution lies in changing this part so that it's only checking for :// near the beginning of the url and not anywhere in the entire url:

utbox/utbox/bin/ut_parse_lib.py

Line 15 in f8db838

preg_rfc1808 = re.compile("://")

utbox/utbox/bin/ut_parse_lib.py

Lines 257 to 258 in f8db838

	if not preg_rfc1808.search(url):
	url = "//%s" % url

[mlhdeveloper]

I think this fixes it so that it's only checking for :// at the very beginning of the url or only after a scheme, i.e. only after any number of alphabetical or + characters (based on schemes handled by urllib.parse):

preg_rfc1808 = re.compile("^[a-z+]*://")

[ggokdemir]

Hi @mlhdeveloper,

Thank you so much for bringing this up!

I changed line 15, which theoretically should have fixed the issue, but it still doesn't seem to work. I suspect it’s still referencing the Public Suffix List (PSL) rules and expects the input accordingly.

preg_rfc1808 = re.compile("^[a-z+]*://")
Line 255 clearly states that it follows RFC 1808. Instead of editing the data, I’ll explore other ways to inform the user more effectively. Please let me know if you have any thoughts or suggestions—I’d really appreciate any help or feedback.

    # Following the syntax specifications in RFC 1808, urlparse recognizes
    # a netlog only if it is properly introduced by '//'.

Thank you! I'll keep you posted if I make any changes.