Data URI support, redirect checks and more

[Bhpsngum]

(From issue #25)

Data URI protocol (data:)

• Add basic support for data URI protocol
• Support base64 encoding, allowing ASCII space characters (' ', '\t', '\n', '\v', '\f', '\r') in encoded + non-encoded form, and strict padding checks
• Failure of validation will throw/dispatch "Invalid URL" error

Redirects

• Add support for HTTP codes 301 and 308
• Add new option new XMLHttpRequest({ maxRedirects: 20 /* default is 20 */ })
• Excessive redirects will throw/dispatch an error Too many redirects, and attempting to redirect to any non-secure protocols (other than https and http) or invalid URI throws/dispatches Unsafe redirect error.

Local file system

• Add new option new XMLHttpRequest({ allowFileSystemSources: true /* default is true */ })
• Attempting to load from local filesystem resource while disabling this option throws/dispatches Not allowed to load local resources: file://... error.

Origin

• Add new option new XMLHttpRequest({ origin: "https://example.com" /* default is no origin */ })
• Similar to calling xhr in the browser page, the origin will act as a base URL, if origin is blank or invalid, any URL passed through xhr.open will act as an absolute URL.

Others

• Add xhr.responseURL to reflect XMLHttpRequest spec
• xhr.statusText now returns the correct status text, taken from HttpIncomingMessage.statusMessage
• xhr.dispatchEvent now dispatches additional object to event parameter { type: "event_type" }, e.g { type: "loadend" }
• Fix xhr.getResponseHeader() and xhr.getAllResponseHeaders() throwing errors when no responses or headers are present (for local resources)

Codespace

• Changing from Url.parse() to new WHATWG-compatible new URL() with more safe restrictions
• Load optional options from default options to prevent parameters prototype pollution
• Destroy any redirect responses to prevent hanging
• SSL options are now separated from original option for better code control

[mjwwit]

Thanks for this! Would you be willing to add some tests for this functionality as well?

[Bhpsngum]

sure, give me a bit

[Bhpsngum]

I added the tests, you can have a look around tho

[mjwwit]

Quite a lot of code here, which isn't surprising given the functionality that has been added. This is no problem, but it will take some time to review everything. To not keep you waiting I've reviewed it partially.
I apologize for the number of small comments. It's a messy project already and I'd rather not make it any worse by introducing (more) inconsistencies.

[Bhpsngum]

Quite a lot of code here, which isn't surprising given the functionality that has been added. This is no problem, but it will take some time to review everything. To not keep you waiting I've reviewed it partially. I apologize for the number of small comments. It's a messy project already and I'd rather not make it any worse by introducing (more) inconsistencies.

It's all fine, after all this project is quite old already, and considering the number of dependencies, need to be careful indeed.

[mjwwit]

I'm very sorry, but my time is very limited at this moment. I hope I can find the time to finish the review this weekend. Please bear with me.

[Bhpsngum]

it's all fine, just take your time

[mjwwit]

Just minor comments, overall this looks very good. Thanks a bunch!
I probably missed some usages of == or !=. Also, could you add your tests to the test script in package.json?

[Bhpsngum]

I've fixed the reviews, also due to the thing that we agreed to change from ERR_INVALID_URL to Invalid URL, i also changed other ones from code to message to unify with this:

• ERR_UNSAFE_REDIRECT -->Unsafe redirect
• ERR_TOO_MANY_REDIRECTS --> Too many redirects

[Bhpsngum]

oh wow, i guess i'll just write a nodejs script to automatically run the test case, the current one looks... not that nice to check.

[mjwwit]

oh wow, i guess i'll just write a nodejs script to automatically run the test case, the current one looks... not that nice to check.

It may be a good time to introduce something like that, thanks. Eventually we'll probably need a decent test runner though.

[Bhpsngum]

I might introduce some unit testing and code coverage frameworks like Istanbul + Mocha combination, but as for now, I'm just writing a simple node script. Might be useful in the future though.

[mjwwit]

I'll release later today or tomorrow, thanks again!

[Bhpsngum]

Thanks for the update as well 😄
I think we also need to document the features prior to the updates (including some old options like agent or ssl-related properties), so people can track down what features are available

[mjwwit]

Good idea, I'll make sure that's done before the release.