Default Content-Type should be application/octet-stream

[waylan]

The default fallback Content-Type is currently text/plain. However, is should be application/octet-stream. In fact, RFC 2046 states in section 4.5.1:

The "octet-stream" subtype is used to indicate that a body contains arbitrary binary data.

Additionally, the standard library http.server.SimpleHTTPRequestHandler uses application/octet-stream as the default fallback (as does Tornado and various other third-party static file servers). Interestingly, the standard library http.server.SimpleHTTPRequestHander adds a few custom types defined as text/plain for those few cases were undefined types might actually need to be served as text/plain.

However, in most cases, application/octet-stream is a better fallback. In fact, I have personally encountered the following file types which fail to be properly handled by a browser if not served by either their correct Content-Type or the application/octet-stream fallback.

File Extension	Expected Content-Type	Acceptable Fallback
.woff2	font/woff2	application/octet-stream
.woff	font/woff or application/font-woff	application/octet-stream
.ttf	font/ttf	application/octet-stream
.mustache	text/template	application/octet-stream
.json	application/json	application/octet-stream

In each case, the standard mimetypes library fails to return a proper type on Windows. By default, the mimetypes library uses the types as defined by the OS. Therefore, the results can be different from machine to machine and no given type can be guaranteed to be defined on any given system. While some of the above file types would seem to be common (particularly .json), with the great diversity out there, that assumption should not be made.

For the above reason, the Whitenoise static server library (which also uses application/octet-stream as the default fallback) defines their own Content-Types with the following comment in their code:

We use our own set of default media types rather than the system-supplied
ones. This ensures consistent media type behaviour across varied
environments. The defaults are based on those shipped with nginx, with
some custom additions.

Those custom additions are here. I have to wonder if it would make sense for Static to do the same. Regardless, the default fallback should be application/octet-stream, or at the very least, be easily overridable by the user.