HTTPS Everywhere and xhr from other add-ons

Https Everywhere is incompatible with xhr requests that are made from Firefox addon modules. This is true when using the nsIXMLHttpRequest component and the friendly request module from the Add-on SDK. Often, when a page or a file is being fetched using these methods, Https Everywhere replaces the old channel with a new one, and the request never gets completed.

That's been a major issue for GNU LibreJS in the past few weeks when running along with Https Everywhere, since it relies on getting the code from external scripts using the nsIXMLHttpRequest component to test whether these scripts have proper free licensing and if they are trivial or not.

While Https Everywhere provides a custom notification, "https-everywhere-uri-rewrite", in our case it won't really help unless you are planning on aborting your request and creating a new one with the new URI passed. Even then, it doesn't seem to work properly.

The solution wasn't obvious to me: To create a new channel and get the response data from that channel instead of using XHR at all. This is a small module that does the trick for LibreJS and fetches the contents of JavaScript files.

To run it, you only need to do something like this from main.js:

var request = require("request");

request.Request('http://www.fsf.org/graphics/widget/global/widget.js', function (responseText) {
    console.log(responseText);
}).get();

The URL used, http://www.fsf.org/graphics/widget/global/widget.js, is redirected to https://www.fsf.org/graphics/widget/global/widget.js by Https Everywhere; then it goes to http://static.fsf.org/nosvn/appeal2011/widget.js, and ends up at https://static.fsf.org/nosvn/appeal2011/widget.js

You can try running Https Everywhere and using the Add-on SDK request module with this URL, and you'll see you won't get a response. With the module below, on the other hand, you get the response. All it does is create a channel then doing an asyncOpen and getting the data onDataAvailable from the listener:

var {Cc, Ci, Cu, Cm, Cr} = require("chrome");

var {XPCOMUtils} = Cu.import("resource://gre/modules/XPCOMUtils.jsm");
var {NetUtil} = Cu.import("resource://gre/modules/NetUtil.jsm");

var request = {

    url: null,

    channel: null,

    responseCallback: null,

    /**
     * init variables
     * Called from constructor.
     */
    init: function (url, callback) {

    this.url = url;

    this.responseCallback = callback;

    var iOService = Cc["@mozilla.org/network/io-service;1"]
            .getService(Ci.nsIIOService);

    this.channel = iOService.newChannel(this.url, 'UTF-8', null);

    },
    /**
     * get
     * asyncOpen channel and get response text 
     * when the request is completed.
     */
    get: function () {

    var that = this;

    var responseReceived = function (data) {
        // do some stuff here with the data if needed.
        that.responseCallback(data);
    };

    this.channel.asyncOpen({
                   QueryInterface: XPCOMUtils.generateQI([Ci.nsIRequestObserver, Ci.nsIStreamListener]),

                   data: "",

                   onStartRequest: function(request, context) {},

                   onDataAvailable: function (request, context, stream, offset, count) {
                      this.data += NetUtil.readInputStreamToString(stream, count);
                   },

                   onStopRequest: function (request, context, result) {
                       responseReceived(this.data);
                   }

                   }, null);

    }

};

exports.Request = function (url, callback) {
    var obj = Object.create(request);
    obj.init(url, callback);
    return obj;
};