Some number of years ago web browser developers came up with the idea of allowing JavaScript to pull data from other sites into a web page on-the-fly. That day AJAX was born! Short for Asynchronous JavaScript and XML, this technology provided a mechanism for web pages to change their content without requiring the whole page to be refreshed. It was a great idea and has revolutionized some of the way we work with the web today. Sites like Gmail use it to keep track of new mail in your inbox and news sites use it to pull in the latest data to news or stock market tickers. It goes out to a remote site, grab all (or some) of its content and dumps it into the current page using JavaScript.
Shortly thereafter this mechanism was created, up rose a problem. The data that you are pulling in could be dangerous and people were using the AJAX mechanism to help deliver unsafe content to unsuspecting pages. This code could compromise visitor computers and gain access to info that they might not want to be shared. Maybe to launch unsavory popups or steal credit card data as it is being entered. Web browsers decided to limit this mechanism by blocking all requests made to other sources not under the website’s direct control. To do this they limited AJAX requests to the same domain. Their reasoning was that if the content came from the same domain as the website making the request, it could be assumed that it is reasonably safe. Unfortunately this also means that you can’t make requests from sites you trust elsewhere but might be on a different domain… at least without some trickery described below.
To get a better understanding of what the problem is and how this works we are going to setup a test page on a different website from the one making the AJAX request. This page will be a simple PHP page that will print some test data. Nothing fancy. Below is an example.
<?php echo "This is a test."; ?>
On the website making the request we can make an AJAX call to that page. If you only have one site, put the test script on your site and put the AJAX code below on a site like codepen.io. Since they are on different sites, the security limitation should kick in and either report the error in the browser’s error console along with the message of “XMLHttpRequest Cannot Load __________” or simply will not print our test message at all and report no error. Below is some JavaScript code (done in jQuery format for a simple example) which requests our other page. If you are using codepen.io, just make sure you load the jQuery library by selecting the gear icon in the upper left corner of the JS pane.
$.ajax({
url: 'http://www.ourothersite.com/testpage.php',
success: function (data) {
// Here "source" is the id of a div element on our page which will show our test data.
$("#source").html(data);
}
});
If you have everything setup right, when you make the AJAX call over to your test page, it will trigger the error message and block your test page data. Yeah I know, it is a real bummer. If you were to put this test page on the same website as the requesting page (in other words the test page is on yoursite.com and you are requesting it from another page on yoursite.com), it should work without a problem. But I know what you are saying, how often are you going to be pulling data from your own site? You will want data other websites have. Financial data from the NASDAQ or the top ten list of this week’s hot new music artists for instance.
Well, the tricks below should help you out with getting around some of these limitations. You will need a few special setups but nothing that is outlandish. Let’s start with the simplest and easiest way to get this working.
The simplest method is to go to your PHP page and add a header declaration to the top of the script that tells the web “I am allowing other origins (aka domains or sites) to access this resource.” To do this we simply add a header call to the top of the test file like this…
<?php
header("Access-Control-Allow-Origin: *");
echo "This is a test.";
?>
Now the page will be ok to be pulled into your AJAX script on the other site. This of course assumes one important thing, you actually have access to the other page and can manipulate the source to put the header on it. This trick is the best one if you have another domain you control and simply want to access it from another site you also control. In other words, pulling across data from two sites you manage.
Some big name sites out there also offer their pages with this origin header already set for you. I know that on some of the Yahoo! pages I have accessed this way they have been kind enough to set the header and so I have no issue making AJAX requests to their sources. But some other sites (yeah I am looking at you Google) don’t always do this.
The second trick is one I have used a few times and works really well. If you have a website that has PHP installed and that PHP installation has the allow_url_fopen option enabled on it, then that means that PHP can make requests to any other sites and pull down data into your domain. You can then call your PHP page, give it something like an ID of the source you want and have it spit the source back to you in plain text. Here is how that script might work…
<?php
header("Content-Type: text/plain");
// Load an array full of some pages you want to pull
$sources = array("http://www.google.com", "http://www.yahoo.com", "http://www.someothersite.net");
// Pass an ID to select which source you want from the array
$id = (isset($_REQUEST["id"]) && is_numeric($_REQUEST["id"])) ? (int)$_REQUEST["id"] : 0;
// Make sure the ID index is in the range of the array and get its contents.
if (($id >= 0) && ($id < count($sources))) {
$sourceContent = @file_get_contents($sources[$id]);
// If successful, print the source and flush it.
if ($sourceContent !== false) {
echo $sourceContent;
flush();
}
}
?>
As you can see we essentially set this up with a content-type header of “text/plain” so it just prints source code. Then, with an index we pass to the script from our AJAX call, we specify the source URL we want from an array of sources and have it fetch the page for us. It echos it and being that this page would be on the same domain as our calling AJAX script, all is ok and it will work nicely. This method makes our PHP script a bit of a proxy for one or more AJAX calls from different pages on our site. You can load the sources array with whatever URL’s you find safe enough to use and have it serve them up to AJAX. If the pages you are fetching are going to be quite large, you might instead want to take precautions and read through the site in chunks, flushing as you go. That way you are not burning up a ton of memory resources trying to hold the entire page content.
But again this method requires that you have PHP installed and the allow_url_open setting set to true in PHP.ini. Many hosts don’t have this on for security reasons themselves, but if they have it on (or give you the ability to turn it on), then this option would be open to you. You could do a similar thing with other server-side languages like ASP.NET or Coldfusion (like cfhttp).
This article covered a bit of the problem involved with the “XMLHttpRequest Cannot Load” security error you get when you make AJAX calls across domains. We talked about two main methods for getting around this. The first is to use the access-control-allow-origin header to get your AJAX calls to work without issue. The second technique is to leverage PHP (or a server-side language) to fetch data on your AJAX script’s behalf and serve that to your scripts. But again only if you have a server-side language setup to grab remote site content and with the proper security permissions set.
Thank you again for reading! 🙂
