If the term paywall doesn't mean anything to you, read this article. The continued rise of adblocking apps has put an end to the revenue model that relies on ads served by websites.
So we have seen many news sites use different ways to make money. So big websites like: The Wall Street Journal, Financial Times, The New York Times ή The Washington Post χρησιμοποιούν paywall για να αυξήσουν τα εισοδήματά τους.
There are different types of paywalls but they all have a common feature: they block access to the content of the web pages, either directly or after reading a certain number of articles.
Immediately afterwards, a paywall window appears to the visitors requesting them to subscribe to the site to continue reading.
This practice may seem meaningful from a business perspective, and may be much more lucrative than Google ads blocked by adblockers, but there is a serious disadvantage.
Websites lose a high percentage of their visitors using a paywall system.
Δεν είναι μυστικό ότι τα ειδησεογραφικά sites επιτρέπουν την πρόσβαση σε όλο το περιεχόμενό τους στις μηχανές αναζήτησης. Αν δείτε για παράδειγμα το Google News , you will notice articles from websites that use paywalls.
Are you wondering how sites can block or allow access to content?
There are many different controls, including user-agent control, referrer, that is, from which page you come and of course cookies. All of the above can be used to determine the legitimacy of access.
How can we overcome the obstacles?
Perhaps the best way is to disguise yourself at Googlebot. Let's see how it works:
Referrer: https://www.google.com/
User-Agent: Mozilla / 5.0 (compatible; Googlebot / 2.1; http: //www.google.com/bot.html
In Firefox
Firefox users need two add-ons to help them bypass paywalls: the first is RefControl, which can change the referral page when you visit news websites, and the second one called User Agent Switcher, will change the user-agent of the browser.
Download and install both extensions in Firefox.
Press the Alt key and select Tools - Options - RefControl.
Click "add webpage", and type in a domain, select the custom action, and enter https://www.google.com/ as a reference.
Repeat for each news page you want to access
When finished, close the configuration window.
Press the Alt key again, and select Tools> User Agent Default> Agents User Editor from the menu. Name it Googlebot.
Exit the menu.
Before you try to access these sites, press Alt and select Default User Agent - Googlebot
- The above add-ons will not work on all sites
For Google Chrome
Google Chrome users can also install extensions User Agent Switcher and Referer Control.
There is however another feature, and this is to create a custom extension that automates the process in the browser.
Follow the instructions. All you need is to create a new directory on your computer. Below we describe how to create two files: background.js and manifest.json.
background.js
var ALLOW_COOKIES = ["nytimes", "ft.com"]
function changeRefer(details) {
foundReferer = false;
foundUA = false
var reqHeaders = details.requestHeaders.filter(function(header) {
// block cookies by default
if (header.name !== "Cookie") {
return header;
}
allowHeader = ALLOW_COOKIES.map(function(url) {
if (details.url.includes(url)) {
return true;
}
return false;
});
if (allowHeader.reduce(function(a, b) { return a || b}, false)) { return header; }
}).map(function(header) {
if (header.name === "Referer") {
header.value = "https://www.google.com/";
foundReferer = true;
}
if (header.name === "User-Agent") {
header.value = "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)";
foundUA = true;
}
return header;
})
// append referer
if (!foundReferer) {
reqHeaders.push({
"name": "Referer",
"value": "https://www.google.com/"
})
}
if (!foundUA) {
reqHeaders.push({
"name": "User-Agent",
"value": "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
})
}
console.log(reqHeaders);
return {requestHeaders: reqHeaders};
}
function blockCookies(details) {
for (var i = 0; i
manifest.json
{"name": "Innocuous Chrome Extension", "version": "0.1", "description": "This is an innocuous chrome extension.", "permissions": ["webRequest", "webRequestBlocking", "http: / /www.ft.com/* "," http://www.wsj.com/* "," https://www.wsj.com/* "," http://www.economist.com/* "," http://www.nytimes.com/* "," https://hbr.org/* "," http://www.newyorker.com/* "," http: //www.forbes .com / * "," http://online.barrons.com/* "," http://www.barrons.com/* "," http://www.investingdaily.com/* "," http : //realmoney.thestreet.com/* "," http://www.washingtonpost.com/* "]," background ": {" scripts ": [" background.js "]}," manifest_version ": 2 }
After creating the files (use NotePad ++ to create them and change the extension to .js), place them in a folder.
If you do not have the time you can download the files folder from the following link:
You need to enable “developer mode” from the address
chrome: // extensions /
and then from “load unpacked extension” select the folder with the two files you have created.
Upload the extension to Chrome.
You can modify the files and add new web pages you want to bypass. Just open the files with Notepad ++ and read the code.
Anti Paywall js Elaine
