Note that there are some explanatory texts on larger screens.

plurals
  1. PO
    primarykey
    data
    text
    <h2>Advanced HTML string replacement functions</h2> <p><em>Note for OP, because he requested such a function: Change <code>base_url</code> to your proxy's basE URL in order to achieve the desired results.</em></p> <p>Two functions will be shown below (the usage guide is contained within the code). Make sure that you don't skip any part of the explanation of this answer to fully understand the function's behaviour.</p> <ul> <li><code>rel_to_abs(urL)</code> - This function returns absolute URLs. When an absolute URL with a commonly trusted protocol is passed, it will immediately return this URL. Otherwise, an absolute URL is generated from the <code>base_url</code> and the function argument. Relative URLs are correctly parsed (<code>../</code> ; <code>./</code> ; <code>.</code> ; <code>//</code>).</li> <li><code>replace_all_rel_by_abs</code> - This function will parse <strong>all</strong> occurences of URLs which have a significant meaning in HTML, such as CSS <code>url()</code>, links and external resources. See the code for a full list of parsed instances. See <strong><a href="https://stackoverflow.com/questions/7474710/can-i-load-an-entire-html-document-into-a-document-fragment-in-internet-explorer/7539198#7539198">this answer</a></strong> for an adjusted implementation to <strong>sanitise HTML strings</strong> from an external source (to embed in the document).</li> <li>Test case (at the bottom of the answer): To test the effectiveness of the function, simply paste the bookmarklet at the location's bar.</li> </ul> <p><hr /> <strong><code>rel_to_abs</code> - <em>Parsing relative URLs</em></strong></p> <pre><code>function rel_to_abs(url){ /* Only accept commonly trusted protocols: * Only data-image URLs are accepted, Exotic flavours (escaped slash, * html-entitied characters) are not supported to keep the function fast */ if(/^(https?|file|ftps?|mailto|javascript|data:image\/[^;]{2,9};):/i.test(url)) return url; //Url is already absolute var base_url = location.href.match(/^(.+)\/?(?:#.+)?$/)[0]+"/"; if(url.substring(0,2) == "//") return location.protocol + url; else if(url.charAt(0) == "/") return location.protocol + "//" + location.host + url; else if(url.substring(0,2) == "./") url = "." + url; else if(/^\s*$/.test(url)) return ""; //Empty = Return nothing else url = "../" + url; url = base_url + url; var i=0 while(/\/\.\.\//.test(url = url.replace(/[^\/]+\/+\.\.\//g,""))); /* Escape certain characters to prevent XSS */ url = url.replace(/\.$/,"").replace(/\/\./g,"").replace(/"/g,"%22") .replace(/'/g,"%27").replace(/&lt;/g,"%3C").replace(/&gt;/g,"%3E"); return url; } </code></pre> <p>Cases / examples:</p> <ul> <li><code>http://foo.bar</code>. Already an absolute URL, thus returned immediately.</li> <li><code>/doo</code> Relative to the root: Returns the current root + provided relative URL.</li> <li><code>./meh</code> Relative to the current directory.</li> <li><code>../booh</code> Relative to the parent directory.</li> </ul> <p>The function converts relative paths to <code>../</code>, and performs a search-and-replace (<code>http://domain/sub/anything-but-a-slash/../me</code> to <code>http://domain/sub/me</code>).</p> <p><hr /> <strong><code>replace_all_rel_by_abs</code> - <em>Convert all relevant occurences of URLs</em></strong><br /> URLs inside script instances (<code>&lt;script&gt;</code>, event handlers are <strong>not</strong> replaced, because it's near-impossible to create a fast-and-secure filter to parse JavaScript.</p> <p>This script is served with some comments inside. Regular Expressions are dynamically created, because an individual RE can have a size of <strong>3000</strong> characters. <code>&lt;meta http-equiv=refresh content=.. &gt;</code> can be obfuscated in various ways, hence the size of the RE.</p> <pre><code>function replace_all_rel_by_abs(html){ /*HTML/XML Attribute may not be prefixed by these characters (common attribute chars. This list is not complete, but will be sufficient for this function (see http://www.w3.org/TR/REC-xml/#NT-NameChar). */ var att = "[^-a-z0-9:._]"; var entityEnd = "(?:;|(?!\\d))"; var ents = {" ":"(?:\\s|&amp;nbsp;?|&amp;#0*32"+entityEnd+"|&amp;#x0*20"+entityEnd+")", "(":"(?:\\(|&amp;#0*40"+entityEnd+"|&amp;#x0*28"+entityEnd+")", ")":"(?:\\)|&amp;#0*41"+entityEnd+"|&amp;#x0*29"+entityEnd+")", ".":"(?:\\.|&amp;#0*46"+entityEnd+"|&amp;#x0*2e"+entityEnd+")"}; /* Placeholders to filter obfuscations */ var charMap = {}; var s = ents[" "]+"*"; //Short-hand for common use var any = "(?:[^&gt;\"']*(?:\"[^\"]*\"|'[^']*'))*?[^&gt;]*"; /* ^ Important: Must be pre- and postfixed by &lt; and &gt;. * This RE should match anything within a tag! */ /* @name ae @description Converts a given string in a sequence of the original input and the HTML entity @param String string String to convert */ function ae(string){ var all_chars_lowercase = string.toLowerCase(); if(ents[string]) return ents[string]; var all_chars_uppercase = string.toUpperCase(); var RE_res = ""; for(var i=0; i&lt;string.length; i++){ var char_lowercase = all_chars_lowercase.charAt(i); if(charMap[char_lowercase]){ RE_res += charMap[char_lowercase]; continue; } var char_uppercase = all_chars_uppercase.charAt(i); var RE_sub = [char_lowercase]; RE_sub.push("&amp;#0*" + char_lowercase.charCodeAt(0) + entityEnd); RE_sub.push("&amp;#x0*" + char_lowercase.charCodeAt(0).toString(16) + entityEnd); if(char_lowercase != char_uppercase){ /* Note: RE ignorecase flag has already been activated */ RE_sub.push("&amp;#0*" + char_uppercase.charCodeAt(0) + entityEnd); RE_sub.push("&amp;#x0*" + char_uppercase.charCodeAt(0).toString(16) + entityEnd); } RE_sub = "(?:" + RE_sub.join("|") + ")"; RE_res += (charMap[char_lowercase] = RE_sub); } return(ents[string] = RE_res); } /* @name by @description 2nd argument for replace(). */ function by(match, group1, group2, group3){ /* Note that this function can also be used to remove links: * return group1 + "javascript://" + group3; */ return group1 + rel_to_abs(group2) + group3; } /* @name by2 @description 2nd argument for replace(). Parses relevant HTML entities */ var slashRE = new RegExp(ae("/"), 'g'); var dotRE = new RegExp(ae("."), 'g'); function by2(match, group1, group2, group3){ /*Note that this function can also be used to remove links: * return group1 + "javascript://" + group3; */ group2 = group2.replace(slashRE, "/").replace(dotRE, "."); return group1 + rel_to_abs(group2) + group3; } /* @name cr @description Selects a HTML element and performs a search-and-replace on attributes @param String selector HTML substring to match @param String attribute RegExp-escaped; HTML element attribute to match @param String marker Optional RegExp-escaped; marks the prefix @param String delimiter Optional RegExp escaped; non-quote delimiters @param String end Optional RegExp-escaped; forces the match to end before an occurence of &lt;end&gt; */ function cr(selector, attribute, marker, delimiter, end){ if(typeof selector == "string") selector = new RegExp(selector, "gi"); attribute = att + attribute; marker = typeof marker == "string" ? marker : "\\s*=\\s*"; delimiter = typeof delimiter == "string" ? delimiter : ""; end = typeof end == "string" ? "?)("+end : ")("; var re1 = new RegExp('('+attribute+marker+'")([^"'+delimiter+']+'+end+')', 'gi'); var re2 = new RegExp("("+attribute+marker+"')([^'"+delimiter+"]+"+end+")", 'gi'); var re3 = new RegExp('('+attribute+marker+')([^"\'][^\\s&gt;'+delimiter+']*'+end+')', 'gi'); html = html.replace(selector, function(match){ return match.replace(re1, by).replace(re2, by).replace(re3, by); }); } /* @name cri @description Selects an attribute of a HTML element, and performs a search-and-replace on certain values @param String selector HTML element to match @param String attribute RegExp-escaped; HTML element attribute to match @param String front RegExp-escaped; attribute value, prefix to match @param String flags Optional RegExp flags, default "gi" @param String delimiter Optional RegExp-escaped; non-quote delimiters @param String end Optional RegExp-escaped; forces the match to end before an occurence of &lt;end&gt; */ function cri(selector, attribute, front, flags, delimiter, end){ if(typeof selector == "string") selector = new RegExp(selector, "gi"); attribute = att + attribute; flags = typeof flags == "string" ? flags : "gi"; var re1 = new RegExp('('+attribute+'\\s*=\\s*")([^"]*)', 'gi'); var re2 = new RegExp("("+attribute+"\\s*=\\s*')([^']+)", 'gi'); var at1 = new RegExp('('+front+')([^"]+)(")', flags); var at2 = new RegExp("("+front+")([^']+)(')", flags); if(typeof delimiter == "string"){ end = typeof end == "string" ? end : ""; var at3 = new RegExp("("+front+")([^\"'][^"+delimiter+"]*" + (end?"?)("+end+")":")()"), flags); var handleAttr = function(match, g1, g2){return g1+g2.replace(at1, by2).replace(at2, by2).replace(at3, by2)}; } else { var handleAttr = function(match, g1, g2){return g1+g2.replace(at1, by2).replace(at2, by2)}; } html = html.replace(selector, function(match){ return match.replace(re1, handleAttr).replace(re2, handleAttr); }); } /* &lt;meta http-equiv=refresh content=" ; url= " &gt; */ cri("&lt;meta"+any+att+"http-equiv\\s*=\\s*(?:\""+ae("refresh")+"\""+any+"&gt;|'"+ae("refresh")+"'"+any+"&gt;|"+ae("refresh")+"(?:"+ae(" ")+any+"&gt;|&gt;))", "content", ae("url")+s+ae("=")+s, "i"); cr("&lt;"+any+att+"href\\s*="+any+"&gt;", "href"); /* Linked elements */ cr("&lt;"+any+att+"src\\s*="+any+"&gt;", "src"); /* Embedded elements */ cr("&lt;object"+any+att+"data\\s*="+any+"&gt;", "data"); /* &lt;object data= &gt; */ cr("&lt;applet"+any+att+"codebase\\s*="+any+"&gt;", "codebase"); /* &lt;applet codebase= &gt; */ /* &lt;param name=movie value= &gt;*/ cr("&lt;param"+any+att+"name\\s*=\\s*(?:\""+ae("movie")+"\""+any+"&gt;|'"+ae("movie")+"'"+any+"&gt;|"+ae("movie")+"(?:"+ae(" ")+any+"&gt;|&gt;))", "value"); cr(/&lt;style[^&gt;]*&gt;(?:[^"']*(?:"[^"]*"|'[^']*'))*?[^'"]*(?:&lt;\/style|$)/gi, "url", "\\s*\\(\\s*", "", "\\s*\\)"); /* &lt;style&gt; */ cri("&lt;"+any+att+"style\\s*="+any+"&gt;", "style", ae("url")+s+ae("(")+s, 0, s+ae(")"), ae(")")); /*&lt; style=" url(...) " &gt; */ return html; } </code></pre> <p>A short summary of the private functions:</p> <ul> <li><code>rel_to_abs(url)</code> - Converts relative / unknown URLs to absolute URLs</li> <li><code>replace_all_rel_by_abs(html)</code> - Replaces all relevant occurences of URLs within a string of HTML by absolute URLs. <ol> <li><code>ae</code> - <strong>A</strong>ny <strong>E</strong>ntity - Returns a RE-pattern to deal with HTML entities.</li> <li><code>by</code> - replace <strong>by</strong> - This short function request the actual url replace (<code>rel_to_abs</code>). This function may be called hundreds, if not thousand times. Be careful to not add a slow algorithm to this function (customisation).</li> <li><code>cr</code> - <strong>C</strong>reate <strong>R</strong>eplace - Creates and executes a search-and-replace.<br />Example: <code>href="..."</code> (within any HTML tag).</li> <li><code>cri</code> - <strong>C</strong>reate <strong>R</strong>eplace <strong>I</strong>nline - Creates and executes a search-and-replace.<br />Example: <code>url(..)</code> within the all <code>style</code> attribute within HTML tags.</li> </ol></li> </ul> <h3>Test case</h3> <p>Open any page, and paste the following bookmarklet in the location bar:</p> <pre><code>javascript:void(function(){var s=document.createElement("script");s.src="http://rob.lekensteyn.nl/rel_to_abs.js";document.body.appendChild(s)})(); </code></pre> <p>The injected code contains the two functions, as defined above, plus the test case, shown below. <strong>Note</strong>: The test case does <strong>not</strong> modify the HTML of the page, but shows the parsed results in a textarea (optionally).</p> <pre><code>var t=(new Date).getTime(); var result = replace_all_rel_by_abs(document.documentElement.innerHTML); if(confirm((new Date).getTime()-t+" milliseconds to execute\n\nPut results in new textarea?")){ var txt = document.createElement("textarea"); txt.style.cssText = "position:fixed;top:0;left:0;width:100%;height:99%" txt.ondblclick = function(){this.parentNode.removeChild(this)} txt.value = result; document.body.appendChild(txt); } </code></pre> <p>See also:</p> <ul> <li><a href="https://stackoverflow.com/questions/7474710/can-i-load-an-entire-html-document-into-a-document-fragment-in-internet-explorer/7539198#7539198">Answer: Parsing and sanitising HTML strings</a></li> </ul>
    singulars
    1. This table or related slice is empty.
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. VO
      singulars
      1. This table or related slice is empty.
    2. VO
      singulars
      1. This table or related slice is empty.
    3. VO
      singulars
      1. This table or related slice is empty.
 

Querying!

 
Guidance

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload