html - Check duplicate content without doing a GET -

-

one of main purposes of url normalization avoid get requests on distinct urls produce exact same result.

now, know can check canonical tag , compare 2 url's html see if they're same, have download exact same resource twice in order this, beating point stated before.

is there way check duplicated content doing head request? if not, there way download <head> section of web page without downloading entire document?

i can think of solutions last one, wan't know if there's direct one.

according msdn documentation solution question following

dim myhttpwebrequest httpwebrequest = ctype(webrequest.create(url), httpwebrequest) dim myhttpwebresponse httpwebresponse = ctype(myhttpwebrequest.getresponse(), httpwebresponse) console.writeline(controlchars.lf + controlchars.cr + "the following headers received in response") dim integer while < myhttpwebresponse.headers.count     console.writeline(controlchars.cr + "header name:{0}, value :{1}", myhttpwebresponse.headers.keys(i), myhttpwebresponse.headers(i))     = + 1 end while myhttpwebresponse.close()

let me explain code first line creates httpwebrequest specified url , second line , third line displays headers present in response received uri , fourth eighth line - headers property webheadercollection. use it's properties traverse collection , display each header , tenth close request , if want head portion of web page php class freely available @ http://www.phpclasses.org/package/4033-php-extract-html-contained-in-tags-from-a-web-page.html


Comments

Post a Comment

Popular posts from this blog

c# - how to write client side events functions for the combobox items -

-
<dx:aspxcombobox id="aspxcombobox2" runat="server" clientidmode="autoid" selectedindex="-1" onselectedindexchanged="aspxcombobox1_selectedindexchanged1" > <clientsideevents selectedindexchanged="function(s, e) { grid.selectrows();}" /> <items> <dx:listedititem text="selecte rows" /> <dx:listedititem text="unselectall" /> </items> </dx:aspxcombobox> i want write event "unselect all" item in combobox. how can this? you'll need write javascript function iterates thru combo-box , marks selected items false. see select/unselect checkbox fields using pure javascript , html sample code msdn:
Read more

exception - Python, pyPdf OCR error: pyPdf.utils.PdfReadError: EOF marker not found -

-
pypdf throws exception: pypdf.utils.pdfreaderror: eof marker not found i don't need fix pypdf, need eof error cause "except" block execute , skip on file, doesn't work. still causes program stop running. background: batch ocr program pdfs python, pypdf, adobe pdf ocr error: unsupported filter /lzwdecode ... saga continues. i got 10,000 pdfs in folder. ocrd, not. can't tell 'em apart. step 1 figure out ones not ocrd , ocr (see other threads details). so i'm using pypdf. exceptions related unrecognized characters , unsupported filters when try read text. guestimated if throws exception, it's got text in , doens't go in list. problem solved, right? so: pypdf import pdffilewriter, pdffilereader import sys, os, pypdf, re path = 'c:\users\homer\documents\my pdfs' filelist = os.listdir(path) has_text_list = [] does_not_have_text_list = [] pdf_name in filelist: pdf_file
Read more