Databases
StackOverflow2013
Postsdbo
PORemotely Scrape Page and Get most Relevant title or Description for Images with XPath
 

Note that there are some explanatory texts on larger screens.

plurals
  1. PORemotely Scrape Page and Get most Relevant title or Description for Images with XPath
    primarykey
    data
    text
    <p>What I'm looking at doing is essentially the same thing a Tweet button or Facebook Share / Like button does, and that is to scrape a page and the most relevant title for a piece of data. The best example I can think of is when you're on the front page of a website with many articles and you click a Facebook Like Button. It will then get the proper information for the post relative to (nearest) the Like button. Some sites have Open Graph tags, but some do not and it still works.</p> <p>Since this is done remotely, I only have control of the data that I want to target. In this case the data are images. Rather than retrieving just the <code>&lt;title&gt;</code> of the page, I am looking to somehow traverse the dom in reverse from the starting point of each image, and find the nearest "title". The problem is that not all titles occur before an image. However, the chance of the image occurring after the title in this case seems fairly high. With that said, it is my hope to make it work well for nearly any site.</p> <p>Thoughts:</p> <ul> <li>Find the "container" of the image and then use the first block of text.</li> <li>Find the blocks of text in elements that contain certain classes ("description", "title") or elements (h1,h2,h3,h4).</li> </ul> <p>Title backups:</p> <ul> <li>Using Open Graph Tags</li> <li>Using just the <code>&lt;title&gt;</code></li> <li>Using ALT tags only</li> <li>Using META Tags</li> </ul> <p>Summary: Extracting the images isn't the problem, it's how to get relevant titles for them.</p> <p><strong>Question:</strong> How would you go about getting relevant titles for each of the images? Perhaps using DomDocument or XPath?</p>
    singulars
    1. PO
      singulars
      1. PTAnswer
    1. This table or related slice is empty.
    1. PTQuestion
    1. USstwhite
    1. USstwhite
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PO
      singulars
      1. PTAnswer
    1. VO
      singulars
      1. PORemotely Scrape Page and Get most Relevant title or Description for Images with XPath
      1. This table or related slice is empty.
      1. VTUpMod
    2. VO
      singulars
      1. PORemotely Scrape Page and Get most Relevant title or Description for Images with XPath
      1. This table or related slice is empty.
      1. VTUpMod
    3. VO
      singulars
      1. PORemotely Scrape Page and Get most Relevant title or Description for Images with XPath
      1. This table or related slice is empty.
      1. VTUpMod
    1. COHonestly, after you scrape it with PHP, if you could hand it off via REST calls to a small Java Web server, you could use JSOUP to easily get access to all of those elements and attributes. JSOUP is like jQuery for Java and uses much the same syntax. I wish it was available for PHP as it would make your problem go away in seconds!
      singulars
      1. PORemotely Scrape Page and Get most Relevant title or Description for Images with XPath
      1. USjmort253
    2. COthere are several libraries available that deal with content extraction from pages, although I don't know of one that deals directly with images. but you might get some ideas and directions, or be able to use them. here's one: http://code.google.com/p/boilerpipe/wiki/Components
      singulars
      1. PORemotely Scrape Page and Get most Relevant title or Description for Images with XPath
      1. USNot_a_Golfer
    3. COThanks for your thoughts. I've updated my question to target more of the "logic" behind getting relevant titles or descriptions for each image rather than how to get the images themselves.
      singulars
      1. PORemotely Scrape Page and Get most Relevant title or Description for Images with XPath
      1. USstwhite
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload