Databases
StackOverflow2013
Postsdbo
POmy first perl script: using "get($url)" method in a loop?
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POmy first perl script: using "get($url)" method in a loop?
    primarykey
    data
    text
    <p>So it seemed easy enough. Use a series of nested loops to go though a ton of URLs sorted by year/month/day and download the XML files. As this is my first script, I started with the loop; something familiar in any language. I ran it just printing the constructed URLs and it worked perfect. I then wrote the code to download the content and save it separately, and that worked perfect as well with a sample URL on multiple test cases. But when I combined these two bits of code, it broke, the program just got stuck and did nothing at all. I therefore ran the debugger and as I stepped through it, it became stuck on this one line:</p> <blockquote> <p>warnings::register::import(/usr/share/perl/5.10/warnings/register.pm:25):25:vec($warnings::Bits{$k}, $warnings::LAST_BIT, 1) = 0;</p> </blockquote> <p>If I just hit r to return from the subroutine it works and continues to another point on its way back down the call stack where something similar happens over and over for some time. The stack trace:</p> <blockquote> <p>$ = warnings::register::import('warnings::register') called from file `/usr/lib/perl/5.10/Socket.pm' line 7</p> <p>$ = Socket::BEGIN() called from file `/usr/lib/perl/5.10/Socket.pm' line 7</p> <p>$ = eval {...} called from file `/usr/lib/perl/5.10/Socket.pm' line 7</p> <p>$ = require 'Socket.pm' called from file `/usr/lib/perl/5.10/IO/Socket.pm' line 12</p> <p>$ = IO::Socket::BEGIN() called from file `/usr/lib/perl/5.10/Socket.pm' line 7</p> <p>$ = eval {...} called from file `/usr/lib/perl/5.10/Socket.pm' line 7</p> <p>$ = require 'IO/Socket.pm' called from file `/usr/share/perl5/LWP/Simple.pm' line 158</p> <p>$ = LWP::Simple::_trivial_http_get('www.aDatabase.com', 80, '/sittings/1987/oct/20.xml') called from file `/usr/share/perl5/LWP/Simple.pm' line 136</p> <p>$ = LWP::Simple::_get('<a href="http://www.aDatabase.com/1987/oct/20.xml" rel="nofollow noreferrer">http://www.aDatabase.com/1987/oct/20.xml</a>') called from file `xmlfetch.pl' line 28</p> </blockquote> <p>As you can see it is getting stuck inside this "get($url)" method, and I have no clue why? Here is my code:</p> <pre><code>#!/usr/bin/perl use LWP::Simple; $urlBase = 'http://www.aDatabase.com/subheading/'; $day=1; $month=1; @months=("list of months","jan","feb","mar","apr","may","jun","jul","aug","sep","oct","nov","dec"); $year=1987; $nullXML = "&lt;?xml version=\"1.0\" encoding=\"UTF-8\"?&gt;\n&lt;nil-classes type=\"array\"/&gt;\n"; while($year&lt;=2006) { $month=1; while($month&lt;=12) { $day=1; while($day&lt;=31) { $newUrl = "$urlBase$year/$months[$month]/$day.xml"; $content = get($newUrl); if($content ne $nullXML) { $filename = "$year-$month-$day.xml"; open(FILE, "&gt;$filename"); print FILE $content; close(FILE); } $day++; } $month++; } $year++; } </code></pre> <p>I am almost positive it is something tiny I just dont know, but google has not turned up anything.</p> <p>Thanks in advance,</p> <p>B.</p> <p><strong>EDIT:</strong> It's official, it just hangs forever inside this get method, runs for several loops then hangs again for a while. But its still a problem. Why is this happening?</p>
    singulars
    1. PO
      singulars
      1. PTAnswer
    1. This table or related slice is empty.
    1. PTQuestion
    1. USgnomed
    1. USgnomed
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PO
      singulars
      1. PTAnswer
    2. PO
      singulars
      1. PTAnswer
    3. PO
      singulars
      1. PTAnswer
    1. VO
      singulars
      1. POmy first perl script: using "get($url)" method in a loop?
      1. This table or related slice is empty.
      1. VTUpMod
    2. VO
      singulars
      1. POmy first perl script: using "get($url)" method in a loop?
      1. This table or related slice is empty.
      1. VTUpMod
    1. CONo idea why it's failing (your code looks OK), but please replace the 'while' loops with 'for' loops (i.e. "for ($year=1987; $year<=2006; $year++)" or "for $year (1987 .. 2006)".
      singulars
      1. POmy first perl script: using "get($url)" method in a loop?
      1. USAndrew Medico
    2. COfair call. consider them replaced. bummer about this method though.
      singulars
      1. POmy first perl script: using "get($url)" method in a loop?
      1. USgnomed
    3. COPlease supply 1. output of perl -V, 2. version numbers for LWP::Simple, Socket, and IO::Socket
      singulars
      1. POmy first perl script: using "get($url)" method in a loop?
      1. USAlnitak
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload