StackOverflow2013

Note that there are some explanatory texts on larger screens.

plurals

PO
text
Body
copied!<p>to answer my own question, here's a script that starts up as a cron job every 30 minutes. it dumps the incoming stream in 5-minute chunks (or set by FILE _ SECONDS ) to a particular directory. block borders are synchronised to the clock, and it doesn't start writing until the <em>end</em> of the current time chunk, so the running cronjobs can overlap without doubling up data or leaving gaps. files are named as (epoch time % number of seconds in 24 hours).str . </p> <p>i haven't made a player yet, but the plan was to set the output directory to somewhere web-accessible, and write a script to be run locally that uses the same timestamp-calculating code as here to sequentially access (timestamp 12 hours ago).str, tack them back together again, and then set up as a shoutcast server locally. then i could just point my music player at <a href="http://localhost:port" rel="nofollow noreferrer">http://localhost:port</a> and get it.</p> <p><em>edit: New version with timeouts and better error condition checking, plus nice log file. this is currently running hitch-free on my (cheap) shared webhost, with no problems.</em></p> <pre><code>#!/usr/bin/python import time import urllib import datetime import os import socket # number of seconds for each file FILE_SECONDS = 300 # run for 30 minutes RUN_TIME = 60*30 # size in bytes of each read block # 16384 = 1 second BLOCK_SIZE = 16384 MAX_TIMEOUTS = 10 # where to save the files OUTPUT_DIRECTORY = "dir/" # URL for original stream URL = "http://url/path:port" debug = True log = None socket.setdefaulttimeout(10) class DatestampedWriter: # output_path MUST have trailing '/' def __init__(self, output_path, run_seconds ): self.path = output_path self.file = None # needs to be -1 to avoid issue when 0 is a real timestamp self.curr_timestamp = -1 self.running = False # don't start until the _end_ of the current time block # so calculate an initial timestamp as (now+FILE_SECONDS) self.initial_timestamp = self.CalcTimestamp( FILE_SECONDS ) self.final_timestamp = self.CalcTimestamp( run_seconds ) if debug: log = open(OUTPUT_DIRECTORY+"log_"+str(self.initial_timestamp)+".txt","w") log.write("initial timestamp "+str(self.initial_timestamp)+", final "+str(self.final_timestamp)+" (diff "+str(self.final_timestamp-self.initial_timestamp)+")\n") self.log = log def Shutdown(self): if self.file != None: self.file.close() # write out buf # returns True when we should stop def Write(self, buf): # check that we have the correct file open # get timestamp timestamp = self.CalcTimestamp() if not self.running : # should we start? if timestamp == self.initial_timestamp: if debug: self.log.write( "starting running now\n" ) self.log.flush() self.running = True # should we open a new file? if self.running and timestamp != self.curr_timestamp: if debug: self.log.write( "new timestamp "+str(timestamp)+"\n" ) self.log.flush() # close old file if ( self.file != None ): self.file.close() # time to stop? if ( self.curr_timestamp == self.final_timestamp ): if debug: self.log.write( " -- time to stop\n" ) self.log.flush() self.running = False return True # open new file filename = self.path+str(timestamp)+".str" #if not os.path.exists(filename): self.file = open(filename, "w") self.curr_timestamp = int(timestamp) #else: # uh-oh # if debug: # self.log.write(" tried to open but failed, already there\n") # self.running = False # now write bytes if self.running: #print("writing "+str(len(buf))) self.file.write( buf ) return False def CalcTimestamp(self, seconds_offset=0): t = datetime.datetime.now() seconds = time.mktime(t.timetuple())+seconds_offset # FILE_SECONDS intervals, 24 hour days timestamp = seconds - ( seconds % FILE_SECONDS ) timestamp = timestamp % 86400 return int(timestamp) writer = DatestampedWriter(OUTPUT_DIRECTORY, RUN_TIME) writer_finished = False # while been running for < (RUN_TIME + 5 minutes) now = time.mktime(datetime.datetime.now().timetuple()) stop_time = now + RUN_TIME + 5*60 while not writer_finished and time.mktime(datetime.datetime.now().timetuple())<stop_time: now = time.mktime(datetime.datetime.now().timetuple()) # open the stream if debug: writer.log.write("opening stream... "+str(now)+"/"+str(stop_time)+"\n") writer.log.flush() try: u = urllib.urlopen(URL) except socket.timeout: if debug: writer.log.write("timed out, sleeping 60 seconds\n") writer.log.flush() time.sleep(60) continue except IOError: if debug: writer.log.write("IOError, sleeping 60 seconds\n") writer.log.flush() time.sleep(60) continue # read 1 block of input buf = u.read(BLOCK_SIZE) timeouts = 0 while len(buf) > 0 and not writer_finished and now<stop_time and timeouts<MAX_TIMEOUTS: # write to disc writer_finished = writer.Write(buf) # read 1 block of input try: buf = u.read(BLOCK_SIZE) except socket.timeout: # catch exception but do nothing about it if debug: writer.log.write("read timed out ("+str(timeouts)+")\n") writer.log.flush() timeouts = timeouts+1 now = time.mktime(datetime.datetime.now().timetuple()) # stream has closed, if debug: writer.log.write("read loop bailed out: timeouts "+str(timeouts)+", time "+str(now)+"\n") writer.log.flush() u.close(); # sleep 1 second before trying to open the stream again time.sleep(1) now = time.mktime(datetime.datetime.now().timetuple()) writer.Shutdown() </code></pre>

Querying!

Guidance

An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload