StackOverflow2013

Note that there are some explanatory texts on larger screens.

plurals

POGo file downloader
text
Body
copied!<p>I have the following code which is suppose to download file by splitting it into multiple parts. But right now it only works on images, when I try downloading other files like tar files the output is an invalid file.</p> <p><strong>UPDATED:</strong></p> <p>Used <code>os.WriteAt</code> instead of <code>os.Write</code> and removed <code>os.O_APPEND</code> file mode.</p> <pre><code>package main import ( "errors" "flag" "fmt" "io/ioutil" "log" "net/http" "os" "strconv" ) var file_url string var workers int var filename string func init() { flag.StringVar(&file_url, "url", "", "URL of the file to download") flag.StringVar(&filename, "filename", "", "Name of downloaded file") flag.IntVar(&workers, "workers", 2, "Number of download workers") } func get_headers(url string) (map[string]string, error) { headers := make(map[string]string) resp, err := http.Head(url) if err != nil { return headers, err } if resp.StatusCode != 200 { return headers, errors.New(resp.Status) } for key, val := range resp.Header { headers[key] = val[0] } return headers, err } func download_chunk(url string, out string, start int, stop int) { client := new(http.Client) req, _ := http.NewRequest("GET", url, nil) req.Header.Add("Range", fmt.Sprintf("bytes=%d-%d", start, stop)) resp, _ := client.Do(req) defer resp.Body.Close() body, err := ioutil.ReadAll(resp.Body) if err != nil { log.Fatalln(err) return } file, err := os.OpenFile(out, os.O_WRONLY, 0600) if err != nil { if file, err = os.Create(out); err != nil { log.Fatalln(err) return } } defer file.Close() if _, err := file.WriteAt(body, int64(start)); err != nil { log.Fatalln(err) return } fmt.Println(fmt.Sprintf("Range %d-%d: %d", start, stop, resp.ContentLength)) } func main() { flag.Parse() headers, err := get_headers(file_url) if err != nil { fmt.Println(err) } else { length, _ := strconv.Atoi(headers["Content-Length"]) bytes_chunk := length / workers fmt.Println("file length: ", length) for i := 0; i < workers; i++ { start := i * bytes_chunk stop := start + (bytes_chunk - 1) go download_chunk(file_url, filename, start, stop) } var input string fmt.Scanln(&input) } } </code></pre> <p>Basically, it just reads the length of the file, divides it with the number of workers then each file downloads using HTTP's Range header, after downloading it seeks to a position in the file where that chunk is written.</p>

Querying!

Guidance

An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload