Databases
StackOverflow2013
Postsdbo
POJava * Multithreading with MySQL
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POJava * Multithreading with MySQL
    primarykey
    data
    text
    <p>This is my first post so please let me know if there is anything I should add to the post if needed.</p> <p>I have a program that grabs urls from my MySQL database and sends them to a class that will download the images to my desktop.</p> <p>I have 80,000 images I need to get, and I well I want to utilize multithreading to make it quicker.</p> <p>I have gotten somewhere but I am now stuck and have spent hours researching as I am brand new to multithreading.</p> <p>The problem is when the program runs, it goes through a loop processing the first 5 entries over and over again. This happens to be the ExecutorFixedPoolSize.</p> <p>Can someone help me please.</p> <p>My Code is as follows.</p> <pre><code> public static void main(String[] args) { try{ countAllAltPicsNotSaved(); System.out.println("Not saved: " + totalAltPicsNotSaved); Class.forName("com.mysql.jdbc.Driver"); Connection conn =DriverManager.getConnection(dbUrl + dbName, userName, password); Statement s = conn.createStatement (); s.executeQuery ("SELECT DISTINCT id from productsaltimages WHERE saved != 1"); ResultSet rs = s.getResultSet(); int saveCounter = 0, altImageCount = 0; List&lt;Thread&gt; threads = new ArrayList&lt;Thread&gt;(); while (rs.next()) { //Get ID altImageCount = 0; //Product Alt Image Counter String id = rs.getString("id"); //Product Table ID Statement news = conn.createStatement (); //New Conn for get Alt Images From ID news.executeQuery ("SELECT url from productsaltimages WHERE id ='"+id+"' AND saved != 5"); //Gets query from mySQL ResultSet newrs = news.getResultSet(); //Resultset for AltImges for ID ExecutorService executor = Executors.newFixedThreadPool(5); Runnable task = null; while (newrs.next()){ //Get Images altImageCount++; //Increment ID Alt Image Counter String url = newrs.getString("url"); task = new DownloadImage(url, id, altImageCount); executor.execute(task); } } } catch (Exception e){ } } public static void countAllAltPicsNotSaved() throws Exception { try{ totalAltPicsNotSaved=0; Class.forName("com.mysql.jdbc.Driver"); Connection conn =DriverManager.getConnection(dbUrl + dbName, userName, password); Statement s = conn.createStatement (); boolean sqlExecuteok = false; s.executeQuery ("SELECT * from productsaltimages WHERE saved = '0'"); ResultSet rs = s.getResultSet (); while (rs.next()) { totalAltPicsNotSaved++; } rs.close(); s.close(); conn.close(); } catch (Exception e){ } } For my DownloadImage class code is: public class DownloadImage implements Runnable { String url, id, threadName; private int altImageCount = 0; private static String userName = "owner"; private static String password = "hello123"; private static String dbUrl = "jdbc:mysql://localhost/"; private static String dbName = "shop"; private static String dbClass = "com.mysql.jdbc.Driver"; public DownloadImage(String url, String id, int altImageCount) { this.url = url; this.id = id; this.altImageCount = altImageCount; } public void run() { while(true) { File file=new File("D:\\FBShop_AltImages\\" + id + "\\"); boolean exists = file.exists(); if (!exists) { // Create multiple directories boolean success = (new File("D:\\FBShop_AltImages\\" + id + "\\")).mkdirs(); } String newFilename = "D:\\FBShop_AltImages\\" + id + "\\" + id + "_" + altImageCount + ".jpg"; try { try { BufferedImage image = null; URL imgurl = new URL(url); URLConnection con = imgurl.openConnection(); con.setConnectTimeout(50 * 10000); con.setReadTimeout(50 * 10000); InputStream in = con.getInputStream(); image = ImageIO.read(in); ImageIO.write(image, "jpg", new File(newFilename)); } catch (Exception e) { System.out.println("Error getting image: " + url); } try { //UpdateTable Class.forName("com.mysql.jdbc.Driver"); String updatesql = "UPDATE productsaltimages SET saved = '1' WHERE id = '"+id+"'"; Connection conn =DriverManager.getConnection(dbUrl + dbName, userName, password); Statement ups = conn.createStatement (); int val = ups.executeUpdate(updatesql); System.out.println("Task Complete: " + url); try { Thread.sleep(5000); } catch (Exception e) { } } catch (Exception e) { } } catch (Exception e) { } //System.out.println("Thread Finished: " + threadName); } } } </code></pre>
    singulars
    1. PO
      singulars
      1. PTAnswer
    1. This table or related slice is empty.
    1. PTQuestion
    1. USfkerber
    1. USJeffrey Holmes
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PO
      singulars
      1. PTAnswer
    1. VO
      singulars
      1. POJava * Multithreading with MySQL
      1. This table or related slice is empty.
      1. VTApproveEditSuggestion
    2. VO
      singulars
      1. POJava * Multithreading with MySQL
      1. This table or related slice is empty.
      1. VTUpMod
    1. This table or related slice is empty.
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload