Databases
StackOverflow2013
Postsdbo
POBittorrent implementation in java && need some info on swarm behaviour
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POBittorrent implementation in java && need some info on swarm behaviour
    primarykey
    data
    text
    <p>I'm developing a bitTorrent client in Java. I know there are a lot of libraries out there, online, but I can't help it; I want my own. Anyway, I noticed some weird behaviors and maybe you guys know something that I'm missing: </p> <ul> <li>About 80% of all peers I'm trying to connect to result in unsuccessful connections (either <code>socketTimeOut</code> or "can't connect" errors). Obviously, the list of peers is received from the trackers. I also tested randomly some IPs by trying to ping them; the ping is usually successful.</li> <li>When I do connect: <ul> <li>50% drop connection after HandShake, </li> <li>on 30% I noticed a weird behaviour: I receive Handshake, I receive BitField (they have all pieces), I get bombarded with +20 Have messages (I checked the index of piece they already mentioned this in BitField), then they drop connection, which is weird.</li> </ul></li> </ul> <p>(For all statistics, figures are not precise.)</p> <p>Some BitTorrent questions:</p> <p><strong>UPDATE #4:</strong> im cutting off some questions due to considering answer found</p> <ul> <li><p>this was the '80% failed connect rate question': <del> What could be reason of my 80% failed to connect rate? This can't be bad luck, in the sense that every client I tried to connect had no more room for me. I'm listening on 6881, but also tested with other ports. Yesterday I had great success, a bunch of connections accepted (same code, a few changes in past week), Piece messages started flowing.. so my code is not totally useless. </del></p></li> <li><p>Do torrent clients send, before closing, a last message to tracker with <code>event=stopped</code> to make it update its internal database with peer info so that it won't send, as a response, a list with useless peer info? Or just they should.. because it really seems I'm receiving dead peers.</p></li> <li>Is the order of received peers of any importance? Maybe percentage of completion.. or really random. </li> <li>Also, every now and then I receive a peer with port 0, which makes my Socket constructor throw an exception. What does port 0 mean ? Can I contact it on any port?</li> <li>Can my PeerId (that I send in Handshake or announce my self to the tracker) influence if the torrent clients I' trying to communicate will continue a started connection? Meaning what if I lie and say that I'm an Azureus client by using '-AZ2060-' as my ID?</li> <li>this was the 'piece availability scaring off peers question': <del> Does my piece availability scare off peers? I'm trying to connect, and I send a empty bitfield (I have no pieces, <code>[length: 1][Id = 5][payload: {}]</code>); it seems that they send bitfield, I send bitfield.. (some send like crazy Have messages), they realise I'm poor, they drop me.. some drop connection after handshake. (How rude.) </del></li> <li>Is there a benefit of not using the classic port interval: 6881 - 6889?</li> <li>this was the 'list of bad peers question': <del> Do torrent clients keep internally a list of bad peers (like a black list)? Sometimes after finding a nice peer, I continually used its info in my tests but only 1/3 connection was accepted. Sometimes 10 minutes had to pass to have a successful connection again. </del></li> </ul> <p><strong>UPDATE #1:</strong> it seems that connections with &mu;Torrent clients behave in the aforementioned pattern (BITFIELD, HAVE bombardment, close connection). I tested locally with a bunch of bitTorrent clients (&mu;Torrent, BitTorrent, Vuze, BitCommet, Deluge) and only noticed this pattern on &mu;Torrent. On the others, communication was fine (HS, BITFIELD, UNCHOCE &amp; happy piece sharing). Now, this &mu;Torrent is probably the most popuar bitTorrent client (6/8 connections started were &mu;Torrent), so&hellip; any ideas?</p> <p><strong>UPDATE #2:</strong> In terms of keeping a <code>"bad list,"</code> it does seem so (and it actually makes sense to do so). For example, with &mu;Torrent, I noticed the following no-connect intervals (30s, 1min, 1min30s, 2min.. ). By "no-connect" in mean, after previous connection ended, for <code>x</code> seconds no new connection was accepted.</p> <p><strong>UPDATE #3:</strong> That HAVE message bombardment might have been the so-called "lazy bitfield" (did a couple of tests, each piece mentioned in HAVE was not present in BITFIELD). I see that &mu;Torrent and BitTorrent use this approach.</p> <p><strong>Another conclusion</strong>: Some clients are more restrictive in terms of respecting the BitTorrent specs and will close connection if you break a rule. <strong>Ex:</strong> I noticed with BitTorrent and BitTornado that if you send a bitfield message but have no pieces they will close connection (no pieces = empty bitfield.. but specs say "It is optional, and need not be sent if a client has no pieces"), while others close connection if you send any type of msg before they send a UNCHOKE msg (not even INTERESTED).</p> <p><strong>UPDATE #4:</strong> Since I'm mostly interested in first question (What could be reason of my 80% failed to connect rate?.. the striked questions are more than probably liked), here are some explanations of why sometimes connections were unsuccesfully:</p> <p>1) if I start a connection with peer shortly after stopping a previous connection (by stop - I mean close socket): the peer on the other side wont know until next read/write.</p> <p>Details: - I noticed this a bunch of times, this is more obvious after finishing a download.. if I close connection peer won't realise this until it tries to send a new KEEP_ALIVE (~2 minutes). But if I close while in an exchange REQUEST-PIECE, peer will realise pretty fast.. In first scenario after closing connection, I am still present in uTorrent peer tab. If I look inside the logger tab, after about 2 minutes, it will realise that I am gone.</p> <p>2) it seems that uTorrent sees my BITFIELD message corrupted (&amp; obvious should close connection after receiving it) (this doesn't happen always.. also I checked &amp; rechecked, msg is OK &amp; with other BT client there were no such problems).</p> <p>Details: - if I look inside uTorrent logger tab, it displays "Disconnected: Bad packet" right after I send bitfield - I'm planning to try an implementation of lazzy bitfield, maybe I can escape this (also I see that majority of BT clients do this)</p> <p>3) (more than probably linked to #1) when uTorrent doesn't allow me to re-connect, I see in logger tab: "Disconnect: already have equal connection (dropped extra connection)".. Currently i choose random local port when initing a new connection (saw this implemented in the majority of BT clients), but this doesn't trick it, he still sees that im a peer already present in his "peer list" (probably does ip match).. Buuut: in 30% of the tests, same scenario, it does allow me to reconnect :) .. I have no explanations why yet</p> <p>4) one more thing: it seems that the 'listener for incomming connexions' is still alive after you close a torrent in uTorrent (by close I mean: right click + stop). This means that I can still start a connection, send HANDSHAKE.. after this, I'm disconnected (it doesn't HANDSHAKE back). Message in uTorrent logger: "Disconnect: No such torrent: 80FF40A75A3B907C0869B798781D97938CE146AE", this long string being my info hash.. seen this while testing with other BT clients too.</p> <p><strong>Some more info</strong>:</p> <ul> <li>scenarios with uTorrent of type full-upload/partial-upload &amp; full-download are successful, those of partial-download not so much.. probably due to #2</li> <li>I still get with uTorrent that bitField + have bombardment + close connection.. as I remember the same msg in logger tab "Disconnected: Bad packet".. probably due to #2</li> <li>besides uTorrent, I've tested with: BitTorrent, BitTornado, BitCommet, qBitTorrent, FlashGet (communication was OK) &amp; with Vuze, FrostWire, Shareaza (with these guys, it was super OK).</li> <li>not all clients behave the same. Ex: FlashGet &amp; uTorrent (&amp; BitCommet?) don't unchoke until you send INTERESTED.. while others seem to unchoke right after BITFIELD.. in this sense I'm planning somehow to treat clients differently (i really think this is necessary).. probably guess their name from the bitfield (there are only 2 naming conventions) &amp; start from there.. I already have something implemented, this is how I know that I connected to client of type uTorrent..</li> </ul>
    singulars
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PTQuestion
    1. This table or related slice is empty.
    1. USpulancheck1988
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PO
      singulars
      1. PTAnswer
    1. VO
      singulars
      1. POBittorrent implementation in java && need some info on swarm behaviour
      1. This table or related slice is empty.
      1. VTUpMod
    2. VO
      singulars
      1. POBittorrent implementation in java && need some info on swarm behaviour
      1. This table or related slice is empty.
      1. VTUpMod
    3. VO
      singulars
      1. POBittorrent implementation in java && need some info on swarm behaviour
      1. This table or related slice is empty.
      1. VTUpMod
    1. COLots of questions. Is there a torrent specification somewhere?
      singulars
      1. POBittorrent implementation in java && need some info on swarm behaviour
      1. USSotirios Delimanolis
    2. COhttps://wiki.theory.org/BitTorrentSpecification, http://www.bittorrent.org/beps/bep_0003.html ..
      singulars
      1. POBittorrent implementation in java && need some info on swarm behaviour
      1. USpulancheck1988
    3. COI think digging through a successful Bittorrent client´s sources is a key to success. There are so many rules other clients implicitely enforce to get rid of leechers, and so on, that it might be impossible to implement a universal client by just following the spec...
      singulars
      1. POBittorrent implementation in java && need some info on swarm behaviour
      1. USTheBlastOne
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload