Databases
StackOverflow2013
Postsdbo
PO
 

Note that there are some explanatory texts on larger screens.

plurals
  1. PO
    primarykey
    data
    text
    <p>Hmm, </p> <p>This is a bit like a math problem. The question of how two computers <strong>establish a connection</strong> once they find each other is fairly straightforward. You can use any number of P2P or client-server protocols. <a href="http://www.apache-ssl.org/" rel="nofollow noreferrer">SSL</a> is almost universally available but you could also serve <a href="http://www.openssh.com/" rel="nofollow noreferrer">SSH</a>, run <a href="http://freenetproject.org/" rel="nofollow noreferrer">Freenet</a> or whatever. Once you establish a connection through one of these protocols, a publish/Subscribe model for <strong>exchanging data</strong> could work well. But there is </p> <p>The question of how the computers FIND each other is where things get tricky. There are essentially three cases:</p> <ol> <li><p><strong>All your computers will be on your own local area network.</strong> In this case, any computer that goes online can broadcast a message that it is online and get message back concerning what other machines are online. Remember any broadcast/multicast message has to every machine on the network since it doesn't know what its target. You cannot do this on the internet since you cannot send a message to every machine on the net. </p></li> <li><p><strong>Your computers are on arbitrary nodes on the internet ** <em>and</em> ** there will always be at least one of your computers connected to the web ** <em>and</em> ** all machine go online on a regular basis.</strong> In this case, each machine can keep a running list of IP addresses. When a machine that's been offline for a while comes back online, it checks for the known addresses, connecting to the first valid address - this is how the emule protocol finds servers. </p></li> <li><p><strong>Your computers are on arbitrary nodes on the internet ** <em>and</em> ** all machines offline together ** <em>or</em> **quite a few go offline for long periods while otheers switch IP addresses.</strong> Here, I don't think that you can demonstrate that there is no way for new machines coming on to find each other <em>without</em> some central server to links to common IPs. There is no way to broadcast a message throughout the Internet because it is big. In these circumstances, computers have no identifying information for the other machines coming online so you need to use a central, common resource: the email address you mentioned, a website, an ftp server, an IRC channel. A dynamic DNS is just one more instance of a centralized information store. If you must use such a store, naturally you have to evaluate all the stores available for their reliability, speed and permanence. Unless you give more information about what your application needs in this regard, I don't think anyone else can decide which permanent store you need. </p></li> </ol> <p>I think this covers all the possibilities. What you have to do is decide which of these general cases your application falls under and then decide on one of protocols which fits this case. </p> <h2>Edit:</h2> <p>From feedback, it seems case 3 applies. From the reasoning you can see above, it should be clear that there is no way to avoid some form of external server - the only way to find a needle in a haystack is by keeping track of where it is. The qualities one would want in such a provider are:</p> <ul> <li>Reliability: How often is the thing up in a given day</li> <li>Speed: How quickly does the thing respond</li> <li>Permanence: How long do you expect the thing to last? Will you lose access to it as the Internet evolves?</li> </ul> <p>As mentioned, there are many easily available resources that more or less fit this bill. Email servers (as you are presently using), web servers, FTP servers, DNS servers, IRC channels, Twitter accounts, web forums....</p> <p>The problem of applications that come to life after a while and require updating without a central server is a common problem but it common mostly among virus writer - just about any organization which has the resources to create a distributed application also has resources to maintain a central server. That said, standard solutions over the years included email, http servers, ftp servers, IRC channels and dynamic DNS servers. Different servers in each of these categories vary in their speed, reliability and permanence so the task of choosing one goes back to your judgment. IRC channels deserve mention because they fast and easy to set up but these might indeed fade as the internet evolves. </p> <p>As an example of distribution application which uses a variety of "client finding" techniques, you can download <a href="http://www.bo2k.com/software/bo2k11.html" rel="nofollow noreferrer">the source to BO2K</a>, a, uh "remote administration utility". This might provide some insight into all of the functionality of your remote updating client. </p> <p>Just to repeat. I imagine there are three parts to your problem:</p> <ol> <li>The machines finding each other (see above)</li> <li>The machines establishing a connection (again, SSL, SSH and others are readily available)</li> <li>The machines exchanging data. You could use a "publish/subscribe" model or just roll your own simple <a href="http://en.wikipedia.org/wiki/Communications_protocol" rel="nofollow noreferrer">protocol</a>. I worked for company which had an auto-updating client and that's what we did. The reasons for creating your own protocol are 1) in even the simplest situations, the requirements for speed and reliability will vary as will the data exchanged, 2. The simplest exchange of data requires just a few lines of code and so no one bothers following protocols for really simple data exchange, 3. Since so different applications use different methods and languages, no protocol for simple data exchange is dominant. For more complex situations, there indeed a whole forest of protocols but their vary complexity would make them awkward to use for simple data exchange. The way that the <a href="http://git-scm.com/" rel="nofollow noreferrer">git scm</a> sends data is one example of a updating protocol. If it is just so happens that you database resembles the source code that git sends, you could use git to maintain your database. But the chances are your updating approach won't resemble what git does that closely. Another example protocol is one or another version of <a href="http://en.wikipedia.org/wiki/Web_service" rel="nofollow noreferrer">web services</a> such as SOAP. These protocols just <em>wrap</em> the process of calling a function on one machine using xml and http. If you can already establish socket communication between your applications, then there's no reason to do this. Remember, to implement web services you'd need to run an run an http server and parse the xml that an http clients into raw data. Considering you can send your data directly through a socket, there's no reason to do this. So you are back to rolling your own. </li> </ol> <p>Anyway, an example of a simple protocol might be: </p> <ul> <li>one app first sends an index of the data it has as an array of indexes. </li> <li>the other app sends a list of the items which are new it </li> <li>and then the first app sends those actual items. </li> </ul> <p>Then the apps change roles and exchange data the other way. A given "handshake" in your protocol would look like this in pseudo code:</p> <pre><code>void update_database(in_stream, out_stream) { Get_Index_Of_Other_Machines_Items(in_stream); Send_Index_Of_Items_You_Need(out_stream); Get_Items_You_Need(in_stream); } </code></pre> <p>With another function to implement the opposite side of these data exchanges. </p> <p>A further consideration is that if each entry in the database is being generated independently, you will need to generate an ID which is unique to this without being able to reference all of the items in the distributed database. You can generate a <a href="http://en.wikipedia.org/wiki/Globally_Unique_Identifier" rel="nofollow noreferrer">GUID</a> or just a large random number for this purpose. </p> <p>You will no doubt have to tweak and develop this further if you are going to use it.</p> <p>Keep in mind, however, that <em>if your applications are all only occasionally updating</em>, there will no way of being certain that any given instances will have any data item. For example, suppose, in a day, half the machines only go online <em>after</em> 5:00pm and the other half only go online <em>before</em> 5:00 pm. In this case, the two groups of machines will share no data whatsoever if they directly updating each other. If, on the other hand, your machines really do on at even distributed times (rather than according to pattern like I described), it is reasonably likely that each machine will eventually get all of the updates (at least all of the old updates). </p> <p>Given all this, I think you should consider exactly <em>how infrequently</em> your applications will be connecting and how important it is for all the data to be fully synchronized. In situations of extreme infrequency, you may need to use an external server to move your data as well as finding your clients. <strong>Regular email</strong> is natural solution to this problem. It would be one thing to use if no computer is ever online when another is online. If you are worried about access to a particular email accounts, you could start each app with a list of multiple email addresses, with all the addresses being checked and you having the option of updating the application with even more addresses as given addresses fail. Using email for everything would have the virtue of simplicity. You could use </p> <p>On a lighter side note, <a href="http://en.wikipedia.org/wiki/Numbers_station" rel="nofollow noreferrer">numbers stations</a> are a pre-Internet effort to solve the information updating problem but they also wouldn't fit your case (they do broadcast to a large portion of the world, however). </p>
    singulars
    1. This table or related slice is empty.
    1. POHow to do private comms between private apps over network?
      singulars
      1. PTQuestion
    1. PTAnswer
    1. USJoe Soul-bringer
    1. USJoe Soul-bringer
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. POHow to do private comms between private apps over network?
      singulars
      1. PTQuestion
    1. This table or related slice is empty.
    1. VO
      singulars
      1. PO
      1. This table or related slice is empty.
      1. VTUpMod
    2. VO
      singulars
      1. PO
      1. This table or related slice is empty.
      1. VTUpMod
    3. VO
      singulars
      1. PO
      1. This table or related slice is empty.
      1. VTUpMod
    1. COCase 3 applies. There are make-works available, but I would have thought this not to be such a unique problem. Surely there must be techniques available? or am I seeking snowballs in hell...
      singulars
      1. PO
      1. USslashmais
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload