Databases
StackOverflow2013
Postsdbo
PO
 

Note that there are some explanatory texts on larger screens.

plurals
  1. PO
    primarykey
    data
    text
    <p>After the outline setup by Mark I figured out the last part by looking at <a href="http://git.openstreetmap.nl/index.cgi/pbf2osm.git/tree/src/main.c?h=35116112eb0066c7729a963b292faa608ddc8ad7" rel="nofollow">http://git.openstreetmap.nl/index.cgi/pbf2osm.git/tree/src/main.c?h=35116112eb0066c7729a963b292faa608ddc8ad7</a></p> <p>Here is the final code.</p> <pre><code>using System; using System.Diagnostics; using System.IO; using crosby.binary; using OSMPBF; using PerlLLC.Tools; using ProtoBuf; using zlib; namespace OpenStreetMapOperations { class OpenStreetMapParser { static void Main() { using (var file = File.OpenRead(StaticTools.AssemblyDirectory + @"\us-pacific.osm.pbf")) { // from http://wiki.openstreetmap.org/wiki/ProtocolBufBinary: //A file contains a header followed by a sequence of fileblocks. The design is intended to allow future random-access to the contents of the file and skipping past not-understood or unwanted data. //The format is a repeating sequence of: //int4: length of the BlockHeader message in network byte order //serialized BlockHeader message //serialized Blob message (size is given in the header) int length, blockCount = 0; while (Serializer.TryReadLengthPrefix(file, PrefixStyle.Fixed32, out length)) { // I'm just being lazy and re-using something "close enough" here // note that v2 has a big-endian option, but Fixed32 assumes little-endian - we // actually need the other way around (network byte order): length = IntLittleEndianToBigEndian((uint)length); BlockHeader header; // again, v2 has capped-streams built in, but I'm deliberately // limiting myself to v1 features using (var tmp = new LimitedStream(file, length)) { header = Serializer.Deserialize&lt;BlockHeader&gt;(tmp); } Blob blob; using (var tmp = new LimitedStream(file, header.datasize)) { blob = Serializer.Deserialize&lt;Blob&gt;(tmp); } if (blob.zlib_data == null) throw new NotSupportedException("I'm only handling zlib here!"); HeaderBlock headerBlock; PrimitiveBlock primitiveBlock; using (var ms = new MemoryStream(blob.zlib_data)) using (var zlib = new ZLibStream(ms)) { if (header.type == "OSMHeader") headerBlock = Serializer.Deserialize&lt;HeaderBlock&gt;(zlib); if (header.type == "OSMData") primitiveBlock = Serializer.Deserialize&lt;PrimitiveBlock&gt;(zlib); } blockCount++; Trace.WriteLine("Read block " + blockCount.ToString()); } Trace.WriteLine("all done"); } } // 4-byte number static int IntLittleEndianToBigEndian(uint i) { return (int)(((i &amp; 0xff) &lt;&lt; 24) + ((i &amp; 0xff00) &lt;&lt; 8) + ((i &amp; 0xff0000) &gt;&gt; 8) + ((i &gt;&gt; 24) &amp; 0xff)); } } abstract class InputStream : Stream { protected abstract int ReadNextBlock(byte[] buffer, int offset, int count); public sealed override int Read(byte[] buffer, int offset, int count) { int bytesRead, totalRead = 0; while (count &gt; 0 &amp;&amp; (bytesRead = ReadNextBlock(buffer, offset, count)) &gt; 0) { count -= bytesRead; offset += bytesRead; totalRead += bytesRead; pos += bytesRead; } return totalRead; } long pos; public override void Write(byte[] buffer, int offset, int count) { throw new NotImplementedException(); } public override void SetLength(long value) { throw new NotImplementedException(); } public override long Position { get { return pos; } set { if (pos != value) throw new NotImplementedException(); } } public override long Length { get { throw new NotImplementedException(); } } public override void Flush() { throw new NotImplementedException(); } public override bool CanWrite { get { return false; } } public override bool CanRead { get { return true; } } public override bool CanSeek { get { return false; } } public override long Seek(long offset, SeekOrigin origin) { throw new NotImplementedException(); } } class ZLibStream : InputStream { // uses ZLIB.NET: http://www.componentace.com/download/download.php?editionid=25 private ZInputStream reader; // seriously, why isn't this a stream? public ZLibStream(Stream stream) { reader = new ZInputStream(stream); } public override void Close() { reader.Close(); base.Close(); } protected override int ReadNextBlock(byte[] buffer, int offset, int count) { // OMG! reader.Read is the base-stream, reader.read is decompressed! yeuch return reader.read(buffer, offset, count); } } // deliberately doesn't dispose the base-stream class LimitedStream : InputStream { private Stream stream; private long remaining; public LimitedStream(Stream stream, long length) { if (length &lt; 0) throw new ArgumentOutOfRangeException("length"); if (stream == null) throw new ArgumentNullException("stream"); if (!stream.CanRead) throw new ArgumentException("stream"); this.stream = stream; this.remaining = length; } protected override int ReadNextBlock(byte[] buffer, int offset, int count) { if (count &gt; remaining) count = (int)remaining; int bytesRead = stream.Read(buffer, offset, count); if (bytesRead &gt; 0) remaining -= bytesRead; return bytesRead; } } } </code></pre>
    singulars
    1. This table or related slice is empty.
    1. POProtobuf-net Deserialize Open Street Maps
      singulars
      1. PTQuestion
    1. PTAnswer
    1. This table or related slice is empty.
    1. USjonperl
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. CODid you have any problems reading nodes in during de-serialization? This code runs for me without error, however when look for data in primitiveBlock I get nothing.
      singulars
      1. PO
      1. USninehundredt
    2. COSorry, I never got a notification. Did you figure this out? I remember being able to access the data. Although we are no longer using this code.
      singulars
      1. PO
      1. USjonperl
    3. COI did finally get the code working after looking at another project, but we decided to go with another solution after running into more problems with open street maps.
      singulars
      1. PO
      1. USninehundredt
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload