Databases
StackOverflow2013
Postsdbo
POParse Python's serialized object in Perl
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POParse Python's serialized object in Perl
    primarykey
    data
    text
    <p>I need to make my Perl code read some Python's serialized object for later processing. I came with parser based on Parse::MGC, but it's slow. May be I did it wrong or may be someone knows better way to convert Python's serialized object in some sort of Perl structure?</p> <p>Here is my Parse code:</p> <pre><code>package Room::HandParser; use base qw( Parser::MGC ); my @poker_cards_string = ( '2h', '3h', '4h', '5h', '6h', '7h', '8h', '9h', 'Th', 'Jh', 'Qh', 'Kh', 'Ah', '2d', '3d', '4d', '5d', '6d', '7d', '8d', '9d', 'Td', 'Jd', 'Qd', 'Kd', 'Ax', '2c', '3c', '4c', '5c', '6c', '7c', '8c', '9c', 'Tc', 'Jc', 'Qc', 'Kc', 'Ac', '2s', '3s', '4s', '5s', '6s', '7s', '8s', '9s', 'Ts', 'Js', 'Qs', 'Ks', 'As' ); sub parse_declaration { my $self = shift; [ $self-&gt;any_of( sub { $self-&gt;token_int }, sub { $self-&gt;token_string }, ), $self-&gt;expect(":"), $self-&gt;parse, ] } sub parse_hash { my $self = shift; my %ret; $self-&gt;list_of(",", sub { my $res = $self-&gt;parse_declaration; $ret{$res-&gt;[0]} = $res-&gt;[2]; }); return \%ret; } sub parse_cards { my $self = shift; my $card = $self-&gt;token_int; return $poker_cards_string[$card &amp; 0x3F]; } sub parse { my $self = shift; $self-&gt;any_of( sub { $self-&gt;scope_of( "[", sub { $self-&gt;list_of(",", \&amp;parse) }, "]" ) }, sub { $self-&gt;scope_of( "(", sub { $self-&gt;list_of(",", \&amp;parse) }, ")" ) }, sub { $self-&gt;scope_of( "{", sub { $self-&gt;parse_hash }, "}" ) }, sub { $self-&gt;scope_of( "PokerCards([", sub { $self-&gt;list_of(",", \&amp;parse_cards) }, "])" ) }, sub { $self-&gt;token_float }, sub { $self-&gt;token_int }, sub { $self-&gt;token_string }, sub { $self-&gt;token_kw( qw(None True False) ) }, ); } 1; </code></pre> <p>Here is example of serialized Python object I need to parse:</p> <pre><code>[('game', 0, 195, 0, 0.0, 'holdem', '100-200-no-limit', [50312, 50313, 50314, 50315, 50316, 50317, 2], 0, {2: 1000000, 50312: 200000, 50313: 200000, 50314: 200000, 50315: 200000, 50316: 200000, 50317: 200000}), ('position', 1), ('blind', 50313, 10000, 0), ('position', 2), ('blind', 50314, 20000, 0), ('position', -1), ('round', 'pre-flop', PokerCards([]), {2: PokerCards([226, 208]), 50312: PokerCards([223, 206]), 50313: PokerCards([221, 233]), 50314: PokerCards([222, 211]), 50315: PokerCards([235, 216]), 50316: PokerCards([209, 236]), 50317: PokerCards([237, 243])}), ('position', 3), ('call', 50315, 20000), ('position', 4), ('call', 50316, 20000), ('position', 5), ('call', 50317, 20000), ('position', 6), ('call', 2, 20000), ('position', 0), ('fold', 50312), ('position', 1), ('call', 50313, 10000), ('position', 2), ('check', 50314), ('position', -1), ('round', 'flop', PokerCards([7, 21, 46]), {2: PokerCards([226, 208]), 50313: PokerCards([221, 233]), 50314: PokerCards([222, 211]), 50315: PokerCards([235, 216]), 50316: PokerCards([209, 236]), 50317: PokerCards([237, 243])}), ('position', 1), ('check', 50313), ('position', 2), ('check', 50314), ('position', 3), ('check', 50315), ('position', 4), ('check', 50316), ('position', 5), ('check', 50317), ('position', 6), ('check', 2), ('position', -1), ('round', 'turn', PokerCards([7, 21, 46, 38]), None), ('position', 1), ('check', 50313), ('position', 2), ('check', 50314), ('position', 3), ('check', 50315), ('position', 4), ('check', 50316), ('position', 5), ('check', 50317), ('position', 6), ('check', 2), ('position', -1), ('round', 'river', PokerCards([7, 21, 46, 38, 20]), None), ('position', 1), ('check', 50313), ('position', 2), ('check', 50314), ('position', 3), ('check', 50315), ('position', 4), ('check', 50316), ('position', 5), ('check', 50317), ('position', 6), ('check', 2), ('position', -1), ('showdown', None, {2: PokerCards([226, 208]), 50313: PokerCards([29, 41]), 50314: PokerCards([222, 211]), 50315: PokerCards([43, 24]), 50316: PokerCards([209, 236]), 50317: PokerCards([45, 51])}), ('end', [50317], [{'serial2delta': {2: -20000, 50313: -20000, 50314: -20000, 50315: -20000, 50316: -20000, 50317: 100000}, 'player_list': [50312, 50313, 50314, 50315, 50316, 50317, 2], 'serial2rake': {50317: 0}, 'serial2share': {50317: 120000}, 'pot': 120000, 'serial2best': {2: {'hi': [101154816, ['FlHouse', 46, 20, 7, 34, 21]]}, 50313: {'hi': [50841600, ['Trips', 46, 20, 7, 38, 21]]}, 50314: {'hi': [50841600, ['Trips', 46, 20, 7, 38, 21]]}, 50315: {'hi': [50842368, ['Trips', 46, 20, 7, 38, 24]]}, 50316: {'hi': [50841600, ['Trips', 46, 20, 7, 38, 21]]}, 50317: {'hi': [101171200, ['FlHouse', 46, 20, 7, 51, 38]]}}, 'type': 'game_state', 'side_pots': {'building': 0, 'pots': [[120000, 120000]], 'last_round': 3, 'contributions': {0: {0: {2: 20000, 50313: 20000, 50314: 20000, 50315: 20000, 50316: 20000, 50317: 20000}}, 1: {}, 2: {}, 'total': {2: 20000, 50313: 20000, 50314: 20000, 50315: 20000, 50316: 20000, 50317: 20000}, 3: {}}}}, {'serials': [50313, 50314, 50315, 50316, 50317, 2], 'pot': 120000, 'hi': [50317], 'chips_left': 0, 'type': 'resolve', 'serial2share': {50317: 120000}}])] </code></pre> <p>For such structure it takes several seconds and 100% CPU to parse this object which is not acceptable in my case.</p> <p>EDIT: here I am NOT looking for workaround like writing python script for eval'ing this strucutre and output it JSON, or rewrite original Python app with added functions to store data as JSON. I am looking into parsing this data with Perl with reasonable performance, since this format is pretty close to JSON and it should be possible to parse it in similar time.</p>
    singulars
    1. PO
      singulars
      1. PTAnswer
    1. This table or related slice is empty.
    1. PTQuestion
    1. USPavel Karoukin
    1. USPavel Karoukin
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PO
      singulars
      1. PTAnswer
    2. PO
      singulars
      1. PTAnswer
    1. VO
      singulars
      1. POParse Python's serialized object in Perl
      1. This table or related slice is empty.
      1. VTUpMod
    2. VO
      singulars
      1. POParse Python's serialized object in Perl
      1. This table or related slice is empty.
      1. VTUpMod
    1. CO`repr(ob)`'s output is not a serialization format, it's a representation made for the ease of the programmer. Using `repr`/`eval` combos is not a good idea even in Python. Use a real serialization format (e.g. JSON) that's supported by both Python and Perl.
      singulars
      1. POParse Python's serialized object in Perl
      1. USRosh Oxymoron
    2. COThat's what I have. I am not working on Python part, but need to deal with data generated with it.
      singulars
      1. POParse Python's serialized object in Perl
      1. USPavel Karoukin
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload