Databases
StackOverflow2013
Postsdbo
POUsing FILTER after FOREACH in Pig-Latin failed
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POUsing FILTER after FOREACH in Pig-Latin failed
    primarykey
    data
    text
    <p>I am a beginner in Pig-Latin and I found a problem about FILTER statement. Look at the example:</p> <p>Suppose we have a data file(test.txt) whose content is:</p> <pre><code>1,2,3 2,3,4 3,4,5 4,5,6 </code></pre> <p>I want to select the records whose the 1st field is '3'. The Pig script is:</p> <pre><code>t = LOAD 'test.txt' USING PigStorage(','); t1 = FOREACH t GENERATE $0 AS i0:chararray, $1 AS i1:chararray, $2 AS i2:chararray; f1 = FILTER t1 BY i0 == '3'; DUMP f1 </code></pre> <p>The task runs well but the output result is nothing. EXPLAIN f1 shows:</p> <pre><code>#-------------------------------------------------- # Map Reduce Plan #-------------------------------------------------- MapReduce node scope-27 Map Plan f1: Store(fakefile:org.apache.pig.builtin.PigStorage) - scope-26 | |---f1: Filter[bag] - scope-22 | | | Equal To[boolean] - scope-25 | | | |---Project[chararray][0] - scope-23 | | | |---Constant(3) - scope-24 | |---t1: New For Each(false,false,false)[bag] - scope-21 | | | Project[bytearray][0] - scope-15 | | | Project[bytearray][1] - scope-17 | | | Project[bytearray][2] - scope-19 | |---t: Load(file:///Users/woody/test.txt:PigStorage(',')) - scope-14-------- Global sort: false ---------------- </code></pre> <p>However, if I change the head 2 lines into:</p> <pre><code>t1 = LOAD 'test.txt' USING PigStorage(',') AS (i0:chararray, i1:chararray, i2:chararray) </code></pre> <p>(i.e. assign the schema in LOAD statement)</p> <p>The task works well and the result is also correct. In this case, the EXPLAIN f1 shows:</p> <pre><code>#-------------------------------------------------- # Map Reduce Plan #-------------------------------------------------- MapReduce node scope-33 Map Plan f1: Store(fakefile:org.apache.pig.builtin.PigStorage) - scope-32 | |---f1: Filter[bag] - scope-28 | | | Equal To[boolean] - scope-31 | | | |---Project[chararray][0] - scope-29 | | | |---Constant(3) - scope-30 | |---t1: New For Each(false,false,false)[bag] - scope-27 | | | Cast[chararray] - scope-19 | | | |---Project[bytearray][0] - scope-18 | | | Cast[chararray] - scope-22 | | | |---Project[bytearray][1] - scope-21 | | | Cast[chararray] - scope-25 | | | |---Project[bytearray][2] - scope-24 | |---t1: Load(file:///Users/woody/test.txt:PigStorage(',')) - scope-17-------- Global sort: false ---------------- </code></pre> <p>Is it a Pig's bug? or is there any good way to avoid it ?</p> <p>pig --version on my computer is:</p> <pre><code>Apache Pig version 0.9.2 (r1232772) compiled Jan 18 2012, 07:57:19 </code></pre>
    singulars
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PTQuestion
    1. USWoody Wang
    1. USWoody Wang
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PO
      singulars
      1. PTAnswer
    2. PO
      singulars
      1. PTAnswer
    1. This table or related slice is empty.
    1. COIt may be immature, but I LOL'd.
      singulars
      1. POUsing FILTER after FOREACH in Pig-Latin failed
      1. USVladislav Zorov
    2. COWhat's so funny?:D
      singulars
      1. POUsing FILTER after FOREACH in Pig-Latin failed
      1. USThomas Jungblut
    3. COIs there anything preventing you from doing with the second route and assigning the schema in the LOAD statement with AS? That seems like not only a viable solution but also more efficient coding.
      singulars
      1. POUsing FILTER after FOREACH in Pig-Latin failed
      1. USc.gutierrez
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload