Note that there are some explanatory texts on larger screens.

plurals
  1. PO
    primarykey
    data
    text
    <p>I think I understand the question... here's what I would do: wrap another RecordReader and buffer the keys/values from it into a local queue.</p> <pre><code>public class MyRecordReader extends RecordReader&lt;LongWritable, Text&gt; { private static final int BUFFER_SIZE = 5; private static final String DELIMITER = ":"; private Queue&lt;String&gt; valueBuffer = new LinkedList&lt;String&gt;(); private Queue&lt;Long&gt; keyBuffer = new LinkedList&lt;Long&gt;(); private LongWritable key = new LongWritable(); private Text value = new Text(); private RecordReader&lt;LongWritable, Text&gt; rr; public MyRecordReader(RecordReader&lt;LongWritable, Text&gt; rr) { this.rr = rr; } @Override public void close() throws IOException { rr.close(); } @Override public LongWritable getCurrentKey() throws IOException, InterruptedException { return key; } @Override public Text getCurrentValue() throws IOException, InterruptedException { return value; } @Override public float getProgress() throws IOException, InterruptedException { return rr.getProgress(); } @Override public void initialize(InputSplit arg0, TaskAttemptContext arg1) throws IOException, InterruptedException { rr.initialize(arg0, arg1); } @Override public boolean nextKeyValue() throws IOException, InterruptedException { if (valueBuffer.isEmpty()) { while (valueBuffer.size() &lt; BUFFER_SIZE) { if (rr.nextKeyValue()) { keyBuffer.add(rr.getCurrentKey().get()); valueBuffer.add(rr.getCurrentValue().toString()); } else { return false; } } } else { if (rr.nextKeyValue()) { keyBuffer.add(rr.getCurrentKey().get()); valueBuffer.add(rr.getCurrentValue().toString()); keyBuffer.remove(); valueBuffer.remove(); } else { return false; } } key.set(keyBuffer.peek()); value.set(getValue()); return true; } private String getValue() { StringBuilder sb = new StringBuilder(); Iterator&lt;String&gt; iter = valueBuffer.iterator(); while (iter.hasNext()) { sb.append(iter.next()); if (iter.hasNext()) sb.append(DELIMITER); } return sb.toString(); } } </code></pre> <p>Then for example, you can have a custom InputFormat that extends TextInputFormat and overrides the <code>createRecordReader</code> method to call <code>super.createRecordReader</code> and return that result wrapped in a <code>MyRecordReader</code>, like this:</p> <pre><code>public class MyTextInputFormat extends TextInputFormat { @Override public RecordReader&lt;LongWritable, Text&gt; createRecordReader( InputSplit arg0, TaskAttemptContext arg1) { return new MyRecordReader(super.createRecordReader(arg0, arg1)); } } </code></pre>
    singulars
    1. This table or related slice is empty.
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. VO
      singulars
      1. This table or related slice is empty.
    2. VO
      singulars
      1. This table or related slice is empty.
    3. VO
      singulars
      1. This table or related slice is empty.
 

Querying!

 
Guidance

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload