Databases
StackOverflow2013
Postsdbo
POMoving values between rows without a for loop in R
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POMoving values between rows without a for loop in R
    primarykey
    data
    text
    <p>I have written some code used to organize data sampled at different frequencies, but I made extensive use of for-loops, which slow the code's operation down significantly when the data set is large. I've been going through my code, finding ways to remove for-loops to speed it up, but one of the loops has got me stumped.</p> <p>As an example, let's say the data was sampled at 3Hz, so I get three rows for every second of data. However, the variables A, B, and C are sampled at 1Hz each, so I will get one value every three rows for each of them. The variables are sampled consecutively within the one second period, resulting in a diagonal nature to the data.</p> <p>To further complicate things, sometimes a row is lost in the original data set.</p> <p>My goal is this: Having identified the rows that I wish to keep, I want to move the non-NA values from the subsequent rows up into the keeper rows. If it weren't for the lost data issue, I would always keep the row containing a value for the first variable, but if one of these rows is lost, I will be keeping the next row.</p> <p>In the example below, the sixth sample and the tenth sample are lost.</p> <pre><code>A &lt;- c(1, NA, NA, 4, NA, 7, NA, NA, NA, NA) B &lt;- c(NA, 2, NA, NA, 5, NA, 8, NA, 11, NA) C &lt;- c(NA, NA, 3, NA, NA, NA, NA, 9, NA, 12) test_df &lt;- data.frame(A = A, B = B, C = C) test_df A B C 1 1 NA NA 2 NA 2 NA 3 NA NA 3 4 4 NA NA 5 NA 5 NA 6 7 NA NA 7 NA 8 NA 8 NA NA 9 9 NA 11 NA 10 NA NA 12 keep_rows &lt;- c(1, 4, 6, 9) </code></pre> <p>After I move the values up into the keeper rows, I will remove the interim rows, resulting in the following:</p> <pre><code>test_df &lt;- test_df[keep_rows, ] test_df A B C 1 1 2 3 2 4 5 NA 3 7 8 9 4 NA 11 12 </code></pre> <p>In the end, I only want one row for each second of data, and NA values should only remain where a row of the original data was lost.</p> <p>Does anyone have any ideas of how to move the data up without using a for-loop? I'd appreciate any help! Sorry if this question is too wordy; I wanted to err on the side of too much information rather than not enough.</p>
    singulars
    1. PO
      singulars
      1. PTAnswer
    1. This table or related slice is empty.
    1. PTQuestion
    1. This table or related slice is empty.
    1. USPatrigon
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PO
      singulars
      1. PTAnswer
    2. PO
      singulars
      1. PTAnswer
    3. PO
      singulars
      1. PTAnswer
    1. VO
      singulars
      1. POMoving values between rows without a for loop in R
      1. This table or related slice is empty.
      1. VTUpMod
    2. VO
      singulars
      1. POMoving values between rows without a for loop in R
      1. This table or related slice is empty.
      1. VTUpMod
    3. VO
      singulars
      1. POMoving values between rows without a for loop in R
      1. This table or related slice is empty.
      1. VTUpMod
    1. COTo clarify: Can two or more consecutive rows be lost? If, say, instead of dropping 6 and 10, you dropped 4, 5, and 6, how would you detect that that had happened?
      singulars
      1. POMoving values between rows without a for loop in R
      1. USgoodside
    2. COYes, multiple consecutive rows can be lost. I've identified which rows to keep after taking that into account elsewhere in my code, and I would have ended up with something like the "rows_to_keep" variable that I gave in my example being generated by the code instead of being user-defined. I'm not sure if that would cause an issue in the solutions given, since the "rows_to_keep" variable wasn't implemented.
      singulars
      1. POMoving values between rows without a for loop in R
      1. USPatrigon
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload