Databases
StackOverflow2013
Postsdbo
PO
 

Note that there are some explanatory texts on larger screens.

plurals
  1. PO
    primarykey
    data
    text
    <p>You could use <a href="http://en.wikipedia.org/wiki/Bootstrapping_%28statistics%29" rel="noreferrer">bootstrapping</a> to estimate <a href="http://en.wikipedia.org/wiki/Confidence_interval" rel="noreferrer">confidence intervals</a>. MATLAB provides <a href="http://www.mathworks.com/help/stats/bootci.html" rel="noreferrer"><code>bootci</code></a> function in the Statistics toolbox. Here is an example:</p> <pre><code>%# generate a random cell array of 400 sequences of varying length %# each containing indices from 1 to 5 corresponding to ACGTE sequences = arrayfun(@(~) randi([1 5], [1 randi([500 1000])]), 1:400, ... 'UniformOutput',false)'; %# compute transition matrix from all sequences trans = countFcn(sequences); %# number of bootstrap samples to draw Nboot = 1000; %# estimate 95% confidence interval using bootstrapping ci = bootci(Nboot, {@countFcn, sequences}, 'alpha',0.05); ci = permute(ci, [2 3 1]); </code></pre> <p>We get:</p> <pre><code>&gt;&gt; trans %# 5x5 transition matrix: P_hat trans = 0.19747 0.2019 0.19849 0.2049 0.19724 0.20068 0.19959 0.19811 0.20233 0.19928 0.19841 0.19798 0.2021 0.2012 0.20031 0.20077 0.19926 0.20084 0.19988 0.19926 0.19895 0.19915 0.19963 0.20139 0.20088 </code></pre> <p>and two other similar matrices containing the lower and upper bounds of confidence intervals:</p> <pre><code>&gt;&gt; ci(:,:,1) %# CI lower bound &gt;&gt; ci(:,:,2) %# CI upper bound </code></pre> <p>I am using the following function to compute the transition matrix from a set of sequences:</p> <pre><code>function trans = countFcn(seqs) %# accumulate transition matrix from all sequences trans = zeros(5,5); for i=1:numel(seqs) trans = trans + sparse(seqs{i}(1:end-1), seqs{i}(2:end), 1, 5,5); end %# normalize into proper probabilities trans = bsxfun(@rdivide, trans, sum(trans,2)); end </code></pre> <p>As a bonus, we can use <code>bootstrp</code> function to get the statistic computed from each bootstrap sample, which we use to show a histogram for each of the entries in the transition matrix:</p> <pre><code>%# compute multiple transition matrices using bootstrapping stat = bootstrp(Nboot, @countFcn, sequences); %# display histogram for each entry in the transition matrix sub = reshape(1:5*5,5,5); figure for i=1:size(stat,2) subplot(5,5,sub(i)) hist(stat(:,i)) end </code></pre> <p><img src="https://i.stack.imgur.com/f6JUf.png" alt="bootstrap_histograms"></p>
    singulars
    1. This table or related slice is empty.
    1. POEstimating confidence intervals of a Markov transition matrix
      singulars
      1. PTQuestion
    1. PTAnswer
    1. This table or related slice is empty.
    1. USAmro
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. POEstimating confidence intervals of a Markov transition matrix
      singulars
      1. PTQuestion
    1. This table or related slice is empty.
    1. VO
      singulars
      1. PO
      1. This table or related slice is empty.
      1. VTUpMod
    2. VO
      singulars
      1. PO
      1. This table or related slice is empty.
      1. VTUpMod
    3. VO
      singulars
      1. PO
      1. This table or related slice is empty.
      1. VTAcceptedByOriginator
    1. COThis is fantastic! Thank you so much! Exactly why it's not in the stats forum! Giving bounty as soon as available! However, I do have a concern that actually row entries to Phat are not univariate normal distributions because they cannot vary individually. In fact each row forms a multivariate normal distribution don't you think? e.g. as one entry increases, at least one other in the same row must necessarily decrease to maintain sum(P,2)=1
      singulars
      1. PO
      1. USHCAI
    2. COWow, lots of effort given here!
      singulars
      1. PO
      1. USHugh Nolan
    3. COGlad I could help. As for `P_hat` the rows should indeed sum to one (up to a certain precision). In the calculation function `countFcn`, I first accumulate the counts of co-occurrences from each sequence (using a less-known syntax of `sparse` function but we could have also used `accumarray`), then I divide by the row sums (the `bsxfun` call). One caveat: if a symbol doesn't occur in a sequence (say we only had `[1 2 4 5]`), then we could be dividing by zero, which would cause `NaN` values in the transition matrix. A common solution is to add +1 to the entire matrix before normalizing the rows
      singulars
      1. PO
      1. USAmro
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload