Subset Replacement of U(C) and Storage in Cell Array

Back to Fan's Matlab Examples Table of Content

Cell Array Store U(C) to Avoid Duplicate Computation over Iteration: u(c) across iterations often shares many common values

During Iteration Solution Procedure, sometimes only a subset of rows/columns need to be updated for some core matrix after each iteration.

Potentially, there could be significant speed gains if one does not need to fully recompute based on some N by M matrix, but can compute based on some N-n by M-m matrix, and update values in the N by M matrix with new values. See this file for examples when we fully reuse matrixes.

As stated here, we will store existing calculations in cell arrays.

Cell array based index updating is time saving because no additional array copying during slicing is needed.

One should not store data in larger tensors or matrixes and slice subsets when needed, that will not lead to speed improvements as shown here.

Files

  • Cell matrix part update function testing: ipynb | html
  • Cell matrix part update function: m | html
  • Direct evaluation and interpolate speed comparison: m | ipynb | html
    • core: cell{}, cl_u_store{i}(ar_it_update,:) = f_u(f_c(ar_coh, ar_kp(ar_it_update), ar_bp(ar_it_update)));

Below I invoke the best subset replace function out of the ones tested with some different parameters and look at resulting speeds. The code below shows the core codes contained in the data subset updating file here.

In [25]:
% %% Full Replace Standard Function
% function ffs_full_replace(ar_coh, ar_kp, ar_bp, f_u, f_c, it_iter, it_z)
% for it_iter_n=1:1:it_iter
%     for it_z_m=1:1:it_z
%         mt_u = f_u(f_c(ar_coh, ar_kp, ar_bp));
%     end
% end

In [26]:
% %% Partial Replace with Cell Indexing
% function ffs_cellpart_replace(ar_it_rows_replace, ar_coh, ar_kp, ar_bp, f_u, f_c, it_iter, it_z)
% % This is the most efficient method
% cl_u_store = cell([it_z, 1]);
% tic;
% for it_iter_n=1:1:it_iter
%     for it_z_m=1:1:it_z
%         if (it_iter_n == 1)
%             mt_c = f_c(ar_coh, ar_kp, ar_bp);
%             mt_u = f_u(mt_c);
%             cl_u_store{it_z_m} = mt_u;
%         else
%             cl_u_store{it_z_m}(ar_it_rows_replace,:) = f_u(f_c(ar_coh, ar_kp(ar_it_rows_replace), ar_bp(ar_it_rows_replace)));
%             mt_u = cl_u_store{it_z_m};
%         end
%     end
% end
% end

Shifting the State and Choices Sizes

Shift Matrix Size

In [27]:
% Limited States
param_map = containers.Map('KeyType','char', 'ValueType','any');
param_map('it_rown') = 50;
param_map('it_rown_update') = 10;
param_map('it_coln') = 500;
ff_u_c_partrepeat(param_map)
    'it_coln'    'it_iter'    'it_rown'    'it_rown_update'    'it_z'

    [500]    [50]    [50]    [10]    [15]

                                                                  speedmat    speedfull
                                                                  ________    _________

    Recompute u(c) every time: mt_u=f_u(f_c(coh,k,b))                0.103     77.248  
    Update u(c): str{i}(rows,:)=f_u(f_c(coh,k(rows),b(rows)));    0.063922     47.942  

Elapsed time is 0.078838 seconds.

In [28]:
% More states/choices
param_map = containers.Map('KeyType','char', 'ValueType','any');
param_map('it_rown') = 300;
param_map('it_rown_update') = 60;
param_map('it_coln') = 3000;
ff_u_c_partrepeat(param_map)
    'it_coln'    'it_iter'    'it_rown'    'it_rown_update'    'it_z'

    [3000]    [50]    [300]    [60]    [15]

                                                                  speedmat    speedfull
                                                                  ________    _________

    Recompute u(c) every time: mt_u=f_u(f_c(coh,k,b))              5.2018      3901.3  
    Update u(c): str{i}(rows,:)=f_u(f_c(coh,k(rows),b(rows)));     2.2137      1660.3  

Elapsed time is 2.226605 seconds.

Shifting the Proportion of Values Requiring Updating

Shift the number of values that requiring updating. During iteration solution, sometimes the proportion of u(c) values, for fixed grid one choice problem, is 0 percent.

For two dimensional choice problem converted to one dimensional problem, the proportion of u(c) that requires changing decreases with each iteration quickly.

In [29]:
% Update 99 percent.
param_map = containers.Map('KeyType','char', 'ValueType','any');
param_map('it_rown') = 300;
param_map('it_coln') = 3000;
param_map('it_rown_update') = param_map('it_rown')-1;
ff_u_c_partrepeat(param_map)
    'it_coln'    'it_iter'    'it_rown'    'it_rown_update'    'it_z'

    [3000]    [50]    [300]    [299]    [15]

                                                                  speedmat    speedfull
                                                                  ________    _________

    Recompute u(c) every time: mt_u=f_u(f_c(coh,k,b))              5.1461      3859.6  
    Update u(c): str{i}(rows,:)=f_u(f_c(coh,k(rows),b(rows)));     6.5884      4941.3  

Elapsed time is 6.562584 seconds.

In [30]:
% Update half
param_map = containers.Map('KeyType','char', 'ValueType','any');
param_map('it_rown') = 300;
param_map('it_coln') = 3000;
param_map('it_rown_update') = param_map('it_rown')/2;
ff_u_c_partrepeat(param_map)
    'it_coln'    'it_iter'    'it_rown'    'it_rown_update'    'it_z'

    [3000]    [50]    [300]    [150]    [15]

                                                                  speedmat    speedfull
                                                                  ________    _________

    Recompute u(c) every time: mt_u=f_u(f_c(coh,k,b))              5.2352      3926.4  
    Update u(c): str{i}(rows,:)=f_u(f_c(coh,k(rows),b(rows)));     3.8007      2850.5  

Elapsed time is 3.795854 seconds.

In [31]:
% Update 1/4
param_map = containers.Map('KeyType','char', 'ValueType','any');
param_map('it_rown') = 300;
param_map('it_coln') = 3000;
param_map('it_rown_update') = round(param_map('it_rown')/4);
ff_u_c_partrepeat(param_map)
    'it_coln'    'it_iter'    'it_rown'    'it_rown_update'    'it_z'

    [3000]    [50]    [300]    [75]    [15]

                                                                  speedmat    speedfull
                                                                  ________    _________

    Recompute u(c) every time: mt_u=f_u(f_c(coh,k,b))              5.2528      3939.6  
    Update u(c): str{i}(rows,:)=f_u(f_c(coh,k(rows),b(rows)));     2.5051      1878.8  

Elapsed time is 2.515015 seconds.

In [32]:
% Update 1/8
param_map = containers.Map('KeyType','char', 'ValueType','any');
param_map('it_rown') = 300;
param_map('it_coln') = 3000;
param_map('it_rown_update') = round(param_map('it_rown')/8);
ff_u_c_partrepeat(param_map)
    'it_coln'    'it_iter'    'it_rown'    'it_rown_update'    'it_z'

    [3000]    [50]    [300]    [38]    [15]

                                                                  speedmat    speedfull
                                                                  ________    _________

    Recompute u(c) every time: mt_u=f_u(f_c(coh,k,b))               5.186      3889.5  
    Update u(c): str{i}(rows,:)=f_u(f_c(coh,k(rows),b(rows)));     1.3847      1038.5  

Elapsed time is 1.395232 seconds.

In [33]:
% Update 1
param_map = containers.Map('KeyType','char', 'ValueType','any');
param_map('it_rown') = 300;
param_map('it_coln') = 3000;
param_map('it_rown_update') = 1;
ff_u_c_partrepeat(param_map)
    'it_coln'    'it_iter'    'it_rown'    'it_rown_update'    'it_z'

    [3000]    [50]    [300]    [1]    [15]

                                                                  speedmat    speedfull
                                                                  ________    _________

    Recompute u(c) every time: mt_u=f_u(f_c(coh,k,b))              5.1574        3868  
    Update u(c): str{i}(rows,:)=f_u(f_c(coh,k(rows),b(rows)));    0.16243      121.82  

Elapsed time is 0.167445 seconds.