Generate Statistics from Prob Mass Function over States
back to Fan's Dynamic Assets Repository Table of Content.
Contents
- FF_AZ_DS_POST_STATS post ff_az_ds statistics generation
- Default
- Parse
- f(y), f(c), f(a): Generate Key Distributional Statistics for Each outcome
- f(y), f(c), f(a): Find p(outcome(states)), proability mass function for each outcome
- f(y), f(c), f(a): Compute Statistics for outcomes
- f(y), f(c), f(a): Store Statistics Specific to Each Outcome
- Covariance and Correlation
- f(y), f(c), f(a): Store Statistics Shared Table All Outcomes
function [result_map] = ff_az_ds_post_stats(varargin)
FF_AZ_DS_POST_STATS post ff_az_ds statistics generation
Having derived f(a,z) the probability mass function of the joint discrete random variables, we now obtain distributional statistics. Note that we know f(a,z), and we also know relevant policy functions a'(a,z), c(a,z), or other policy functions. We can simulate any choices that are a function of the random variables (a,z), using f(a,z)
parameter structure provides a list of
- from result_map('ar_st_pol_names'), get list of outcome matrix on state space
- simulate each outcome using f(a,z) for probability draws
- compute key statistics: (1) mean (expectation=sum) (2) sd (3) min and max (4) iqr (5) fraction = 0 (6) percentiles including: 99.9, 99, 95, every 5 in between 5, 1, 0.01.
Uses fake binomial data when file is invoke with defaults.
@param param_map container parameter container
@param support_map container support container
@param result_map container contains policy function matrix, value function matrix, iteration results
@param mt_dist_az matrix N by M where N are asset states and M are shock states, the f(a,z) probability mass function derived earlier in ff_az_ds or ff_az_ds_vec
@return result_map container with statistics added to result_map
- the first element of each of these cell array is y(a,z), the outcome/choice at the state space points
- the second element of the cell is another container, which contains statistics computed for f(y) based on y(a,z) and f(a,z), f(y) is the probability mass function for outcome y given the stationary distribution f(a,z). The second element container also includes f(y) itself as well as f(y,z).
- additionally, result_map also stores some of the statistics for different variables jointly together. (a) tb_outcomes_meansdperc: where each row is a different outcome of the model, and each table column stores a different statistics of interest. (b) tb_outcomes_fracheld: which measures the fraction of asset held by different people.
@example
bl_input_override = true; result_map = ff_az_ds_post_stats(support_map, result_map, mt_dist_az, bl_input_override);
@include
Default
use binomial as test case, z maps to binomial win prob, remember binom approximates normal.
if (~isempty(varargin)) % if invoked from outside overrid fully [support_map, result_map, mt_dist_az] = varargin{:}; bl_display_final_dist_detail_local = false; else clear all; close all; it_states = 6; it_shocks = 5; fl_binom_n = it_states-1; ar_binom_p = (1:(it_shocks))./(it_shocks+2); ar_binom_x = (0:1:(it_states-1)) -3; % f(z) ar_binom_p_prob = binopdf(0:(it_shocks-1), it_shocks-1, 0.5); % f(a,z), mass for a, z mt_dist_az = zeros([it_states, it_shocks]); for it_z=1:it_shocks % f(a|z) f_a_condi_z = binopdf(ar_binom_x - min(ar_binom_x), fl_binom_n, ar_binom_p(it_z)); % f(z) f_z = ar_binom_p_prob(it_z); % f(a,z)=f(a|z)*f(z) mt_dist_az(:, it_z) = f_a_condi_z*f_z; end % y(a,z), some non-smooth structure rng(123); mt_pol_a = ar_binom_x' - 0.01*ar_binom_x'.^2 + ar_binom_p - 0.5*ar_binom_p.^2 + rand([it_states, it_shocks]); mt_pol_a = round(mt_pol_a*3); rng(456); mt_pol_c = 10 -(mt_pol_a) + 15*(rand([it_states, it_shocks])-0.5); % Generate result_map result_map = containers.Map('KeyType','char', 'ValueType','any'); result_map('cl_mt_pol_a') = {mt_pol_a, zeros(1)}; result_map('cl_mt_pol_c') = {mt_pol_c, zeros(1)}; result_map('ar_st_pol_names') = ["cl_mt_pol_a", "cl_mt_pol_c"]; % support_map support_map = containers.Map('KeyType','char', 'ValueType','any'); support_map('bl_display_final_dist') = true; support_map('bl_display_final_dist_detail') = true; bl_display_final_dist_detail_local = true; end
Parse
% support_map params_group = values(support_map, {'bl_display_final_dist', 'bl_display_final_dist_detail'}); [bl_display_final_dist, bl_display_final_dist_detail] = params_group{:}; if (bl_display_final_dist_detail) bl_display_drvstats = true; else bl_display_drvstats = false; end % result_map params_group = values(result_map, {'ar_st_pol_names'}); [ar_st_pol_names] = params_group{:};
f(y), f(c), f(a): Generate Key Distributional Statistics for Each outcome
Loop over outcomes, see end of ff_az_vf_vecsv where these are created
for it_outcome_ctr=1:length(ar_st_pol_names)
f(y), f(c), f(a): Find p(outcome(states)), proability mass function for each outcome
Using from tools: fft_disc_rand_var_mass2outcomes, compute unique sorted outcomes for y(a,z) and find:



note: sum(mt_dist_az, 2) = result_map('cl_mt_pol_a'){2}, but not at small simulation grids. These two might be different because pol_a is based on a choices, mt_dist_az is based on a states
see end of ff_az_vf_vecsv outcomes in result_map are cells with two elements, first element is y(a,z), second element will be f(y) and y, generated here.
st_cur_output_key = ar_st_pol_names(it_outcome_ctr);
cl_mt_choice_cur = result_map(st_cur_output_key);
mt_choice_cur = cl_mt_choice_cur{1};
% run function from tools: fft_disc_rand_var_mass2outcomes
% <https://fanwangecon.github.io/CodeDynaAsset/tools/html/fft_disc_rand_var_mass2outcomes.html>
bl_input_override = true;
[ar_choice_prob_byY, ar_choice_unique_sorted_byY, mt_choice_prob_byYZ, mt_choice_prob_byYA] = ...
fft_disc_rand_var_mass2outcomes(st_cur_output_key, mt_choice_cur, mt_dist_az, bl_input_override);
f(y), f(c), f(a): Compute Statistics for outcomes
Using from tools: fft_disc_rand_var_stats, compute these outcomes:
- $\mu_Y = E(Y) = \sum_{y} p(Y=y) \cdot y $



- percentiles:

- fraction of outcome held by up to percentiles:

% run function fft_disc_rand_var_stats.m from tools: % <https://fanwangecon.github.io/CodeDynaAsset/tools/html/fft_disc_rand_var_stats.html> [ds_stats_map] = fft_disc_rand_var_stats(st_cur_output_key, ar_choice_unique_sorted_byY', ar_choice_prob_byY', bl_display_drvstats); % prcess results % retrieve scalar statistics: fl_choice_mean = ds_stats_map('fl_choice_mean'); fl_choice_sd = ds_stats_map('fl_choice_sd'); fl_choice_coefofvar = ds_stats_map('fl_choice_coefofvar'); fl_choice_min = ds_stats_map('fl_choice_min'); fl_choice_max = ds_stats_map('fl_choice_max'); fl_choice_prob_zero = ds_stats_map('fl_choice_prob_zero'); fl_choice_prob_below_zero = ds_stats_map('fl_choice_prob_below_zero'); fl_choice_prob_above_zero = ds_stats_map('fl_choice_prob_above_zero'); fl_choice_prob_min = ds_stats_map('fl_choice_prob_min'); fl_choice_prob_max = ds_stats_map('fl_choice_prob_max'); % retrieve distributional array stats ar_choice_percentiles = ds_stats_map('ar_choice_percentiles'); ar_choice_perc_fracheld = ds_stats_map('ar_choice_perc_fracheld');
----------------------------------------
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
Summary Statistics for: cl_mt_pol_a
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
----------------------------------------
fl_choice_mean
-0.1108
fl_choice_sd
4.1239
fl_choice_coefofvar
-37.2149
fl_choice_prob_zero
0.0643
fl_choice_prob_below_zero
0.5487
fl_choice_prob_above_zero
0.3871
fl_choice_prob_max
0.0273
tb_disc_cumu
cl_mt_pol_aDiscreteVal cl_mt_pol_aDiscreteValProbMass CDF cumsumFrac
______________________ ______________________________ ______ __________
-7 0.051764 5.1764 3.2699
-6 0.050217 10.198 5.9889
-5 0.10978 21.176 10.942
-4 0.0014875 21.324 10.996
-3 0.11706 33.031 14.165
-2 0.0080324 33.834 14.31
-1 0.21033 54.867 16.208
0 0.064259 61.293 16.208
2 0.049682 66.261 15.311
3 0.096388 75.9 12.702
cl_mt_pol_aDiscreteVal cl_mt_pol_aDiscreteValProbMass CDF cumsumFrac
______________________ ______________________________ ______ __________
-1 0.21033 54.867 16.208
0 0.064259 61.293 16.208
2 0.049682 66.261 15.311
3 0.096388 75.9 12.702
4 0.085679 84.468 9.6092
5 0.065337 91.002 6.6611
6 0.057231 96.725 3.5623
7 0.0054218 97.267 3.2198
8 3.7187e-06 97.267 3.2196
9 0.027329 100 1
tb_prob_drv
percentiles cl_mt_pol_aDiscreteValPercentileValues fracOfSumHeldBelowThisPercentile
___________ ______________________________________ ________________________________
0.1 -7 3.2699
1 -7 3.2699
5 -7 3.2699
10 -6 5.9889
15 -5 10.942
20 -5 10.942
25 -3 14.165
35 -1 16.208
50 -1 16.208
65 2 15.311
75 3 12.702
80 4 9.6092
85 5 6.6611
90 5 6.6611
95 6 3.5623
99 9 1
99.9 9 1
----------------------------------------
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
Summary Statistics for: cl_mt_pol_c
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
----------------------------------------
fl_choice_mean
8.8423
fl_choice_sd
6.5845
fl_choice_coefofvar
0.7447
fl_choice_prob_zero
0
fl_choice_prob_below_zero
0.0273
fl_choice_prob_above_zero
0.9727
fl_choice_prob_max
0.0465
tb_disc_cumu
cl_mt_pol_cDiscreteVal cl_mt_pol_cDiscreteValProbMass CDF cumsumFrac
______________________ ______________________________ ______ ___________
-6.3772 0.015232 1.5232 -0.010985
-4.4805 0.011621 2.6853 -0.016874
-0.72091 0.00047599 2.7329 -0.016913
0.14102 0.057119 8.4448 -0.016002
0.27238 0.085679 17.013 -0.013362
0.50318 0.023242 19.337 -0.01204
2.7525 0.02975 22.312 -0.0027791
3.5617 3.7187e-06 22.312 -0.0027776
4.0352 0.0059499 22.907 -6.2407e-05
5.1855 0.0054218 23.449 0.0031172
cl_mt_pol_cDiscreteVal cl_mt_pol_cDiscreteValProbMass CDF cumsumFrac
______________________ ______________________________ ______ __________
13.231 0.028917 65.473 0.36675
13.357 0.018593 67.332 0.39484
13.799 0.12852 80.184 0.59539
13.901 0.000119 80.196 0.59558
15.71 0.024097 82.605 0.63839
16.255 0.0080324 83.409 0.65316
16.887 0.092967 92.705 0.8307
18.136 0.022848 94.99 0.87756
19.35 0.0036146 95.352 0.88547
21.786 0.046484 100 1
tb_prob_drv
percentiles cl_mt_pol_cDiscreteValPercentileValues fracOfSumHeldBelowThisPercentile
___________ ______________________________________ ________________________________
0.1 -6.3772 -0.010985
1 -6.3772 -0.010985
5 0.14102 -0.016002
10 0.27238 -0.013362
15 0.27238 -0.013362
20 2.7525 -0.0027791
25 5.2138 0.041007
35 6.2166 0.1181
50 6.5321 0.1893
65 13.231 0.36675
75 13.799 0.59539
80 13.799 0.59539
85 16.887 0.8307
90 16.887 0.8307
95 19.35 0.88547
99 21.786 1
99.9 21.786 1
f(y), f(c), f(a): Store Statistics Specific to Each Outcome
see intro section
% Append prob mass functions to ds_stats_map ds_stats_map('mt_choice_prob_byYZ') = mt_choice_prob_byYZ; ds_stats_map('mt_choice_prob_byYA') = mt_choice_prob_byYA; ds_stats_map('ar_choice_unique_sorted_byY') = ar_choice_unique_sorted_byY; ds_stats_map('ar_choice_prob_byY') = ar_choice_prob_byY; % ds_stats_map is second element of cell for the key for the variable % in result_map cl_mt_choice_cur{2} = ds_stats_map; result_map(st_cur_output_key) = cl_mt_choice_cur; % key stats ar_keystats = [fl_choice_mean fl_choice_sd fl_choice_coefofvar fl_choice_min fl_choice_max ... fl_choice_prob_zero fl_choice_prob_below_zero fl_choice_prob_above_zero ... fl_choice_prob_min fl_choice_prob_max ar_choice_percentiles]; cl_outcome_names(it_outcome_ctr) = st_cur_output_key; if (it_outcome_ctr == 1) mt_outcomes_meansdperc = ar_keystats; mt_outcomes_fracheld = ar_choice_perc_fracheld; else mt_outcomes_meansdperc = [mt_outcomes_meansdperc; ar_keystats]; mt_outcomes_fracheld = [mt_outcomes_fracheld; ar_choice_perc_fracheld]; end
end if (bl_display_final_dist || bl_display_final_dist_detail) tb_outcomes_meansdperc = array2table(mt_outcomes_meansdperc); ar_fl_percentiles = ds_stats_map('ar_fl_percentiles'); cl_col_names = ['mean', 'sd', 'coefofvar', 'min', 'max', ... 'pYis0', 'pYls0', 'pYgr0', 'pYisMINY', 'pYisMAXY', ... strcat('p', string(ar_fl_percentiles))]; tb_outcomes_meansdperc.Properties.VariableNames = matlab.lang.makeValidName(cl_col_names); tb_outcomes_meansdperc.Properties.RowNames = matlab.lang.makeValidName(cl_outcome_names); if (bl_display_final_dist_detail_local) disp('xxx tb_outcomes_meansdperc: mean, sd, percentiles xxx') disp(rows2vars(tb_outcomes_meansdperc)); end % Process Aset Held by up to percentiles tb_outcomes_fracheld = array2table(mt_outcomes_fracheld); cl_col_names = [strcat('fracByP', string(ar_fl_percentiles))]; tb_outcomes_fracheld.Properties.VariableNames = matlab.lang.makeValidName(cl_col_names); tb_outcomes_fracheld.Properties.RowNames = matlab.lang.makeValidName(cl_outcome_names); if (bl_display_final_dist_detail_local) disp('xxx tb_outcomes_fracheld: fraction of asset/income/etc held by hh up to this percentile xxx') disp(rows2vars(tb_outcomes_fracheld)); end end
xxx tb_outcomes_meansdperc: mean, sd, percentiles xxx
OriginalVariableNames cl_mt_pol_a cl_mt_pol_c
_____________________ ___________ ___________
'mean' -0.11081 8.8423
'sd' 4.1239 6.5845
'coefofvar' -37.215 0.74466
'min' -7 -6.3772
'max' 9 21.786
'pYis0' 0.064259 0
'pYls0' 0.54867 0.027329
'pYgr0' 0.38707 0.97267
'pYisMINY' 0.051764 0.015232
'pYisMAXY' 0.027329 0.046484
'p0_1' -7 -6.3772
'p1' -7 -6.3772
'p5' -7 0.14102
'p10' -6 0.27238
'p15' -5 0.27238
'p20' -5 2.7525
'p25' -3 5.2138
'p35' -1 6.2166
'p50' -1 6.5321
'p65' 2 13.231
'p75' 3 13.799
'p80' 4 13.799
'p85' 5 16.887
'p90' 5 16.887
'p95' 6 19.35
'p99' 9 21.786
'p99_9' 9 21.786
xxx tb_outcomes_fracheld: fraction of asset/income/etc held by hh up to this percentile xxx
OriginalVariableNames cl_mt_pol_a cl_mt_pol_c
_____________________ ___________ ___________
'fracByP0_1' 3.2699 -0.010985
'fracByP1' 3.2699 -0.010985
'fracByP5' 3.2699 -0.016002
'fracByP10' 5.9889 -0.013362
'fracByP15' 10.942 -0.013362
'fracByP20' 10.942 -0.0027791
'fracByP25' 14.165 0.041007
'fracByP35' 16.208 0.1181
'fracByP50' 16.208 0.1893
'fracByP65' 15.311 0.36675
'fracByP75' 12.702 0.59539
'fracByP80' 9.6092 0.59539
'fracByP85' 6.6611 0.8307
'fracByP90' 6.6611 0.8307
'fracByP95' 3.5623 0.88547
'fracByP99' 1 1
'fracByP99_9' 1 1
Covariance and Correlation
Having computed elsewhere E(X), E(Y), and SD(X), SD(Y), and given X(a,z) and Y(a,z), which are the optimal choices along the endogenous state space grid a, and the exogenous state space grid z, and given also f(a,z), the probability mass function over (a,z), we compute covariance and correlation between outcomes X and Y.
- Covariance

- Correlation

for it_outcome_x_ctr=1:length(ar_st_pol_names) st_cur_output_x_key = ar_st_pol_names(it_outcome_x_ctr); cl_mt_choice_cur = result_map(st_cur_output_x_key); ds_stats_map = cl_mt_choice_cur{2}; cl_mt_choice_cur = result_map(st_cur_output_x_key); mt_choice_x_bystates = cl_mt_choice_cur{1}; fl_choice_x_mean = ds_stats_map('fl_choice_mean'); fl_choice_x_sd = ds_stats_map('fl_choice_sd'); ar_covvar = zeros([1,length(ar_st_pol_names)*2]); ar_st_covvar = strings([1,length(ar_st_pol_names)*2]); for it_outcome_y_ctr=1:length(ar_st_pol_names) st_cur_output_y_key = ar_st_pol_names(it_outcome_y_ctr); cl_mt_choice_cur = result_map(st_cur_output_y_key); ds_stats_map = cl_mt_choice_cur{2}; cl_mt_choice_cur = result_map(st_cur_output_y_key); mt_choice_y_bystates = cl_mt_choice_cur{1}; fl_choice_y_mean = ds_stats_map('fl_choice_mean'); fl_choice_y_sd = ds_stats_map('fl_choice_sd'); covvar_input_map = containers.Map('KeyType','char', 'ValueType','any'); covvar_input_map('mt_choice_x_bystates') = mt_choice_x_bystates; covvar_input_map('mt_choice_y_bystates') = mt_choice_y_bystates; covvar_input_map('mt_dist_bystates') = mt_dist_az; covvar_input_map('fl_choice_x_mean') = fl_choice_x_mean; covvar_input_map('fl_choice_x_sd') = fl_choice_x_sd; covvar_input_map('fl_choice_y_mean') = fl_choice_y_mean; covvar_input_map('fl_choice_y_sd') = fl_choice_y_sd; [fl_cov_xy, fl_cor_xy] = fft_disc_rand_var_mass2covcor(covvar_input_map); % only include the y name, x name is from the row st_x_y_cov = strjoin(["fl_cov_" st_cur_output_y_key], ''); st_x_y_cor = strjoin(["fl_cor_" st_cur_output_y_key], ''); ds_stats_map(st_x_y_cov) = fl_cov_xy; ds_stats_map(st_x_y_cor) = fl_cor_xy; ar_covvar(it_outcome_y_ctr*2-1) = fl_cov_xy; ar_covvar(it_outcome_y_ctr*2) = fl_cor_xy; ar_st_covvar(it_outcome_y_ctr*2-1) = string(st_x_y_cov); ar_st_covvar(it_outcome_y_ctr*2) = string(st_x_y_cor); cl_mt_choice_cur{2} = ds_stats_map; result_map(st_cur_output_y_key) = cl_mt_choice_cur; end if (it_outcome_x_ctr == 1) mt_outcomes_covvar = ar_covvar; else mt_outcomes_covvar = [mt_outcomes_covvar; ar_covvar]; end end if (bl_display_final_dist || bl_display_final_dist_detail) tb_outcomes_covvar = array2table(mt_outcomes_covvar); tb_outcomes_covvar.Properties.VariableNames = matlab.lang.makeValidName(ar_st_covvar); tb_outcomes_covvar.Properties.RowNames = matlab.lang.makeValidName(cl_outcome_names); if (bl_display_final_dist_detail_local) disp('xxx tb_outcomes_covvar: variance correlation xxx') disp(rows2vars(tb_outcomes_covvar)); end end
xxx tb_outcomes_covvar: variance correlation xxx
OriginalVariableNames cl_mt_pol_a cl_mt_pol_c
_____________________ ___________ ___________
'fl_cov_cl_mt_pol_a' 17.007 -22.084
'fl_cor_cl_mt_pol_a' 1 -0.81327
'fl_cov_cl_mt_pol_c' -22.084 43.356
'fl_cor_cl_mt_pol_c' -0.81327 1
f(y), f(c), f(a): Store Statistics Shared Table All Outcomes
% Add to result_map mt_outcomes = [mt_outcomes_meansdperc, mt_outcomes_covvar, mt_outcomes_fracheld]; result_map('mt_outcomes') = mt_outcomes; if (bl_display_final_dist || bl_display_final_dist_detail) tb_outcomes = [tb_outcomes_meansdperc, tb_outcomes_covvar, tb_outcomes_fracheld]; result_map('tb_outcomes') = tb_outcomes; if (bl_display_final_dist_detail) disp(rows2vars(tb_outcomes)); else disp(tb_outcomes); end end
OriginalVariableNames cl_mt_pol_a cl_mt_pol_c
_____________________ ___________ ___________
'mean' -0.11081 8.8423
'sd' 4.1239 6.5845
'coefofvar' -37.215 0.74466
'min' -7 -6.3772
'max' 9 21.786
'pYis0' 0.064259 0
'pYls0' 0.54867 0.027329
'pYgr0' 0.38707 0.97267
'pYisMINY' 0.051764 0.015232
'pYisMAXY' 0.027329 0.046484
'p0_1' -7 -6.3772
'p1' -7 -6.3772
'p5' -7 0.14102
'p10' -6 0.27238
'p15' -5 0.27238
'p20' -5 2.7525
'p25' -3 5.2138
'p35' -1 6.2166
'p50' -1 6.5321
'p65' 2 13.231
'p75' 3 13.799
'p80' 4 13.799
'p85' 5 16.887
'p90' 5 16.887
'p95' 6 19.35
'p99' 9 21.786
'p99_9' 9 21.786
'fl_cov_cl_mt_pol_a' 17.007 -22.084
'fl_cor_cl_mt_pol_a' 1 -0.81327
'fl_cov_cl_mt_pol_c' -22.084 43.356
'fl_cor_cl_mt_pol_c' -0.81327 1
'fracByP0_1' 3.2699 -0.010985
'fracByP1' 3.2699 -0.010985
'fracByP5' 3.2699 -0.016002
'fracByP10' 5.9889 -0.013362
'fracByP15' 10.942 -0.013362
'fracByP20' 10.942 -0.0027791
'fracByP25' 14.165 0.041007
'fracByP35' 16.208 0.1181
'fracByP50' 16.208 0.1893
'fracByP65' 15.311 0.36675
'fracByP75' 12.702 0.59539
'fracByP80' 9.6092 0.59539
'fracByP85' 6.6611 0.8307
'fracByP90' 6.6611 0.8307
'fracByP95' 3.5623 0.88547
'fracByP99' 1 1
'fracByP99_9' 1 1
end
ans =
Map with properties:
Count: 5
KeyType: char
ValueType: any