Impact of Correlation Matrix Filtering on QSLS Outputs

Introduction

The QSLS methodology employs matrix mathematics and linguistic correlations to quantify system architecture support for various quality attributes and business drivers. A key feature of this approach is the ability to filter correlation matrices by removing values below a specified threshold. This filtering capability significantly impacts the resulting outputs and provides important insights for Program Managers and stakeholders. This paper examines how correlation matrix filtering affects QSLS outputs and how decision-makers can leverage these effects.

Understanding Correlation Matrix Filtering

In the QSLS methodology, correlation matrices capture the linguistic relationships between different architectural concepts:

• Architectural Mechanisms to Part Components

• Part Components to Characteristic System Attributes

• Characteristic System Attributes to Quality Attribute Sub-Attributes

• Quality Attribute Sub-Attributes to Business Drivers

Filtering allows users to establish a minimum threshold (e.g., 0.4) below which correlation values are treated as zero. This effectively removes weak or potentially noise-level correlations from consideration.

Key Effects of Correlation Filtering

1. Signal-to-Noise Ratio Enhancement

The level where correlations can be considered "actual" in nature is found to be around .4 to 1.0. Lower values are less reliable in terms of actual relationship to true correlation. It is suggested that filters be set around .4 therefore accepting correlation values at .4 to 1.0 and setting all other correlation node points to 0 (no real correlation).

When filtering is applied:

• Increased Output Clarity: By removing weak correlations, the outputs focus on stronger, more meaningful relationships within the system architecture.

• Reduced Statistical Noise: Lower correlation values, which may represent incidental linguistic similarities rather than true conceptual relationships, are eliminated.

• More Focused Results: The filtered outputs highlight the primary relationships driving system behavior.

2. Changes to Statistical Measures

Filtering correlation matrices causes significant changes to the statistical measures produced by QSLS:

Minimum Values

• Typically increase as weak correlations are removed

• May become zero for some elements if all correlations fall below the threshold

• Create more distinct separation between supported and unsupported elements

Maximum Values

• Generally remain stable unless very high threshold filters are applied

• Represent the strongest relationships that typically survive filtering

• Provide consistent reference points across different filtering levels

Average Values

• Increase as weak correlations are removed from calculations

• Become more representative of strong, meaningful relationships

• Show greater differentiation between well-supported and poorly-supported elements

Median Values

• Shift upward as the distribution of values is truncated at the lower end

• Become more indicative of the central tendency of meaningful relationships

• Provide better representation of typical support levels

Max-Min Differential

• Often decreases as minimum values rise while maximum values remain stable

• Indicates increased consistency in support levels

• Suggests higher reliability in system performance for well-supported elements

3. Impact on Architectural Understanding

The problem of measuring system architecture is the complexity of comparing relatable terms in such a way as to compute values which can represent level of support.

Filtering affects how architects and stakeholders understand the system:

• Clearer Architectural Priorities: Filtering highlights the most significant relationships, helping to identify the architectural elements with the greatest impact.

• More Distinct Trade-offs: With weaker correlations removed, the relationships between quality attributes become more pronounced, making trade-offs more visible.

• Enhanced Focus on Critical Paths: Filtering reveals the primary pathways through which architectural mechanisms support business drivers.

Practical Applications for Program Managers

1. Setting Appropriate Filter Thresholds

Program Managers must carefully consider filter threshold settings:

• Low Thresholds (0.2-0.3): Include more potential relationships but introduce more noise

• Medium Thresholds (0.4-0.6): Balance between inclusivity and focus, generally recommended for most analyses

• High Thresholds (0.7+): Focus only on the strongest relationships but may exclude important secondary effects

Accepting correlations below .2 can be done but is close to accepting noise in the computation. The decision is left to the Architect as to what the correlation filter is set to in the QSLS tool.

2. Comparative Analysis with Multiple Filter Levels

By analyzing the same system with different filter thresholds, Program Managers can:

• Identify which relationships persist across all filter levels (core relationships)

• Detect secondary relationships that appear only at lower thresholds

• Understand the sensitivity of the system architecture to different relationship strengths

3. Risk Assessment Based on Filter Sensitivity

The sensitivity of outputs to filtering provides valuable risk insights:

• High Sensitivity: If output values change dramatically with different filter settings, the architecture may rely on many weak relationships, indicating higher risk.

• Low Sensitivity: Outputs that remain stable across different filter settings suggest the architecture is built on strong, reliable relationships.

Optimizing Output Presentations for Stakeholders

Given the impact of filtering, stakeholders need appropriately designed output presentations:

1. Multi-level Filtering Views

• Present analyses at different filter thresholds (e.g., 0.3, 0.5, 0.7)

• Show how key metrics change across these thresholds

• Highlight which relationships persist at higher thresholds

2. Confidence Indicators

• Use filter sensitivity as a basis for confidence ratings

• Indicate which results are most reliable (stable across filter settings)

• Flag areas where results are highly dependent on filter settings

3. Relationship Strength Visualization

• Color-code relationship lines by correlation strength

• Provide interactive filtering capabilities in visualization tools

• Enable stakeholders to dynamically adjust filter thresholds and observe changes

Implementation Guidance

To effectively leverage correlation matrix filtering:

1. Standard Filter Profiles

Establish standard filter profiles for different analysis purposes:

• Inclusive Analysis: Lower thresholds (~0.3) for early exploration

• Balanced Analysis: Medium thresholds (~0.5) for typical assessments

• Critical Focus: Higher thresholds (~0.7) for identifying core relationships

2. Comparative Reporting

Always present filtered results in context:

• Include information about the filter threshold applied

• Provide comparison to results at other threshold levels when appropriate

• Explain how filtering affects interpretation of the results

3. Sensitivity Analysis

Make filter sensitivity analysis a standard part of architecture reviews:

• Calculate how much key metrics change across different filter thresholds

• Identify areas where results are highly dependent on filter settings

• Use this information to guide architecture refinement

Technical Considerations

The implementation of filtering affects computation processes:

This filtering approach can also be applied to the starting vector of Mechanism weights. This again must be approached with care as at the extremes can result in no Mechanisms being available to compute results. By applying filters, we are focusing on the key relationships and taking away the less important relationships found in the math.

1. Matrix Sparsity Effects

• Higher filter thresholds create sparser matrices

• Sparse matrix mathematics may require different computational approaches

• Performance optimizations become more effective with higher sparsity

2. Normalization Adjustments

• Filtering changes the normalization factors in calculations

• Outputs must be appropriately scaled to maintain comparability

• Special handling may be needed for cases where all correlations fall below the threshold

3. Statistical Validity Considerations

• Very high filter thresholds may reduce the statistical validity of some measures

• Maintain minimum thresholds for the number of active relationships

• Consider weighted approaches that scale rather than eliminate lower correlations

Conclusion

Correlation matrix filtering is a powerful capability within the QSLS methodology that significantly impacts the resulting outputs. By selectively focusing on stronger relationships, filtering enhances the signal-to-noise ratio and provides clearer insights into system architecture.

For Program Managers and stakeholders, understanding how filtering affects outputs is crucial for proper interpretation and decision-making. By appropriately leveraging different filter thresholds, decision-makers can gain more nuanced insights into architectural strengths, weaknesses, and risks.

The ability to adjust filter thresholds also offers a valuable mechanism for sensitivity analysis, helping to identify which architectural relationships are most critical and which may represent more tenuous connections. This capability transforms QSLS from a static analysis tool into a dynamic exploration platform that supports deeper architectural understanding and more informed decision-making.

As QSLS implementations mature, establishing best practices around filter threshold selection, multi-level filtering analysis, and filter-aware visualization will be key to maximizing the value of this powerful approach to quantitative system architecture evaluation.