Module 6 · Benchmarks

A preliminary evaluation of the disparity-study landscape

Boston measured itself in 2020. The peer studies below show where the methodological frontier has moved since: who conducts this work, what their methods do well, and where a more rigorous, more reproducible approach strengthens both the analysis and its legal defensibility.

Who does this work

A handful of firms produce most disparity studies nationally: BBC Research, which conducted Boston's 2020 study, alongside MGT, Keen Independent, Colette Holt & Associates, Griffin & Strong, and Mason Tillman. House Strategies Group is pursuing the Boston study directly, as prime, meeting that bar with a modern data and analytics core and an academic bench whose members have led and defended disparity studies nationally. The point of the evaluation below is not that these studies are wrong. It is that the methodological frontier has moved, and Boston can get a more controlled, more transparent, and more defensible study than the incumbent norm.

City of Boston, MA

2021

BBC Research & ConsultingFY2015-FY2019 (July 1, 2014 through June 30, 2019)

Minority- and woman-owned businesses were available for 16.9 percent of City contract and subcontract dollars but received 11.0 percent, a disparity index of 65.

BBC's 703-page final report, released February 2021, used a custom census approach with dollar-weighted availability across construction, professional services, goods and other services, and engaged 570 local businesses through surveys, interviews and public forums. MWBE firms considered together showed a disparity index of 65 overall, well below the substantial-disparity threshold of 80, alongside documented barriers in human capital, financial capital and business ownership. The findings drove a 2021 executive order setting a 25 percent overall MWBE utilization target, with 15 percent for woman-owned and 10 percent for minority-owned firms.

Relevance to Boston: This is the anchor study the current portal updates. It fixes the baseline metrics, the FY15-19 data window, and the methodological precedent (custom census, dollar-weighted availability, index-of-80 threshold) that any Boston follow-on study will be measured against.

Preliminary evaluation: A custom census study that went beyond the surface ratio with marketplace barrier analyses, but its data window ends in June 2019, before the Supplier Diversity Program and the 2021 executive order goals took effect, which is the core case for an update.

View study

State of New York (Empire State Development)

2024

MGT of America Consulting, LLCApril 1, 2016 through March 31, 2022

MWBE firms were 38.81 percent of available firms under MGT's custom census but received 30.08 percent of the roughly $303.7 billion in state contract dollars analyzed.

MGT applied a dual test: whether a group's disparity index falls at or below 80 percent of its availability, and whether the gap passes a t-test for statistical significance. The study found statistical evidence of underutilization of MWBEs at both the prime and subcontract levels, with group-by-sector findings across construction, construction-related services, non-construction services and commodities, and it underpins the reauthorization of the Article 15-A statewide MWBE program.

Relevance to Boston: The largest recent Northeast statewide study and the template for availability-based goal-setting at scale. Boston's update will be compared against how New York pairs a custom census with significance testing and reports no-disparity cells honestly. New York City's own next citywide study, by CUNY ISLG for the Department of Small Business Services, is still in data collection and unpublished.

Preliminary evaluation: A custom census study with an explicit dual significance test; its summary tables report binary disparity or no-disparity findings by group and sector rather than leading with numeric indices, so the availability and utilization tables carry the headline evidence.

View study

City of Chicago, IL

2021

Colette Holt & Associates2015 through 2019 construction contracts (approximately $1.55 billion)

M/WBE firms received 54.2 percent of City construction prime and subcontract dollars, but over 90 percent of the work performed by Black-, Hispanic- and white woman-owned firms came as subcontracts.

CHA found expected availability of 16.8 percent for MBEs and 9.5 percent for WBEs in the six-county Chicago market, concluded that strong topline utilization was largely an artifact of the program's 26 percent MBE and 6 percent WBE goals, and documented that dollars were concentrated among a small number of firms in each NAICS code. The study recommended reaffirming the goals and shifting support toward M/WBE prime contracting.

Relevance to Boston: The court-tested national counterweight to the BBC and MGT templates. Authored by the co-author of the NCHRP disparity study guidance, it shows how a study defends a program even where topline utilization is high, by looking under the surface at subcontract dependence and dollar concentration.

Preliminary evaluation: Goes well beyond the surface utilization ratio with prime-versus-subcontract and firm-concentration analyses, and derives but-for expected availability from Census ACS data rather than a survey-based custom census.

View study

City of Philadelphia, PA

2022

Econsult Solutions, Inc. and Milligan Consulting, LLCFY2021 ($700.3 million in City contracts; annual series required by city code)

MWDSBEs were 30.5 percent of firms located in the city but received 15 percent of City contract dollars, a disparity ratio of 0.49.

The FY2021 Annual Disparity Report, dated November 30, 2022, also found overall MWDSBE participation of 32.4 percent when firms from any location are counted, and an MSA-level all-MWDSBE disparity ratio of 0.88, down from 0.90 in FY2016. Philadelphia is unusual in producing a disparity study every year under Title 17 of the Philadelphia Code rather than once every five or so years.

Relevance to Boston: The annual-cadence model: a standing measurement instrument tied to goal-setting every year. For Boston it demonstrates what continuous monitoring looks like after a one-time study, exactly the gap between Boston's 2021 study and its FY2023-forward annual reporting requirement.

Preliminary evaluation: A lighter-weight annual instrument: availability rests on Census Survey of Business Owners data rather than a custom census, and the core report explicitly notes it excludes the deeper analyses a full Croson study would include.

View study

Massachusetts Bay Transportation Authority (MBTA)

2023

MBTA Office of Diversity and Civil Rights (federal DBE goal methodology, not an independent disparity study)Federal fiscal years 2024-2026 (methodology submitted July 31, 2023)

The MBTA set an overall DBE goal of 22 percent of federally funded contract dollars for FFY 2024-2026, up from 20 percent, with 18 points race- and gender-conscious and 4 points race-neutral.

This is a 49 CFR Part 26 goal-setting methodology rather than a Croson disparity study: a two-step calculation of the relative availability of ready, willing and able DBEs in the local market area, then an adjustment weighing past DBE participation and broader MWBE availability evidence, applied to roughly $1.84 billion in projected federal contract spending. In October 2025 a federal interim final rule removed race- and gender-based presumptions, and MassDOT and the MBTA cannot set or enforce DBE contract goals while certified firms are reevaluated.

Relevance to Boston: The closest thing to a Boston-market availability analysis outside the City's own study, and the regional example of federal DBE availability math. No Massachusetts state or transportation agency has commissioned a full Croson-style disparity study since 2019, which leaves Boston's 2020 study as the region's benchmark.

Preliminary evaluation: Ready-willing-and-able availability logic built on real bidder and directory data, but a goal-setting exercise only: no utilization-to-availability disparity testing, no significance tests and no anecdotal record.

View study

Commonwealth of Virginia

2025

BBC Research & ConsultingJuly 1, 2019 through June 30, 2024 (approximately $27.1 billion in contracts)

Minority-owned businesses combined were available for 23.3 percent of state agency contract dollars but received 10.7 percent, with disparity indices of 0.16 for Black-owned and 0.24 for Hispanic-owned firms.

BBC's second consecutive statewide study for Virginia, delivered to the General Assembly as report RD248 under the 2025 Appropriation Act. It follows BBC's 2020 Virginia study (FY2014-FY2019), which found substantial disparities for every racial, ethnic and gender group and led the Commonwealth to set a 23.1 percent discretionary spending goal with woman- and minority-owned businesses. White woman-owned firms showed a disparity index of 0.30 at state agencies in the new study.

Relevance to Boston: The same consultant as Boston's 2020 study, running the same custom census playbook statewide five years later. It previews what a BBC-style Boston update would look like methodologically and maps the consultant landscape Boston would procure from.

Preliminary evaluation: A consistent custom census template repeated across two study cycles, which makes trend claims possible; like most BBC work it reports survey-based custom census availability and leads with the utilization-to-availability ratio.

View study

Why the method matters in Massachusetts

The leading appellate case already drew the line

H.B. Rowe Co. v. Tippett, 615 F.3d 233 (4th Cir. 2010) is the leading federal appellate treatment of a disparity-study record, and counsel on our bench litigated it. The court upheld race-conscious contracting measures only for the groups the evidence actually supported, and struck them down where the record lacked statistically significant, properly controlled findings. Massachusetts sits in the First Circuit, which has no equally developed modern disparity precedent, so Croson's own requirements are the design standard. A disparity study that holds up in Boston is the one built, from the first design choice, to clear that bar. Every improvement below is made for exactly that reason.

The methodological frontier

Where our method goes further, and is more legally defensible

Each row pairs a common limitation of the published studies with what House Strategies Group would do instead, why it produces a more defensible result, and a documented example from the case law and the research where that limitation has surfaced. These are not hypothetical concerns.

Common limitation

Availability is often an equal-weight count of firms in a geography, which inflates the denominator with firms that never bid or cannot perform.

Our method

A ready, willing, and able custom availability survey, anchored to actual public-work registration and bid behavior, not a raw headcount.

Why it is more defensible

Defeats the inflated-availability attack, the most contested step under Croson, by measuring the pool a court actually recognizes.

In the record: research by George R. La Noue (University of Maryland, Baltimore County) finds studies routinely count firms that never bid or lack the capacity to perform, inflating the pool the comparison rests on.

Common limitation

A raw availability-to-utilization ratio is treated as a finding, with no control for the legitimate factors that also drive who wins work.

Our method

Controls for firm size, age, experience, and capital, plus a private-sector but-for regression, so a gap is reported as a barrier only when it survives those controls.

Why it is more defensible

Separates a barrier from a capability difference, which is the core of a strong-basis-in-evidence showing and the gap critics most often exploit.

In the record: in H.B. Rowe Co. v. Tippett, 615 F.3d 233 (4th Cir. 2010), the leading federal appellate treatment of a disparity-study record, the court struck the contracting measures for the groups whose findings were not backed by controlled, statistically significant evidence.

Common limitation

A single headline index can mask which specific groups and categories the evidence actually supports.

Our method

Group-by-category findings with significance testing on every cell, remediated only where the evidence holds.

Why it is more defensible

Matches H.B. Rowe, which upheld measures only for the groups the record supported and struck them where it did not.

In the record: H.B. Rowe upheld remedies for African American and Native American firms but struck them for Asian, Hispanic, and women-owned firms, and Croson warned against treating unlike groups as one.

Common limitation

Anecdotal evidence is sometimes anonymous, unverified, and self-selected.

Our method

A verified, attributed, and retained anecdotal record, with structured instruments and a documented protocol.

Why it is more defensible

Meets the corroboration courts require and survives the discovery that has undone studies whose underlying records could not be produced.

In the record: La Noue's congressional testimony warns that anonymous or unverified anecdotes cannot, on their own, sustain a race- or gender-based preference.

Common limitation

The deliverable is frequently a static PDF, and the underlying data and code are not published.

Our method

Published data, documented model specifications, and a live, interactive, Section 508-compliant platform delivered to the City.

Why it is more defensible

Makes the strong-basis-in-evidence showing auditable and reproducible, the failure mode that sank disparity evidence in other jurisdictions.

In the record: La Noue documents disparity-study litigation in which a regression analysis could not be reviewed (Montana) and a consultant could not produce its underlying data (Shelby County, Tennessee).

Common limitation

Recommendations can be boilerplate, leading with race-conscious goals.

Our method

Race-neutral, opportunity-focused remedies first, with race-conscious measures reserved for the narrow, significant, group-specific gaps neutral tools cannot close.

Why it is more defensible

The narrow-tailoring discipline Croson and 49 C.F.R. 26.51 require, and the posture most durable after Students for Fair Admissions.

In the record: Croson and 49 C.F.R. 26.51 require race-neutral measures be considered first, and La Noue documents the institutional pressure on for-profit firms to find disparity rather than test for it.

The common thread, read honestly

Across every peer study, minority- and women-owned firms are a substantial share of the available market and a smaller share of the dollars. Boston's current preliminary reads point the same way: roughly 0.64 for minority-owned firms and 0.16 for women-owned firms, computed from certified-firm spend only, so they are a floor on true participation, not a verdict. That is the case for a full study, not a substitute for one, and the value is in measuring the gap correctly and explaining what causes it.

See the approach behind these claims

The methodology page draws the line between what public data can show and what the full study adds. The approach module sets out how we move from a measured gap to its cause.

Our approach Methodology & legal framework The preliminary index