California Air Resources Board

Scope of Work

Objective

The objective of this project is to develop a deep understanding of how Metropolitan Planning Organizations (MPOs) select the projects that appear in their Regional Transportation Plans (RTP) project list and Sustainable Communities Strategies (SCS) implementation. The project will also examine how selection processes has evolved over time and identify how project performance is tracked after implementation. Research results will help to inform future Senate Bill 150 Reports (SB 150, Allen Chapter 646, Statues of 2017).

Background

Geographic location, along with additional unique characteristics of large and small MPOs can result in a broad array of limitations when it comes to adopting and implementing RTPs. Large MPOs tend to have more capacity to develop elaborate processes to inform the development of RTPs and performance measures to evaluate their efficacy. These MPOs have benefitted from having implemented performance-based measures for years,[1] prior to recent prescribed federal[2] and or state mandates; some even going beyond the assigned requirements. Despite the MPOs’ achievements, these regional agencies continue to face multiple adversities.

In 2017, a national survey of 104 MPOs from 42 states found “significant variation” in how MPO’s apply performance measures. For instance, some MPOs limit to satisfy the required minimums and miss on the opportunity to assess specific projects and improve unmet experienced conditions.[3] Following the passage of the Sustainable Communities and Climate Protection Act (SB 375) (Chapter 728, Statutes of 2008), RTPs developed by California MPOs included an SCS that enumerated how the long-range transportation plan, if implemented, supports the state’s climate goals and meets CARB-set GHG reduction targets. Since 2008, Californian MPOs’ have also enhanced and broadened their performance-based measures to characterize how the RTP/SCS and the projects therein impact land use, the regional economy, social equity, and the natural environment. While performance measures have become stronger with SB 375, California MPOs use varying techniques of performance measures, which makes it difficult to assess MPOs plans and their success.[4]

SB 150 requires CARB to produce a report for the legislature every four years on progress made in implementing SB 375, including the status of each of California’s 18 MPOs in meeting their regional GHG emission reduction targets. The 2018 SB 150 Progress Report (Report), used 24 data-supported indicators to assess what on-the-ground change has occurred since SB 375 was enacted. The Report found that “California is not on track to meet the greenhouse gases (GHG) reductions expected under SB 375 for 2020, with emissions from statewide passenger vehicle travel per capita increasing and going in the wrong direction” in addition to evidence of widening gaps in many communities’ access to clean transportation, opportunities, and healthy sustainable communities.[5] In response to these finding, CARB is committed to finding solutions to help reverse this trend and advance the adoption and implementation of best programs and strategies in future RTPs/SCSs.

Furthermore, one opportunity identified by the Report, is to shift transportation investments toward more sustainable transportation choices, something that the Report found has not happened since SB 375 became law. Specifically, the report notes that the amount of funding planned for transit, walking, and biking in Southern California, the San Francisco Bay Area, Sacramento, and San Diego has not changed significantly in spite of the implementation of this landmark law. Collecting additional information on how MPOs select which transportation project investments are included in the RTP/SCSs, as well as on whether and how they ultimately get implemented could help build upon the information in the Report to help the state identify best practices that could provide guidance and build on past successes to improve state-wide GHG emission and vehicle-miles-traveled (VMT) reductions.

Proposed Scope of Work Elements

Project performance assessments[6] (PPA)may become useful tools for other agencies and or jurisdictions that hold decision-making authority over transportation funds. These type of performance evaluation measures grant the opportunity to achieve more with less, not only by offering the opportunity to showcase the benefits of transportation investments, but also to disclose unmet challenges. Further, PPAs provide an added value to increase transparency, accountability, and build strategic relationships that bring additional stakeholders and the public to collectively address the unmet challenges from diverse perspectives.[7]

To achieve the project objectives, the researchers will review publically available documents, existing practices, including analytical methods and criteria used by MPOs across the state as part of its transportation project selection process for the RTP/SCS. This project will also use interviews, focus groups, surveys, or other qualitative methods to develop a deep understanding of project selection and SCS implementation.

The project results will provide recommendations on best practices to select, evaluate, and strengthen SCS development through the transportation project selection process for the RTP/SCS, as well as any measures to strengthen implementation of selected projects. Results will also provide preferred practices and metrics to evaluate equity in conjunction with recommendations on how MPOs can strategically balance climate and equity co-benefits as part of the project selection process. The results will be shared publicly and will help accelerate progress toward meeting SB 375 goals with equitable outcomes.

Research questions driving this project

1. What methods are currently being used to screen, rank, assess, and/or prioritize transportation projects in the RTP/SCS project list? (see Tasks 1-3) 
2. Are there best practices that can be scaled up, opportunities to be explored, or barriers that need to be overcome to improve the way transportation projects are selected and implemented? (see Tasks 1-3)

Task 1: Review and analyze existing tools and practices—collective measures—MPOs employ in their RTPs/SCSs around plan performance and PPAs around project selection

Researchers will work with CARB and other stakeholders to identify existing tools, methods, processes for analyzing the project impacts and outcomes—collective measures—(e.g. processes used by the Metropolitan Transportation Commission, Sacramento Area Council of Governments (SACOG), San Diego Association of Governments, Fresno Council of Governments, the California Life-Cycle Benefit/Cost Analysis Model (Cal-B/C), and other currently available and relevant tools).[8] To the extent possible, researchers will review and analyze any publicly available information on project selection and evaluation from as many MPOs as feasible. The sample size should include a diverse selection of MPOs—in addition to the big four—that vary in size, region, demographics, and other relevant characteristics. 

Task 2: Perform a qualitative analysis to capture the process MPOs undergo when developing the RTP project list for RTPs/SCS

Researchers will build upon Task 1 by incorporating a qualitative and quantitative approach in the project. Under this task, researchers are encouraged to do a thorough analysis and evaluation from all identified collective measures. Under this task, researchers are expected to conduct a mix-method analysis, including, but not limited to interviews, focus groups, surveys, case studies, observations, and other methods to not only analyze available data, studies, and or plans but also engage MPOs, stakeholders, and community members.

Researchers will be evaluating the strengths, similarities, and differences of developing the RTP project list to help understand what has propelled and held back (e.g. resources, decision-making) some MPOs from adopting their projects. Also, it will be valuable to analyze how MPOs’ tools, methods, and or practices—collective measures—are used to make adjustments to funding priorities, and how have these measures influenced past decision-making, and what were the outcomes. In addition, through interviews, focus groups, publically available records, and or surveys, researchers should look into how MPOs’ adopted collective measures and outcomes have been received in the stakeholder community and local governments, and how these two groups have responded. Researchers and CARB staff will coordinate to select the region/s to capture the stakeholder and local government reactions.   

Insight into the process for developing these methods and how they may have evolved over time will inform other regions about common pitfalls to avoid.  Information on how past investment have achieved more equitable outcomes will be especially helpful to inform best practices for advancing equity throughout the state.

Task 3: Provide recommendations to assess future transportation projects and identify sources to obtain metric information from regions to inform future SB 150 Reports

Researchers will use the findings to develop recommendations and best practices for the selection, implementation, and evaluation of transportation projects. CARB understands that there will not be a one-size-fits-all solution approach. In spite of this limitation, the recommendations should focus on assessing projects in SCSs with assessments beyond mobility and accessibility-based metrics to examine impacts on climate protection, equity, affordable housing, safety, and others under the SCS’s purview. Or improve upon existing or related tools like Cal-B/C, SACOG, and other existing and relevant. The offered recommendations should also include lessons learned from current and past projects and performance measures to inform combinations of optimized assessment metrics and performances frameworks that could be used across MPOs or at the state level, in existing or emerging tools.

Further, the researchers should explore offering recommendations on policies and levers that the state—via direct state agencies’ influence and beyond state jurisdictions—can employ to facilitate better project-level investments and strategies to reduce GHG emissions and VMT. Opportunities to encourage more localized decision-making to support equity, climate and housing goals are also important to include in the final recommendations. Under this task, researchers are also encouraged to offer suggestions on identifying the best strategies to use in different contexts and how communities can be consulted to identify what is best needed. In their final recommendations, researchers may also present opportunities for how difficult to quantify elements (e.g. equity, quality of life, opportunities for advancement) and resulting co-benefits (e.g. improved social networks, mental health, regional resilience) of SB 375 can strengthen future SCS. Lastly, researchers need to offer recommendation on data sources CARB can use to obtain metric information from regions to inform MPOs success and progress in future SB 150 Reports.

Deliverables

Beginning this fiscal year, 2021-2022, CARB’s Research Division (RD) will require its contracted research projects to embody racial equity components. The newly instituted initiative calls for all RD contracted projects to adopt racial equity components into their research. In light of this commitment to advance racial equity in research, RD will work closely with contractors to not only increase solicitation awareness, but also find opportunities to connect researchers with non-academic partners. Moving forward, RD staff will work with multiple interested stakeholders to develop additional well-established racial equity components that reflect the policies, programs, and interests of CARB to achieve equitable and healthy sustainable community strategies that meet California’s climate goals. The equity components that we hope to see included in the proposals that we receive through this solicitation include those listed in this section below.

• Quarterly Progress Reports and conference calls including public facing updates posted to CARB website (Template will be provided);
• Consultation calls with CARB and key stakeholders;
• Researchers must undergo a cultural competency training (e.g. implicit bias training, racial equity training) before work begins on community impact assessments or engagement;
• Draft final report;
• Final Report;
• Present research findings at an in-person or virtual seminar (pending health requirements);
• Draft 2-page project overview for public outreach;
• Final 2-page project overview for public outreach;
• All data, analyses and analytical tools generated through the course of this project;
• In addition to the above deliverables, the researchers are encouraged to include the following:
  • Equity implications section in the final 2-page overview and report;
  • Non-academic partners in the project budget, such as non-profits or community advocates, or smaller universities with experience in community engagement;
  • Compensation for engaged partners (non-academic partners);
  • Include a community engagement plan;
  • All researchers must undergo a cultural competency training (examples include implicit bias training, racial equity training) and provide a cultural competency statement in the proposal;
  • Peer reviewed publications should be publicly available (please budget for this expense);
• Additional deliverables to be determined in consultation with CARB staff.

 Timeline

It is anticipated this project will be completed in 24 months from the start date (early 2022). The estimated budged for this project is $200.000.

Scoring Criteria

In order to increase transparency of how winning pre-proposals are chosen, scoring criteria have been included for each project. Please note that scoring criteria is tailored to each project included in CARB's fiscal year 2021-2022 solicitation. If you are submitting pre-proposals for more than one project in the solicitation, please be sure to review the scoring criteria to get a better understanding of what components of the project are most valued for selection.

1. Responsiveness to the Goals and Objectives Outlined in the Proposal Solicitation(15 points) Proposers should demonstrate a clear understanding of the policy objectives and research needs that CARB seeks to address with this project, and should convey their knowledge of the subject. The proposal should spell out, in adequate detail, exactly what the Proposer proposes to do to satisfy the requirements of the Solicitation. The draft proposal must propose work that would satisfy the following two objectives stated in the Research Solicitation: First, develop a deep understanding of how Metropolitan Planning Organizations (MPOs) select the projects that appear in their Regional Transportation Plans (RTP) project list and Sustainable Communities Strategies (SCS) implementation. Second, examine how the selection processes has evolved over time and identify how project performance is tracked after implementation.
2. POLICY RELEVANCE/BENEFITS TO THE STATE(10 points) – Does the proposal describe how the project will provide data, information, and/or products to help CARB accomplish its mission? Findings from this project are intended to accelerate progress toward meeting Senate Bill 375 goals with increased equitable outcomes and inform future Senate Bill 150 Reports. 
3. PREVIOUS WORK (10 points)
   1. Policy expertise: (5 points): Does the research team discuss how they will build upon previous relevant work that was funded by CARB, other state federal, regional, and local agencies (e.g. departments of transportation, metropolitan planning organizations, transportation agencies, or other related agencies)? Researchers should describe their level of experience with regional transportation plans and the impacts their previous work has generated. This project will require utilizing mix-methods of analysis (e.g. interviews, focus groups, surveys, etc.) as an important tool to offer comprehensive results.
   2. Experience and proven dedication to community based engagement approaches (5 points): Does the research team have relevant experience to perform the proposed work? It is expected researchers have experience engaging diverse stakeholders, including community organizations and resident perspectives. The researchers and research partners should describe previous experience with community engagement, present letters of support from previous community partners, references, or a community impact statement describing the impact previous work has had in communities.
4. EXPANDING EXPERTISE (10 points) – Is the team composed of a multidisciplinary set of experts? Researchers are required to partner with non-academic partners, such as non-profits or community based organizations, or smaller universities/colleges with relevant experience in this area.
5. TECHNICAL MERIT (20 points) - Describe the submission's technical strengths and/or weaknesses. Proposers should demonstrate the logic and feasibility of the methodology and technical approach to the project, spell out the sequence and relationships of major tasks, and explain methods for performing the actual work. Please factor in how well the draft proposal describes these areas:
   1. Effective use of interviews, surveys, and focus groups.
   2. Familiarity with regional transportation plans, existing tools MPOs employ in their regional plans.
   3. Incorporation of equity and climate components in the proposal. Including opportunities to engage diverse communities and stakeholders.
   4. The review team will be selecting only one draft proposal for development into a full proposal. If this draft proposal has potential, what areas or topics should be prioritized or better explained in the full proposal?
6. LEVEL AND QUALITY OF EFFORT TO BE PROVIDED(15 points) – Does the proposal allocate time and resources in such a way that the objectives of the study will be met? Is supervision and oversight adequate for ensuring that the project will remain on schedule? Is the distribution of workload appropriate for activities such as research, evaluation and analysis, data reduction, computer simulation, report preparation, meetings, and travel?
7. COST EFFECTIVENESS (15 points) - Does the cost seem appropriate for the proposed work?  Does the proposed work seem feasible within the requested budget?  Projects that provide co-funding should be evaluated more favorably.

Back to FY21-22 Solicitation Landing Page

[1] Transportation for America. (2017). How Metropolitan Planning Organizations Across the United States Use Performance Measures. https://t4america.org/wp-content/uploads/2017/02/MPO-Performance-Measures-Survey.pdf

[2] Two most recent legislations include the Moving Ahead for Progress in the 21st Century Act and Fixing America's Surface Transportation Act.

[3] Transportation for America. (2017).

[4] Deakin, E. (2020). Barbour, E. Chapter 20. Transportation, Land Use, and Environmental Planning. Elsevier.

[5] CARB. (2018). 2018 Progress Report: California’s Sustainable Communities and Climate Protection Act, (page 4). https://ww2.arb.ca.gov/sites/default/files/2018-11/Final2018Report_SB150_112618_02_Report.pdf  

[6] Project performance assessments (PPA), summarized as a process to evaluate the impacts of transportation investments. Performance measures are considered an equivalent of PPAs in this document.

[7] Improvements may include; creating feedback loops, engaging multiple stakeholders, developing a circular planning and evaluation process that assess past and current plan performance.

[8] This task will evaluate what is publically available. The task may be relatively quick and possibly build on work that CARB staff have done.