Civic Bridge 2024: Strengthening San Francisco’s Coordinated Street Response System

Since early 2023, we at the Mayor’s Office of Innovation (MOI), along with our partners in the Department of Emergency Management (DEM), have been working to overcome the challenge of data silos in the City’s homelessness street response system. Utilizing a novel legal framework, MOI has integrated individual client encounter data from across the City’s nine street response teams to produce a single comprehensive dataset — the All Street Integrated Database (ASTRID). ASTRID includes individual encounter histories for 15,000+ unique clients. The robust dataset is designed to level-up the City’s homelessness response through improved individual care, system-level tracking, analysis, and modeling.

The 2024 Civic Bridge cohort of projects were strategically aligned to support ASTRID City Department partners and, in turn, enhance the ASTRID dataset. This year’s projects leveraged cross-sectoral expertise to improve data quality, enhance cross-departmental coordination in response to overdoses, and refine street team objectives and key performance indicators (KPIs) to better understand and improve the City’s coordinated street response.

Healthy Street Operations Center & US Digital Response: Utilizing iterative design to update HSOC’s client engagement form

The Healthy Streets Operations Center (HSOC) helps people living in large encampments (6+ tents) by offering them shelter, health and behavioral support services. The HSOC team needs accurate and complete client information to make data-informed client and system-level decisions. Accurate and standardized data is also integral to develop analytics and performance metrics to strategize and prioritize outreach methods based on previous performance.

Prior to this project, the HSOC client engagement process was manual, messy, and vulnerable to human error:

• The client intake process required outreach workers to fill out a 20+ question Microsoft Forms Survey for every client they encountered in the field.
• An analyst would then copy and paste the form’s raw data into a SharePoint excel sheet for cleaning.
• Many of the form’s fields were open input (as opposed to fixed/drop down), which led to a lack of standardization across the data collected.

HSOC partnered with US Digital Response (USDR) to refine the questions in its encounter form to streamline the intake process, as well as recommendations for an updated form with fixed fields to standardize the data being collected.

Process

The USDR volunteer team conducted user research with HSOC outreach workers and departmental partners to understand the challenges in the existing data intake process. Utilizing insights gleaned from their user research, the USDR volunteers developed a comprehensive stakeholder map of the form data. The journey map visually represented how different users throughout the data intake process were interacting with the data collected in the outreach form.

The key project goals identified through USDR’s user research and stakeholder mapping were:

• Automate data cleaning processes to reduce manual work.
• Optimize data formatting and reporting to facilitate easier analysis and answering of common policy questions.
• Improve data entry process by addressing issues with redundant questions and free-form answers.

After identifying these key goals, the USDR team conducted an audit of the existing form to identify potential opportunities for improvement in standardization. The main optimization areas identified were:

• Date
• Location
• Refusal/ Acceptance of shelter
• General form structure.

Deliverables

USDR developed two prototypes for a new, improved intake form. These prototypes focused on:

• User-friendliness
• Improved standardization of questions
• Data quality
• Mobile compatibility

With multiple ways to test a single problem area, the team decided to conduct a two-phased (A/B) test. The goals of testing various phases for form improvements were two-fold:

• Improve the overall quality of the data received so that Data Analysts could spend less time manually cleaning dirty data.
• Improve the overall user experience of the form to be less tedious for ERT members to fill out while in the field.

Impact

The form changes reduced the percentage of submissions containing dirty data from 15.38% to 2.49% — an 84% improvement! Overall, the changes made during both Phase 1 and Phase 2 tests reduced the need for manual database cleanup. Though cleanup was still needed for the ‘Date’ and ‘Manually Entered Locations’, the addition of helper text and a fixed dropdown for Date and Location respectively significantly reduced the amount of cleaning needed.

The data intake standardization both directly improved the HSOC data quality being regularly uploaded into the ASTRID dataset, and provided scalable insights for other street team data leads to standardize and improve their own field data intake processes.

SF Fire Department & Adobe: Developing a logic model and client well-being measurement tool for the EMS-6 program

SF Fire’s EMS-6 program provides intensive wraparound care to the most frequent users of the City’s 9–1–1 /emergency system (four or more times in a 30-day period, or ten or more times in a 12-month period). The wrap-around care EMS-6 provides is long-term and inherently personalized to the unique needs and circumstances of each client.

To improve the program’s operations and corresponding client & system outcomes, SF Fire sought to:

1. Develop a set of KPIs that capture the balance between client stabilization and reduction in clients’ cost to the emergency system; and
2. Formalize and optimize the EMS-6 team’s processes for providing direct care to clients and partnering with the City’s network of emergency and non-emergency care providers.

Process

The Adobe volunteer team conducted extensive field research and stakeholder engagement for this project to ensure a comprehensive understanding of the Coordinated Street Response ecosystem. This included:

• 10+ ride-alongs with EMS-6 captains
• Interviews with all six EMS-6 Captains and Chiefs
• Weekly meetings with EMS-6 leadership.
• Meetings with various City departments including DataSF, DPH, DEM, MYR, and SF Fire.

Leveraging the findings from their user research, the Adobe volunteers developed an EMS-6 service blueprint. The blueprint provided a visual representation of the EMS-6 service journey from emergency notification to client handoff. The blueprint’s comprehensive overview of the EMS-6 service journey was then used to facilitate working sessions amongst EMS-6 leadership about how to enhance operational efficiency and effectiveness. The blueprint also enabled the project team to identify how to approach developing KPIs to measure the program’s overall success.

Deliverables

• A prototype framework for a client wellbeing tracker to monitor improvements in client health. The measurement tool benchmarks a client’s well-being upon their first interaction with an EMS-6 Captain using common client themes identified by the Adobe team through user research with the EMS-6 team and tracks client improvements over time.
• A prototype form for EMS-6 Captain’s to use in the field to track when there is a mismatch in client resources available versus resources requested/needed.

Impact

The client well being measurement tool is intended to help SF Fire better understand EMS-6’s impact on clients and reflect the effort Captains invest in connecting clients to needed resources. Due to the inherently long-term nature of the EMS-6 Captain’s relationship with clients, being able to identify and measure over time the incremental successes and areas for improvement is integral to high functioning program operations.

Additionally, the in-field resource tracker addresses a significant barrier to getting EMS-6 clients the care they need — a lack of available client resources. Tracking when mismatches in resources occur provides crucial feedback to City leadership on resource demands, which in turn can inform tactical resource investment.

Both the client well being measurement tool and in-field resource tracker have the potential to scale to other teams involved in the City’s systemwide street response. Additionally, the tool’s characteristics and weighting scale can applied directly to the ASTRID dataset to automatically provide “well-being” scores for all clients included in the dataset.

Department of Public Health & ZS Associates: Utilizing rapid iteration to enhance a data driven system of care for survivors of non-fatal overdoses

San Francisco, like many cities across the country, faces an ongoing opioid crisis, with a high number of non-fatal overdoses despite a harm reduction strategy and expanded access to treatment services. The Department of Public Health (DPH) identified challenges in effectively coordinating response efforts among the various overdose prevention teams, including SORT (Street Overdose Response Team), POET (Post Overdose Engagement Team), and UCSF’s HOPE (Home Outreach, Prevention & Engagement). Key issues included:

• Fragmented communication
• Inconsistent models of care across teams
• Care gaps that hinder timely interventions for overdose survivors

DPH partnered with ZS Associated to develop a data-driven, coordinated care response system to streamline service access and enhance support for individuals in recovery.

Process

ZS Associates utilized a multi-phase approach for this project:

1. Project Kickoff and Immersion Phase: The team conducted shadowing sessions and interviews with stakeholders from each response team to gain an in-depth understanding of workflow pain points and potential areas for improvement. This immersion allowed the team to build rapport, identify specific barriers, and align on intervention concepts.
2. Iterative Workshops and Intervention Development: Through a series of workshops, the Civic Bridge team and stakeholders brainstormed, prioritized, and developed interventions based on their impact and feasibility. Key interventions were designed to improve data sharing, streamline handoff processes between teams, and increase general coordination among response teams.
3. Agile Implementation in Sprints: The project followed an agile framework, implementing interventions in iterative sprints that allowed rapid feedback loops, enabling continuous improvement and flexibility. Interventions were tested and refined in real-time to meet evolving needs.
4. Final Synthesis and Reporting: After implementing the interventions, the project team conducted a final synthesis session to document outcomes and lessons learned. The Civic Bridge team then provided recommendations for ongoing improvements and inter-agency collaboration.

Deliverables

Priority interventions identified and implemented during the project timeline by the Civic Bridge team and partner teams included:

1. Standardized Training on Overdose Prevention

• Intervention: Conducted comprehensive training sessions on medications for addiction treatment and overdose prevention, attended by over 60 community paramedics.
• Objective: Targeted inconsistent levels of knowledge among community paramedics regarding substance use disorder treatments and overdose response.
• Outcome: Increased paramedics’ comfort in recognizing opioid withdrawal signs and administering necessary treatments, contributing to a higher rate of Buprenorphine starts and improved support in overdose cases.

2. Warm Handoff Process Between SORT and POET

• Intervention: Established a protocol for “warm handoffs” from SORT to POET, facilitating smoother transitions for clients ready for medication-assisted therapy (MAT).
• Objective: Aimed to improve coordination and ensure clients interested in starting MAT received seamless support from SORT to POET without gaps in care.
• Outcome: The warm handoff process improved continuity in client care, leading to increased transitions to treatment and enhanced MAT uptake among clients.

3. EPIC Access Expansion and Documentation Standardization for HOPE

• Intervention: Expanded access to DPH’s database, EPIC, for the HOPE team and established a standardized documentation workflow to streamline client information sharing.
• Objective: Targeted issues with inconsistent data entry and duplicative efforts, aiming to integrate HOPE’s workflows within a common platform.
• Outcome: Improved efficiency in supporting clients with unclear housing status and centralized client information on EPIC, which saved substantial administrative time and allowed for more direct service.

4. Collaboration and Visibility Enhancements through Regular Meetings

• Intervention: Implemented a regular meeting cadence between SORT, POET, and HOPE leadership to discuss workflows, review client cases, and coordinate efforts for clients with transient housing.
• Objective: Addressed challenges in collaboration and duplicated efforts, especially in tracking clients with unstable housing.
• Outcome: Enhanced inter-agency collaboration, reduced duplicative outreach efforts, and improved tracking of client engagement, especially for high-risk clients.

Additionally, ZS Associates developed a Response Journey Map: A detailed journey map outlining each step in the overdose response process, highlighting key interaction points and potential drop-off areas to streamline the client journey from overdose to treatment.

Impact

Implemented interventions have enhanced the responder experience across SORT, POET, and HOPE by improving training, coordination, and documentation processes. The teams plan to continue utilizing an iterative approach to refine interventions from the Civic Bridge project, as well as implement new interventions. The goals of these interventions are to:

• Strengthen warm handoff processes, ensuring clients starting Buprenorphine with SORT paramedics receive seamless follow-up care through POET.
• Equip 100% of community paramedics with training to recognize signs of opioid withdrawal and connect clients to substance use resources, following the successful training of 60+ paramedics on addiction treatment and overdose prevention.
• Increase access to community resources, aiming to ensure 90% of clients receive guidance on available substance use services.
• Enhance continuity of care through standardized EPIC documentation, with 90% of clients flagged to POET having a care coordination note, supporting follow-up care for at least 100 clients annually.
• Improve efficiency in client tracking, particularly for POET and HOPE teams, supporting clients with unclear housing status and minimizing duplication of efforts to identifying / locating clients
• Optimize data management within EPIC to capture 80% more client documentation, centralize monitoring, and save 100+ administrative hours for redeployment to direct client service activities.