A Framework for Collecting Useful Feedback from Diverse Building Users
Commonly referred to as Post-Occupancy Evaluation (POE), feedback from building users helps architects and building managers understand how projects are performing, what could be fine-tuned, and what to address in future projects. But many design projects lack a formal process for collecting and structuring occupant feedback into a useful format. The Environmental Design Research (EDR) class that I taught at Roger Williams University (RWU) addressed this challenge through a three-way collaboration in which students engaged architects and owners to collect and process feedback from building occupants.
Each year, our Environment Design Research Class studied three buildings, with sponsorship from the architects.
A model launched over 15 years ago by my co-instructor Lefteris Pavlides, this academic-practice partnership benefits all three collaborators:
For owners, the class provides a forum for building inhabitants to voice their experiences.
For students (future architects), the class provides students the opportunity to see how design decisions impact actual building occupants.
For architects, the collaboration provides information about design outcomes.
These three parties also contribute to the process:
Owners provide access to their buildings and help coordinate and recruit interview participants.
Students create the instruments for the study and conduct the interviews.
Architects provide background on the project to help interpret interview feedback.
The semester-long building study involves a 3-way collaboration between the architect, owner and students.
POE methods vary widely. What distinguishes our approach is a technique which the academic literature calls Ethnographic Photo-elicitation. Simply put, this process involves using photographs to generate spontaneous comments. “Ethnographic” refers to adopting the interviewee’s perspective, allowing participants to comment on their own rather than directing their attention with specific questions. Much in the way an anthropologist studying a culture sets aside his or her outsider’s perspective, our goal is to set aside the architect’s, owner’s or student researcher’s perspective to better hear occupants’ concerns. “Photo-elicitation” refers to presenting photographs of a building to facilitate comments.
The Process
Phase I: Background Research
The semester-long study starts with a lecture by the architects, a building walkthrough and four types of research activities to develop a background understanding of the building:
Architecture Field: This research involves students visiting, photographing and analyzing design aspects of the building being studied.
Architecture Literature: Students examine precedents buildings to become familiar with the building type being studied.
Social Field: This research involves documenting the different types of users and the range of activities that occur in the building.
Social Literature: The students collect social science studies relevant to the activities in the building.
Initial background research includes steps to better understand the design of the building, user demographics and range of activities that users engage in.
Phase II: Creating the Research Instrument
Students divide up the responsibilities for assembling the research instrument, the tools for the study:
Assembling floor plans and designing recording forms for the interviews and observational studies
Planning participant recruitment
Writing an informed consent form
Selecting building photographs for the interview
Setting up the interview recording form
Preparing the matrix (spreadsheet) for entering and processing the interviews.
Pre-testing the study
Students divide up responsibility to compile components of the research instrument.
Phase III: Data Collection & Analysis
Often conducted individually or in pairs, interviewers show participants photographs of a building and record their responses to a broad question about what works well in a space and what could be improved.
The researchers process the interviews by breaking down responses into separate comments, entering them into a matrix (spreadsheet) and tagging each comment with demographic characteristics of the participant. A typical class section with 15 students, each student conducting 2 or 3 interviews, can generate about 3000 comments. To summarize this information, we group similar comments and label each group with a summary sentence.
Comments might express different and contradictory views, but we welcome this diversity of opinion. Sometimes, the demographic information that we collect helps clarify reasons for these differences.
Student researchers enter and sort participant comments in a data matrix and then review summarized comments with the architect sponsors.
A critical step for understanding occupant feedback involves reviewing these summarized comments with the architects. Building occupants are often unaware of the reasons behind design decisions. By providing background information about the project history, existing conditions, constraints (e.g., code, budget, client requests) or subsequent alterations by the owner, the architects help provide a fuller understanding of occupant feedback.
The architects provide background information about the project to better understand the building occupant interviews.
Phase IV: Final Presentation
The semester study concludes with students assembling the findings in a slide presentation. Each finding is accompanied by summarized, relevant social science studies (Social Literature), precedent buildings (Architecture Literature), illustrative quotes from study participants and interpretations from the architects.
At the end of the semester study, students present findings and deliver a set of slides summarizing the findings to the architect and owner.
Examples of Useful, Actionable Findings
In collecting feedback from building users, our goal has been to uncover information that is useful and actionable.
Often, this information helps building owners and managers enact immediate operational improvements. For instance, in an orthopedic center, comments about an entryway mat that presented a trip hazard to patients in walkers prompted the facility manager to quickly remove it.
Other times, learnings accrue over successive projects. Notably, the Providence, RI firm Vision 3 Architects has evolved their work with their client partner Thundermist Community Health Centers, for instance, successively improving their approach to the design of the patient check-in and check-out process.
Timeline showing how patient check-in and check-out stations at Thundermist Community Health Centers evolved, informed by building occupant interviews.
In some projects, the needs and use patterns of client organizations evolve over time. In one Thundermist project, staff members commented that a particular group office felt crowded and noisy. The architects explained that the space was originally designed for 12 people, but now held 18 or more. To accommodate more staff, the owners also experimented with hotelling, but with the pandemic, the architects speculated that the client might move away from shared desks. Findings are often time-dependent. The same building, studied again years later, can provide different responses as culture evolves and an organization’s needs change.
The semester research is summarized in a set of slides that include individual findings, the architect’s interpretive comments and interview quotes illustrating the finding.
Conclusions: A Framework for Capturing Diverse User Responses
Our framework for processing building occupant feedback has been developed with architects sponsoring the Environmental Design Research class over many years. These methods could also be used in a purely professional setting.
Our ethnographic photo-elicitation methods facilitate collecting input from the building user’s perspective. When systematically coded, summarized and interpreted with the project architect, this qualitative approach can provide a richer narrative, compared to typical direct queries, multiple choice questions or rating scales. Specifically:
This listening-focused process also allows unexpected findings to surface, less biased by researcher pre-conceptions.
Second, buildings are used by many types of users, and our approach better reveals those diverse user responses. Our methods highlight important minority needs (e.g., the challenges of wheelchair users), contrasting with other approaches that focus on the average response or majority view.
The usefulness of our approach has been validated by the architects and owners who have sponsored the class multiple times and implemented changes based on our findings. This “feed-forward” approach to collecting occupant input closes the loop between design intentions and outcomes, informs operational changes and can serve as a launching point for future building projects.