The Sign of the City

Machine Learning, Data Visualization, Urban Analytics, Web Development
Role:
Designer
Researcher
Timeline:
Jan. 2024- April 2024
Team:
Individual
Tool:
D3.js
Mapbox.js
JavaScript
HTML/CSS
Project Website  ↗
Project overview
This project is an interactive data visualization web application that won the 2024 GSAPP Visualization Award for its innovative use of computing media in architectural and urban research, design, and fabrication.

The application employs machine learning algorithms(CNN and OCR) to detect signage in New York City’s Street View images, analyzing their types, languages, and content. This data is then presented through a JavaScript-based interactive visualization, showcasing information in a carousel-like format. Users can explore sign locations, view street images, and access details about the sign’s types, languages, and content. Additionally, demographic data related to the signs’ locations is included to provide further context.

The web application features robust search and filter functions, allowing users to search for specific keywords that appear on signs and filter results by language and sign type. This functionality offers a fun and insightful way to explore the city through the lens of its signage, revealing financial, cultural, lexical, and political information about the city. By analyzing and visualizing these aspects, the project provides a deeper understanding of New York City’s diverse urban landscape and the various narratives conveyed through its signage.
Background
The development of urban visual information study
The tradition of incorporating visual information runs through the entire history of modern urban studies. In the late 1900s, Camilo Sitte and Ebenezer Howard started to study the sensory experience of citizens, emphasizing the shape and proportion of beauty in the public space. In the 1970s, Kevin Lynch started to study structure and human perceptions in the city. As they lay the foundation of many urban visual information studies, most of these are more focused on the physical aspects like shape, proportion, and human perceptions but do not much extend beyond these traditional dimensions to explore other aspects such as cultural, economic, and political aspects. With the development of AI, which could potentially open new opportunities in urban visual study, how can we utilize it to build up on the previous studies?
Adding more to the Image of the City?
In the book The Image of the City by Kevin Lynch, he stated that a well-designed city incorporating 5 elements - paths, edges, districts, nodes, and landmarks - would obviate the need for additional signages, as the urban structure itself would inherently guide and inform. As long as people need signage, the image of the city fails...However in modern cities, are there any cities with no signs? As one of the few elements in urban street views containing words and texts, signs offer a unique opportunity to glean insights beyond physical attributes, extending into more cultural, commercial, and political aspects. Therefore, this thesis seeks to explore a new way of urban visual information study, by taking what previous research overlooked and what Kevin Lynch considers a "failure" - signage - as an advantage, and leveraging new AI technology, to add more to the images of the city.
Research Questions
What AI technologies can help extract more information from urban images?
What do signs tell us about the city?
What types of information signs can add to our understanding of the city(Cultural, Financial, Temporary, Informal, Lexicon...)?
Methodology
The research methodology consists of two steps: first, web-scraping to gather Street View images from Google Maps of the designated research site; second, analyzing these images using Deep Learning algorithms to classify the signs. After classification, the sign areas are extracted for further analysis. Optical Character Recognition (OCR) algorithms are then employed to decipher the content and language displayed in the signs.
Findings
19 signage types were identified in New York City and can be grouped into three main categories. The first group is street signs, which include Don't Block signs, Do Not Enter signs, and All Traffic signs, etc. The second group consists of business-related signs, including Shop signs, For rent signs, Promotion signs, etc. The third category is public infrastructure signs, including Bus stop signs, Scaffolding signs, Subway signs, etc. 7 signage languages were identified, including 6 non-English languages: Spanish, Chinese, French, Italian, Korean, and Russian.