Artificial intelligence–based assessment of built environment from Google Street View and coronary artery disease prevalence

Abstract

Background and Aims

Built environment plays an important role in the development of cardiovascular disease. Tools to evaluate the built environment using machine vision and informatic approaches have been limited. This study aimed to investigate the association between machine vision–based built environment and prevalence of cardiometabolic disease in US cities.

Methods

This cross-sectional study used features extracted from Google Street View (GSV) images to measure the built environment and link them with prevalence of coronary heart disease (CHD). Convolutional neural networks, linear mixed-effects models, and activation maps were utilized to predict health outcomes and identify feature associations with CHD at the census tract level. The study obtained 0.53 million GSV images covering 789 census tracts in seven US cities (Cleveland, OH; Fremont, CA; Kansas City, MO; Detroit, MI; Bellevue, WA; Brownsville, TX; and Denver, CO).

Results

Built environment features extracted from GSV using deep learning predicted 63% of the census tract variation in CHD prevalence. The addition of GSV features improved a model that only included census tract-level age, sex, race, income, and education or composite indices of social determinant of health. Activation maps from the features revealed a set of neighbourhood features represented by buildings and roads associated with CHD prevalence.

Conclusions

In this cross-sectional study, the prevalence of CHD was associated with built environment factors derived from GSV through deep learning analysis, independent of census tract demographics. Machine vision–enabled assessment of the built environment could potentially offer a more precise approach to identify at-risk neighbourhoods, thereby providing an efficient avenue to address and reduce cardiovascular health disparities in urban environments.

Structured Graphical Abstract

Extracted features from street view images via artificial intelligence (AI) demonstrate that 63% of the variation in coronary heart disease (CHD) prevalence can be explained by these environmental factors, highlighting the significant potential of this data in informing cardiovascular health assessments. AIC, Akaike information criterion; BIC, Bayesian information criterion; DSE, demographic and socio-economic; GSV, Google Street View; LMEM: linear mixed-effects model; LRT, likelihood ratio test.

Cardiovascular risk, Neighbourhood, Built environment, Google Street View, Machine learning

See the editorial comment for this article ‘Artificial intelligence-enhanced exposomics: novel insights into cardiovascular health’, by R. Khera, https://doi.org/10.1093/eurheartj/ehae159.

Translational perspective

This cross-sectional study utilizes deep learning and machine vision to examine the association between the built environment and coronary heart disease at the neighbourhood level. Street view–based built environment factors, analysed using deep learning, explain a significant portion of coronary heart disease prevalence independently of socio-demographics. These findings have implications for targeted interventions, urban planning, and public health policies. Incorporating machine vision–enabled identification of urban features associated with cardiovascular risk can guide the creation of heart-healthy cities, reducing the burden of coronary artery disease and benefiting population health.

Introduction

Coronary heart disease (CHD) accounts for over 50% of mortality from heart disease in the USA, responsible for nearly 400 000 deaths in 2020.¹ Despite advances in prevention and treatment over the past decade in the USA,² CHD remains the leading cause of death in the USA since 1950 with increasing evidence for non-conventional risk factors playing a larger than anticipated role than previously suspected.^1,3

Socioenvironmental factors are amongst the leading non-traditional risk factors increasingly implicated in CHD development.^4–6 These factors include social determinants such as race, income, education, and culture as well as the factors in the built environment and factors in the ambient environment such as noise, temperature, and air pollution, all of which have been shown to exert significant effects on CHD.^5–9

Large-scale integrated assessment of the environment at the neighbourhood level can facilitate rapid and complete assessment of its impact on CHD. Such data are however scarce, partly because of the costly and time-consuming nature of neighbourhood audits and inconsistent measurements and standards for data collection. Machine vision approaches such as Google Street View (GSV) have become an increasingly popular approach for virtual neighbourhood audits since its launch in 2007. Google Street View image coverage has been consistently expanding in recent years achieving almost full coverage in the USA.¹⁰ Previous studies have shown GSV results are comparable with field assessments and have been used to assess the built environment features such as greenspace,^11,12 buildings,¹³ and roads.¹⁴

Google Street View images have gained popularity as a preferred data source for large-scale studies due to the widespread availability of this data set. It represents one of the largest repositories for machine vision–enabled assessments of extensive geographical areas, facilitated by standardized data collection approaches. Deep learning approaches such as convolutional neural networks (CNN) have been widely used in many studies and applications, given their excellent performance in tasks such as image classification, object detection, and image segmentation.¹⁵ The use of such approaches to rapidly assess and extract built environment features from GSV images using deep learning can help facilitate integrated assessment and capture other aspects that may not be otherwise included. The goal of this study is to use GSV images to assess the built environment and use them to estimate CHD prevalence at the census tract level.

Methods

The University Hospitals Institutional Review Board exempted this cross-sectional study from review and the need for informed consent in accordance with 45 CFR §46, as the data were deidentified and thus not considered human participant research.

Data source for coronary heart disease

The prevalence of census tract CHD was obtained from the 2018 Centers for Disease Control and Prevention (CDC) Population Level Analysis and Community Estimates (PLACES), a project that provided chronic disease risk factors, health outcomes, and clinical preventive services. This project, a collaboration between the CDC Foundation and the Robert Wood Johnson Foundation, measures CHD prevalence using data (2015, 2016) from Behavioral Risk Factor Surveillance System (BRFSS), where people aged ≥18 are surveyed to report whether or not they have been told by a doctor, nurse, or other health professional that they had angina or CHD. We collected the CHD prevalence data for 789 census tracts in seven cities: Bellevue, WA; Brownsville, TX; Cleveland, OH; Denver, CO; Detroit, MI; Fremont, CA; and Kansas City, KS. For each city, we calculated the mean CHD prevalence and its interquartile range (IQR), showing the range within which the middle 50% of prevalence lies. The seven-city selection relies on population, disease burden, and geography, as outlined in the online supplementary material.

Google Street View data

Environment information was derived from ∼0.53 million GSV images for the seven cities (143 K for Detroit, 59 K for Kansas City, 70 K for Cleveland, 65 K for Brownsville, 38 K for Fremont, 35 K for Bellevue, and 120 K for Denver). The GSV images were downloaded via GSV static application programming interface (API) from 2020 to 2021. Google Street View API provides users with street-level panoramic imagery which captures the visual domain of pedestrians in thousands of cities worldwide. The GSV images of each census tract were downloaded in a grid pattern in the corresponding tract with an interval of 100 m. At each location where GSV images were retrieved, four images were gathered from different directions (i.e. the cardinal directions: N, E, S, and W), which composes a panoramic view of the surroundings at that location. When latitude and longitude coordinates are provided, the API searches within a 50 m radius for a photograph closest to this location. The API would not return any images if no available images could be found.

To process these images and gain environment information from them, a pre-trained deep CNN (DCNN) Places365 CNN¹⁶ was used as the feature extractor to obtain the deep features of the image. Here, the deep features are the outputs of the deep layers in the hierarchy of the network. Compared with the shallow features in the shallow layers, these deep features represent the semantic information of the GSV images. Details of how the extraction was performed can be found in Supplementary data online, Figure S1. We used Places365 CNN as the feature extractor because the images trained on Places365 CNN are more similar to that of GSV. Places365 CNN was trained on the subset of Places Database, which contains more than 10 million images consisting of 400+ unique scene categories such as towers, soccer fields, streets, swimming pools, and train station platforms. Compared with the ImageNet database, the diversity of environmental features found in the Places Database was believed to be representative of what is contained in GSV images. Through feature extraction, we obtained 4096 features representing the average built environment information for each census tract. It is noteworthy that Google emphasizes considering various factors, including weather conditions, in their image acquisition process.¹⁰ Additionally, upon examination of our data set, only a very small proportion of the images were captured under conditions where only a limited amount of snow was present, with the observation that this has minimal influence on the computer vision algorithm.

Traditional demographics, socio-economic factors, and composite indices for social determinant of health

In addition to the GSV features, we obtained traditional demographic and socio-economic (DSE) factors: age (median), sex (male %), race (Black %), income (median $), and education (<high school %) from the 2018 American Community Survey 5-year estimates.¹⁷ We also collected the established composite indices for social determinants of health (SDoH). Specifically, we considered three widely used SDoH indices: 2018 Social Deprivation Index (SDI),¹⁸ 2015 Area Deprivation Index (ADI),¹⁹ and 2018 Social Vulnerability Index (SVI).²⁰ Social Deprivation Index, provided by Robert Graham Center, is a composite measure capturing seven demographic characteristics gathered from the American Community Survey to assess the overall socio-economic disparities at the area level. In contrast, ADI focuses on either the state or national level, ranking socio-economic (e.g. income, education, employment, and housing quality) disadvantage in those neighbourhoods. Additionally, CDC’s SVI provides insights into the vulnerability of communities by considering socio-economic status, housing, transportation, and other factors. These composite indices collectively contribute to a nuanced understanding of the SDoH in a neighbourhood.

Statistical analysis

Machine learning model using raw convolutional neural network–extracted features

We utilized three machine learning (ML) models to explore the association between the raw (4096) CNN-extracted features of GSV images and the tract-level CHD prevalence. The models for this analysis included extratrees regressor (ET),²¹ random forest regressor (RF),²² and light gradient boosted machine regressor (LGBM).²³ All models were estimated using a 10-fold cross-validation technique for a more robust result. For 10-fold cross-validation, the data set is split into 10 equal-sized subsets, and the model is trained on 9 subsets and tested on the remaining 1 subset. This process is repeated 10 times until all 10 subsets were used once as the testing set. R² values were reported as the model quality, quantifying the extent to which the CNN-extracted features of GSV images fit tract-level CHD prevalence. The performance of each model was also evaluated using the mean absolute error (MAE) and root mean squared error (RMSE).

Multilevel modelling with traditional demographics, socio-economic factors, and composite indices for social determinant of health

We analysed the effects of common DSE factors as well as CNN-extracted features of GSV images associated with the CHD. Details of this analysis are provided in the online supplementary material. Briefly, we followed an entirely supervised two-step modelling strategy. In the first step, a multivariate sparse partial least squares (SPLS) regression²⁴ was applied to the CNN-extracted features to reduce the dimensionality issue and the effect of noise on the error rates of our inferences. In the second step, we compare the effects of CNN-extracted features with recognized factors by building multilevel regression models allowing to account for both fixed and random effects. Specifically, linear mixed-effect regression models were fitted with the first few selected SPLS components augmented with DSE factors as well as other composite indices for SDoH, all treated here as fixed effects, and where city was treated as a random effect. The models were all adjusted for fixed effects including age, sex, race, income, and education. We then compared the regression estimates and goodness-of-fit measures between the reduced models and the combined linear mixed-effects models (LMEMs), where each set of independent variables enter simultaneously or individually. Two sets of three models were compared in this analysis: (i) a model containing both DSE factors and SPLS components (DSE + GSV model), (ii) a model with DSE factors alone (DSE model), and (iii) a model with the SPLS components alone (GSV model). Similarly, we compared three other models: (i) a model that incorporated the three SDoH indices alongside SPLS components (SDoH + GSV model), (ii) a model comprising only the three SDoH indices (SDoH model); and (iii) a model solely featuring the SPLS components (GSV model). In all comparisons, model performance was assessed using goodness-of-fit measures such as likelihood ratios tests (as it applied), Akaike information criterion (AIC) and Bayesian information criterion (BIC) criteria, as well as marginal and conditional R² values based on Nakagawa’s R² for mixed models.²⁵ For each model, we calculated the coefficient estimate of each variable with its standard error and confidence intervals (CI), the intraclass (within city) correlation coefficient (ICC) for the city random effects, the within-city random-effect residual variance (σ²), and the between-city random-effect residual variance (τ_00City) to assess the reliability or consistency of the measurements. All P-values less than the α = 0.05 level were considered significant.

Features visualization using Grad-CAM

To understand the deep features of the GSV images that are associated with neighbourhood CHD prevalence, we identified the most influential GSV features contributing to the SPLS components. These top features were discerned by examining the magnitudes and signs of their coefficients in the SPLS regression model, thus helping to understanding how each feature is associated with CHD prevalence. Subsequently, we employed the Grad-CAM technique²⁶ to create the saliency map to highlight these prominent features in the original GSV images. This process provides certain explanations of what environmental features the CNN thinks to be associated with neighbourhood CHD prevalence.

In our study, we employed Python (version 3.8.10) for deep learning and ML tasks, utilizing packages such as PyTorch (version 1.8.2), fastai (version 2.6.3), and scikit-learn (version 1.0.2). For multilevel modelling, we used R (version 4.2.1), spls (version 2.2–3), and library lmer (version 1.1–35.1).

Results

Seven cities revealed varying levels of CHD prevalence at the census tract level. In Bellevue, the median CHD prevalence (%) was 4.70 (IQR: 3.75–5.23), whereas in Brownsville, it was 7.70 (IQR: 6.48–8.63). Cleveland exhibited a median CHD prevalence of 8.70 (IQR: 7.35–10.00). Denver had a median CHD prevalence of 4.30 (IQR: 3.45–5.20), while Detroit’s median CHD prevalence was 8.55 (IQR: 7.40–9.80). Fremont displayed a median CHD prevalence of 3.70 (IQR: 3.35–4.10), and Kansas City reported a median CHD prevalence of 7.20 (IQR: 6.10–8.30).

Machine learning model results with raw convolutional neural network features

The 4096 CNN-extracted features from GSV images were able to explain more than 63% of the variance (R² = 0.634) on the tract-level CHD prevalence in seven cities (Figure 1). The three ML models had the similar performance, with the ET achieving the best result among all models with the lowest average MAE of 1.11 and RMSE of 1.58 (see Supplementary data online, Table S1). The actual estimate from CDC’s CHD prevalence and the model-predicted CHD prevalence were mapped for all census tracts in seven cities (Figure 2). There was a good agreement between the actual estimates and predicted CHD prevalence across all census tracts in seven cities. We found a small number of extreme values that were underestimated by the models in certain census tracts of Detroit and Cleveland. The CHD prevalence of these underestimated census tracts was often more than 12%. When examining the CNN-extracted features using t-SNE, we noticed clustering of census tracts with similar values of CHD prevalence (see Supplementary data online, Figure S2).

Figure 1

Actual estimates (observed) and predicted CHD prevalence. A total of 789 census tracts in seven cities were analysed. Predicted CHD prevalence was from LGBM model trained using CNN-extracted features. The black dotted line represents the y = x line. Values are in percentage. CHD, coronary heart disease; CNN, convolutional neural network; LGBM, light gradient boosted machine

Figure 2

Maps of actual estimates (left) and predicted (right) CHD prevalence. The predicted CHD prevalence is obtained by averaging the results from 100 random trials based on k-fold cross-validation (with k = 10). Values are in percentage. Maps are not on the same scale. CHD, coronary heart disease

Comparison of Google Street View–sparse partial least square components with demographics and socio-economic factors

With SPLS, an optimal model was obtained with h = 6 SPLS components (η = 0.6), yielding a model with 883 CNN-extracted features that explains $R_{XY}^2$ > 64.8% variance of CHD prevalence in the census tracts (see Supplementary data online, Figure S3). We found that the combined model (DSE + GSV) demonstrated a better goodness of fit, with statistically significant higher log-likelihood, lower AIC/BIC, and higher R² (conditional R² = 0.792) when compared with GSV and notably DSE model alone (Table 1). The DSE model had lower AIC and BIC values, with a significant LRT, and higher R² when compared with the GSV model (Table 1). Supplementary data online, Table S2, presents the regression estimates and analysis of variance (ANOVA) results. Notably, four out of six GSV–SPLS components demonstrated statistical significance, while all DSE variables are statistically significant.

Table 1

Model performance and comparison of linear mixed-effects models for coronary heart disease prevalence: Google Street View and demographic and socio-economic variables

LMEM	AIC	BIC	R2		Log. lik.	Test	LRT	P-value
			Marg.	Cond.
DSE + GSV	632.0	697.0	0.760	0.792	−308.0
DSE	706.0	743.0	0.645	0.738	−368.0	DSE + GSV vs. DSE	120	<.001
GSV + DSE	632.0	697.0	0.760	0.792	−308.0
GSV	988.0	1030.0	0.608	0.645	−468.0	DSE + GSV vs. GSV	320	<.001
DSE	706.0	743.0	0.645	0.738	−368.0
GSV	988.0	1030.0	0.608	0.645	−468.0	DSE vs. GSV	201	<.001

Models: GSV, the LMEM with only the selected SPLS components (CHD: h = 6 obtained from the 4096 CNN features); DSE, the LMEM with only the demographics and socio-economic variables (city + sex + age + race + income + education); and GSV + DSE, the LMEM with both sets of independent variables from GSV and DSE.

LMEM, linear mixed-effects model; GSV, Google Street View; AIC, Akaike information criterion; BIC, Bayesian information criterion; LRT, likelihood ratio test.

Comparison of Google Street View–sparse partial least square components with social determinant of health indices

As expected, the LMEM that incorporates both GSV–SPLS components and SDoH indices, referred to as SDoH + GSV, achieved the best goodness of fit and overall performance. It exhibited the lowest AIC/BIC and the highest log-likelihood and marginal/conditional R² (conditional R² = 0.785), compared with GSV and notably SDoH model alone (Table 2). Remarkably, the SDoH model performance was comparable with the GSV model with non-significant LRT (P = .60), while AIC/BIC criteria and R² remained not significantly different. All three SDoH indices were statistically significant in the models, and the three GSV–SPLS components were statistically significant when adjusting for either DSE variables (see Supplementary data online, Table S2) or SDoH indices (see Supplementary data online, Table S3).

Table 2

Model performance and comparison of linear mixed-effects models for coronary heart disease prevalence: Google Street View and social determinants of health indices

LMEM	AIC	BIC	R2		Log. lik.	Test	LRT	P-value
			Marg.	Cond.
SDoH + GSV	796.0	852.0	0.739	0.785	−379.0
SDoH	924.0	952.0	0.624	0.680	−467.0	SDoH + GSV vs. SDoH	279	<.001
GSV + SDoH	796.0	852.0	0.739	0.785	−379.0
GSV	988.0	1030.0	0.608	0.645	−468.0	SDoH + GSV vs. GSV	178	<.001
SDoH	924.0	952.0	0.624	0.680	−467.0
GSV	988.0	1030.0	0.608	0.645	−468.0	SDoH vs. GSV	1.89	.60

Models: GSV, the LMEM with only the selected SPLS components (CHD: h = 6 obtained from the full CNN features); SDoH, the LMEM with only the SDoH indices (SDI + SVI + ADI); and GSV + SDoH, the LMEM with both sets of independent variables from GSV and SDoH.

LMEM, linear mixed-effects model; GSV, Google Street View; AIC, Akaike information criterion; BIC, Bayesian information criterion; LRT, likelihood ratio test; SDOH, social determinants of health; SDI, Social Deprivation Index; SVI, Social Vulnerability Index; ADI, Area Deprivation Index.

Visualization of the most influential convolutional neural network–Google Street View features

Grad-CAM was utilized to visualize top CNN-extracted features identified from SPLS regression. The saliency maps generated by the Grad-CAM suggested that feature #2017, which seemed to highlight deteriorated buildings (suggesting neighbourhood blight), had a positive association with CHD prevalence (Figure 3A). Interestingly, other images for feature #2017 seemed to also highlight wooden utility poles (see Supplementary data online, Figure S4). Another feature (feature #458) that was positively associated with CHD was found to be highlighting road cracks as shown in Figure 3B. In contrast, feature #2873 in Figure 3C had a negative association with CHD prevalence, and its heatmap highlighted trees along the road. Feature #237, seeming to focus on well-built houses, also had a negative association with CHD prevalence (Figure 3D). More examples of Grad-CAM on the CNN-extracted GSV features are provided in the Supplementary data online, Figures S4–S8, with ‘noises’.

Figure 3

Feature interpretations using Grad-CAMs. (A and B) Two pairs of GSV images (left) and their activation maps (right) for the features associated with higher CHD prevalence. (C and D) Two pairs of GSV images (left) and their activation maps (right) for the features associated with lower CHD prevalence. CHD, coronary heart disease; Grad-CAMs, gradient-weighted class activation mapping; GSV, Google Street View

Discussion

While many epidemiological studies have examined associations between cardiovascular disease and individual built environmental features (e.g. greenspace, urban architecture, street connectivity, and food availability), our approach focused on machine vision–derived physical environment, relying on CNN and its related techniques to extract features.

Our results showed a strong model fit (R² = 0.634), indicating that the raw CNN-extracted features from GSV images effectively predict CHD prevalence at the census tract level in seven cities (Structured Graphical Abstract). This indicated that the CNN-extracted features could capture neighbourhood features related to cardiovascular health. The predicted CHD prevalence using CNN-extracted features tended to be underestimated in certain areas compared with observed CHD prevalence especially in Detroit and Cleveland. This may be caused by the limited number of samples for these extreme values in the data set; the model may struggle to accurately distinguish and predict such rare occurrences. This may also suggest that certain CHD-related factors may either not be embedded in these environments at these locations or that perhaps features not captured by street view images, such as demographic factors, ambient factors, and other demographic and traditional variables, may play a much larger role in these environments.

Previous studies have used CNN to detect pre-defined built environmental features from GSV and found that dilapidated buildings and visible wires were associated with an increased risk of cardiovascular diseases.^27,28 Our approach employed a data-driven approach, taking the advantage of the knowledge that fully connected layers in the CNN contain condensed information of the input imagery that can be extracted and utilized for a variety of purposes. We utilized a pre-trained DCNN Places365 CNN,¹⁶ so that the deep features from the CNN may be more representative of the built environment. One advantage of this approach is that pre-defined relevant features in the built environment are not required. The 4096-dimensional deep features embed all essential information of the built environment in the imagery so that we could retain relevant features as much as reasonably possible. Conversely, the disadvantage of using deep features from a pre-trained CNN is that it becomes difficult to identify corresponding physical features that impact CHD at the neighbourhood level. To alleviate this issue and provide certain interpretations of the deep features, we utilized Grad-CAM techniques to visualize the CHD-related features with a saliency map.

The results of multilevel modelling using demographics and socio-economic factors indicate that DSE variables were still better predictors of CHD prevalence than GSV features. One explanation is obviously the fact that physical environmental feature even if they represent a ‘meta’ framework for other mediators may not be sufficient to convey the risk conveyed by other factors which may be sparsely represented. However, by incorporating GSV features into the model with regular DSE variables, one could help improve the prediction of CHD prevalence at the neighbourhood level (Table 1). Further comparisons between GSV features and existing SDoH indices reveal that GSV features can be on par with these established SDoH indices. Given the fact that these existing composite indices encompass a wild variety of social economic and environmental factors, our single-source GSV features offer a valuable and efficient perspective on the built environment’s potential impact on health outcomes. Our results further suggest that GSV features indeed may be helpful in highlighting specific built environment information related to CHD prevalence at the neighbourhood level as illustrated by Grad-CAM methods, which provided a potential way of identifying built environment information.

Grad-CAM highlighted several potential built environment features that are either associated with higher or lower CHD at the neighbourhood level. Deteriorated houses and roads are a feature of urban blight associated with higher CHD. This feature may in turn embody other features in the neighbourhood that drive cardiovascular risk, including lack of space for physical activity,^7,29 limited access to nutritionally balanced food,³⁰ and lack of access to health care.³¹ Street greenery on the other hand was highlighted as associated with lower CHD prevalence. This agrees with previous studies that showed a robust association between green space and decreased cardiovascular risks.^32,33 It should be noted that some Grad-CAM results on a feature revealed different representations of built environment features. For instance, the feature that highlighted deteriorated houses also highlighted wooden utility poles in the images (see Supplementary data online, Figure S4). Other features may show combined physical features, an exemplified in Supplementary data online, Figure S8, where it seemed to show an amalgamation of tree canopy and nearby sky. This analysis identifies potential environmental features correlated with CHD, but it is crucial to note that these correlations do not establish causality. It is possible that underlying factors such as socio-economic status, which might influence living conditions and health behaviours, play a significant role in these observed correlations.

Implications

Our study carries significant implications for the field of health research clinical practice. Firstly, we have pioneered the utilization of street view features in assessing cardiovascular risk, marking a novel approach that introduces new dimensions to our comprehension of the impact of the built environment on health. Furthermore, our research, while primarily conducted at the census tract level, holds the potential for analysis at an even more granular level, down to the level of individual patients. This extended analysis could yield highly precise and personalized insights into the intricate interplay between the built environment and individual health, thereby enabling tailored interventions and healthcare strategies. As we look ahead, future research in this field may explore advanced methods such as semantic segmentation to extract specific built environment features, including elements like green space, blue space, and sidewalks. This approach promises a more intricate understanding of how the environment influences cardiovascular health, offering an avenue for even more finely targeted interventions in public health.

Our findings also underscore the utility of such data in broadening our understanding of environmental influences on health. The associations identified between street view features and cardiovascular risk not only open avenues for generating new hypotheses but also serve as a valuable reference for shaping public health policies. These insights, while emphasizing correlations, guide us in identifying potential areas for intervention and in designing studies aimed at exploring causal relationships. This approach, therefore, contributes to a more informed and targeted strategy in public health planning, with a focus on mitigating cardiovascular risks associated with specific environmental factors.

Study limitations

There are multiple limitations of this study that should be noted. Firstly, the GSV images used in the study are only available along major streets and roads, and there are some populations who do not live in such neighbourhoods. However, given the fact that most population live around the urban neighbourhood where GSV are abundant, we believe this would not significantly affect the results for majority of census tracts. Further, although Places365 database contains 400+ unique scene categories, it may not include all features that can be found in the built environment. Small objects such as trash, other environmental pollutants, and physical domains that may translate into better urban quality of life may be difficult for computer vision techniques like CNN to detect in a GSV image.³⁴ It's important to note that interpreting Grad-CAM maps can involve a degree of speculation, and the visualizations may not always provide a definitive understanding of the exact features being detected by the model. Additionally, the census tracts with CHD prevalence data are from seven representative US cities of CDC PLACES data set and may not generalize to all census tracts in the USA, especially rural areas.³⁵ Future work is needed to examine the disparities of urban and rural areas and its cardiovascular-related built environment features.

Conclusion

Built environment impacts cardiovascular health outcome. In this study, we used GSV and a scene-pre-trained CNN to assess the built environment. We found CNN-extracted features explain significant portion of CHD prevalence at the census tract level. Compared with traditional DSE factors or composite indices for SDoH, GSV provides unique information that may relate to CHD such as buildings, greenspace, and roads as suggested by the activation maps from Grad-CAM technique. The outcomes of our study provide proof of concept for machine vision–enabled identification of urban network features associated with risk that in principle may enable rapid identification and targeting interventions in at-risk neighbourhoods to reduce cardiovascular burden.

Supplementary data

Supplementary data are available at European Heart Journal online.

Declarations

Disclosure of Interest

All authors declare no disclosure of interest for this contribution.

Data Availability

The data used in this analysis are publicly available. The analytic code can be made available upon request.

Funding

This work was funded by the National Institute on Minority Health and Health Disparities Award nos. P50MD017351 and 1R35ES031702-01 awarded to S.R.

Ethical Approval

Ethical approval was not required.

Pre-registered Clinical Trial Number

None supplied.

Author notes

Zhuo Chen and Jean-Eudes Dazard contributed equally to the study.

Sadeer Al-Kindi and Sanjay Rajagopalan Co-senior authors.