Data & Reproducibility

Public data sources, Git artifacts, and reproducibility information

Public Data Sources

NYC Open Data 311 Service Requests

https://data.cityofnewyork.us/

Over 30.9M geocoded service requests used as the primary signal for urban service demand

NOAA GHCN-Daily

https://www.noaa.gov/

Daily weather observations aggregated to weekly NTA-level features

OpenStreetMap Contributors

https://www.openstreetmap.org/

Points of interest data for urban context features

NYC Planning PLUTO/MapPLUTO

https://www.nyc.gov/planning/

Built environment features including parcel density, lot count, and land use

Git Repository Artifacts

Precomputed artifacts stored in the Git repository and used by this dashboard.

Paper & Figures4 files
  • paper_springer/main.pdf
  • paper_springer/figures/method_pipeline_overview_sample.svg
  • paper_springer/figures/shap_group_importance.png
  • paper_springer/figures/shap_summary_bar.png
Paper Tables (8 CSV files)8 files
  • paper_table_01_final_model_comparison.csv
  • paper_table_02_final_model_summary.csv
  • paper_table_03_final_model_confusion.csv
  • paper_table_04_category_performance.csv
  • paper_table_05_category_thresholds.csv
  • paper_table_06_hyperparameter_best_models.csv
  • paper_table_07_baseline_candidates.csv
  • paper_table_08_lift_at_k.csv
Ensemble Category Thresholds8 files
  • category_thresholds.csv
  • confusion_matrices.csv
  • final_model_comparison.csv
  • final_model_comparison_long.csv
  • metrics_by_category.csv
  • score_summary.csv
  • threshold_grid_validation.csv
  • threshold_selection_summary.csv
SHAP Explainability8 files
  • shap_group_importance.csv / .png
  • shap_global_importance.csv
  • shap_importance_by_complaint_category.csv
  • shap_local_cases.csv
  • shap_local_case_feature_contributions.csv
  • shap_beeswarm.png
  • 8 SHAP dependence plots
  • 18 SHAP local case plots

Excluded Large Files

Large raw files, the full processed modeling table, row-level intermediate files, and model binaries are not stored in this Git repository. These include:

  • final_nyc_urban_service_demand_dataset.csv.gz (173 MB, 1.35M rows, 254 columns)
  • scored_validation_test.csv.gz (row-level prediction scores)
  • LightGBM and XGBoost model .joblib binaries
  • Full raw 311, NOAA, OSM, PLUTO, and NTA data