How FactorPrism® analyzed 50+ million records to uncover hidden patterns in New York City's service requests—in under an hour.
New York City's 311 service handles millions of non-emergency calls annually, from noise complaints to pothole reports. With over 50 million records spanning multiple years, the NYC Open Data initiative invited the community to help discover patterns in this vast dataset.
Traditional analysis would require teams of analysts spending months to identify meaningful trends. Even then, subtle patterns and interaction effects would likely remain hidden. We decided to demonstrate how FactorPrism® could surface these insights automatically.
Using FactorPrism®'s advanced algorithms, we analyzed the period from September 2013 to March 2017—a timeframe showing clear growth trends. Our goal was to understand whether this growth was uniform across all service types or driven by specific hidden factors.
While 311 usage grew 11% overall during the period, this headline number obscured crucial patterns:
FactorPrism® isolated a dramatic spike in pothole complaints in 2014 that was distinct from general street condition issues:
Pothole complaints surged to 3x normal levels in 2014. This spike was independent of other street condition complaints—a pattern that would have been missed by looking at aggregate street complaints. This type of granular insight allows city planners to understand specific infrastructure failures rather than general trends.
Water system complaints showed fascinating seasonal patterns with anomalies:
What makes these findings remarkable isn't just their value—it's how they were discovered.
FactorPrism®'s algorithms excel at this type of analysis because they separate overlapping effects (the 25% decline in housing complaints was invisible in aggregate data), identify pure signals (the 2014 pothole spike was isolated from general street trends), and detect anomalies in patterns (the mild winter 2016 water system spike stood out against historical patterns).
If FactorPrism® can find these needles in NYC's 50-million-record haystack, imagine what it can uncover in your data:
Separate true product performance from category trends, seasonal effects, and marketing impacts
Identify which customer segments are actually churning while overall growth looks healthy
Detect when normal patterns break—indicating either problems to fix or improvements to replicate
Don't let critical insights stay buried in your data. What would have taken NYC months to discover, FactorPrism® found in under an hour.
Get it on Snowflake Marketplace