FINRA earned the top prize in two of seven categories at the second annual Cloudera 2014 Data Impact Awards, becoming the first organization to walk away with more than one award. FINRA swept the “Business Impact” and “Government, Nonprofit and NGO” categories at the Strata + Hadoop World conference in New York in October 2014. It was our first Data Impact nomination and recognized our use of Apache Hadoop in the cloud.
FINRA is engaged in technology platform redevelopments, shifting our market monitoring applications to modern big data platforms in the cloud. Moving our big data operations to the cloud yields faster more reliable analytics, more comprehensive management for data and reduced technology expenses while supporting future market growth. In a major upgrade to the Order Audit Trail System (OATS), we use Hadoop to link 30 billion events per day into a holistic picture of market activity. This vast volume of data is made available to users by the new FastOLA application, which uses Hadoop and Hbase to respond to user queries that formerly took hours and were reduced to mere minutes. Hadoop, along with Hive, is also the foundation of the large library of surveillance analysis patterns that analyze trade sequences and relationships over different time periods to identify instances of non-compliance or market manipulation.
“As a regulator, it is very important that people recognize we are doing cool stuff with technology,” said Brink Trammell, Program Manager of FINRA’s Market Regulation Cloud Initiative. “Going to work for FINRA is not like going to work for some Washington bureaucracy. We are one of the biggest data users in the country. These awards support a broader effort to increase the level of awareness outside of FINRA and outside of the financial services industry about the kind of technological challenges we are addressing.”
"As a regulator, it is very important that people recognize we are doing cool stuff with technology."
Redeveloping our market regulation systems on a big data/cloud platform is projected to bring a net cost benefit of $10 to $20 million annually. Using Amazon Web Services (AWS) and incorporating Cloudera and a host of open source technology were all keys to this cost reduction. “In the cloud not only do you get horizontal scalability, there are automated data management and infrastructure provisioning tools,” Trammell said. “You don’t have to take a forklift to the datacenter. There are software APIs and script managed tools that accomplish the same thing.”
Other Data Impact 2014 winners include MasterCard (Security & Compliance), Costco Wholesale Corporation (Pervasive Integration), Cloudwick (Community Contribution), Children’s Healthcare of Atlanta (Social Impact) and TIBCO Software (Advanced Analytics). The seven categories were judged on established, creative and results-oriented use cases for Cloudera’s Distribution Including Apache Hadoop (CDH). Winners were chosen by an independent group of 13 judges representing the enterprise data management industry. This year’s nominations doubled to 60, up from 30 last year. “In both years they represented a wide range of industries, company sizes and locations,” said Karina Babcock, Cloudera’s Senior Manager of Corporate Communications. Fifteen percent of this year’s nominations were from organizations outside of North America.
The date for this year’s Data Impact Awards has not yet been set. According to Babcock, Data Impact nominations will likely open up in May 2015 with the award presentations coinciding with the Strata + Hadoop World conference in New York in late September 2015.