Commerce Department Builds Broad Support for Data Analytics

Published: July 13, 2016

Big DataCloud ComputingDOCInformation TechnologyNatural Resources/EnvironmentShared Services

The Department of Commerce aims to strengthen its roles as “America's Data Agency” through a number of current efforts.

Reportedly collecting, processing, and disseminating approximately 36% of the federal open data, the Commerce Department is looking to drive cost efficiencies while improving capabilities and promoting public access to data. New approaches to funding, including joint venture arrangements, as well as a strategic view for future shared services potential are fueling the investment in data analysis products and services.

NOAA gathers over 20 terabytes of data every day, but only a small percentage of this data has historically been easily accessible to the public. In April 2015, the Commerce Department’s National Oceanic and Atmospheric Administration (NOAA) launched its Big Data Project. Leveraging Cooperative Research and Development Agreements (CRADAs), the effort established collaborative alliances with Amazon Web Services, Google Cloud Platform, IBM, Microsoft, and the Open Commons Consortium. The most recent addition to various previously available weather, water, ocean, and climate data is NOAA’s NEXRAD L2 data, which became available in June 2016. The effort to unleash NOAA’s data is also likely to spur investment in cloud computing to offer scalability and drive efficiencies for real-time data analysis. In fact, in early June 2016, the Commerce Department’s Economics and Statistics Administration (ESA) referenced cloud adoption as a critical step in facilitating powerful data initiatives like NOAA’s Big Data Project and the Commerce Data Service.

In November 2015, the Department of Commerce launched the Commerce Data Service, a start-up composed of a skilled cadre of designers, developers, software engineers and data scientists tasked with partnering with the twelve bureaus that comprise the Commerce Department to deliver products and services that will improve government agencies’ delivery of information to their customers. To date, the group reports a total of 14 projects: 

Commerce Data Service Projects

Status

Bureau

Initiative

Services

Objective

Completed

ESA

Commerce Data Advisory Council (CDAC)

General

Fueling Economic Growth

Census

Income Inequality

General

Fueling Economic Growth

DOC

Commerce Data Usability Project

UI/UX, Data Science, Back-End

Delivering Data Services

DOC

Data.Commerce.Gov

General

Fueling Economic Growth

DOC

White House Council of Women and Girls

Data Science, UX/UI

Delivering Data Services

DOC

Commerce Data Academy

General

Creating Data-Driven Government

In Progress

BEA

BEA RIMS III

UI/UX

Delivering Data Services

NIST

NetZero

Data Science, Back-End

Creating Data-Driven Government

NIST

MEP

Data Science, Back-End

Creating Data-Driven Government

PTO

Open Data Roadmap

General

Fueling Economic Growth

ITA

Principal Tracker

UI/UX, Back-End

Delivering Data Services

ITA

New Exporters Project

Data Science

Fueling Economic Growth

Discovery

DOC

OMG CAP Project

UI/UX, Data Science

Creating Data Driven Government

Pre-Discovery

NOAA

Data Labs

UI/UX, Back-End

Delivering Data Services

Source: Commerce Data Service

Yet another effort highlighting the departments focus on data capabilities comes from the National Technical Information Service. In June 2016, the NTIS issued a notice requesting proposals for a joint venture partnership to aid federal agencies in developing and implementing innovative ways to collect, analyze, or leverage federal data and data services. Unlike the General Service Administration’s 18F, which deals with IT services, the NTIS initiative is expected to focus solely on data. While NTIS already has joint venture partnerships, those will conclude with the completion of the associated projects. Proposal submissions are due August 1, 2016.

In addition to building application programming interfaces, the department is developing training opportunities for government employees and contractors. Initial offerings have drawn high levels of interest, with 3,500 people reportedly signed up for the first sessions. Federal employees follow the training with apprenticeship at the Commerce Data Service prior to returning to their own agencies. The emphasis on improving data access and usability means that usage is likely to be one metric of the success of these investments. Continuing to grow user demand will likely be key to ensuring that Commerce's focus on these data efforts doesn't waver.