The Need for Robust Data Management Strategies and Organizations in the Oil and Gas Industry
September 12, 2017
With the ongoing changes in both technology and regulation, data is becoming increasingly important for gas transmission, facilities, and distribution operations. Most operators have large quantities of data that drive engineering and field operations decisions residing in asset and work management systems. Pipeline location and characteristics, typically stored in Geographic Information Systems (GIS), feed risk engineering and integrity management programs. In addition to the large quantities of data already being managed by pipeline operators, the Pipeline and Hazardous Materials Safety Administration (PHMSA) has prescribed, through proposed rulemaking (“Safety of Gas Transmission and Gathering Pipelines” [1]), 45 separate data items required for threat identification and inspection programs. Furthermore, operators may be required to validate certain data to meet the requirement of being “traceable, verifiable, and complete.”

The combination of new regulations and business enablement systems require operators to develop and implement data management strategies that were previously not necessary. Gas operators have an additional challenge to develop processes to manage and integrate legacy data of unknown quality with newly generated data. The asset data is only valuable to operators if it can be effectively integrated into business operations, both in consumption and updates.

The objectives of a comprehensive data management strategy include:

  • Ensure that data in enterprise systems are accurate and complete per internal or regulatory requirements
  • Provide a change management process for data structure, process, and systems
  • Promote understanding and utilization of enterprise asset data over offline sources
  • Validate that data delivery systems are accurate and up to date

The combination of tools, processes, and organizations that meet the above objectives can include the following components:

  • Data Ownership and Maintenance
  • Data Delivery
  • Data Governance
  • Collaboration with other business units

Data Ownership and Maintenance

By definition, data ownership means single point accountability to ensure that enterprise system data are accurate and complete. Data owners typically approve data editing processes and prescribe data acceptance criteria for any groups who have responsibility to update, edit, and/or fix data in enterprise systems. For example, GIS mappers update enterprise GIS data based on as-built records using processes approved by the data owner. The data owner subsequently hold GIS mappers accountable for meeting specific accuracy and completeness criteria for all edits made to data.

Generally, the groups responsible to edit the data report to person(s) accountable for the accuracy and completeness of data. However, this is not always have to be the case. It is advisable to have a data quality control and quality assurance group that is separate but collaborative with data editing groups. Data quality control and quality assurance groups can constantly analyze and query enterprise data for systematic issues. They can be responsible for reporting continuous improvement metrics.

Data Delivery

Most operators want a single source of truth for asset information for all of gas operations. However, many operators struggle with multiple, disparate data systems that can contain conflicting information. Data is often maintained off-line by particular groups, which can create confusion on a company level. However, a single source of truth can be promoted on an enterprise level, to minimize the use of legacy and/or offline data sources. Proper implementation and change management is required to ensure adoption of the new data and data format.

A data delivery group can promote the use of enterprise data, deliver formatted data to internal or external requestors, and train users to be self-serving. These objectives can be vital for data intensive groups like Integrity Management (IM).

Data Governance

As operations groups start to use enterprise data, a lot of comments and feedback will arise. Some feedback will relate to data accuracy and/or completeness while others will be requests to change data formats or improve user experience within systems. A data governance process can be implemented to manage these change requests as all change requests require assessment of validity, impact, and priority.


It is valuable for the data owners to collaborate closely with organizations that are accountable for:

  • Data systems Operations and Maintenance
  • Records Management
  • Data intensive processes such as IM programs, risk assessments , and system operations

IT Operations & Maintenance (O&M) groups are generally accountable for addressing defects and enhancements to data systems such as GIS. It is most effective if data management groups and IT Operations & Maintenance (O&M) collaborate closely to provide the best designs, effective testing, and ultimate user experience.

A majority of enterprise asset data should be traceable to records that are in, or will soon be in, enterprise record management systems. Therefore, collaboration between data management and Records Management groups can reduce future issues with Traceable, Verifiable, and Complete (TVC).

Lastly, since data management groups, by definition, are a service and support organization, they are accountable to ensure that data intensive processes and organizations are well supported and have the data they need. Open lines of communication between such organizations (ex. IM) will promote faster data delivery and effective data improvements.

For questions or comments about data management strategies, please contact Dr. Wen Tu.


[1] PHMSA, "Docket No. PHMSA-2011-0023, Pipeline Safety: Safety of GAs Transmission and Gathering Pipelines," Department of Transportation, Washington, DC, 2016.