Natural Gas Infrastructure Firm Modernizes Records Management Processes for Future Growth and Compliance

Natural Gas Infrastructure Firm Modernizes Records Management Processes for Future Growth and Compliance

As a leader in the highly regulated energy industry, this client's Information Technology (IT) team was struggling with how to design and implement a M365 and Azure solution to centrally store documents, allowing advanced search and discovery of its operations files, from new projects to ongoing maintenance. With tens of millions of existing and new files, the team needed a repository that would consolidate content, integrating it with its line of business systems and its existing asset hierarchy. The primary challenge was scaling the solution to allow upwards of 100 million documents for the future enterprise-wide deployment, which involved handling vast amounts of data securely and ensuring seamless integration with existing systems and identifying multiple advanced migration scenarios to populate the system from existing technical and physical locations. This also included the planning for AI integration via automated metadata extraction and LLM integration with the documents in the repository.

A three-phased strategy

Following discovery, we determined the project would consist of multiple tracks of build and migration, using three different delivery models to successfully analyze, plan, and build the new operations document repository and migration of operations content to Microsoft 365 (M365).

Protiviti designed for M365 and Azure, including a structured Teams environment for governance, integration into their master data system, unified search, and a migration plan to populate the repository with the appropriate documents. The project tracks included:

Track 1 - Operations Teams Workspaces discovery and design

• Bringing Protiviti's UX/UI and M365 expertise to design modern, effective Teams Workspaces unique to the Operations department requirements.

Track 2 - Migration analysis and planning

• Discovery and analysis of existing operations content in multiple source repositories and recommending tool(s) and processes for migration including AI automated metadata extraction.

Track 3 - Operations repository implementation

• Implementation of the approved design of the operations repository in SharePoint Online and Azure including custom SPFx development and integration with line-of-business systems

Getting to work

We introduced a holistic approach that included strategic planning, technical expertise, and change management:

• Technical execution: We began with a thorough assessment of existing digital infrastructure to ensure compatibility and smooth integrations, developing custom scripts and tools to automate much of the data migration process and minimized errors during data transfer. We built an integration layer on Azure that sat between multiple source systems and SharePoint Online to funnel documents through and perform metadata management both manually and automatically via the new integration layer. This allowed us to handle the scale of uploading large batches of files to SharePoint. We also built a modern end user experience in SPFx on SharePoint for searching and sorting capabilities from such a large repository.
• Risk management: Understanding the risks associated with large-scale data build and migration, the team implemented robust security measures to protect sensitive information during and after the transfer. This included automated code deployment and strong security within Azure for the integration layer.
• Training and support: To address the migration's human aspect, testing and education programs were tailored for different user groups within the organization, delivered through webinars, face-to-face workshops, and detailed manuals.

Separate workstreams were established for each of the three project tracks.

For Track 1, we:

• Configured and implemented the approved design based on
• SPFx web parts for Operations Teams collaboration and discovery
• Created testing scripts for end users and admins​ and tracked issues and enhancements
• Finalized site map and navigation​ and established an organizational change management plan for launch

For Track 2, we:

• Performed detailed analysis in preparation for migrations​
• Established migration technical architecture ​across multiple migration servers using automated migration functionality through Azure pipelines
• Performed functional testing​ and production pilot migrations from source data to SharePoint Online

And, for Track 3, we:

• Developed a strong backlog and executed sprint planning​ based on the custom development plan for the new operations document repository
• Configured and implemented the approved design of the repository​ in Azure and SharePoint Online
• Created and executed end-user and admin testing
• ​Implemented the organizational change management plan​ and launched the new Repository to the operations team
• Maintained backlog for enhancement requests​ which now includes integration with new source systems for more automated document uploads

The solutions delivered ensure the client remains compliant in the highly regulated energy and utility safety industry, providing increased efficiency and productivity with greater accuracy.