Some consider oil and gas exploration and extraction to be the original “Big Data” industry.
In the early days, wells were sited near surface “seeps” where oil or gas naturally bubbled to the surface, indicating significant quantities near ground level. As demand increased and the most accessible sources were tapped out, oil exploration became more sophisticated.
Intensive geological analysis came into play, with tectonic and exploratory data assembled and combed through for clues as to the composition of underlying rock strata. With the cost of test wells sometimes exceeding $1 million, mistakes are expensive, and a few too many could be enough to tank an oil and gas exploration outfit.
With pockets of oil becoming more and more scarce, new techniques like extracting petroleum products from oil shale or tar sands have put even more pressure on data analysts to locate viable fields and calculate recoverable products. A few decimal points in such calculations can make or break a company operating at tight margins in an untapped field.
With so much at stake, exploration companies are looking to hire only the most talented data scientists. But competition for master’s-prepared data scientists is fierce, and the energy sector is scrambling to keep up with the likes of Google and Facebook in recruiting top talent.
Big Data is a Big Player in Big Energy
The U.S. Commerce department tells the story of energy use in the United States:
- Energy is the third-largest industry in the U.S. today
- The U.S. is forecast to become the world’s top oil producer and is on track to become energy-independent in the next two decades
- The U.S. has the world’s largest estimated recoverable reserves of coal
- By 2030, renewable energy will comprise almost 30 percent of U.S. generating capacity
Both nationally and globally, the companies behind the production and generation of all that energy are growing desperate for talented data scientists educated at the master’s level.
Ironically, the answer to this recruiting challenge might also lie in the hands of data scientists. According to a Deloitte Center for Energy Solutions paper, “Oil and Gas Talent Management Powered by Analytics: Adopting Analytics to Effectively Manage Workforce Needs,” advanced data analytics might also be the best chance for the industry to recruit and retain key workers in the face of increasing demand. By putting together labor force data, educational statistics, and demand forecasting, data science is helping oil and gas industry titans meet their hiring needs– and solve their exploration problems.
How Real-Time Metrics Enhance Well-Head Production
In the early days of the industry, a hillbilly could fire off a shotgun in a swamp and strike oil, but harvesting oil and gas today is an intensely high-tech affair. With billions of dollars on the line, immense pressures at play, and the potential for disaster constantly looming, real-time monitoring of wells has become standard.
Take Chevron’s i-field, or “intelligent field,” program, which is using advanced networked sensors at more than 40 drilling sites. Among them are some of the most productive oil fields on Earth. The program will provide an unprecedented level of real-time insight into everything from how drilling hardware is performing to reservoir levels, relying on algorithms to detect problems earlier than ever before possible. The program is expected to save as much as a $1 billion a year when it goes fully online. Chevron’s i-field won’t be completely operational until some time this year (2016), and there will be plenty of monitoring and fine-tuning left to be done even then.
Tiny, millimeter-sized sensors of the sort that Chevron is embedding in wells can dramatically increase safety and enhance production, but only if the data coming out of it is processed rapidly and interpreted correctly. Data scientists are foundational to designing, installing, monitoring, and interpreting sensors and data.
Companies use well monitoring for immediate operational needs like conducting predictive maintenance and maintaining production flows, but there is also a long-term component to collecting all that data. i-field, for example, extends past production data to help analysts look at business trends and improve efficiency. Over time, oil producers expect to use those datasets to help determine everything from optimal well-spacing to optimal production based on market demands.
Renewables Are The Future of The Energy Industry
The U.S. Department of Commerce forecasts that renewable sources will account for nearly 30 percent of U.S. generating capacity by 2030– a 420 percent increase from 2010.
Although the energy industry today is still dominated by petrochemical products, many of the most exciting opportunities in the field are on the cutting edge of green energy projects.
Unlike more portable energy sources, sunshine and wind are heavily dependent on climate. With the climate changing around us, and with enormously variable outcomes derived from relatively minor input conditions, the calculations behind locating the optimal sites for these projects are heavily data driven.
Around $700 billion will be invested in green energy projects in the coming decades. To make sure those investments pay off, data scientists will be hard at work assembling data sets on historic regional wind speeds, direction and intensity of sunshine, terrain models, and hundreds of other factors that impact the performance of renewable resources.
Analyzing Consumption is Key to Meeting the Energy Needs of the Future
Although most of the focus of the energy industry is on the production side, the fact is that the real gains of the future might come more from cutting energy consumption than increasing production.
This is the area where data science might have most to offer to the energy industry. Consumption patterns may only reveal opportunities for efficiency when aggregated over years of monitoring and millions of users. Everyone agrees that there is waste in energy consumption— it takes data analysis on a massive scale to identify where it is happening and how to reduce it. According to the National Institutes for Science and Technology, by 2030, smart grid data analysis could be generating savings of up to $2 trillion.
The savings are not theoretical. Even as the economy expanded year-over-year, according to the Energy Information Administration, energy consumption in the U.S. actually showed a year over year decline from 2014 to 2015, and again from 2015 to 2016.
The Emergence of Demand Management
Improvements in efficiency and waste reduction are happening on both the consumer and producer sides of the slate. Home technologies like Nest thermometers feed back into the analytics engine to shave unnecessary consumption at the individual level, reducing home heating and cooling bills. But on the utility side of the scale, the same data allows refined predictive modeling of peak demand periods, which allows power plants to operate on something closer to a “just-in-time” model, rather than being spun up proactively based on basic trend forecasting.
Cooperative efforts such as those being spearheaded by California’s AutoGrid may be the future of demand management. AutoGrid’s software allows utilities to coordinate with their customers to alleviate demand and balance loads on the fly. Automated controls tied to the software can then shut down non-critical loads at customer premises in order to improve generating efficiency at the utility.
How the 21st century plays out is largely dependant on our ability to solve the energy problem, and how well we balance divergent factors that run the gamut from cost to cleanliness. From the old guard of oil and gas exploration – which no doubt has decades left before it gives up the ghost – to the future of renewables, data science will play an increasingly important role in seeing to it the energy industry is able to keep pace with the world’s growing thirst for affordable energy while adhering to the logic of sustainability.