Lauren wants to do more with data. Her current job requires her to focus on rig work (well workovers) and pumping schedules (artificial lift). Today the tools she has access to are limited to trends and alarms. Lauren has learned to use Excel, Spotfire and a few other reporting tools. She is not a strong programmer, but that doesn’t mean she doesn’t aspire to be a great analyst.
Lauren is a production engineer with a regional operator in the Rocky Mountains. She has about five years of experience in the industry, so she missed the data science craze that is going around universities right now. But she has a pretty good petroleum engineering background and now is in a position where she can make a difference in field/production optimization. A small group of younger production engineers at her company have similar interests. They have lots of ideas for operations improvement, but some barriers include getting access to data, few standards (so finding the right data for analysis is always a challenge), and lack of resources from the business intelligence team to help her out. Lauren feels that current algorithms she has for artificial lift optimization are very basic — she and her colleagues can do better internally. Her group needs a data prep middleman, not necessarily more sophisticated applications. But she needs more training on advanced tools.
Lauren is an Excel master. It is sometimes said that the best Visual Basic programmers in the oil industry are production engineers. Everyone uses Excel. Analysts also describe that “Excel Hell” that companies have unintentionally created for themselves. Inefficient workflows are often riddled with dueling spreadsheets and emails with Excel attachments instead of modern workflow automation tools. Coding and decoding these spreadsheets take a lot of time for Lauren, but it is the tool she has for all the field problems she faces.
Recently Lauren built a model with a lot of wells (in the thousands), a lot of data from the SCADA systems, and a lot of historical data to look at production performance and decline (three to five years’ worth). The model isn’t running too well and won’t accept new data. Lauren thinks that she may have broken Excel. Before we look at the challenge of life beyond Excel limits, let’s look at a little history of spreadsheets before Lauren’s time.
It all started with VisiCalc. In 1978, Harvard Business School student Dan Bricklin developed a program called VisiCalc. It was a relatively small program with few basic capabilities. It could only calculate data within a matrix of 5 columns by 20 rows. To make VisiCalc more powerful, Bricklin hired Bob Frankston, who is also known as the co-creator of VisiCalc. Frankston made the program fast and with better arithmetic. VisiCalc was an instant success, and the duo were able to sell around 1 million copies of the program.
And then came Lotus 1-2-3. After the phenomenal success of VisiCalc, a team headed by Mitch Kapor in 1983, developed a new spreadsheet program called Lotus 1-2-3. Mitch and his team power-packed Lotus 1-2-3 with charting, graphing and rudimentary database capabilities along with the basic arithmetic. This made Lotus 1-2-3 a new favorite in the industry. Although, before this, in 1982 Microsoft had already launched Multiplan but it was outshined by Lotus 1-2-3. And this thing provoked Microsoft to come up with Microsoft Excel, and the rest is history
If you aren’t the Excel wizard that Lauren is, here is what Wikipedia says about Excel: Microsoft Excel is a spreadsheet application developed by Microsoft for Windows, macOS, Android and iOS. It features calculation, graphing tools, pivot tables, and a macro programming language called Visual Basic for Applications. Microsoft released the first version of Excel for the Mac in 1985, and the first Windows version in November 1987. It has been a very widely applied spreadsheet for these platforms, especially since version 5 in 1993, and it has replaced Lotus 1-2-3 as the industry standard for spreadsheets. Excel forms part of the Microsoft Office365 suite of software. Version 15 of Excel was released in 2013. The latest version comes with Office 2016.
So, back to Lauren’s dilemma, can you break Excel? I looked it up, and found the following Excel limitations:
- Total number of rows and columns on a worksheet - 1,048,576 rows by 16,384 columns
- Column width - 255 characters
- Total number of characters that a cell can contain - 32,767 characters
- Hyperlinks in a worksheet - 66,530 hyperlinks
- Sheets in a workbook and a lot of other information are limited only by the memory of your computer
In theory one can build a relational data model in Excel (Powerpivot) that has over 20M rows and 200 columns of data that are used as the foundation of a model. Then you can have macros that grab data out of 100+ workbooks and process over 180M data points. In theory (again): the 32-bit environment is subject to 2 gigabytes (GB) of virtual address space, shared by Excel, the workbook, and add-ins that run in the same process. A data model’s share of the address space might run up to 500 – 700 megabytes (MB), but could be less if other data models and add-ins are loaded. But is this what you really want to do? Lauren is starting to think otherwise. Maybe there is a better way of solving her problem. I am sure that there are a lot of bright folks out there who have suggestions. Lauren would like to talk to you.
Consultants and tech evangelists are captivated by today’s and future need for Data Scientists. Can Lauren be one? Often, dubbed the ‘engineer of the future,’ this individual is said to lead the Digitalization revolution. Before you go on about emerging technology, Lauren wants you to slow down and think about the challenge from her perspective.
A torrent of data from the Industrial Internet of Things (IIoT) needs to be interpreted by experienced engineering and subject matter experts (SME – in our case Ayush). Even if you want to go with a chatbot or an autonomous agent where the decision-making entity is non-human, a human SME must have some oversight. The current issue with Boeing 737 MAX 8 aircraft is but one example where maybe the analytics got ahead of the subject matter expert (and the pilots).
Lauren lives in the world of the intersection of field automation and emerging digital tools. The data volume can overwhelm her Excel tool and the time it takes to find all the best data from a number of sources can drag her productivity down. Is anyone at home office listening? Lauren can use some help out there in the field. Lauren is on the front line for the effective use of analytics for production optimization. I think she is a stakeholder worth listening to.
