Big Data Methods For Energy Systems

A Big Data methodology for the optimal selection of a subset

What you'll learn
Data Scientists
Software Engineers
Quants
Big Data professionals
Requirements
No pre-requisites
Description
What is the course about:Big data refers to data sets that are too large or complex to be dealt with - and this is true for electricity demand data particularly after the arrival of Smart Meters.  Smart Meters are electronic devices that record information such as the consumption of electricity, known also as electricity demand. Imagine that 5 different locations (e.g. houses) have smart meters and every minute these devices measure the consumption of the household. Then, the amount of data produced would be too large to really perform analysis on. When the data is too large usually a smaller dataset is selected.  And the question is then how large this dataset should be? If for example we have 1 million data points (measurements) then how much shall we select to perform some analysis? What is a "representative" dataset that we can select out of the entire dataset? In other words, we want to minimize the loss of information. We do not want to randomly select a dataset, but rather we want to select in such a way that the information loss is minimized. This course presents a representative Big Data methodology for application to electricity systems. This methodology can be transferred to other quantities (e.g. electricity generation) or other energy vectors (e.g. natural gas).Who:I am a research fellow and I lead industry projects related to energy investments using mathematical optimisation and data science. Specialized in the Data Science aspect of the Green Energy transition, focused on algorithmic design and optimisation methods, using economic principles. Doctor of Philosophy (PhD) in Analytics & Mathematical Optimization applied to Energy Investments, from Imperial College London , and Master of Engineering (M. Eng.) degree in Power System Analysis (Electricity) and Economics .Special Acknowledgements:To Himalaya Bir Shrestha, senior energy system analyst, who has been contributing to the development of Python scripts for this course and who regularly posts on medium. Important:No pre-requisites and no experience required.Every detail is explained, so that you won't have to search online, or guess. In the end you will feel confident in your knowledge and skills. We start from scratch, so that you do not need to have done any preparatory work in advance at all.  Just follow what is shown on screen, because we go slowly and understand everything in detail.

Overview

Section 1: A Big Data Method for Electricity to reduce size and retain key information

Lecture 1 Summary

Lecture 2 Description of the Methodology & Associated risks

Lecture 3 Structure of Big Data on Electricity Demand

Lecture 4 Step1: Calculation of seasonal statistics for the load factor using EXCEL

Lecture 5 Step 1 using Python

Lecture 6 Step2 - Calculation of daily average Load Factor time-series

Lecture 7 Step 2 - using Python

Lecture 8 Step3 - Finding the error of daily average from seasonal load factors

Lecture 9 Step3 using Python

Lecture 10 Step4 - Calculation of the minimum Load Factor Error

Lecture 11 Step4 using Python

Lecture 12 Step5 - Finding the day where the minimum error occurs

Lecture 13 Step5 using Python

Lecture 14 Step6: Calculation of the small dataset that has the typical days

Lecture 15 Step6 using Python

Enterpreneurs,Data Scientists,Investment Bankers,Consultants,Quants,Energy professionals,PhD students