Fight in Florida

The fight in Florida is the beginning of what will be a growing riff between utilities and advocates of solar energy. To frame the present situation in Florida, it is currently illegal in most of the state to sell solar to a third party. If you sold electricity to a third party, you would be considered a utility and subject to the rules of a utility. This becomes an issue when you look at the very popular business model for solar companies to install and own panels on a home and then sell the electricity to the residence. The homeowner doesn’t have to stake the upfront capital costs, but can still reap the rewards of having solar. Utilities are against this because it propagates solar use which cuts into their revenues. While, there are ways around this issue, such as PWRStation’s program of leasing or renting portable units, it’s a huge blow to the proliferation of solar in Florida.

The Floridians for Solar Choice effort is trying to change this. It is an effort primarily backed by the Southern Alliance for Clean Energy and other environmental, clean energy, and social organizations. They have sponsored the Florida Right to Produce and Sell Solar Initiative Amendment. The Amendment limits the ability of the government or utilities to impose barriers on the sale of solar electricity to customers at the same or contiguous site as long as it’s under 2 MW per day. This would not only allow solar companies to own solar installations on people’s homes, but allow for other forms of solar financial models such as community cooperatives.

The opposition to this is the Consumers for Smart Solar backed by the utility companies. They have counteracted by proposing a bill of their own. It is called the Florida Right to Solar Energy Choice Amendment. It states that it ensures the right to produce solar for your own use. This you can already do, so in essence, it changes nothing. You will still not be able to sell solar in Florida. The more important part of this campaign is to confuse voters in an effort to torpedo the effort by Floridians for Solar Choice. It is meant to protect utilities from the spread of solar systems.

                The latest on this is that the Floridians for Solar Choice effort will most likely fall short of the 639,149 signatures required by Feb. 1, 2015 to get on the November 2016 ballot. As of Dec. 25, 2015, Floridians for Solar Choice has 271,000 signatures. The signatures are good for 24 months, so they can continue the effort for the 2018 ballot. Moreover, what’s more troubling is the big money lobbying and disinformation campaign put on by the utility industry. $5.9 million was raised by Consumers for Smart Solar (the utilities) versus $1.9 million by the Floridians for Solar Choice effort. The utilities were clearly successful in their confusion campaign creating enough mix-up to derail the original effort.

          The fight in Florida perfectly exemplifies the issues that solar and distributed Energy in general will face. A new energy structure needs to be molded that takes into account solar and renewable’s unique characteristics. Solar's unique financial business models need to be allowable and utilities need to be properly compensated. There are lots factors to consider which will play out in very opaque ways, as exemplified by the fight in Florida.

               

Sources:

• https://ballotpedia.org/Florida_Right_to_Produce_and_Sell_Solar_Energy_Initiative_%282016%29
•  http://www.flsolarchoice.org/
• https://ballotpedia.org/Florida_Right_to_Solar_Energy_Choice_Amendment_%282016%29
• https://smartsolarfl.org/
• http://www.pwrstation.com/
• http://www.miamiherald.com/news/state/florida/article33164295.html
• http://www.utilitydive.com/news/second-ballot-initiative-emerges-to-challenge-floridas-solar-choice-measur/402439/

Home Energy Data: Breaking down Components and Cyclicality

    We're going to try a different way of trying to predict the Energy Production of a home solar panel. The technique I want to look at uses the seasonal_decompose function in Statsmodel python package.
    However, to start it will help to see how the different houses compare across the months. I graphed each house's energy production using the same x-axis of dates. The median is shown as the thick, red line.

    As you can see, every house seems to have a similar change in each month at its own level. However, it's tough to get a trend out of this that would be useful in predicting future months.
    Let's attempt to break it down, though. We can use StatModel's seasonal_decompose function to try and get a trend, seasonal, and residual component. Passing the median timeseries into the function is necessary because we can only give it one time series. The median will be less influenced by outliers than the mean because it only takes the middle value and not the weighted average. Although, in this dataset, it doesn't seem to be much of an issue because of the lack of large outliers. Put simply, the seasonal_decomposition function uses a convolution filter to take out the trend. A convolution filter is a type of weighted average which not only looks at previous values, but also subsequent. After taking out the trend, it finds the seasonal pattern. This is essentially the averages for the particular periods. For example, the averages of all the Julys, the averages of all the Augusts, etc... What's then left is the residual. The last component of what would make up the data.

   The reason there is trend information lacking for 6 months on the ends is that the convolution filter uses an average of 6 months before and after the position. It runs out of data on either side. The useful information that pops out of these plots is the cyclical pattern of the seasonal information. EnergyProduction seems to be dependent on the seasons. This is similar to what we say when we broke the data down by months. Let's see if we can make use of this.
     I combined all the trend and seasonality components into one model. This model contains a sine function for cyclicality and a linear component for overall trend. I used the least square function from scipy to optimmize the parameters in the model function. This function requires initial guesses for which I used: constant = mean, linear component's slope = slope of linearly fitted line, amplitude = 3 * standard deviation / sqrt(2),  phase = pi/6. These seemed to be the most appropriate guesses that resulted in the best parameter estimates by the least square optimization.

The red line of the model shows a pretty good fit to the data, visually. A sinuisoidal component with a slight upward trend. Let's see how it does on test data in predicting new data.

Very similar looking to the training data. However, this is not surprising since even the test data has 500 data points. The MAPE score comes out to be 19.1. This is a quite a bit worse than our previous best prediction of 12.48. most likely due to the fact that we are only predicting using the time component and not the additional factors of Temperature and Daylight that the previous model included. Still, makes for a good exercise for manipulating timeseries, breaking down components of the data, and fitting cyclical data.


References:
 -GitHub Code: https://github.com/262globe/Blog.git
 -http://stackoverflow.com/questions/26470570/seasonal-decomposition-of-time-series-by-loess-with-python
-https://searchcode.com/codesearch/view/86129185/
-http://statsmodels.sourceforge.net/devel/generated/statsmodels.tsa.filters.filtertools.convolution_filter.html
-http://www.cs.cornell.edu/courses/cs1114/2013sp/sections/s06_convolution.pdf
-http://stackoverflow.com/questions/16716302/how-do-i-fit-a-sine-curve-to-my-data-with-pylab-and-numpy

Home Energy: Linear Regression

     I'm going to start out by trying out some linear regression. The hope is that this will allow us to input new data and output the Energy Production. Should be straightforward. The first model regresses Energy Production on Temperature, the second regresses Energy Production on Daylight, and the third regresses Energy Production on both Temperature and Daylight. We'll start with these and see what we get.
     Energy Production on Temperature gives us a model which is not very good. The good news is that the p value is very low at 2.32e-195. The means the model is a good predictor. Technically, it tells us to reject the null hypothesis that the coefficient is 0. However, R Squared is very low at .07, meaning a high variance in the prediction. This is bad. It doesn't seem to be very useful because of this.

The Daylight vs Energy Production seems to be slightly better. It also has a very low p value (0), but a slightly higher R Squared (.28). This means that more of the variance can be explained by the model.

So, what happens when we combine both Temperature and Daylight? We get a p value of 0.0, once again. But a marginally improved R-squared value of .37 as compared to the .28 for the Daylight data.
 

	coef	std	err	t	P>|t|	95% Conf. Int.
-------------------------------------------------------------------------------
Intercept	40.3892	7.144	5.654	0.000	26.386	54.392
Temperature	5.0511	0.124	40.856	0.000	4.809	5.293
Daylight	2.6425	0.036	74.090	0.000	2.573	2.712

   The next thing I tried was to break the data down by month before doing the regression. The hope being that we could better predict EnergyProduction when we take into account what month it is. So, each month has a separate linear model that it uses. Only Daylight is used to predict. The plot shows different color dots and regression lines for each month. The p values are once again very low. The R Squared values vary to a good degree month to month from a low of .004 in September to .46 in August. This seems to suggest that there could be a large spread in what our model predicts in certain months, while being a much tighter error in other months.

How good the model succeeds will be based on how well it does on new data. The measurement we will use with this will be the MAPE (Mean Absolute Percentage Error). I already broke the data down into training and test. The models were created with the training data and will be measured with the test data. Let's compare the regression model with Temperature and Daylight vs. the model breaking it down into months.

Model   MAPE Score Temperature and Daylight Regression    15.06 Daylight Regression by Month     12.88

And just for fun, I expanded the month by month regression to include both Temperature and Daylight. This gave a very small improvement.

Model	MAPE Score
Temperature and Daylight by Month	12.48

Creating a separate model for each month that includes both the Temperature and Daylight factors gives the best predictions. Although, it's only a marginal improvement on the Daylight model for each month, I'll stick with it, because it is the best and the cost to implement is low enough. Let's see if we can improve on this score on future analysis!

Home Energy: Testing and Measuring of Prediction Models

Before we continue, it is important to clarify a couple prediction model concepts. This will allow us to test our models.

This is a side post on how we will evaluate the performance of the models.

We will break the data down into 2 sets. The first set is what will be used to train our model and give it the parameters. This will include the first 80% of the entries. Think of this as teaching the model. The 2nd set will test the model to see how well it performs on new data. This will subsist of the last 20% of the data.

     The performance of the models will be compared using Mean Absolute Percentage Error (MAPE). MAPE is calculated by taking the mean of the standardized absolute errors and turning it to a percentage.

http://support.minitab.com/en-us/minitab/17/png/measures_of_accuracy.dita_dctm_Chron0900045780196e20_0.png

     This will give us a standardized measurement with which to compare two models. This score will be calculated on the new test data. MAPE percentage error is an intuitive way of understanding the error statistic. As Minitab's online resource explains, a MAPE score of 5 means that the forecast is off by 5% on average. Other options to measure the error of models would have been R-Squared, Mean Absolute Deviation, or Mean Squared Deviation. I won't go into the details of these except to explain that R-Squared is the percentage of the response variable variation explained by the model on a scale of 0%-100%. In other words, it is a deviation measure between the model and actual values over a deviation measure between the mean and actual values.
     In summary, in order to improve something we have to measure it. So, we want to make sure we have a system in place to systematically measure and compare our models.

Home Energy: Data Exploration

         My first project is to predict energy production of a solar panel. I got my hands on an energy dataset that compares temperature and daylight to energy production (I'm assuming this is energy production of solar panels) of houses. It is monthly frequency over two years. This is from Cambridge Energy Lab's Challenge. The ultimate goal is to be able to predict Energy Production from the data on an entirely new house. To start off, we're going to do some data exploration to get a better sense of the data.

         The data itself looks like as follows:

There are 12,000 rows. Let's go through the columns -

Label: 0-22 index repeated for each house

House: 500 houses represented

Year/Month: 2011/7-2013/5

Temperature: Degrees Celsius

Statistic	Value
Count	11,500
Mean	14.37
Stdev	8.49
Minimum	0.8
25th Percentile	5.3
50th Percentile	13.2
75th Percentile	22.8
Maximum	29

Daylight: How much daylight. Not sure of the units.

Statistic	Value
Count	11,500
Mean	189.12
Stdev	29.43
Minimum	133.7
25th Percentile	169.1
50th Percentile	181.8
75th Percentile	205.2
Maximum	271.3

EnergyProduction: (kWh)

Statistic	Value
Count	11,500
Mean	612.75
Stdev	142
Minimum	254
25th Percentile	509
50th Percentile	592
75th Percentile	698
Maximum	1254

These preliminary statistiscs give us a nice idea of what the magnitude and range of the data is, but quite frankly, nothing we couldn't have gotten from a quick browse through of the raw data.

And to get a preliminary scatter plot with Temperature vs. Energy Production:

There's is not a very clear linear relationship here. There might be some clustering to be done as there seems to be diferent groups the data could be put into.

And graphing the Daylight variable:

The Daylight dependent scatterplot seems to show data that is ripe for linear regression. This is something that should be investigated.

An important part of the data that I originally overlooked are the months. Graphing the Energy Production month by month over a year, should give a fair amount of information. This is because weather is very dependent on what month it is, and Energy Production is very dependent on weather. In theory.

There seems to be some correlation month to month. I'm not sure how able a month would be able to tell us what Energy Production to expect, but the better information might be what type of change to expect from the previous month. I'll have to think about this and see what can be done.

    In summary, this post gives a cursory information of what the data is we are working with. The key will be in what steps we take next. There is a great number of different analysis we could perform from linear regression relating Temperature or Daylight to EnergyProduction, to K Means clustering to group the houses in an effort to try and differentiate them. We'll have to be smart about the next steps, but in the end need to try something and see how it works.

Introduction

This blog is meant to assist me in learning, to educate others, and to demonstrate what knowledge I've gained. I hope to explore the areas of data analysis and its application to renewable energy. Data analysis will include the application of different statistics techniques in the python programming language. I have a background in statistics and have great familiarity with R from a previous job, but would like to apply python to in practical problems. The world of renewable energy is ripe with opportunity to analyze data. It is a field I'm personally interested in that gets me excited becuase of the great opportunity and potential to exct positive change. I hope you might learn a little something. I know that I surely will!
~Andrew Bray