Solarx: Solar Panel Segmentation and Classification
Solarx: Solar Panel Segmentation and Classification
5.1.2 Optimizer
4.2. Preprocessing
The data set required a significant amount of preprocess-
ing to get it in an acceptable form to pass into our U-Net
(see Appendix Figure 4). Initially, we passed in the im-
ages as well as their corresponding PV coordinates into a
mask generator that created a 5000x5000 boolean numpy
array mask with 0’s where solar panels were not present
and 1’s where they were present for each image (figure 3b). Next we experimented with optimizers (see table 2).
The images were also converted to 3x5000x5000 arrays. To RMSProp preformed significantly worse than Nadam and
generate more data and to allow our model to train more Adam. We hypothesize that this is because RMSProp
doesn’t account for momentum of the weights and instead 5.2. PV Segmentation
just decays the update. Nadam and Adam both incorporate
5.2.1 Loss Function
momentum. We opted to choose Adam since the ROC-AUC
score outperformed Nadam. Nadam is a slightly more opti-
mized version of adam that works by looking ahead at the
next x to preform our current update. The update rule for
Nadam can be seen in figure 5.
5.2.3 Hyperparameters
5.3. Visualization
6. Conclusion
In this paper, we present a solar panel segmentation model
that works to classify and segment solar PV’s in a given im-
age. The model divides the training portion into two phases:
a pre-trained Resnet34 model for classification and a U-net
model for segmentation. We experiment with multiple clas-
sification architectures, such as different forms of Resnet.
In addition, we experiment with other variables such as the
segmentation batch size, the optimizer, and segmentation
loss function. Our model produced promising results and if
we compare it with the likes of DeepSolar as a benchmark,
our model is able to classify solar PV’s more effectively. hyperparameters, loss functions, and architectures. He did
the majority of training for the models. Additionally, Ethan
6.1. Future Work Hellman was chiefly responsible for the sourcing of data.
While our methodology seeks to converge on an optimal This includes the final dataset used in this study. He also
model for solar panel segmentation from satellite imagery, supplied coffee for everyone.
there are a few areas which could benefit from further
exploration. Given that we choose to use AUC-ROC as our Rodrigo Nieto was responsible for the election of the PV
standardized evaluation metric for different classification segmentation and selection model with Spencer Paul. In ad-
models, we deemed an Adam optimizer to be better than a dition, Rodrigo Nieto researched and procured the literature
NAdam optimizer. That being said, if instead, the F1 score and state-of-the-art models of current solar PV segmenta-
was used as a standardized evaluation metric, a NAdam tion projects to be able identify promising techniques, archi-
optimizer could be considered more optimal. As such, in tectures, and to build a better understanding of our bench-
the future, further exploration should be done to evaluate mark. He was also involved in the data pre-processing por-
the performance of a classifier using a NAdam optimizer tion of the project.
with the various other hyperparameters that were tuned. By
the same logic, a different architecture may have proven 8. Appendix
more optimal as well. Table 1. shows how a Resnet50
may have outperformed other Resnet architectures if the
evaluation were the F1 score. Therefore, further tests
should be done to evaluate this architecture compared to a
Resnet34 with optimally tuned parameters.
7. Contributions
This research would not be possible without code provided
from fastai for initial models as well as gabrieltseng for data
preprocessing (see refrences).
We also could not have done this work without the follow-
ing python libraries (numpy, pytorch, sklearn, matplotlib,
PIL). [23] [24] [25] [26] [27] References
[1] Zemel R. S. He, X. and Carreira-Perpiñán. Multiscale con-
ditional random fields for image labeling. IEEE, 2004. 2
Spencer Paul was responsible for initial research into
[2] Winn J. Rother C. Shotton, J. and A. Criminisi. Textonboost
PV segmentation and selection of the model architectures
for image understanding: multi-class object recognition and
alongside Rodrigo Nieto. He also was responsible for writ- segmentation by jointly modeling texture, layout, and con-
ing the abstract, methods and results section of the paper. text. Int. J. Comput, 2009. 2
Additionally, he wrote the evaluation and visualization
code for graphics in results section. He helped with training [3] Zhang X. Ren S. He, K. and J. Sun. Deep residual learning
for image recognition. IEEE, 2016. 2
and experimentation although the majority was done by
Ethan Hellman. [4] Shelhamer E. Long, J. and T. Darrell. Fully convolutional
networks for semantic segmentation. IEEE, 2015. 2
Ethan Hellman was responsible for model experimentation [5] Fischer P. Ronneberger, O. and T. Brox. U-net: convolu-
and designing the research process so as to evaluate tional networks for biomedical image segmentation. Medical
image computing and computer-assisted intervention (MIC- [23] Charles R. Harris, K. Jarrod Millman, Stéfan J. van der Walt,
CAI), 2015. 2 Ralf Gommers, Pauli Virtanen, David Cournapeau, Eric
Wieser, Julian Taylor, Sebastian Berg, Nathaniel J. Smith,
[6] Leslie M. Collins Jordan M. Malof, Kyle Bradbury and
Robert Kern, Matti Picus, Stephan Hoyer, Marten H. van
Richard G. Newell. Automatic detection of solar photo-
Kerkwijk, Matthew Brett, Allan Haldane, Jaime Fernández
voltaic arrays in high resolution aerial imagery. Applied En-
del Rı́o, Mark Wiebe, Pearu Peterson, Pierre Gérard-
ergy, 2016. 2
Marchant, Kevin Sheppard, Tyler Reddy, Warren Weckesser,
[7] K. Bradbury J. M. Malof, L. M. Collins and R. G. Newell. A Hameer Abbasi, Christoph Gohlke, and Travis E. Oliphant.
deep convolu-tional neural network and a random forest clas- Array programming with NumPy. Nature, 585(7825):357–
sifier for solar photovoltaic arraydetection in aerial imagery. 362, September 2020. 8
IEEE, 2016. 2
[24] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer,
[8] Arun Majumdar Jiafan Yu, Zhecheng Wang and Ram Ra- James Bradbury, Gregory Chanan, Trevor Killeen, Zem-
jagopal. Deepsolar:a machine learning framework to effi- ing Lin, Natalia Gimelshein, Luca Antiga, Alban Desmai-
ciently construct a solar deploymentdatabase in the united son, Andreas Kopf, Edward Yang, Zachary DeVito, Mar-
states. Joule, 2018. 2, 5 tin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit
[9] O. A. Omitaomu J. Yuan, H. L. Yang and B. L. Bhaduri. Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch:
Large-scalesolar panel mapping from aerial images using An imperative style, high-performance deep learning library.
deep convolutional networks. IEEE, 2016. 2 In Advances in Neural Information Processing Systems 32,
pages 8024–8035. Curran Associates, Inc., 2019. 8
[10] Li Zhuang, Zijun Zhuang, and Long Wang. The automatic
[25] Fabian Pedregosa, Gaël Varoquaux, Alexandre Gramfort,
segmentation of residential solar panels based on satellite
Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu
images: A cross learning driven u-net method. ELSEVIER,
Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg,
2020. 2
et al. Scikit-learn: Machine learning in python. Journal of
[11] Fastai python library. “https : / / github . com / machine learning research, 12(Oct):2825–2830, 2011. 8
fastai/fastai”, 2022. 2
[26] John D Hunter. Matplotlib: A 2d graphics environment.
[12] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Computing in science & engineering, 9(3):90–95, 2007. 8
Deep residual learning for image recognition, 2015. 2 [27] P Umesh. Image processing in python. CSI Communica-
[13] Vincent Feng. An overview of resnet and its variants. To- tions, 23, 2012. 8
wards Data Science, 2017. 2
[14] Diederik P. Kingma and Jimmy Ba. Adam: A method for
stochastic optimization, 2014. 3
[15] Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net:
Convolutional networks for biomedical image segmentation,
2015. 4
[16] Shruti Jadon. A survey of loss functions for semantic seg-
mentation. IEEE TPAMI, pages 1–6. 4, 6
[17] Kyle Bradbury. Distributed solar photovoltaic array location
and extent data set for remote sensing object identification.
Sci Data, 2016. 5
[18] “https://github.com/gabrieltseng/solar-
panel-segmentation”, 2019. 5
[19] Kaiqi Huang Kongming Liang Yizhou Yu Huikai Wu,
Junge Zhang. Fastfcn: Rethinking dilated convolution in the
backbone for semantic segmentation. 2019. 8
[20] Varun Jampani Sanja Fidler Towaki Takikawa, David Acuna.
Gated-scnn: Gated shape cnns for semantic segmentation.
2019. 8
[21] Iasonas Kokkinos Kevin Murphy Alan L. Yuille Liang-
Chieh Chen, George Papandreou. Deeplab: Semantic image
segmentation with deep convolutional nets, atrous convolu-
tion, and fully connected crfs. 2017. 8
[22] Piotr Dollár Ross Girshick Kaiming He, Georgia Gkioxari.
Mask r-cnn. 2017. 8