This is my current plan on my current project and I have a lot of progress to make… It has been a path of a lot of roadblocks and months of research. This isn't a topic I’m an expert in by any manner and nor do I know any expert to consult so a lot of what I often conclude or plan doesn't go as I expect it to. I have spent countless hours researching, planning, coding, debugging, and learning new technologies for this project and this project itself could be broken down into 4 or 5 heavy projects.
I finished the first part of my program earlier this year. This part is what collects and stores information. I learnt SQL, SQLAlchemy, and some other python libraries in order to comfortably interface the database in python. I designed a mySQL server and wrote some scripts with SqlAlchemy in order to collect data every 15 mins and store it in the db. I also wrote some python scripts to easily query data by time. I have made the data collection a completely separate entity in which the data in the db cant be edited manually or corrupted by my program or any user accidentally meaning that it can work like a rest api if worked on a little more and can be used by other projects too. I wish to improve this in the future with the use of a time series database which would be a lot more efficient to store, and easy to store and query. I have been working on the prediction algorithm, forecasting, and anomaly detection algorithm for a while now. I have tried multiple different techniques, algorithms, and logics. Some of them have shown promise but none of them have given me the output I would like. In order to understand and write these algorithms I have spent countless hours learning pandas, tensorflow, keras, statistics, logic behind the algorithms, and some data science. I haven't had a formal education in statistics until this year(I am taking AP Stat in school this year) but there are many algorithms that I have tried to understand and haven't had any luck in doing so, because my understanding of the basics isn't strong. I hope this year I get a good foundation and understanding of statistics. The next section would be a way for the user to interface and access the data from my program. While I haven't written or planned this section yet I think this would probably be the easiest section for me. The first project I did for energize andover I struggled with flask and front end dev but since then I have worked on countless projects outside Energize and set a goal for myself to become proficient in Flask and gain a basic knowledge in front end dev. I have worked on multiple web applications since then and have designed some very high level flask applications making something like a login supported user interface very easy to write. During this project I have had to learn and understand some topics that aren't directly related to my project or mentioned above. In order to keep the data collection a separate entity I have had to get a good understanding of multithreading by language and by OS. I have even written some code in C, scripted in bash, and some js, all languages I am not an expert in but something I have picked up along the way. I have also had to learn to use cloud computing because my system isn't powerful enough to process and run many of the algorithms on a whim and I have gained a lot of knowledge on using the Google Cloud Platform architecture to host applications, use api’s, etc. I have also had to learn many topics that while I never came to apply in this project, I think that it has advanced me a lot in my overall growth as a student and a programmer. Working on such a project on a long term basis has also taught me a lot about patience, planning/strategizing, and prototyping. Hopefully in time I will understand everything better and come to an output that I like. I hope my long writeup hasn't lost you or put you to sleep… I have also been working on many other projects at hackathons and even got an opportunity to explore different fields of software in an internship I got over the summer. I have mentioned it all in my resume which I have attached in the email. If you have any questions or doubts please feel free to contact me at: Ayush Zenith https://ayushzenith.tech [email protected]
0 Comments
My current project has to do with making predictions of electricity usage in a building. The predictions could then be compared to real time data. Any anomalies presented between the prediction and real time data must indicate irregular use of electricity. Irregular use of electricity can indicate either a shift in schedule of electricity use or a potential source of electricity wastage. Buildings like our school often operate on a pretty consistent period. With higher electricity usage during the weekdays, specifically during school days and hours… Some classrooms have after school activities which also operate on a periodic schedule in certain parts of the building only on certain days and certain hours… If these periods could be studied and automated predictions could be made for electricity in advance using historical data which was collected every 15 mins for maybe 2-3 years, then we would be able to make accurate predictions accounting hourly, daily, weekly, monthly, and yearly periods… The below image from a study in an academic journal documenting “Real-time detection of anomalous power” (https://ayushzenith.tech/files/Renewable%20reviews%20-%20Real-time%20detection%20of%20anomalous%20power%20consumption.pdf) summarizes my current plan to tackle this issue quite well. For data acquisition we have sensors and a server communicating on the BACnet protocol through which I can receive data. I am collecting data every 15 minutes and storing it in a mySQL database(for now… In the near future might switch to a more time series effective database. I have done some research and my current method of storing time series data in a SQL db isn't very efficient but there are time series db’s which make storing and querying a lot easier.) This data is then queried by the ARIMA neural network. ARIMA is an acronym that stands for AutoRegressive Integrated Moving Average and is often used to understand or predict future points in time series data. I am currently in the process of writing my ARIMA neural net but there are multiple simple interpretations to writing ARIMA NN to predict electricity usage data(Some of these popular interpretation to which I am referring to include: https://towardsdatascience.com/predict-electricity-consumption-using-time-series-analysis-4650284e40aa which is an article focused on predicting electricity consumption using time series analysis and https://www.machinelearningplus.com/time-series/arima-model-time-series-forecasting-python/ which is just a complete guide to time series forecasting in python). I have looked at some other models too but the ARIMA model is the only I have so far been able to get any remotely accurate results with. While I am currently focused on the prediction model, once I figure it out I would need to work on anomaly detection. While I haven't done a lot of research on this yet I have currently decided to code this based on the plan highlighted by the same study from where I got the above image (https://ayushzenith.tech/files/Renewable%20reviews%20-%20Real-time%20detection%20of%20anomalous%20power%20consumption.pdf). The below image from the same study highlights the plan on finding anomalies between the prediction and real time data(I haven't gotten to this yet and this is just my plan and may change depending on the performance of my prediction model). If interested in more detail on the anomaly detection flowchart please refer to the study I picked it up from as it is explained in detail in the study.
Once I have the backend for the anomaly detection working I plan on writing a front end so that the respective people are able to identify which room/sensor is currently presenting with anomalous data should be taken a look at. The plan for the front end is something I picked up from an academic study on “Early-warning application for real-time detection of energy consumption anomalies in buildings” (https://ayushzenith.tech/files/Journal%20of%20cleaner%20production%20-%20Early-warning%20application%20for%20real-time%20detection%20of%20energy.pdf). I would possibly like to integrate the front end with Daniel’s dashboard which he has been working on. The below images are from the just mentioned study and summarize my plan for the front end quite well After that project for a while I wrote a program that generated weekly reports as to which rooms were the hottest, coldest, and used most energy. I tried to use different differentiators to highlight other data too. Ex - If there any outliers from the data (I would like the data to have a low standard deviation and remain close). I was working on making this project into a modular web application but I learnt that there were other students working on similar projects and I also started to gain interest in different topics and technologies thus I started my current main project.
My first application(https://github.com/Energize-Andover/BACnet-AHS-electric-gauge) was mostly me exploring and learning more about programming. It was a live gauge for monitoring electricity usage around any building using BACnet protocol to communicate electricity usage(such as our high school). The gauges could potentially be used to realize during what time of day is the most electricity used or sometimes to just seem as a visual aid to all the numbers. The application was a web app where I mostly learnt many of the basics of programming and explored possible ways to use data like electricity and water use. This was my first experience collaborating with someone on a programming project and I learnt a lot about working with others effectively too and how to use version control, delegating work efficiently, etc.
I finally found 2 logic errors thus it wasn't showing up in the IDE I have corrected them and tommorow I will know if it is working
Right now you can find the updated project at
https://github.com/Energize-Andover/BACnet-AHS-electric-gauge I have the kW, kWh, and dollar meter ready and working. My kWh Hourly is currently buggy and thus I havent written any documentation If anyone thinks they can help please fork it look at my code and put in a pull request Thank you I have documented my first version of the gauge
https://energize-andover.github.io/BACnet-AHS-electric-gauge/ I will soon update this as soon as I know that the new update I made to it works fine.(Testing takes 11 hours) _____________________________________________________________________ oI started working on the gauge and I think this seemed a lot more difficult than it is. It is still a in progress project but my goal was to make a gauge in python that will operate on live info(5-10 second intervals) on electricity info. I started with the plotly library. This library wasn't very clear when I went through the documentation. Me being a PyCharm professional user spend about an hour trying to figure out why I cant run any of the demo programs that were demoed on the documentation. I took me a while but then I found out that all those commands are made specially for ipython notebook and then I found another section for users of other ide's. Learning this I went forward and found a Gauge template after making a few changes to this template and playing around I think I had the grasp of the library. So for the library you wrote all your code in libraries and it made the graphical interface by converting your code to html and made an html file that you could open in you browser. After this I had a though on how I would be updating this html file. Me never having worked with html was not sure how but after spending some time on w3school I learnt a command with which I can make the html page force refresh it self every 5 seconds and when it refreshes the html page would have the new live data. I implemented this command after plotly produced the html file. Python would open it delete the first line and then retype it with the command to refresh every 5 seconds(update-I am still in the process of writing this blog but I just came across something called ajax that might be worth giving a look at in the next blog). Next I started working on moving the pointer of the gauge. I learnt about this my learning of the command M for move, L for line to, and Z to end. This is how the shape would be made in css and html and if I got python to update the html file every 5 seconds by changing the location or coordinates of the pointer it would update on the webpage to have a moving pointer. After learning that I went on to figure how what value would correspond to how much movement. To do this I tried breaking it between percentage and the coordinates on the circle. This took a look of math(geometry, trignometry, and algebra). I figured out how to scale percent to how much this meant on the scale by doing the math on paper. I am yet to write the algorithm for this on python. When I do I will blog about it and hopefully by success with AJAX. I believe I may use the math library SymPy to simplify my work.
_____________________________________________________________________ For the first half we just worked on improving our pandas knowledge by coding small programs that would filter through csv files with many data points(million) and try making different kinds of filters as we explored many of the issues and interesting things in pandas like the data types the library has and the different functions for the different ones and how it has its own format and way to do things like the format of a date or such. Later we had a presentation by Janet Nicosia and I found this presentation very interesting she gave us a small sneak peak or just a few things about the solar business and how it functioned. The bills she gave us showed really high profits(70% interest) and I also found how AHS is actually a fully clean energy powered school yet I have never heard any talk or brag about this. The net metering system seems something very cool and complicated at the same time and I just never though it would be possible to turn a solar plant into a largely profitable source of income.
We all went over the Carbon Dioxide Emission diagrams and how 60 Quads is Rejected Energy. I found this very surprising and 21 quads were from transportation and I found this very surprising as that seems like a very large amount considering only 5 quads are go under actual energy service. We then moved on to heat or natural gas and how the unit is therms and such. We also went over gas bills and national grid bills
|
AuthorHi! My name is Ayush Zenith! I am currently a senior at Andover High School. I have been part of the Energize Andover Program since June, 2017 (8th grade). I have since been working on improving my knowledge in programming and working on writing better applications in order to save and conserve resources in buildings... Archives
November 2020
Categories |