End of week 4/20

This fourth week went by extremely fast. I’ve mostly been stuck on some technical issues this week. I’ve spent most time designing the runExperiments.sh-script and debugging issues with various Kubernetes components. I’m a bit behind my planning: according to the time plan, this week should have been spent implementing and trying anomaly detection methods. I have not tried a single method yet.

I think that I will start implementing anomaly detection methods at the second half of the next week.

Summary of week [4/20]

• Monday
  • More work on runExperiments.sh: fixed errors with logging and authentication/authorization issues in helm/kops/kubernetes
• Tuesday
  • More work on runExperiments.sh: added functionality for the script to save metrics from your k8s cluster to your local computer.
• Wednesday
  • More work on runExperiments.sh: added functionality for installing the sock shop among other things
• Thursday
  • More work on runExperiments.sh: ran into a lot of technical issues with some metric collectors. Added and later removed support for istio.
• Friday
  • More work on runExperiments.sh: added primitive methods of visualizing metrics in Grafana
  • Had a meeting with my supervisor regarding what to prioritize going forward
  • Wrote this post

Experiences

There isn’t very much to say about this week when it comes to experiences. I managed to get out of bed early and come to work early, which is nice. Also, I had a doctors appointment and dentist appointment on Wednesday which was distracting. Besides from that, the week has been rather mundane in terms of experiences.

Next week

As I said in the introduction of this post, I’m a bit behind on my planning. However, according to the time plan, I should spend next week implementing anomaly detection methods and start writing on the project report. I think it is feasible to do that next week, aside from that I actually should have started implementing anomaly detection methods this week.

Upcoming tasks

I have some upcoming events soon: in week 6, I should send in my work for a peer review. And then I should attend the peer review in week 7. Therefore it would be good if I have a first draft of my project report by the end of next week. Also, I have some technical tasks.

The following is a list of tasks, ordered by how I prioritize them:

Reproduce earlier anomaly detection results by fellow employees at Elastisys

I still have not had the time to reproduce the anomaly detection work that Fredrik Liv and Jakob Lindqvist did during the summer here at Elastisys. I’ve read most of the code and documentation, what is left now is just to actually run the code. I will probably be done with this by monday

Establish a baseline for normal operations of the sock shop

In order for machine learning-approaches to discern anomalies from normal operations, I need a data-set of normal operations. Preferably, this means running the sock shop with moderate load for a long time.

I will try to do record some of these “normal operations” during the weekend, but I will probably need to re-do this work later. Now I’m a bit uncertain what metrics to collect, and I’m worried that I may need to modify some of the code of the sock shop to gather application specific metrics.

Determine what metrics are going to be used for anomaly detection

I need to visualize and make decisions about what metrics will be used for anomaly detection. For this, I will use Grafana and Influxdb.

Design and implement an anomaly detection method

Obviously, this needs to be done.

Org-mode

As I think I’ve mentioned on the blog, I’m an Emacs user and Emacs comes with the awesome organizational tool called org-mode. I just wanted to include a screenshot of my planning document for this master’s thesis. :)