Short&Long-form writing by Iverson(ZhuZheng) ZHOU
I’m thrilled to share some exciting news: Very recently I’ve officially become a certified SRE Professional and Observability Professional! Passing both the SRE Foundation and Observability exams with an exceptional score of 95%,99% is a testament to my commitment to excellence and continuous learning.
Recently I have been busy juggling with big data. It is such a deep ocean with wide range of domain knowledge and was particularly difficult for someone who does not have a comsci background. Although there are a lots of resource floating around the internet but not all of them are intuitive. After all the filtering and trial and error I have done through the journey of learning big data ecosystem, thought I should write a comprehensive tutorial not only as a way for my own documentation, but also for people who are in the same boat I was because I never found one single learning resource where it explains things in detailed from the ground up and from scratch! .Hope the stuff I share here would be of use for someone starting the big data journey, as the tech world is moving fast for sure. I remember when I finished my undergraduate degree, we were using SPSS as a “database” and things certainly have changed drastically with new technalogy. Not sure what the future looks like but it is very exciting and scary at the same time.
Some may argue as a statistician or data scientist, one does not need to learn the engineering side of things. Indeed, of course in the real world these tasks will be divided amongst, software engineer, system architect , system admin, ML engineer, data engineer, ETL people, database administrator ,data scientist. However, learning some or even all of them will give you a cohesive view of the entire eco system: e.g. from a single meta data to a fully deployed ML solution. Which might come in handy sooner or later.
Let’s dive in! (Github repository to my work can be found here)
(more…)
docker on windows (nightmare)
docker with linux with r (still nightmare but not as much)
steps: pull software, run linux package installation, install shiny, permission configuration, r packages and dependencies
issues encountered: (e.g. Java configuration, different built being run with AMD64）
link:
https://github.com/Z-ingdotnet/Cov_19_dockerized
Although R shiny app was around for a long time. I didn’t get the chance to try it out until recently
A confession to be made, I was late on understanding cryptocurrency and by no means am I any expert of the block chain technology. But as I continue to read more about the technology, the more intriguing it gets . I was already developing an interest in cryptography and knowing something that combines cryptography with various other knowledge domains just really excites me. Hence I thought it would be a great opportunity to learn about block chain but learning how its proof of work system works, and thus mining it would be a great place to start.
a side note. I don’t think I am qualified to tell you that cryptocurrency are the best investment you could make. In fact, I’m still very skeptical of it. Although mining cryptocurrencies sounds pretty cool, but don’t be fool by thinking there will be no initial investment. Whether or not cryptocurrency mining is profitable is largely depending on the computing power of your device. And to the best of my knowledge, the current game for popular cryptocurrencies like bitcoins are very large, there are already people making thousands of upfront investment on devices that are specifically designed for cryptocurrency that give them very high hash rates. And mining with CPU just never going to give you the same computational power. However, there are still other cryptocurrencies that are still worthy of a try. At least for educational purposes or just some passive income that you let it run 24/7 along with your website.
I love the “decentralized” concept so lets get started:
中国有句古话：“读万卷书不如行万里路”. It is an old Chinese saying equivalent to “The most valuable lessons in life can never be expressed in black and white, but must be experienced”
I never thought I could just pick up a bag, take my car and disappear. But now looking back, I couldn’t have been more thankful as it was probably one of the best decisions I made in recent years.
In university, you get a false perception that everything of importance can be found in books. But actually, many things in life are very hard to put in black and white. There is a reason why many agree that experience is the greatest teacher of all.
I was lucky enough that when I quit my old job, the lease was also about to expire so I took advantage of the situation at the time; hired a storage unit and moved my man cave into it within a day, booked a ferry ticket the same night, took my car and disappear from the city the next day. From then on, for a period of three months I was constantly on the move, from one backpacker to another, one camp to the next, one city to a new one…….This traveling/backpacking experience opened up my eyes and had me lived in a different world that forced me to think and reevaluate what’s important,what to value more and how to live the life I wanted to live. It was a priceless lesson that I think every individual with the opportunity should attend. As Buddha saids: “There are only two mistakes one can make along the road to truth; not going all the way, and not starting.”
Here are the vital lessons traveling taught me:
wesearch.co.nz was launched back in 2010 as a personal project for learning about the web. I was really proud of the fact that I took something from imagination to execution. However, due to various constrains and other personal commitments at the time, unfortunately the site/idea didn’t receive the maintenance and improvement it deserves. Although the site initially received a fair amount of attention from the public , but there have only been around 300 listings posted to date.
I realized I never had the chance to look at the actual classified ads data stored behind the site. Up until this week, I decided learn some text mining techniques and thought it would be a great idea to extract ads data from the site and use it as sample data.
Disclaimer: This is my first attempt , so by no means I’m a text mining expert, if there are errors or better way to do things please let me know. Below is what I have done:
This weekend I learned something about HTTPS and the reasons for forcing all requests to redirect to https rather than letting requests going through unencrypted http traffic.
In a nutshell, it is about security. Https stands for “Hyper Text Transfer Protocol” with Secure Sockets Layer (SSL). It essentially provides critical security and data integrity for the communications between the website and the users’ browsers. Which in turn protects users’ personal information.
In the absence of a SSL encryption, intruders can exploit unprotected communications through almost every unprotected resource such as images, cookies or scripts to trick users into giving up sensitive information and create security vulnerabilities. They may also passively listen to unprotected HTTP communications between the website and users.
Here is how you set up TLS/SSL certificate on Linux(Ubuntu).
update local apt package indexes and install the Let’s Encrypt client:
sudo apt-get update
sudo apt-get install python-letsencrypt-apache
this is probably a litmus test that most so called leaders fail
note: plot produced using R, plate boundary and earthquake data(01/Jan/2014-20/Dec/2016) with minimum magnitude of 5 or greater
P.S…never really looked into earthquake data until the recent Kaikorua event happened in NZ and it elevated my curiosity in this space. In a nutshell, although it was obvious that NZ and South America are sitting on several fault lines and that they are in the deforming plate boundary zone. but looking at the number of earthquakes illustrated graphically gave me a bit of a shock.
Have anyone previously applied machine learning in catastrophe/risk management, more specifically, in attempting to predict earthquakes (e.g. using a large number of time series data to put into machine learning algorithms to output a predicted classification)? please let me know
First time learning these networking stuff ,VPS,socks5 protocols, very interesting learning curve, thoroughly enjoyed it
after generating a public and a private key to ssh, installed the required packages and done some configuration on ports and the firewall etc, I was able to go on Instagram
Conference Abstract:
PLS17 – Abstract – Final
Below is the Preso I prepared aims to give a quick overview of the entire thesis. Here I am trying to explain the thesis in the form of customer journey mapping, incorporated with some creative and design thinking stuff.
it touches the following domains of knowledge:
It has been a really tough and long journey to pull this piece of work together which required me to step into uncharted territories to learn and apply new knowledge. Most importantly, the thesis would not have been possible in the absence of the willingness and effort survey participants had put in.
Thankfully the work received high recognition and an A+ grade. My supervisor and I are in the process of turning it into a journal article. Will send all participants relevant link(s) once the publication is complete.
I would like to thank all the participants once again. Prize draw winners have been contacted and all prizes have been sent to the provided address details.
Link to the original survey page is here
