Python: Automation & data analysis of Covid 19


Below we will see how we can get data from a website (Web scraping) and then analyze it and draw statistical conclusions using the programming language Python

I will not go into the process of explaining the code, the scripts are ready and available for free on GitHub and can be used by anyone or customized as they see fit!

What is Web scraping?

By Web scraping we mean extracting data from a website using a software program. There are many programs of this type, such as ScrapeStorm which does a great job, but it's not free .. With the free version you can only use it once a day and it can only export 100 lines of one table at a time. Otherwise you have to pay about 50 € per month to have full operation! Of course not!

Obviously this did not cover me either so I wrote my own script in Python that exports everything from its page wordometers.

You can see the code here.

How does the scrap script work?

The script is simple, as you will see it is only 15 lines of code along with the spaces. Uses a package, the covid 2.2.6th, created by Johns Hopkins University and helps us gather the information we want from the above website. What I did was compile all this data into a .csv file so that we could later process it in some programming language. In short, I create a database on a daily basis, with just the push of a button and above all for free! I save this database on GitHub and can use it from anywhere.

Here you can see it .csv file with all the data by country, continent etc.

Script for data analysis with Python.

The script takes as input the data from the above .csv file and makes any analysis I specify. I have assigned him to calculate the following.

  • Mortality calculation for each country
  • Mortality calculation for each country
  • Calculation of infection percentage of the population for each country
  • Percent increase in cases compared to yesterday
  • Percent increase in deaths compared to yesterday
  • Percent of serious cases
  • Percentage of those who recovered
  • Percent active cases
  • Cases per ten thousand of the population of each country
  • Cases per one million population of each country
  • Prints a Pie chart with the important data for each country
  • How many tests were done and the percentage of tests that came out positive

Here you can see the script code GitHub

Example of the script output for Greece.

python

Basically what we do with Python and all programming languages ​​is, once we write the code of what we want a program to do, it will do the same calculations every second, depending on the data we enter each time!

python

For Windows users

You can download from here the .exe script for Windows computers. Just double-click on the .exe you download and it will take the data of whichever of the following countries or continents you choose:

  • WORLD
  • ITALY
  • SPAIN
  • GERMANY
  • NETHERLANDS
  • BELGIUM
  • GREECE
  • FRANCE
  • NORWAY
  • PORTUGAL
  • SWEDEN
  • DENMARK
  • LUXEMBOURG
  • FINLAND
  • AUSTRIA
  • CZECHIA
  • IRELAND
  • ICELAND
  • CYPRUS
  • SWITZERLAND
  • UK
  • NORTH AMERICA
  • EUROPE
  • ASIA
  • SOUTH AMERICA
  • OCEANIA
  • AFRICA
  • USA

The input of the country / continent must always be in Latin characters and in capitals as shown above. To see the diagram you need to install the following library pymetasploit3 simply by opening the Windows terminal and writing the following.

pip install pymetasploit3

Press enter and wait for it to finish.

For Linux users

Linux users can run the script directly from the terminal by downloading this file and after they have primarily installed Python on their computer.

sudo apt install python3
sudo apt install pythonpy
pip install pymetasploit3

After all of the above are installed (and whatever else the terminal warns you is missing), open the terminal on the tab where you saved the script, run the following

py worldmeters_analysis.py

Press enter and if there is no error it will run and ask you to enter a country or continent exactly as described for Windows users.

GitHub

worldometers

Registration in iGuRu.gr via Email

Enter your email to subscribe to the email notification service for new posts.


Read them Technology News from all over the world, with the validity of iGuRu.gr

Follow us on Google News iGuRu.gr at Google news