Data wrangling exercises

The best way to know whether you enjoy working with data — or to learn how to work with data — is to handle many different forms of messy, real-world datasets, i.e. perform the necessary data cleaning and transformations to derive meaningful insights from them.

This post shares two sample Jupyter notebooks that demonstrate various data transformation steps to achieve clean data for two different data formats:

- JSON output obtained from calling the Developer API of the SingStat Table Builder (Singapore Department of Statistics): https://github.com/hxchua/datadoubleconfirm/blob/master/notebooks/SingStat%20API%202025.ipynb 

- Raw CSV file hosted on GitHub: https://github.com/hxchua/datadoubleconfirm/blob/master/notebooks/githubraw.ipynb

The example data relates to Graduates from University First Degree Courses by Type of Course and Sex in Singapore.

JSON output from calling the Developer API of SingStat Table Builder from Department of Statistics Singapore

Cleaned dataframe from parsing the JSON output

Raw CSV file hosted on Github: https://github.com/hxchua/datadoubleconfirm/blob/master/datasets/unigradsg.csv

Cleaned transformed dataframe ready for plotting chart

.Chart created using matplotlib. Code can be found in the first notebook link above