DATA DIVE DAYS

Selected as Top 100 Data Science Resources for 2018/2019

Aug 22

data.gov.sg API

data.gov.sg has openAPI and for each dataset, it provides a sample Python code to query for the dataset. The output is a JSON and here's...

Jul 6

Extracting data from tables in PDF [Updated] - Python

The previous post on Extracting data from tables in PDF was many years old and requires an update as it didn't cover a Python package...

Jul 6

Singapore Traffic Updates

Traffic Updates, Road Closures & Road Works in Singapore are published on LTA OneMotoring website. To understand the traffic conditions...

May 24

OneMap API

If you're looking for the latitude and longitude of places in Singapore, you can make use of OneMap API. It is free for public use. Below...

Dec 4, 2023

Simple Retrieval Augmented Generation Use Case

This blogpost shows how Retrieval Augmented Generation can be done easily using Dataiku (without any code!). We want to ask some...

Apr 19, 2023

Monetary Authority of Singapore (MAS) API

This post shows an example of how to query data using the MAS API. The results returned are in bytes format and has to be transformed to...

Apr 14, 2023

SkillsFuture Singapore (SSG) API

Public can create an account to get data related to courses published on the SkillsFuture Singapore website via their API. This notebook...

Sep 1, 2022

Scraping table data off website - Python

This notebook presents two ways of scraping table data off website, one using pandas and the other using BeautifulSoup:...

Jan 23, 2022

Data integrity

Data integrity is vital to enable robust analysis. In the process of open-sourcing Singapore's Covid-19 data, I ran into several data...

Dec 19, 2021

OneMap SG Latitude/ Longitude conversion

It's very often to come across datasets from data.gov.sg or open data from Singapore government's portals with X- and Y-coordinates...

1