DATA DIVE DAYS

Process and product of various data science tasks— from data collection, data preparation, data visualization, to basic statistical analysis and modelling. Datasets for practice available.

Selected as Top 100 Data Science Resources for 2018/2019

on MastersInDataScience.com

Updating data in google sheets using API and Python

The Singapore government publishes locations visited by COVID-19 patients in the last 14 days, and removes any data earlier. I decided to...

Jul 31, 2020

Engineering job openings in Singapore

The following information for over 5000 job opening listed on a government portal for virtual career fairs was scraped: title, company,...

May 30, 2020

Extracting text data from online image

Here's the link to the exact notebook: https://github.com/hxchua/datadoubleconfirm/blob/master/notebooks/pytesseract_onlineimage.ipynb.

Mar 26, 2020

Analysing updates on COVID-19 local situation on MOH's website

Looking at the number of updates published on the Ministry of Health's website, we might be able to get a sense of the severity of the...

Mar 25, 2020

COVID-19 Singapore Data

There are various open-source datasets on Python and R relating to global cases but here's sharing a more detailed dataset focusing...

Feb 1, 2020

SingStat API

SingStat (Singapore Department of Statistics) has made quite some data publicly available but not in the most analyst-friendly format....

Nov 10, 2019

Toughest part in a data science project

Was surprised to see my post trending. Since it caught on, I would cross-share it over here. While the post was meant to answer a...

Oct 30, 2019

Cross-sharing data on data.world

While I mainly host my datasets on my Github repository, I have also cross-shared some datasets on data.world as the platform is...

Jun 23, 2019

Emoji data

I got to chance upon the emoji python package that allows printing emojis in Python and decided to collect some data relating to emojis...

Apr 24, 2019

YouTube data analysis - Python

There are various data items, such as channel name, title of video, and number of views, likes, dislikes, and comments, that can be...

{"items":["5f6da1bae932b900183c86f6","5f6da1bae932b900183c86f9","5fcc95b8b3157700171d443f","5f6da1ba4eda0d0017df0be8","5f6da1ba4eda0d0017df0be9","5f6da1badb77650017225c9a","5f6da1badb77650017225ca2","5f6da1bb2afe96001736ee52","5f6da1bb2afe96001736ee5b","5fcc95b8812ee60017e52bf8"],"styles":{"galleryType":"Columns","groupSize":1,"showArrows":true,"cubeImages":true,"cubeType":"max","cubeRatio":1.7777777777777777,"isVertical":true,"gallerySize":30,"collageAmount":0,"collageDensity":0,"groupTypes":"1","oneRow":false,"imageMargin":32,"galleryMargin":0,"scatter":0,"rotatingScatter":"","chooseBestGroup":true,"smartCrop":false,"hasThumbnails":false,"enableScroll":true,"isGrid":true,"isSlider":false,"isColumns":false,"isSlideshow":false,"cropOnlyFill":false,"fixedColumns":0,"enableInfiniteScroll":true,"isRTL":false,"minItemSize":50,"rotatingGroupTypes":"","rotatingCropRatios":"","columnWidths":"","gallerySliderImageRatio":1.7777777777777777,"numberOfImagesPerRow":3,"numberOfImagesPerCol":1,"groupsPerStrip":0,"borderRadius":0,"boxShadow":0,"gridStyle":0,"mobilePanorama":false,"placeGroupsLtr":true,"viewMode":"preview","thumbnailSpacings":4,"galleryThumbnailsAlignment":"bottom","isMasonry":false,"isAutoSlideshow":false,"slideshowLoop":false,"autoSlideshowInterval":4,"bottomInfoHeight":0,"titlePlacement":"SHOW_BELOW","galleryTextAlign":"center","scrollSnap":false,"itemClick":"nothing","fullscreen":true,"videoPlay":"hover","scrollAnimation":"NO_EFFECT","slideAnimation":"SCROLL","scrollDirection":0,"scrollDuration":400,"overlayAnimation":"FADE_IN","arrowsPosition":0,"arrowsSize":23,"watermarkOpacity":40,"watermarkSize":40,"useWatermark":true,"watermarkDock":{"top":"auto","left":"auto","right":0,"bottom":0,"transform":"translate3d(0,0,0)"},"loadMoreAmount":"all","defaultShowInfoExpand":1,"allowLinkExpand":true,"expandInfoPosition":0,"allowFullscreenExpand":true,"fullscreenLoop":false,"galleryAlignExpand":"left","addToCartBorderWidth":1,"addToCartButtonText":"","slideshowInfoSize":200,"playButtonForAutoSlideShow":false,"allowSlideshowCounter":false,"hoveringBehaviour":"NEVER_SHOW","thumbnailSize":120,"magicLayoutSeed":1,"imageHoverAnimation":"NO_EFFECT","imagePlacementAnimation":"NO_EFFECT","calculateTextBoxWidthMode":"PERCENT","textBoxHeight":240,"textBoxWidth":200,"textBoxWidthPercent":50,"textImageSpace":10,"textBoxBorderRadius":0,"textBoxBorderWidth":0,"loadMoreButtonText":"","loadMoreButtonBorderWidth":1,"loadMoreButtonBorderRadius":0,"imageInfoType":"ATTACHED_BACKGROUND","itemBorderWidth":0,"itemBorderRadius":0,"itemEnableShadow":false,"itemShadowBlur":20,"itemShadowDirection":135,"itemShadowSize":10,"imageLoadingMode":"BLUR","expandAnimation":"NO_EFFECT","imageQuality":90,"usmToggle":false,"usm_a":0,"usm_r":0,"usm_t":0,"videoSound":false,"videoSpeed":"1","videoLoop":true,"jsonStyleParams":"","gallerySizeType":"px","gallerySizePx":1000,"allowTitle":true,"allowContextMenu":true,"textsHorizontalPadding":-30,"itemBorderColor":{"themeName":"color_15","value":"rgba(0,0,0,1)"},"showVideoPlayButton":true,"galleryLayout":2,"calculateTextBoxHeightMode":"MANUAL","textsVerticalPadding":-15,"targetItemSize":1000,"selectedLayout":"2|bottom|1|max|true|0|true","layoutsVersion":2,"selectedLayoutV2":2,"isSlideshowFont":true,"externalInfoHeight":240,"externalInfoWidth":0},"container":{"width":560,"galleryWidth":592,"galleryHeight":0,"scrollBase":0,"height":null}}