Tried out textgenrnn, to create a text-generating neural network, using text from Singapore's Budget 2020 - Resilience Budget/ Supplementary Budget Statement as the training set. There's over 10,000 words in the statement/ text file but there are some Chinese text within the file as well, so in total, 808 texts are collected by the algorithm and it started training on 59,436 character sequences, with 464 treated as vocab where they are recognised as steps in the training. As the number of words is not exactly a lot, there's still room for improvement for the trained model in generating a more logical flow of words but in general, it got the context right and is using terms specific to the situation!
Below is some samples of text generated by the model. The notebook can be found here.
