A cloze deletion test is a form of language test where a sentence (or paragraph) is given to the test taker with blanks for missing words. The student is expected to fill in a “correct” word in the blanks. I compare the difference between an LSTM (Long-Short term memory) neural network with that of a Bidirectional LSTM. Later the two news sources (described in Section 2) are compared to see which data set is easier to predict. Then I explore tuning the dropout parameter to see how overfitting can be improved. Finally the predictions are analyzed to see which sentences are easy to predict.

In this project I look at a dataset that provides some demographic information for 440 of the most populous counties in the United States in years 1990-92. The crime rate for each of the countries is predicted using linear regression and then using negative binomial regression and the results are compared.

Spinning lights. Full example here

We will create the correlation matrix using Hmisc::rcorr to get the correlations along with the p-values and knitr::kable to print the table. Using R-Studio, we can knit directly to word and format it the way we like.

We have a file with one valid JSON objects per line which we will load using jsonlite