CS for Language Arts: Use Machine Learning to Analyze Writing

Introduction and Background

For this project you will use machine learning tools to evaluate the sentiment of reviews on an article, story, or movie. You will also use machine learning tools to evaluate a question and answer application. These evaluation will give you an experience much like a natural language processing (NLP) researcher might do to evaluate the validity of an artificial intelligence product.

An Introduction to Natural Language Processing (NLP)

This video gives a quick introduction to NLP and the breadth of applications.

Recommended Reading

Introduction to sentiment analysis: What is sentiment analysis?Algorithmia Blog

The article below goes into a lot of depth, but reading up to, and including, the section "What are Word Embeddings?" provides a good introduction.

Introduction to Word Embeddings and its ApplicationsMedium

Student Objectives:

• Prepare text to be processed with natural language processing

• Analyze the text using sentiment analysis and/or sentence encoding techniques

• Evaluate the results for the potential value and risk it brings

Subject Areas: Computer Science, Language Arts and Social Sciences

Instructions

1. Prepare text to be processed with natural language processing

We want to prepare two pieces of text for this project.

• A Collection of Reviews on a Work - This can be simple movie reviews, book reviews, or article reviews. You can collect reviews yourself using a Google form, for example. Your collection doesn't have to be huge. Twenty, or so, reviews collected in a simple document will work great for this project. We will use this to experiment with sentiment analysis, to determine if each comment is positive or negative.

• A Short Body of Text for Automating Questions and Answers - This can be a short story, company descriptions, or brief article. We will use this text to create an automated question and answer application, where the computer will attempt to answer a user's question based on the source text.

2. Analyze the text using sentiment analysis and/or sentence encoding techniques

These online tools will perform best in a Google Chrome browser.

• Sentiment Analysis Use this online tool to complete a sentiment analysis on your collection of reviews. Once your process the reviews, copy the results of the analysis into a text document for your own records.
• Automated Question and Answer Use this online tool to conducted an experiment of automating answers to questions about your chosen body of text. Prepare at least 7 questions about the body of text and record the provided answer for each question.

3. Evaluate the results for the potential value and risk it brings

• Evaluate the Sentiment Analysis

  • Performance: How accurate was the analysis? Did the algorithm do a good job? What seems to give it problems? Write a short summary of your findings.

  • Applications: What would be some appropriate applications of this technology? What might be some less appropriate uses for it?

• Evaluate the Question and Answer Tool

  • Performance: How accurate was the answers? Did the algorithm do a good job? What seems to give it problems? Write a short summary of your findings.

  • Applications: What would be some appropriate applications of this technology? What might be some less appropriate uses for it?

• Prepare a Presentation of Your Project In addition to the evaluation summaries, include a summary of the source texts you used and samples of the results from the NLP tools.