Workflow
Content can come from any source that you can download the full-text.
Examples:
- Gale Digital Scholar Lab
- After building a collection set, download the content. For this project, we will not use the metadata, so only the content is required.
- See here for help downloading a content set.
- Options to download content from a typical library database
- Manually download articles to .TXT, .PDF, or .DOCX format
- Use Zotero to download content.
- Tips on using Zotero:
- Create a new Collection for this project.
- When finished adding documents in the Collection, export the Collection by right-clicking the Collection name and select Export Collection.
- Export format should be RIS. This will create a new folder containing the metadata in RIS format (which we will not use for this project) and a folder named Files that contains all of the attached full-text files. This is what we are after.
There are two methods to share your content with the Jupyter Notebook discussed in the next tab.
- Upload from your Hard Drive - This is highly convenient as you can simply upload files. However, there are two reasons why using the second option, Google Drive, may be preferable.
- Uploading directly to the Jupyter Notebook is painfully slow. If you have more than a handful of files, the wait may be long-ish.
- If you decide to modify your content set (i.e., adding or removing a document or needing to re-run a different process) you will then need to upload those same document again.
- Upload your Content to a Folder in Google Drive - This is the preferred method for the two reasons provided above.
- Google Drive Requirements (for this project):
- You can use any Google Drive account. It does not need to be associated with your Baylor email.
- Create a new folder (directory) in the top-level (root directory) of your Google Drive. The Jupyter Notebook will prompt you for which top-level folder contains your content. All .TXT, .DOC, .DOCX, and .PDF files in this folder will be analyzed. This included sub-folders.
- Google Drive Requirements (for this project):
There are two types of Excel output files from the Jupyter Notebook:
- Analysis of content divided by sentiment (named sentiment.xlsx)
- Analysis of content divided by keyword proximity (named keywords.xlsx)
If a file with the same name is already in the directory, add something to the name to make it unique. For example, change sentiment.xlsx to sentimentJB1.xlsx. Do not remove the words sentiment or keywords as this is what tells Power BI what type of file it is.
I ran a sample of 90 documents from the Gale Digital Scholar Lab.
Keyword Search: Catholicism (Keywords: freedom, church, roman, feeling, protestantism, reason, enter, fear, christian, scandinavian)
Keyword Search: Christianity (Keywords: christianity, christian, china, church, religion, chinese, people, life, christians, christ)
Sentiment: High (Keywords: christian, church, god, great, china, men, life, work, chinese, christ)
Sentiment: Low (Keywords: christian, church, people, china, war, life, death, christianity, evil, sin)
<iframe title="ReligionEnlightenment - Page 1" width="600" height="373.5" src="https://app.powerbi.com/view?r=eyJrIjoiOTIyOGFlYzctNjFmZS00YWQ4LTlhNjktNTVlZDI3YjFmNjZlIiwidCI6IjIyZDJmYjM1LTI1NmEtNDU5Yi1iY2Y0LWRjMjNkNDJkYzBhNCIsImMiOjN9" frameborder="0" allowFullScreen="true"></iframe>
