Tag Archives: Guide

Using Palladio: A Reflection and User Guide

Palladio is a free web-based network tool. It allows users to upload their own data to create both maps and network graphs.  Users don’t need an account and can download project to their computers or save the url to return to their projects.

When embarking upon a networking project, I think it’s important to be conscientious about what kinds of relationships networking can best represent. The idea is to see relationships between data that otherwise are hard to conceptualize. My trials with Palladio have some good examples where networks effectively convey information, and good negative examples as well. As I describe how to use the tools, I’ll included some commentary about what I learned in the process.

Getting Started

 photo Palladio_Home_zpsom8mcrq9.jpg
After entering http://palladio.designhumanities.org/#/ into your address bar all you have to do is press “start.”
 photo Palladio_Load_Data_zpsntqiyodx.jpg
From there you’ll have to load some data. As far as I can tell you can’t create data in Palladio. My data were stored in .csv documents and prepared for me by my instructor. Hopefully you’ve organized your data before you’ve come to this point. I loaded the primary data set I would be drawing from- in other words the spread sheet all other sheets would relate to. From there press “Load.”

Your Data

 photo Palladio_Data1_zpsahjlvh4n.jpg
Your primary dataset will appear in the “data” tab. Here you’ll connect other data tables to make relationships. In this step you’ll already start making decisions about what kinds of relationships you want to show by what data you choose to add extensions to. In my project we were concerned with showing relationships between people and places, so this is the data we linked.
 photo Palladio_Add_Table_zpsscsdgkns.jpg
By clicking on a data field (in my case “where interviewed”) the editing window opens. Here I selected “Add a new table” to start creating relationships. I uploaded my “locations” spread sheet. Then I selected the “subject interviewed” data field from my primary dataset and added the locations of their enslavement. Finally, in the Enslavement table that I just uploaded I clicked on the drop down “Extension” menu and selected “Locations” to link the two datasets. At the end I had this set of data:
 photo Palladio_Data_Complete_zpsez9o611u.jpg
With all my data uploaded and connected, I was ready to start exploring Palladio’s visualizations.

Mapping with Palladio

 photo Palladio_Add_Layer_zpsvknycxra.jpg
It is possible to map with Palladio, but that doesn’t mean you should. Creating the map is fairly easy. Select “Add new layer” then the above editing window appears. Just select what kinds of points you want and the two datasets you’d like displayed (source for me was interview location, target where_ensalved). You might have to “ctrl -” to zoom out enough to see the “add layer” button.
 photo Palladio_Map_zpsxhnldfar.jpg
For some projects the mapping feature is probably sufficient, but for mine, it was far from. Although this map does show the movement of individuals from their places of enslavement to the location of their interview, the directionality is not clear. Palladio does allow for layers, but the information available for display is not nearly as rich or customizable as CartoDB and other GIS specific applications. This map does convey that there was a significant movement of people after slavery ended, but other questions can be better explored and asked in GIS applications.

Networking Tools

 photo Palladio_Settings_zpszjspy6hr.jpg
The networking tool is pretty intuitive. All you have to do is select that data you want to relate (a source and a target). The “facet” tab allows users to focus in on certain aspects of their data (for example in the above visualization I could limit the interviewees to those over the age of 80). In the above example I selected the interviewer as the source data and the interviewee as the target. The resulting visualization shows which interviewers interviewed which ex-slaves. I would say this is a good visualization because the viewer can see the intended information with ease. So, instead of looking at a spreadsheet organized by interviewer, I see all the their interviewees in one page. This visualization doesn’t elicit many novel questions, but it does provide solid information.
WPA M/F photo WPA M_F_zpsykydcfv4.jpg
Within the source data, you can select other facets to relate. In the above example I visualized which interviewers met with male and/or female informants. This helps me ask questions about gender bias in the interviews, or how gender norms in 1936 influenced the interviews.
WPA Type/Topics photo WPA Type_Topics_zpstz0kwiiz.jpg
Networking the relationships regarding topics had mixed results. The graph above, showing the relationship between the type of work the ex-slave was engaged in during their enslavement and what topics they discussed is a rather successful graph. I would have expected there to be more variation in the topics, but upon seeing this graph I was reminded that the interviewers used a script, asking about particular topics and really engaging in natural conversations. The few outliers probably represent occasional spontaneity arising from the script.
WPA Topics photo WPA Topics_zpsdlumm6pm.jpg
The above graph is a fairly clear example of an instance where networking doesn’t work well. The relationship between which interviewees and which topics they discussed. The result is too highly clustered and the size of the nodes too large to make anything out. This unfortunate grouping is due to the afore mentioned scripted. Almost every person discussed the same things sine they answered the questions they were being asked. This consistency also reflects what Mark Twain refers to as “corn-pone,” when enslaved or previously enslaved people who tell white people what they wanted to hear. It could be said that there was already a script, even before the WPA produced one.

A Brief Reflection

Networking visualization can be a very powerful tool when the investigator is conscientious of what information the graphs can provide. Palladio  only computes bimodal networks, which is usually the best thing. Looking at these networks allowed me to ask questions about why these relationships looked the way they did. So, why did female interviewers tend to meet with female informants? Why were ex-house slaves the only people to discuss “mammy?”

I was also able to draw some preliminary conclusions. The WPA script was, arguably, effective. Conversations stayed relatively on script and recorded consistent types of information. These observations would lead me to look at the scripts themselves to see to see if I’m correct. Which leads to another take-away: no tool replaces close reading.

Using Voyant: A Refelction and Guide

Voyant (voyant-tools.org) is a web-based tool set for text mining and analysis. I utilized the service to glean information about the nature of the WPA Slave Narratives. These narratives are the result of interviewers from the Worker’s Progress Administration seeking out ex-slaves from 1936 to 1938.

Voyant Tools: Getting Started

 photo Voyant_Start_zpsetdnpink.jpg
Upon arriving at the home page, I was given the option to either upload text files, enter urls, or enter text directly. I entered 17 urls, one for each state that participated in the WPA project, and clicked “Reveal.”

Here I ran into my first hick-up. Some times, the quantity of data I was loading into the web tool seemed to much and either Voyant would go on “fetching corpus” forever, or it would give up with an “error” and no explanation. Luckily, there’s an easy fix. Just visit http://docs.voyant-tools.org/resources/run-your-own/voyant-server/  and download the Voyant Server. Nearly all my problems were solved after I downloaded the server, so I do suggest it.

Voyant Tools: the Tour

Voyant Home photo Voyant_Home_zpsnqmr7bd9.jpg
Once my corpus was “fetched” I instantly saw visualizations for my text. I already saw that I’d have to do some adjusting. Just looking at the word cloud, I saw that Voyant was including information that I knew wasn’t useful. In my corpus, there was not standard transliteration for dialect, so the most common words “dey” and “dat” were not significant since dialectical variations were reliably recorded. There was an easy solution for this, but first I’ll go through each of the five tools: “Cirrus,” “Reader,” “Trends,” “Summary,” and “Contexts.”

1 Cirrus: This tool provides the old standard word cloud, with the largest words representing the most common words. The word count appears when your hover over the word. By sliding the “terms” bar you can adjust how many words appear in the cloud. The “scale” drop-down menu allows users to look at clouds representative of the entire corpus, or just a particular document. When you click on a word, the “trends” section displays the graph for that word. I found this tool helpful for getting a big picture idea of the interviews.

2 Reader: The reader allows for contextualization and some degree of close reading. The text from your documents is displayed. When I first came to the tools page the first lines of the first text in my corpus were displayed. The colorful boxes along the bottom of the window represent the different documents in your corpus. The width of the boxes represent how much of the total corpus they make up. The line going through the boxes is a representation of the trend of the word you are looking at. When clicking in the boxes, the reader displays the text at that spot. If you select a word from the “contexts” tool it will show that instance of the word (more on that in the “contexts” discussion).

3 Trends: This window displays a line graph of the frequency the term you’re exploring. Much like “cirrus”, users may adjust the scale of the graph from the whole corpus to a specific document. I found this tool useful in gaging how word use changed across states and allowed me to ask those rich “why?” questions.

4 Summary: This box provides the metadata of the document. The first line provides document count, word count, and number of unique word forms. It also conveys how long ago the session was started. Then the tool further breaks down information about each document, first with document length (longest, then shortest), vocabulary density (highest, then lowest), most frequent words, and distinctive words (by document). The “document” tab displays much of the same information in the main tab about each document. If you’re explore one word the “phrases” tab will display phrases the word under investigation is found in. I found the summary useful in, first, getting a sense of the magnitude of the text I was working with. Having never seen the volumes or even read the text, I was able to understand just how much text was being processed. Secondly, the summary conveyed the wide variety of language used across the text.

 5 Context: This tool essentially does what it claims. Once you’ve selected a word in either the Reader or Trends, context displays the documents the word occurs in as well as texts to the right and left. If you click on the term in one of the lines, that line will appear in the reader with the surrounding text. I found this helpful for, well, putting floating terms in context by doing a little close reading.

Voyant Tools: Stoplist

Voyant Settings photo Voyant_Settings_zpsudlfyfnf.jpg

My corpus had a long list of words that weren’t helpful and likely almost any text analysis project will. Luckily, it’s very easy to adjust the stoplist in Voyant. In any of the tools, when you hover over the question mark (but do not click) more options appear (pictured about). Click on the slider icon to call up this text box:
Voyant Options photo Voyant_Options_zpsid7dtkin.jpgThere are several adjustments that can be made, but for the stoplist, just click the “edit list” button next to the “stopwords” dropdown menu. Another text box will appear in which you can enter your next terms and edit the auto-detected list if you choose.

The little arrow  coming out of the box icon allows you to export your visualization in several formats. Below, I chose one that could be embedded in a web page:

As you can see, this is a fully interactive word cloud. Each of the tools allows for this utility. This word cloud is also the result of adding words to the stoplist. This word cloud is much more representative of the corpus than the previous one you can see in the screen shot of the home page.

A Brief  Reflection:

AHaving used Voyant Tools, I have a much better appreciation for the anaylitic power of text mining. I was able to see patterns and outliers much more readily than a close reading. I was also able to ask novel questions that I doubt I would have been able to had I read each interview one at a time. As for using Voyant as that text mining tool, I have mixed feelings. The fact that the service is completely free is a huge boon, but there’s the old saying, you get what you pay for. With project looking at several million words, Voyant might be too slow. Although the export tool allows users to share their visualizations, you can’t save your work. So every time you close the program, you have to re-enter the text. Which, again, for larger projects would be a major hindrance.