Writing the Lit Review for Research Methods

Research. Olin Warner (completed by Herbert Ad...

I recently finished a pretty big, for me, literature review that totaled about 17 pages, including the title page and two pages of references. Here are some scattered thoughts and lessons learned, at my customarily hideous length:

  • I saw the wisdom of The Scholarly Cassidy's advice to begin the search haphazardly. I spent much early time floundering but tried various keywords that eventually led me to articles of interest. Have to get used to the feeling of confusion at beginning and make friends with it.
  • As with most of the work at SILS, what I did wasn't really hard so much as it was time consuming. The keys are starting early (a lesson I'm always re-learning) and letting the work marinate. Because I'm deeply into self-justification, I am obliged to tell you that I started late because I was finishing up a different assignment and dealing with my full-time job, of course, so my research was tucked into the margins of my daily schedule (i.e., at night before bedtime) or relegated to weekends.
  • I remembered advice to break the writing into three fairly equal time-sized chunks: a third searching, a third compiling and sifting, and a third writing. I altered that to make the writing take only one day, but this division let me know when to stop active searching and when to start writing. Although I did occasional follow-up searches, the bulk of my active searching had stopped days before I started writing.
  • I adapted Cal Newton's Newport's Excel-based research database. I added a worksheet to track the lists of keywords I searched against. I kept a list of all of my sources in the main tab, with their citation (if it was easy to get), a URL to the abstract or document, the year it was published, its abstract, a theme or category to which the article belonged (such as "Community Attachment" or "Personal Networks") and a link to a PDF of the full-text article I'd downloaded to my hard drive. I pretty quickly compiled about 125 sources (plus some duplicates). I started scanning for quotations, but discerned that precise quoting wasn't called for (though page references to specific ideas were). I didn't need quotes so much as synthesis. That said, I still had way too many quotes -- the old reporter habits of tucking the evidence into the story die hard.
  • I used the spreadsheet to scan the abstracts and judge immediately whether an article had relevance to me. (I kept reminding myself this was a short paper, not written to last 20 years.) Instead of deleting those rows, I colored the citation cell red. If I liked the abstract, I assigned a theme or category (and duplicated the row if the article fit into more than one category). This got me familiar with the breadth of my article grabs. Then I sorted on the Year Published column (earliest at the top), and auto-filtered by theme. I could then see this haphazard list snap into place: all the articles for the themes sorted from earliest to most recent, and the progression of thought visible in the abstracts. I'd already decided I only needed about 3-4 themes for this paper, so this process helped me identify weak themes (only one or two articles) and combine similar themes for later processing.
  • When it was clear that I had too many articles for a category (about 25 for the Sense of Community theme, for example), I reduced the number to 3-5, which forced me to generate selection criteria and think about how they would fit into the story I was telling. I then printed out only these articles and read them more closely since they would form the spine of the lit review.
  • I spent most of the days leading up to my writing in working this spreadsheet, finding new sources until I reached saturation (the same titles or authors cropping up), and in thinking about the story -- or as some may call it, "building an argument." Same thing, really. Set up the foundation with the themes you'll come back to, remind the reader of them as you go into the middle introducing new concepts, and by the end, you twine and braid the concepts, draw analogies, point out disagreement or overlap, and so on. As always, I found that these connections leapt out at me as I was writing or during my editing. They weren't there to start with.
  • I took a day of vacation to do the actual writing, and the day went smoothly, without much stress. (Had there been an emergency that had taken me away from my home office, though, I would have been doomed.) I suppose, though, that I had a secret weapon, which is that I've been writing in one form or another since 1984. Most of the lit review writing advice I researched struck me as assuming you don't have much writing experience. I, however, know my writing process pretty well. I figured that if I soaked myself in the literature, and could come up with a logical storyline, then the writing would take care of itself. And I'm relieved to say that is, indeed, what happened (to my satisfaction, anyway).
  • One section I left out of the first draft was the conclusion. I felt it was better to wait and do that after I had let the paper cool down and I had put in my edits. Having spent the day intensely with my chosen material, I was able to write a more coherent conclusion that reflected connections that developed during the first-draft writing.
  • Still, I was up till, oh, the wee small hours. After I finished my draft, I took a two-hour break to do a workout, eat, watch a TV show, and practice my banjo. I edited a hardcopy printout, made notes to myself, and then typed in the edits and my conclusion. Ensuring the paper adhered to the APA style guide (and formatting my citations accordingly) actually was more time-consuming or felt like it.
  • The night of the day I finished the assignment, still tired but unable to sleep, I started reading my assignments for the next week. Taking time to pause and rest was probably as much celebration as I could emotionally afford. The best thing to do, I'm learning, is to have another project or task to pour that nervous energy into. (And this has implications for the night of graduation day, whenever that will arrive.)
  • I realized afterward my brain can make the connections between ideas all on its own without me having to force them, and that's rather a relief to discover. If I've stated the problem correctly, I'm interested in the question, and I'm not in a hurry, then all goes well. I don't have to be an expert, but I can be a sense-maker.
  • Always interesting to reflect that any piece of writing is the tip of an iceberg hiding the hours and pages of thinking and drafts. Would be interesting to study the ratio of material/effort expended for a paper to the final page count, so you could calculate that a page of manuscript will require 12-24 hours of effort, or something like that. I imagine someone's already done that.

We tend to think of the task of regulation as one of making systems hard to break. An alternative to consider is making systems easy to fix. Think of a computer. You can try to use firewalls and anti-virus software to make your computer hard to break. But it still pays to back up your data to make it easy to fix.

My Talk on Financial Regulation, Arnold Kling | EconLog | Library of Economics and Liberty

Source: econlog.econlib.org

“The real troubles in your life are apt to be things that never crossed your worried mind, the kind that blindside you at 4 pm on some idle Tuesday.” – Kurt Vonnegut

You will need to divide your current possessions into four major categories.

1. Beautiful things.
2. Emotionally important things.
3. Tools, devices, and appliances that efficiently perform a useful function.
4. Everything else.

Typology of New Yorker cartoons

Given the diversity of talents who over the years contributed cartoons to The New Yorker, it may be surprising to learn that everything in our large cartoon bank has, for the sake of easy reference, been reduced to a dozen or so categories...The categories were as follows: arts and galleries; bars and drinking; birds, fish, and animals; businessmen; cars and road signs; cavemen; children, babies, clergymen; cocktail parties; criminals, cops, jails, and judges; doctors and hospitals; heaven, hell, gods, devils, and so forth; olden times (royalty); old people; politicians and generals; musicians; restaurant and food; tourists, vacations; and finally TV and movies. As far as I know, we've never had anything in the bank that couldn't be easily fit into one of those categories. I'm sure the current bank contains a disproportionate number of TV and movie gags. whereas the cocktail party category is rather thinner than it used to be. Desert island and caveman jokes, of course, go on forever.

Lee Lorenz, The Art of The New Yorker 1925-1995, Knopf, 1995

Two projects, two fuzzy ideas, two lit review processes

The 696 independent study is starting out as a literature review of risk in institutional repositories -- where it's perceived to lie, and, what's interesting to me, who makes the actual decisions? The OAIS model defines the functions of an archival process but leaves the specifics of implementation to each institution. So, for various managerial functions within an archive (archival storage and data management, for example), those functions could be carried out by one person or teams of people. It depends on resources and staffing. Carolyn has advised me to contextualize the risks within the OAIS model and within institutional repositories, which provides me with a good basis from which to select my sources and also (we hope) prevent me from flying off in all sorts of different directions (such as defining risk, decision making algorithms, how risk is managed in other contexts, and so on). I've collected a mass of documents and web pages that I now need to sort through, skim/read, and decide what the current picture of the situation is like. She reminded me at today's meeting that the goal is not to solve a problem, just to describe the situation.

She liked my abstract and suggested headings/subheadings, so she's assured that I seem to be moving in the right direction. The precise path I'm still working out, but the direction is fine.

For the 780 Research Methods course, we received very good comments and annotations on our Problem Statements, which were intended to help us think through the research problem we're proposing, start looking for some literature to support it, and define the research questions that will drive our projects. The key here is to ask the right questions and make sure they're right-sized, so to speak.

As I was writing my statement, I could feel the question and underlying assumptions change under my fingers. That's OK, that's part of the process. (And the value of deadlines, it must be said, is that they focus one's mind powerfully. Damn them.) The professor started out liking my topic and then seemed to veer toward, well, maybe what you're really asking is this. And I have to agree with her.

So, I need to work on that section some more.

Upcoming is a literature review that has to include at least 8 pieces, at least 4 of which need to be empirical studies. Based on my 696 and problem statement experiences, I can tell that I'll need to review/download about 25-40 items to find references that inform what I want to do. The trick here is being sure in my mind what it is I want to do.

I spent this afternoon at the library and found 4 books on community networks that I hope will have either good info I can use or leads on studies. Generally, once you've found a good article or lit review on the topic, that's the mother lode that can lead to more and better items.

Must keep in mind, though, that the finished piece is due in about 10 days, which isn't much time, given the day job, doing our taxes, getting my car worked on, and other obligations. I've reluctantly realized that I'll never get a whole day to just sit and do this work, so I will have to find a way to fit what I have to do into the interstices of my day. Next weekend, though, will need to be devoted to the writing up of whatever I've found so I can discover whether what I've got will support my research ideas.

It always amazes me how few people go to graduate school, who actually enjoy studying.

The day I got no research done

  1. I unpack my stuff in the SILS liberry [1] and start researching.
  2. The Maternalistical Cassidy wheels in with Anastasia and asks if I have lunch plans.
  3. I pack up my stuff and we go to lunch (very pleasant).
  4. I unpack my stuff in the SILS liberry and start researching.
  5. People people people walk by and want to chat. Very pleasant but no work is done. [2]
  6. I get an email from Dr. T saying I've been accepted into SILS' doctoral program (!) and I was granted a DigCCurr II Fellowship (!!).
  7. I sit there stunned and forward her mail to various folks, like Liz and Cassidy. I also send her a thank-you mail.
  8. Not really knowing what else to do, and wanting to settle myself down, I go back to my research. About a minute later, Cassidy comes down and hugs my neck and is giddier and more excited about the news than I am. We chat a bit and process the news.
  9. She leaves to go back to her work and I return to my research. It's a little after 3pm.
  10. Dr. T finds me in the liberry and wants me to walk with her over to Daily Grind so she can get a coffee-booster before her 3:30pm talk.
  11. I pack up my stuff and we walk and talk about the offer.
  12. I finally give up and go home after getting about 20 minutes of research work done. This will be a hard semester.

[1] Many and many a year ago I worked in one of the tech-writing gulags of Northern Telecom. A young Southern lady who managed the Interleaf publishing resources often told us about the templates and files stored in the "liberry." Sorry, but that pronunciation just stuck in my head and I don't want it to leave.

[2] Lori says I should get used to this.

Assorted links

Why I don't use "lol" in emails. Example 2 is the killer.

Have you ever wondered what would happen if a nuclear bomb goes off in your city? Wonder no more with the Ground Zero Calculator. [via Librarian of Fortune]