literature review challenges: search terms and depth

As I build my preliminary bibliography for the games & tourism literature review, I’m running into a number of challenges.

First, as I mentioned in my workflow post, the ubiquity of the word “game” makes it difficult to construct a search that yields the right results. A search that returns 70,000 results is almost as useless as one that yields only two or three. What has been more productive has been identifying an initial set of strong articles (both theoretical and case study) from both computing and tourism literatures, and then reviewing the bibliographies of those articles as well as the list of citing articles (via Google Scholar searches). I began with about 150 articles that I’d tentatively identified as relevant, and have been working my way through them, alphabetically by title, moving the useful items to a “vetted” collection, and adding the new items from bibliographies/citations as I go. I’m nearly through the articles beginning with G, and now have over 300 articles in the preliminary collection.

This leads to the next problem, which is knowing how far I should follow down the rabbit hole of prior work. For instance, if a case study on a tourism game cites an article on co-creation in tourism, and that article in turns cites an earlier theoretical work, do I need to acknowledge and include all three articles? This is a little easier for me to answer in the computing and game studies literature, since I have a better sense of what needs to be explicitly acknowledged/summarized, and what can be implied. But I don’t have that same understanding of the tourism literature, especially when it comes to core concepts like experience design and co-creation.

For now, I’m erring on the side of inclusion, since it’s easier to remove or ignore the articles when I start writing than it would be to track them back down again. But it’s making the preliminary evaluation of citations a LOT slower than I’d initially hoped.

literature review tools & workflow

Since a number of people asked me on Facebook about my literature review process, I decided to write up a detailed version of my answer.

First, the tools I’m using. On my Mac: Zotero citation manager (stand-alone software plus Chrome browser extension), university VPN for access to my university library’s database subscriptions. And on my iPad: PaperShip for access to my Zotero library, VPN for library database access, Zotero bookmarklet for mobile Safari.

(All of those tools are free, although I paid $10 to upgrade PaperShip so that I could annotate PDFs inside the app, and I pay $20/year for 2GB of Zotero cloud storage so that I have access to the PDFs of articles from any of my devices.)

On to the process…

I started with a series of searches in Google Scholar for terms related to my research (the intersection of games and tourism). This turned out to be particularly challenging (even for a trained librarian like me), since the word “game” can be used in so many ways–i.e. olympic games, game animals, game theory. As I encountered citations that looked like they’d be relevant, I used the Zotero extension in Chrome to add those items to a collection I called “Preliminary Lit Review.” In most cases, I made decisions based on title and abstract, since this was a first pass. (I did exclude works that were from known predatory publishers, like igi-global.) When I’m logged into our university’s VPN, the plugin does an excellent job of retrieving the full text of the article along with the citation. In the few cases where it didn’t, I attempted to find the PDF myself using our library’s online databases, and then ordered the remaining items via interlibrary loan.

When I found articles that looked particularly relevant, I used Google Scholar’s “cited by” feature to find related work. Once I started running out of relevant works in Scholar, I re-ran my searches in a number of databases that our university subscribes to, including ACM DL and IEEExplore for computing literature, and SpringerLink, Web of Science, and ProQuest for tourism literature. For each of these searches, I again used the Zotero plugin to add the citations. At the end of this process, I had about 180 citations, complete with full text of the item, stored in my “preliminary” collection.

I then began working my way through the preliminary collection. I open the full text of the article, and scan through it to see if it has clear relevance to the review I’m doing. I’m excluding games that are entirely virtual/digital, with no connection to real world exploration. I’m also excluding items that mention games so peripherally that they add no additional information to the review. For each of these items, I’m adding a note to the Zotero entry so that I can remember later why it wasn’t included.

If the article seems relevant, I then add tags to help me categorize it for later steps. While I’ll need to add more tags once I start writing and develop a structure for organizing the literature, right now I’m primarily focused on simple descriptors like “case study”, “mobile game”, and “augmented reality”. I also review the article’s bibliography to see if there are additional citations that look useful. For articles that seem particularly useful, I’ll check Google Scholar so that I can look over the “cited by” data. I then copy the citation to the “final” review collection.

Working my way through the preliminary collection in this way, I seem to have gotten close to a saturation point at only about the 20% mark. The preliminary collection has grown to ~240 citations, but increasingly the bibliographies and referencing works are items that I already have in the preliminary collection. I suspect there won’t be a lot more that need to be added.

I’m using my iPad when I’m away from my apartment. PaperShip gives me access to my Zotero library, and I can download any of the full text papers from my Zotero cloud storage. PaperShip also provides annotation tools that I can use to either add notes using the iPad keyboard, or directly on the page using the Apple Pencil. Annotations are synced with the PDF, and available on my computer later. This makes it easy for me to do the more intensive reading wherever I am, without carrying the computer with me (something that’s more of a challenge as I recover from my broken foot).

I’m also using Microsoft Word to take notes as I work; I save the file to Dropbox, and can then access it with Word on my iPad as well. I didn’t list Word in the tools above, since notetaking can be done with any editor.

iPad Pro 9.7″: first impressions

(This was originally posted as a Facebook note on 3 April.)

Okay, here’s the lengthier version of my first impressions of the iPad Pro, after having gotten it fully set up and playing with a variety of applications.

Specs: 128GB wifi-only, rose gold (to match my phone), $729 after edu discount. Bought the Pencil ($99) and Applecare ($99), but not a case or keyboard–I’m holding out for the Zagg folio case, which hasn’t shipped yet. I traded in my iPad Mini 2 for a $115 gift card to apply to the purchase, which was a seamless process and netted me more than Gazelle would have given ($70). Total, with edu discount and no tax, was $812.

tl;dr: I love it. It’s been a long time since I’ve gotten a new device and thought “Wow, this is going to significantly change (and improve) the way I work.”

Keyboard: I’m able to use it with my old iPad3 Logitech keyboard. It doesn’t work as a case, but I can set the new one in it and use it as a keyboard until I get my Zagg case, and it makes using the iPad for writing completely feasible. I’ve got all the Office apps installed (Word is my word processor of choice), and can keep files synced between iPad and computer using Dropbox or OneDrive. (This was a huge issue for me with the Mini; even with my tiny hobbit hands, a mini-sized keyboard was unusable for real typing.)

Pencil: I could not love it more. My biggest frustration with styluses on the mini has been that my palm regularly triggers unwanted input. “Palm rejection” works beautifully with this setup, making it possible to write and draw in a comfortable way without sabotaging my own work. It’s a game changer for usability of the iPad in research work.

Research Tools: My goal with this purchase was to have an easily portable device that makes it easy for me to carry research papers with me wherever I go, and then to be able to annotate them while reading. All my research papers are in my Zotero database (citations and PDFs). I was using PaperShip on the mini to access and read them, but couldn’t find a way to effectively annotate them. With the Pencil added to the setup, PaperShip’s annotation tools are well worth the one-time $9.99 cost–I can draw, highlight, add notes, in a very intuitive way. The annotations are saved to the PDF in my Zotero library, and are reflected when I look at the paper in the Zotero program on my computer. I could not be happier with this workflow!
Screen: The TrueTone makes more of a difference than I’d thought–if I turn it off, I can see how blue the screen seems when I’m working in incandescent light. And while it’s not matte, it definitely reflects much less light than any iPad I’ve owned previously.

Miscellaneous Other Stuff: (1) I’m quite intrigued by Adobe Comp, which allows me to hand draw wireframes and have them automatically convert to vector objects (image placeholders, text boxes, etc) so that I can do page layout on my iPad and then easily send it to Photoshop, Illustrator, or inDesign on my computer. Haven’t really put it through its paces yet, but I’m excited about it. (2) I have the Texture app for magazine reading, and magazines look fabulous on it. Much easier to read than on the mini. (3) For the first time ever, I’m actually thinking about starting to play with freehand drawing and sketching on a digital device. Not sure what tools to use for this; open to suggestions from friends who have more experience.

back to blogging

Yesterday, I was explaining in a Facebook post’s comment thread what my literature review workflow currently looks like.

Last month, I wrote up a lengthy Facebook “note” about my first impressions of the iPad Pro 9.7″, which I’m using heavily as part of my workflow these days.

But neither of those are easily searchable or shareable, and I’m realizing that if I really want these posts to be helpful, I need to go back to blogging them. My attempts to kickstart my blog (, which I started in 2002 and maintained actively for over a decade) have been unsuccessful, so I’ve started this new blog to document my academic work.

We’ll see how it goes.