Sunday, February 17, 2008

Is PDQ a good idea? An academic perspective

I was invited to join this blog (not sure why), so I guess that means I can put in my two cents on issues under discussion. I don’t understand the need for a pseudo-journal whose rationale is “providing a citeable format for people uncomfortable with citing weblogs.” Citing blogs is not difficult, see How to Cite Weblogs and Weblog Comments in MLA Style. From the perspective of research and scholarship, the most important thing about journals is that they are peer-reviewed. From my perspective as an archaeologist, I am most interested in knowing the difference between rigorous research and data on the one hand, and opinions and interpretations on the other. When I look at a an archaeology blog, in contrast to the Feb 8 entry by Alun, I am far more interested in whether it is from a recognized authoritative source than whether it is just interesting prose. Of course this is different from non-academic blogs or websites, where I am not an authority and not looking for authoritative information. I do enjoy reading some interesting archaeology blogs by nonacademics, but I try to keep the difference between scholarly and non-scholarly ini mind.

I should probably point out that that I take a strongly scientific approach to the past. I have criticized colleagues for confusing interpretive speculation with empirical findings (e.g., Smith, Michael E., 2005, Did the Maya Build Architectural Cosmograms? Latin American Antiquity 16:217-224.), and these publications have met with rather polarized receptions within Mesoamerican archaeology.

The problem of archiving content may or may not be important. In my view of the world of publishing (writ large), I don’t see much reason for having a good archive of blogs. If there are important and serious research contributions, they should be presented in peer-reviewed journals, monographs, in research-oriented web sites, and the like. I certainly don’t intend anyone to worry about my blogs after a rather short lifespan.

There are other perspectives and needs out there, and I respect the different goals and interests of others who use the internet to discuss and learn about the ancient world. Perhaps something like PDQ would serve some of those interests. But I am wary about trying to make non-peer-reviewed and ephemeral material try to look like a scholarly journal. To my mind much better ideas would be to set up a more permanent web site to archive good blog text or set up a site to link to specific blog entries, etc.

The open and democratic nature of the internet is important in countless ways, but it can be a problem for rigorous research and scholarship. Maybe my perspective is limited and limiting, but I'll end with Vince Gill, "that's my story and I'm stiking to it."


Charles Ellwood Jones said...

I invited you because you write interesting stuff ( relating to the ancient world. Welcome aboard.

Shawn Graham said...

'Pseudo-journal' is not really the appropriate word for what PDQ is trying to accomplish, and I think unnecessarily pejorative. The rationale regarding citation is only one purpose behind PDQ. Certainly, MLA-style citations for blogs exist; but what happens when the blog itself is no longer available, or the author decides he or she has had enough? It takes an enormous amount of energy to try to put quality thought and reflection out there. One niche the PDQ is envisioned to fill is a permanent open repository for these things.

The other niche is the one concerning 'authority'. We teach our students to be wary of websites for which they cannot determine the author. We forbid them to use the Wikipedia. But the fact remains that our students will turn first to the internet, to blogs and wikis, before they wander down to the library and try to find a copy of the Bolletino Communale. JSTOR is fantastic: but I've had maybe six students in the past two years of my intro to Roman culture course actually dig their way through the Library website to gain access to it. It is up to us then to devise ways of providing authority to good solid writing about the past, in the places where our students and the public will find it most easily. PDQ is one answer to this problem.

Traditional peer-reviewing works well, or else it would have been jettisoned years ago. However, I think there is room for alternative approaches to peer-review. I am attracted to the idea of letting it all hang out for the world to see - the evolution of the discussion of the PDQ is in itself a model for a new kind of peer-review.

For me, the greater attraction of something like PDQ is the fact that I write about, and research with, quickly evolving digital tools. Some of my agent-modeling work has been in press for two years now, but the platform I used then is already two or three major version changes out of date. My code is already a relic. Something like PDQ is necessary to get information out there pretty darned quickly. I'm also quite interested - though I don't blog about it personally - in the political uses and abuses of archaeology, archaeology's appearances in the popular press, and how that all plays out. There are issues there that need to be discussed, and *are* discussed on great blogs. These discussions however do not find their way into academic journals (at least not at the time they have contemporary relevance). Again, something like PDQ has a role in legitimizing the discussion.

Finally, and I may be being a bit flip here, I am reminded of the recording industry. No doubt, many record industry executives felt that cds and albums were perfectly good existing ways of getting serious music to its listeners, so who would want to download a single song? The point here is about gate-keeping, and deciding what gets out, and how it gets out, to the public. All of us involved with PDQ are serious academics, who want to make our subject, our interests, and our energies available to a wider public. We want to include that wider public serious about the past, in what traditionally is an exclusive project. We want to lower the barriers to participation, but do it in such a fashion to allow authority to emerge.

Alun said...

I think the original post raises serious questions and if the PDQ is purely for the converted then there isn’t be a lot of point to it. I’ll tackle some of the issues raised and skip others which are discussed elsewhere. First is there a need for an archive?

Web materials are being cited. It may be best that work should be in a journal, but the reality is that for one reason or another not all cited work is subjected to peer-review. I put up a paper on the IScience weblog for the purposes of getting feedback. I promoted it to the statisticians I could find and plan to re-write it into a peer-reviewed paper. In the mean-time it has already been cited in the OJA. The paper’s author and the OJA are trusting me not to tamper with the blog entry. I am happy to leave it as is, even though my opinions have shifted quite a bit. They’re also trusting me to leave in the same location and I cannot promise that.

Unfortunately today I learned that Dr Jim West’s weblog has been deleted, possibly maliciously. Given the way works it’s quite possible it will not be able to be restored. IScience is hosted on the same system and could well face the same problem. It’s not just independent bloggers that have this problem. The December issue of World Archaeology has a paper by by Christopher Witmore which cites the Stanford Symmetrical Archaelogy site. When I visited the server was not responding and the new site was lacking any meaningful text. Thankfully it’s back, but universities are prone to deleting sites when academics leave.

The proposal to have a central website would make sense, from a North American perspective. Recent decisions by the AHRC in the UK dealt a body blow to the credibility of the UK’s digital infrastructure. It would be optimistic to see one website as a secure solution. There will be an XHTML archive, but other formats will improve the availability of the content.

Peer-review is a serious issue. I would disagree with Shawn in that I’m not sure that peer-review has survived because it works. Peer-review is not a single concept. For instance recent research suggests that double-blind review for journals helps tackle gender bias, which among other things makes me wonder how many journals are practicing double-blind review and how many are not. Additionally many academic presses ask for the author to provide reader reports on their monograph. Does supplying your own reviews count as peer-review? As far as I’m concerned it doesn’t but I accept I appear to be a minority. I think the reason we haven’t seen more complaints along the lines of Archaeological Finds is that scholars do seem to be doing the best job they can in getting feedback. For more on peer-review see K. Kris Hirst’s blog entry on ‘Seeing the Light’.

Is the term pseudo-journal fair? Prof. Smith anticipates one of the questions I wanted to throw out to discuss in Vol 1.2. Journal is possibly a bad word for PDQ. It does conjure up images of gold-standard peer-review to some people. The fact that some journals may be trading on this ideal without merit doesn’t mean we should do the same intentionally or unintentionally. I’ve raised the idea that blogging can be seen as a perpetual conference. Ideas are put out and discussed but just as a conference is not an alternative to publishing in a journal or monograph nor should a weblog post be seen as an alternative either. Conference proceedings may be the better level to aim at initially. If blogging becomes more widely adopted then Hirst’s open review model becomes feasible, but even then I think a Topaz system would be better than PDF/XHTML.

As for authority, my views on the authority of some academic monograph publishers are clear. I’m also aware that some professors feel they are authoritative by proxy in other fields as well as their own. Ideally a journal would send interdisciplinary papers to multiple referees expert in all the relevant disciplines. This does not always happen in journals and never happens in weblogs. Additionally I’ve read weblogs by people claiming to be professors or holders of PhDs who plainly aren’t. As a result I read all weblogs critically on a case-by-case basis and really only give value to the words in the entries rather than claimed credentials. If their arguments are logical, based on sound evidence, refer back to the context of work by other scholars and make clear where they divide fact from speculation I find them interesting. To some extent my definition of interesting is flexible, For instance a logical argument which shows where it draws on other scholars’ work could be interesting even if the evidence is scant.

In contrast no titles or affiliations with major departments are going to persuade me that something demonstrably wrong is true. I tend to find such things dull. Given that I’ll do well to have another 40 years on the planet they’re also a waste of my time, so I tend not to comment on them. Again I appreciate opinion is not universal. For instance I disagree with one professor over whether or not north and south are opposite directions. I think they are, but as he points out he is the professor (and doctor). It’s a line of argument that I find unproductive which is why I haven’t responded to it (until now). I’m not convinced his declarations of authority are as helpful to the study of Greek temple orientations as thirty seconds with a map and a compass would be

Fortunately for my sanity there does seem to be a correlation between whether or not a blogger is an active scholar and whether or not I find them interesting.

So what alterations can be made? I can’t see that anything is gained by reducing the formats the whatever-it-is is published in. I can see that the word journal may cause confusion and I need to re-edit text on site. Unfortunately I’m not sure the term periodical is any better. I’m happy to take suggestions for another word. Webzine comes to mind. It lacks authority and prestige, but as Shawn indicated above we should make our own.

Scribalist said...

I have a couple of comments to chuck in here. First, the thing I've always liked about blogs is that by themselves they are often a form of post-facto peer review. Discussing the implications of a particular publication that has already undergone the peer review process is perhaps not exactly open peer review, but to my mind it certainly is a big step in that direction. The review pieces are signed; and (as Alun points out), the reviews may be written by whoever chooses to do so, regardless of credentials or academic standing, and the reader may make of them what they will.

Secondly, the transient nature of media of all types is a difficult issue being uneasily addressed by librarians everywhere (Bruce Sterling's Dead Media Project comes to mind.) Floppy discs, tape, acid-based paper are not permanent media either. The PDQ project offers an archiving layer of data that arguably adds to our understanding of the past. I (like Martha Stewart) think that's a good thing.

Scribalist (aka) Kris Hirst

Michael E. Smith said...

I don't know whether my phrase "pseudo-journal" was fair or not. I am curious to see where this all ends up. There is a big world out there of electronic archaeology and I wish I had time to explore it. But I do find these developments interesting.