On open access in scientific publications

A recent article in "The scientist"  issued a blunt attack on open access in scientific publications, call it: predatory publishing.  This article claims that open access publications will not allow people to distinguish between "science" and "non-science", since it will not guarantee the quality of publications through the rigorous process of peer reviews, and then gives the main reason for objection --- "implications for tenure and promotion will be significant".    Some of the comments compared the arguments with those made against open source software.   

I think that there are some orthogonal issues here:  first -- I am for open access, actually every person today can put any paper on a website, and now all we need is to index it so it will be accessible to all.   Scientific publication is not different in principle from newspaper article, blog, TV show, book - in all of them we can find low quality and high quality examples.  

The review process is highly subjective and noisy,  and the amount of false positives and false negatives is quite high, it provides a quality control which is better than nothing, but it does not guarantee anything.  There can be some quality evaluation models, but this is a different discussion. 

I have written before about the fact that  the need to measure everything is taking over work in the corporate world,  this is also the case in academia,  in many cases universities count papers,  or use some other quantitative metrics, which fits the current publication system.    A new system will require to change this criteria which is an earthquake in the academic system (which needs earthquakes from time to time)...

I think that open access in scientific publication is happening and resistance is futile.....

On quality evaluation of scientific publications

Yesterday, the entire Etzion tribe visited the Technion to attend the ceremony in which my daughter Anat received a plaque of being in the president's list.    Since I don't have yet pictures from this ceremony I am posting pictures of the Technion president, Professor Peretz Lavi, and of Anat doing some non-academic tasks while renovating her apartment.

Talking about academia, in a recent posting on  the complexevents forum, Rainer has put a reference to a European project that is aimed to take a new approach to scientific publications.   In the relatively early days of the Internet (somewhere in the 1990-ies) there has been a panel in one of the conference (I think it was VLDB in Zurich) about the future of scientific publications in the Internet era.   The observation is that now a person can just post papers on the web and make them available to people, instead of publication in scientific journals (or conference proceedings, a computer science anomaly - most disciplines don't view it as a real publication).   However, the peer-reviewed publication process also serves as "quality control", moreover, in the academic world,  the promotion metrics are largely depend on publications (how much? where? how many citations?)  lead to metrics like H-Index and G-Index.   Since our world is increasingly metrics oriented, everything is measured, satisfying the metrics becomes the goal, and the way to do it becomes means (instead of vice versa), this is, by the way, true for other areas - not just scientific publications, but now we discuss scientific publications.   The original proposal at the panel, as far as I remember, was that everybody will freely publish paper on the web, but will get certification by peer-reviewed institutes, whose business models is that the author pays for review and they pay reviewers, authors can apply to several levels of certifications (grade A certification paper, grade B certification paper etc...).    Actually this did not happen, the scientific journals are still alive, although most accesses are now through digital libraries and not through hard copies.  I used to have a lot of journal subscriptions to hard copy journals, and when the IBM Haifa Research Lab migrated to the current building 10 years ago, I thought it is a good time to throw away all journals (leaving some journal issues with my papers published in) and move to digital libraries subsciptions.    


Now, the European project mentioned above, tries to re-invent the publication model claiming the failure of the peer reviewed system, saying there is low correlation between the grade achieved in the peer review process by three researchers who happend to be selected as reviewers and the impact of the paper in reality, and replace it by the "wisdom of the crowd", everybody can comment and endorse the paper.


As a side comment, I never liked the metrics of number of citations (always reminds me the old metrics based  on counting   lines of code), this is certainly a metrics for visibility, but not necessarily a quality mertics,  let's assume that somebody quotes my paper saying -- "this paper attempted to approach the same issue, but its solution was based on faulty assumptions", this is counted as a citation the same as another quote saying -- "we are basing our approach upon the seminal work done in that paper", again counted as one citation.  Also when citation number metrics became popular in evaluation processes, it encouraged the development of clicques that cite each other,  all of these are not reflected in metrics.  


Since I am working in an industrial research institute and not in a university,  my interest is to evaluate the impact of a scientific paper on the world,  where citations is only one (and maybe in some cases a minor) factor.   Codd's paper on the relational databases was cited by 1844 other papers (according to the  Microsoft Academic site)  but it is obvious that this is a minor consideraiton in its impact on the world. 
I think that one of the challenges is to measure this impact, which oftern can be done only in retrospect. 


More thoughts about it - later.